Abstract
Background
Studies in cancer and noncancer populations demonstrate lower than expected correlations between subjective cognitive symptoms and cognitive functioning as determined by standardized neuropsychological tests. This paper systematically examines the association between subjective and objective cognitive functioning in patients with low-grade glioma and the associations of these indicators of cognitive function with clusters of sociodemographic, clinical, and self-reported physical and mental health factors.
Methods
Multiple regression analyses with the subjective and 2 objective indicators of cognitive functioning as dependent variables and 4 clusters of predictor variables were conducted in 169 patients with predominantly low-grade glioma.
Results
Correlations between the subjective and the 2 objective cognitive indicators were negligible (0.04) to low (0.24). Objective cognitive deficits were predominantly associated with sociodemographic (older age, lower education, male sex) and clinical (left hemisphere tumor) variables, while lower ratings of subjective cognitive function were more closely related to self-reported mental health symptoms (fatigue, lower mental well-being), physical (motor) dysfunction and female sex. Self-reported communication deficits were associated significantly with both subjective and objective dysfunction.
Conclusions
We recommend that both subjective and objective measures of cognitive functioning, together with a measure of psychological distress, be used for comprehensive neuropsychological assessments of patients with glioma to determine which areas are most affected and which specific intervention strategies are most appropriate.
Keywords: cognitive function, glioma, mental health, predictors, self-reported cognitive symptoms
Problems with cognitive functioning may be experienced subjectively and reported as symptoms on a self-report questionnaire, or they may be assessed more objectively by means of standard neuropsychological tests. A consistent finding from the research literature is that the correlation between subjectively assessed cognitive symptoms and objectively determined cognitive functioning is quite modest (r generally ranging from 0.20 to 0.30, thus explaining 4%–6% of the variance). These lower-than-expected correlations have been observed in patients with stroke,1 traumatic brain injury,2 multiple sclerosis,3 epilepsy,4 bipolar disorder,5 and cancer.6–10 Moreover, there is evidence that subjective cognitive symptoms may be more closely related to other emotional and mental symptoms, such as depression and fatigue, than to objective neuropsychological deficits.1–4,6,8–12
Few studies have examined these associations in patients with primary brain tumors. In an early study by Taphoorn et al,13 in 13 low-grade glioma patients, there was no difference observed between patients with high versus low cognitive impairment in mean levels of self-reported concentration and memory problems. In another study in 68 newly diagnosed high-grade glioma patients, deficits in attention, information processing, and graphomotor speed correlated moderately with self-reported cognitive functioning (r ranging from 0.33 to 0.49).14 In a group of 195 patients with low-grade glioma who were diagnosed an average of 5.6 years before the study, objectively assessed memory, attention, psychomotor speed, and graphomotor speed correlated moderately with self-reported cognitive functioning (r ranging from 0.23 to 0.34).15 Later, a multivariate analysis of the same sample demonstrated that women (r = 0.24) and patients with a greater number of deviant neuropsychological test scores (r = 0.34) reported more cognitive symptoms.16
In this paper, we report the results of a study in which we systematically examined the association between subjective and objective indicators of cognitive functioning, and the ability of 4 sets of predictor variables (ie, sociodemographics, clinical variables, and physical and mental health symptoms) to predict both subjective and objective cognitive functioning in patients with predominantly low-grade glioma. In this way, we hope to identify explanatory factors for, and thereby obtain a better understanding of, the weak association between subjectively reported and objectively assessed cognitive functioning.
Methods
We used the baseline data from our randomized controlled trial of a cognitive rehabilitation program in patients with glioma17–19 for the current analysis.
Patients and Procedure
Eligible patients for the trial were identified via pathology databases or direct referral from 11 Dutch hospitals. Medical inclusion criteria were as follows: histologically proven or presumed (on the basis of clinical and MRI features) diffuse, low-grade (ie, WHO grade II) glioma (ie, astrocytoma, oligodendroglioma, or oligoastrocytoma) and age between 18 and 70 years; or anaplastic glioma (grade III), age younger than 50 years, and good performance status (ie, Karnofsky Performance Score >70). Patients had to be clinically stable (ie, without any evidence of disease progression) for a minimum of 6 months before entry into the study and could not be receiving antitumor treatment during that period.
Exclusion criteria included the following: any additional serious neurological or psychiatric disorder; inability to undergo the neuropsychological assessments or cognitive rehabilitation program because of premorbid IQ score <85; visual, motor, language, or other severe cognitive problems; lack of basic proficiency in Dutch; or participation in a concurrent study with neuropsychological testing and/or health-related quality of life (HRQoL) assessments.
Medically eligible patients were invited by their physicians to undergo screening for cognitive eligibility. They (n = 366) were first screened by telephone interview for the presence of subjective cognitive symptoms. Those who reported at least one cognitive symptom from the Medical Outcomes Study (MOS) Cognitive Functioning Scale (CFS)20 (Table 1) and who indicated interest in participating in a cognitive rehabilitation program were referred for objective neuropsychological testing (n = 207). These baseline data were used for the current analysis.
Table 1.
Variables | Description | Descriptives M & SD/Med & R /count (%) |
r Attention/Executive | r Verbal Memory | r CFS | |
---|---|---|---|---|---|---|
Outcome variables | ||||||
Attention and executive factora | Objective attention/executive score (from PCA) | M: 0.0 | SD: 1.0 | 1 | 0.24d | 0.24d |
Verbal memory factora | Objective verbal memory score (from PCA) | M: 0.0 | SD: 1.0 | 0.24d | 1 | 0.04 |
CFS total scorea,c | Frequency of cognitive complaints (6–36) | M: 22.1 | SD: 4.4 | 0.24d | 0.04 | 1 |
Predictor variables | ||||||
Sociodemographic | ||||||
Sex | Male vs female | 92 (54%) | 77 (46%) | 0.02 | 0.32d | −0.27d |
Age | Years (range 20.8–69.7) | M: 43.2 | SD: 9.6 | −0.27d | −0.15e | −0.07 |
Education | Low/middle vs high | 88 (52%) | 81 (48%) | 0.18e | 0.31d | 0.04 |
Clinical | ||||||
Hemisphere of lesion | Left vs right | 93 (55%) | 76 (45%) | 0.26d | 0.09 | −0.01 |
Use of antiepileptic drugs in last year | No vs yes | 32 (19%) | 137 (81%) | −0.13 | 0.04 | −0.04 |
History of radiotherapy | No vs yes | 62 (37%) | 107 (63%) | −0.03 | 0.07 | −0.06 |
History of chemotherapy | No vs yes | 153 (91%) | 16 (9%) | −0.22d | −0.11 | −0.04 |
Disease duration | Years | Med: 4.7 | R: 0.6–28.9 | −0.16e | −0.01 | −0.03 |
Physical health symptoms | ||||||
SF-36 physical component summary scorea | Physical-health-related quality of life21,22 (0–100) | M: 46.5 | SD: 9.2 | −0.03 | −0.02 | 0.25d |
QLQ BN-20 visual disorderb | Visual disorder26 (3–12) | Med: 4.0 | R: 3.0–12.0 | −0.09 | −0.07 | −0.35d |
QLQ BN-20 motor dysfunctionb | Motor dysfunction26 (3–12) | Med: 4.0 | R: 3.0–9.0 | −0.13 | −0.10 | −0.36d |
Mental health symptoms | ||||||
SF-36 mental component summary scorea | Mental-health-related quality of life21,22 (0–100) | M: 43.7 | SD: 10.1 | 0.12 | 0.02 | 0.33d |
MFI mental fatigueb | Mental aspects of fatigue25 (4–20) | M: 14.3 | SD: 3.6 | −0.10 | −0.05 | −0.42d |
QLQ BN-20 communication deficitb | Communication deficit26 (3–12) | M: 5.8 | SD: 2.0 | −0.32d | −0.15 | −0.39d |
QLQ BN-20 future uncertaintyb | Future uncertainty26 (4–16) | M: 8.1 | SD: 2.9 | −0.07 | 0.00 | −0.20d |
Abbreviations: CFS, Cognitive Function Scale; M, mean; Med, median; MFI, Multidimensional Fatigue Inventory; PCA, principal component analysis; QLQ BN-20, Brain cancer specific Quality of Life Questionnaire; r, Pearson's product-moment correlation; R, range; SD, standard deviation; SF-36, Medical Outcomes Study Short-Form 36.
aHigher score indicates better (subjective/objective) functioning.
bHigher score indicates lower (subjective/ objective) functioning.
cfrom the MOS.20
dCorrelation is significant at the 0.01 level (2-tailed).
eCorrelation is significant at the 0.05 level (2-tailed).
The trial was approved by the institutional review boards of the 11 participating hospitals, and participants provided written, informed consent. The study was registered on ClinicalTrials.gov (NCT00256425).
Measures
The neuropsychological test battery consisted of 11 tests and incorporated 26 variables measuring aspects of attention, verbal memory, and executive function (see Table 2).
Table 2.
Variable | Aspect(s) of Cognitive Function Measured | Pattern Matrixa |
Structure Matrix |
r | ||
---|---|---|---|---|---|---|
Attention/Executive | Verbal Memory | Attention/Executive | Verbal Memory | CFS Total | ||
LDST 90 s Writing | Psychomotor speed and speed of information processing | 0.74 | 0.06 | 0.76 | 0.27 | 0.20* |
LDST 90 s Reading | Letter reading and speed of information processing | 0.77 | −0.04 | 0.76 | 0.18 | 0.19* |
MST Intercept | Time to complete non-memory stages of information processing | −0.62 | 0.08 | −0.60 | −0.09 | −0.17* |
MST Slope | Speed of memory scanning | −0.50 | 0.00 | −0.51 | −0.15 | −0.07 |
CST Card A | Basic attention and information processing speed (numerical) | −0.72 | 0.09 | −0.74 | −0.29 | −0.14 |
CST Card B | Basic attention and information processing speed (alphabetical) | −0.73 | 0.04 | −0.72 | −0.17 | −0.03 |
CST Card C | Alternating attention | −0.70 | −0.12 | −0.73 | −0.31 | −0.12 |
SCWT Card I | Basic attention and information processing speed (reading) | −0.77 | 0.09 | −0.75 | −0.13 | −0.17* |
SCWT Card II | Basic attention and information processing speed (color naming) | −0.85 | 0.18 | −0.80 | −0.06 | −0.15* |
SCWT Card III | Attentional inhibition of a dominant response | −0.86 | 0.06 | −0.84 | −0.18 | −0.07 |
SCWT Interference | Attentional inhibition of a dominant response | −0.56 | −0.05 | −0.57 | −0.20 | 0.04 |
DS Forward span | Immediate verbal recall | −0.53 | 0.04 | 0.54 | 0.19 | 0.15* |
DS Backward | Verbal working memory | −0.55 | 0.12 | 0.58 | 0.27 | 0.14 |
LF Total score | Speed and flexibility of verbal thought process | 0.46 | 0.29 | 0.55 | 0.42 | 0.05 |
CF Total score | Speed and flexibility of verbal thought process, application of strategies | 0.49 | 0.26 | 0.57 | 0.39 | 0.10 |
TEA El Counting | Sustained auditory attention | 0.36 | −0.04 | 0.35 | 0.06 | 0.05 |
TEA El Dis | Auditory selective attention | 0.41 | −0.10 | 0.38 | 0.01 | 0.06 |
TEA El Rev | Auditory working memory | 0.56 | 0.04 | 0.57 | 0.19 | 0.23* |
TEA Tel+Count | Divided attention | −0.35 | −0.09 | −0.38 | −0.19 | −0.03 |
VVLT Trial 1 | Immediate verbal span | 0.00 | 0.83 | 0.22 | 0.83 | 0.04 |
VVLT Total | Verbal learning (total of 5 verbal memory learning trials) | 0.06 | 0.91 | 0.31 | 0.93 | −0.03 |
VVLT Delayed Recall | Verbal memory spontaneous recall after an interval | 0.16 | 0.85 | 0.39 | 0.89 | −0.03 |
VVLT Delayed Recognition | Verbal memory recognition after an interval | −0.10 | 0.71 | 0.10 | 0.67 | 0.09 |
Abbreviations: CF, Category Fluency, from the Groningen Intelligence Test34; CST, Concept Shifting Test29; DS, Digit Span from the WAIS-R32; El Counting, Elevator counting; El Dis, Elevator counting with Distraction; El Rev, Elevator counting with Reversal; LDST, Letter Digit Substitution Test27; LF, Letter Fluency33; MST, Memory Scanning Test28; SCWT, Stroop Color-Word Test30,31; s, second; TEA, Test of Everyday Attention35; Tel+Count, Telephone search while Counting; VVLT: Visual Verbal Learning Test.36
r = Pearson's product-moment correlation.
*Correlation is significant at the 0.05 level (2-tailed).
aRotation converged in 4 iterations. Underlined: factor loadings >0.4.
We used the CFS20 as a measure of subjective cognitive function (Table 1). This 6-item scale assesses day-to-day problems in cognitive functioning including difficulty with reasoning and problem solving, slowed reaction time, forgetfulness, and problems with concentration in the last week. Cronbach's alpha for the total scale was 0.743 in the study sample, indicating good reliability.
The MOS SF-36 Item Health Survey (SF-36)21 was used as a self-report measure of health status and HRQoL. It consists of 36 questions. Summary component scores for physical and mental health were calculated. The SF-36 has been translated, validated, and normed in a large number of languages, including Dutch.22 It has been shown to have excellent reliability and validity when used in the general population and with diverse patient populations, including cancer patients in the Netherlands.22,23
The brain cancer-specific HRQoL questionnaire QLQ BN-2024 was employed as a supplementary questionnaire to assess additional HRQoL problems associated with glioma and its treatment. Thirteen of the 20 items comprising this questionnaire are organized into 4 subscales assessing future uncertainty, visual disorder, motor dysfunction, and communication deficit. The remaining 7 items assess other disease- and treatment-related symptoms common to patients with brain tumors.
Fatigue was measured with the Multidimensional Fatigue Inventory (MFI).25 This is a 20-item self-report instrument that covers the following dimensions: general fatigue, physical fatigue, mental fatigue, reduced motivation, and reduced activity. The mental fatigue scale was used as a predictor in the analyses.
The individual predictor variables constituting the 4 clusters of sociodemographic, clinical, and self-reported physical and mental health variables are listed in Table 1.
Statistical Methods
Preparatory Statistical Analyses
We employed principal component analysis (PCA) with oblique rotation (direct oblimin) to reduce the large set of individual neuropsychological test variables to a more manageable number of factors for use in further analysis.
The cutoff for the number of factors retained from the PCA was based on the criterion of the point of inflexion on the scree plot. We used oblique rotation because the factors were expected to correlate. The overall Kaiser-Meyer-Olkin (KMO) measure was used for verification of sampling adequacy for the analysis.
Individual participant scores were subsequently calculated for the resulting 2 factors by the regression method available in IBM SPSS Statistics 20.0. These scores represent estimates of the scores participants would have received on each of the factors had these scores been measured directly. Finally, we employed the resulting higher order neuropsychological test factor scores as outcome measures of objective cognitive function in the primary statistical analyses.
Primary Statistical Analyses
First, we calculated Pearson's product-moment correlations of the (2) objective neuropsychological test factor scores generated by PCA, with the total score of subjective cognitive symptoms as assessed by the CFS.
To identify variables significantly associated with the subjective and/or objective indicators of cognitive functioning, we carried out 4 multiple regression analyses. The first 3 included the 2 objective neuropsychological test factor scores and the CFS total score as the dependent variables. In each of these analyses, the same clusters of potential predictor variables were entered sequentially. The order of introducing clusters into the regression analysis was: sociodemographics; clinical variables; self-reported physical symptoms; and self-reported mental symptoms (see above). With this sequence, higher priority is given to clusters of variables that are presumed to take causal precedence over other variables (eg, clinical factors are more likely to cause mental health symptoms such as fatigue than vice versa). The fourth and final multiple regression analysis was similar to the third analysis for predicting the CFS total score. However, in this analysis, we included the 2 objective neuropsychological test factors resulting from the PCA as a fifth and final cluster in order to examine the role of these objective measures for explaining variance in subjective cognitive function.
We checked the statistical assumptions before we conducted the multiple regression analyses. Finally, we carried out sensitivity analysis to investigate the possible influence of missing values. We used a multiple imputation technique (estimation maximization) to estimate values for missing data. For all statistical tests, we used IBM SPSS Statistics 20.0.
Results
Of the 207 participants who were initially included in the study sample for screening both subjective and objective cognitive functions, 23 were excluded from the current analysis for the following reasons (see Figure 1): 5 for intervening medical problems, 5 for severe neurological or psychiatric problems, 5 for serious language/reading problems, 5 with too much missing data, 2 with premorbid IQ too low , and 1 who did not attend scheduled neuropsychological assessment. This left data from 184 participants for the analyses.
Results From the Preparatory Statistical Analyses
Three of the initial 26 neuropsychological variables were excluded due to violation of statistical assumptions required for PCA. Thus, PCA was conducted on 23 neuropsychological test variables (Table 2). One participant was excluded from the analysis because of a missing score on one of the neuropsychological variables, resulting in a total 183 participants for this analysis (see Figure 1). The overall KMO statistic of 0.80 indicated adequate sampling.
Two factors (Table 2) were retained, which together explained 46.5% of the variance in test scores. The first factor, predominantly consisting of attention and executive functioning variables (with some components of speed of information processing and working memory), was subsequently labeled “attention and executive factor.” The second factor, based on verbal memory variables, was labeled “verbal memory factor.” We calculated the scores for both factors for all 183 participants.
Results From the Primary Statistical Analyses
Due to missing data on the CFS total score (n = 1) and occasional missing scores on predictor variables (n = 13), data were used from the final 169 participants for the subsequent correlational and multivariate analyses (see Figure 1).
Sample characteristics and questionnaire scores of the 169 participants are reported in Table 1. Additional sample characteristics were as follows. Sixty-five percent of participants had undergone resection and 28% had been biopsied for histopathological diagnosis. Most participants (80%) had been diagnosed with a WHO grade II glioma; the remaining 20% had a grade III glioma. Forty-five percent of participants had been diagnosed with an astrocytoma, 32% with an oligodendroglioma, and 16% with an oligoastrocytoma. The patients who did not have a formal histopathological diagnosis (7%) were presumed to have a (grade II) glioma based on clinical and MRI features. Median time since last treatment was 3.1 years (range, 0.6–20.4 years).
Correlations
There was a low, but significant correlation between the subjective cognitive score as assessed by the CFS and the attention and executive factor (r = 0.24) and a negligible, nonsignificant correlation between the CFS and the verbal memory factor (r = 0.04). Univariate correlations between the 3 dependent variables and the various independent variables are displayed in Table 1. The correlations of the individual neuropsychological test variables with the CFS total score, as displayed in Table 2, were not higher than the highest of the 2 correlations of the factor variables with the CFS. In addition, all 6 individual CFS items (with one exception, item 6 ‘slow reaction’; r = 0.28) correlated less well with the 2 objective cognitive function factors than the total CFS score.
Prediction of Attention and Executive Factor Scores
Of the 4 clusters of predictor variables, the sociodemographic and clinical clusters contributed significantly to explaining the variance in the attention and executive factor (Table 3). In the final model that included all 4 clusters, older age (B = −0.02; 95% CI, −0.04 to −0.01) and lower education (B = 0.45; 95% CI, 0.17–0.73) were the significant sociodemographic predictors of lower attention and executive factor scores. Of the clinical variables, left hemisphere tumors were associated with worse attention/executive function (B = 0.38; 95% CI, 0.08–0.67). Finally, with regard to the (nonsignificant) cluster of mental health symptoms, participants who reported a higher level of self-reported communication deficits (B = −0.09; 95% CI, −0.17 to −0.01) had significantly worse attention/executive function test scores.
Table 3.
Cluster | R2 | Adjusted R2 | Std. Error of the Estimate | R2 change | Sig. F change |
---|---|---|---|---|---|
1. Sociodemographic | 0.11 | 0.09 | 0.94 | 11% | <0.01* |
2. Clinical | 0.23 | 0.20 | 0.88 | 13% | <0.01* |
3. Physical | 0.27 | 0.21 | 0.87 | 3% | 0.08 |
4. Mental | 0.29 | 0.23 | 0.87 | 3% | 0.18 |
Unstandardized Coefficients |
Standardized Coefficients |
||||
Variables of final model (4) | B | Std. Error | Beta | Sig. | |
(Constant) | 2.23 | 1.14 | 0.05 | ||
Sex | −0.12 | 0.14 | −0.06 | 0.42 | |
Age | −0.02 | 0.01 | −0.23 | <0.01* | |
Education | 0.45 | 0.14 | 0.23 | <0.01* | |
Hemisphere | 0.38 | 0.15 | 0.19 | 0.01* | |
Antiepileptic drugs | −0.29 | 0.17 | −0.12 | 0.09 | |
Radiotherapy | −0.09 | 0.14 | −0.04 | 0.55 | |
Chemotherapy | −0.51 | 0.26 | −0.15 | 0.05 | |
Disease duration | −0.01 | 0.01 | −0.05 | 0.51 | |
SF-36 physical component summary score | −0.01 | 0.01 | −0.12 | 0.18 | |
QLQ BN-20 visual disorder | −0.01 | 0.05 | −0.01 | 0.91 | |
QLQ BN-20 motor dysfunction | −0.06 | 0.06 | −0.09 | 0.32 | |
SF-36 mental component summary score | 0.01 | 0.01 | 0.07 | 0.45 | |
MFI mental fatigue | −0.01 | 0.02 | −0.04 | 0.62 | |
QLQ BN-20 communication deficit | −0.09 | 0.04 | −0.18 | 0.03* | |
QLQ BN-20 future uncertainty | 0.01 | 0.03 | 0.03 | 0.73 |
Abbreviations: MFI, Multidimensional Fatigue Inventory; QLQ BN-20, Brain cancer specific Quality of Life Questionnaire; SF-36, Medical Outcomes Study Short-Form 36; Std., standard; Sig., significant.
Prediction of Verbal Memory Factor Scores
Of the 4 clusters of predictors, only the cluster of sociodemographic variables contributed significantly to the model predicting verbal memory functioning (Table 4). In the final model, sex (B = 0.57; 95% CI, 0.27–0.86) and education (B = 0.58; 95% CI, 0.29–0.88) predicted verbal memory test scores. Men had lower verbal memory scores than women, and participants with lower education had lower scores compared with those with higher education levels.
Table 4.
Cluster | R2 | Adjusted R2 | Std. Error of the Estimate | R2 Change | Sig. F Change |
---|---|---|---|---|---|
1. Sociodemographic | 0.21 | 0.20 | 0.90 | 21% | <0.01* |
2. Clinical | 0.23 | 0.20 | 0.90 | 2% | 0.41 |
3. Physical | 0.25 | 0.19 | 0.90 | 1% | 0.53 |
4. Mental | 0.25 | 0.18 | 0.91 | 1% | 0.83 |
Unstandardized Coefficients |
Standardized Coefficients |
||||
Variables of final model (4) | B | Std. Error | Beta | Sig. | |
(Constant) | 0.01 | 10.20 | 0.99 | ||
Sex | 0.57 | 0.15 | 0.28 | <0.01* | |
Age | −0.01 | 0.01 | −0.11 | 0.14 | |
Education | 0.58 | 0.15 | 0.29 | <0.01* | |
Hemisphere | 0.16 | 0.16 | 0.08 | 0.31 | |
Antiepileptic drugs | 0.09 | 0.18 | 0.04 | 0.61 | |
Radiotherapy | 0.13 | 0.15 | 0.06 | 0.40 | |
Chemotherapy | −0.15 | 0.27 | −0.04 | 0.59 | |
Disease duration | −0.01 | 0.01 | −0.06 | 0.47 | |
SF-36 physical component summary score | 0.00 | 0.01 | 0.00 | 0.97 | |
QLQ BN-20 visual disorder | 0.05 | 0.05 | 0.08 | 0.36 | |
QLQ BN-20 motor dysfunction | −0.06 | 0.06 | −0.08 | 0.35 | |
SF-36 mental component summary score | 0.00 | 0.01 | 0.02 | 0.86 | |
MFI mental fatigue | −0.01 | 0.02 | −0.03 | 0.68 | |
QLQ BN-20 communication deficit | −0.04 | 0.04 | −0.09 | 0.32 | |
QLQ BN-20 future uncertainty | 0.02 | 0.03 | 0.04 | 0.64 |
Abbreviations: MFI, Multidimensional Fatigue Inventory; QLQ BN-20, Brain cancer specific Quality of Life Questionnaire; SF-36, Medical Outcomes Study Short-Form 36; Std., standard; Sig., significant.
Prediction of Subjective Cognitive Function
The sociodemographic, physical, and mental health clusters contributed significantly to predicting subjective cognitive functioning (CFS total score; Table 5). The third cluster, consisting of physical symptoms, had the largest contribution.
Table 5.
Cluster | R2 | Adjusted R2 | Std. Error of the Estimate | R2 Change | Sig. F, Change |
---|---|---|---|---|---|
1. Sociodemograpic | 0.08 | 0.07 | 40.27 | 8% | <0.01* |
2. Clinical | 0.10 | 0.05 | 40.30 | 1% | 0.77 |
3. Physical | 0.29 | 0.24 | 30.86 | 19% | <0.01* |
4. Mental | 0.44 | 0.39 | 30.45 | 16% | <0.01* |
Unstandardized Coefficients |
Standardized Coefficients |
||||
Variables of final model (4) | B | Std. Error | Beta | Sig. | |
(Constant) | 26.73 | 4.55 | <0.01* | ||
Sex | −2.14 | 0.57 | −0.24 | <0.01* | |
Age | 0.00 | 0.03 | 0.01 | 0.89 | |
Education | −0.02 | 0.56 | 0.00 | 0.97 | |
Hemisphere | −0.81 | 0.59 | −0.09 | 0.17 | |
Antiepileptic drugs | −0.44 | 0.69 | −0.04 | 0.53 | |
Radiotherapy | −0.27 | 0.57 | −0.03 | 0.64 | |
Chemotherapy | −1.44 | 1.02 | −0.10 | 0.16 | |
Disease duration | 0.02 | 0.05 | 0.03 | 0.67 | |
SF-36 physical component summary score | 0.03 | 0.04 | 0.07 | 0.38 | |
QLQ BN-20 visual disorder | −0.01 | 0.19 | −0.01 | 0.94 | |
QLQ BN-20 motor dysfunction | −0.81 | 0.23 | −0.28 | <0.01* | |
SF-36 Mental component summary score | 0.09 | 0.03 | 0.21 | 0.01* | |
MFI mental fatigue | −0.30 | 0.09 | −0.25 | <0.01* | |
QLQ BN-20 communication deficit | −0.55 | 0.16 | −0.25 | <0.01* | |
QLQ BN-20 future uncertainty | 0.29 | 0.13 | 0.19 | 0.02* |
Abbreviations: MFI, Multidimensional Fatigue Inventory; QLQ BN-20, Brain cancer specific Quality of Life Questionnaire; SF-36, Medical Outcomes Study Short-Form 36; Std., standard; Sig., significant.
In the final model, female sex (B = −2.14; 95% CI, −3.26 to −1.02), more motor dysfunction (B = −0.81; 95% CI, −1.27 to −0.35), worse self-reported mental health (B = 0.09; 95% CI, 0.03–0.16), more mental fatigue (B = −0.30; 95% CI, −0.47 to −0.13), more communication deficits (B = −0.55; 95% CI, −0.88 to −0.23), and less future uncertainty (B = 0.29; 95% CI, 0.05–0.54) were significantly associated with lower subjective ratings of cognitive function.
We conducted additional post hoc multiple regression analyses in an attempt to understand the seemingly contradictory results regarding the relationship between future uncertainty and subjective cognitive functioning based on the multivariate analyses (a positive association) versus the univariate analyses (a negative association, as expected). The results suggested that the mental component summary score, in combination with other mental health variables, may be mediators of the relationship between future uncertainty and self-reported cognitive function (data not shown).
Prediction of Subjective Cognitive Function, Including Objective Cognitive Factors as Predictors
Finally, to examine the contribution of the objective cognitive test results to explaining variance in subjective cognitive functioning, we also included the attention/executive and verbal memory factors as a fifth cluster of predictors in the analysis (Table 6). In general, the cluster of these 2 objective measures of cognitive function did not contribute significantly to the total explained variance in subjective cognitive symptoms (increasing from 44% to 45% by adding the 2 objective cognitive measures).
Table 6.
Cluster | R2 | Adjusted R2 | Std. Error of the Estimate | R2 Change | Sig. F, Change |
---|---|---|---|---|---|
1. Sociodemograpic | 0.08 | 0.07 | 4.27 | 8% | <0.01* |
2. Clinical | 0.10 | 0.05 | 4.30 | 1% | 0.77 |
3. Physical | 0.29 | 0.24 | 30.86 | 19% | <0.01* |
4. Mental | 0.44 | 0.39 | 30.45 | 16% | <0.01* |
5. Objective cognitive factors | 0.45 | 0.39 | 30.44 | 1% | 0.29 |
Unstandardized Coefficients |
Standardized Coefficients |
||||
Variables of final model (5) | B | Std. Error | Beta | Sig. | |
(Constant) | 25.70 | 4.60 | <0.01* | ||
Sex | −2.18 | 0.60 | −0.25 | <0.01* | |
Age | 0.02 | 0.03 | 0.04 | 0.58 | |
Education | −0.32 | 0.60 | −0.04 | 0.59 | |
Hemisphere | −1.01 | 0.60 | −0.11 | 0.10 | |
Antiepileptic drugs | −0.32 | 0.70 | −0.03 | 0.65 | |
Radiotherapy | −0.25 | 0.58 | −0.03 | 0.66 | |
Chemotherapy | −1.18 | 1.04 | −0.08 | 0.26 | |
Disease duration | 0.03 | 0.05 | 0.04 | 0.60 | |
SF-36 physical component summary score | 0.04 | 0.04 | 0.08 | 0.30 | |
QLQ BN-20 visual disorder | −0.02 | 0.19 | −0.01 | 0.92 | |
QLQ BN-20 motor dysfunction | −0.77 | 0.23 | −0.26 | 0.00 | |
SF-36 mental component summary score | 0.09 | 0.03 | 0.21 | 0.01* | |
MFI mental fatigue | −0.29 | 0.09 | −0.24 | <0.01* | |
QLQ BN-20 communication deficit | −0.51 | 0.16 | −0.23 | <0.01* | |
QLQ BN-20 future uncertainty | 0.29 | 0.13 | 0.19 | 0.02 | |
Attention and executive factor | 0.46 | 0.32 | 0.10 | 0.16 | |
Verbal memory factor | 0.17 | 0.31 | 0.04 | 0.59 |
Abbreviations: MFI, Multidimensional Fatigue Inventory; QLQ BN-20, Brain cancer specific Quality of Life Questionnaire; SF-36, Medical Outcomes Study Short-Form 36; Std., standard; Sig., significant 1.
A summary of the general findings of each regression analysis is presented in Table 7.
Table 7.
Objective |
Subjective | ||
---|---|---|---|
Clusters | Attention and Executive Factor | Verbal Memory Factor | CFS Total Score |
Sociodemographic | 11%: Age+, Educ+ | 21%: Sex+, Educ+ | 8%: Sex− |
Clinical | 13%: HemR | ns | ns |
Physical symptoms | ns | ns | 19%: MotDysf− |
Mental symptoms | 3%: CommDef- | ns | 16%: MCS+, MF−, Comm−, FutUnc+ |
Objective cognitive | N/A | N/A | ns |
Abbreviations: CFS, cognitive function scale; CommDef, communication deficits; Educ, education; FutUnc, future uncertainty; Hem, hemisphere; MCS, mental component summary score; MF, mental fatigue; MotDysf, motor dysfunction; N/A, not applicable; ns, not significant.
+higher (score) indicates better (subjective/objective) functioning; −higher (score) indicates lower (subjective/objective) functioning; Sex+: females better performance than males; Sex−: females lower subjective cognitive function than males; Rright hemisphere associated with better cognitive functioning.
Sensitivity Analyses
Thirteen (7.1%) of the 183 participants for whom factor scores were available had some missing data on the predictor variables for the multiple regression analyses, and one participant (0.5%) had missing scores on the CFS. There were no single variables with 5% or more data missing.
Participants with missing data did not differ from those with complete data except for the CFS total score, which was significantly higher (better) for the participants with missing data (t (181) = −2.53, P = .01), and for use of antiepileptic drugs (χ2 (1) = 5.41, P = .02), which were used less frequently by participants with missing data. Correlations of attention and executive factor (r = 0.18) and verbal memory factor (r = 0.01) with CFS total score were lower in the group in which these data were available (N = 182). When multiple imputation was done for estimation of the missing data, repetition of the 4 multiple regression analyses yielded pooled regression parameters comparable with those resulting from the analyses with standard listwise exclusion of the cases with missing values (data not shown).
Discussion
Similar to previous reports in various populations, the correlations between subjective and objective indicators of neuropsychological functioning were negligible to low in our sample of glioma patients. Correlations found in our study were somewhat lower than those reported in some, but not all, other studies of patients with glioma.15,16
Whereas objective cognitive deficits were predominantly associated with sociodemographic (older age, lower education, male sex) and clinical (left hemisphere tumor) variables, lower subjective cognitive function was more closely related to self-reported physical (motor) and mental health functioning (mental fatigue, lower mental well-being, and lower future uncertainty), and female sex. Communication deficits were associated with both subjective and attention/executive dysfunction.
Of the observed predictors, (older age) and (lower education) are probably the most well-known variables affecting objective cognitive function and are commonly taken into account in the clinical interpretation of neuropsychological performance based on normative comparison. Consistent with our findings, aging has been found to negatively influence processing speed and performance on tests of attention and executive function, whereas these effects may be smaller for verbal memory.37 It is also a typical finding that women have better verbal memory than men and may preserve verbal skills better after brain damage.37
Worse objective attention/executive functioning was more strongly related to left hemisphere tumors than to those in the right hemisphere. In general, performance on many of the tests that were aggregated in the attention and executive factor tends to be poorer in the participants with left hemisphere lesions (Digit Span tasks, Stroop Color-Word Test, Letter Fluency37). The same observation has also been reported in patients with glioma in particular.15 At the same time, the women in our sample reported lower subjective cognitive function than their male counterparts. Previous studies investigating the relationship between sex and subjective cognitive function have reported inconsistent results: some find the same association (ie, female participants report more cognitive complaints),38 but many others do not find any association at all.2,6,39,40
Self-reported communication deficits were associated with measures of both subjective and objective attention and executive function. The relationship with subjective functioning is perhaps not surprising, given that self-reported communication deficits may be a relevant aspect of cognitive complaints. The univariate correlation of self-reported communication deficits with the objective attention and executive factor, however, was even stronger than the relationship between global subjective cognitive function and objective attention and executive function.
In line with the previous reports, lower ratings of subjective cognitive function were significantly associated with self-reported mental fatigue and lower mental well-being. Although it was not surprising to find a clear relationship between subjective cognitive function and mental fatigue since both measure aspects of attention/concentration, this was not the case for the association with the mental well-being scale and the BN-20 subscales, which measure issues very different from cognitive functioning. Although our study did not include a direct measure of depressive symptoms, the mental well-being scale includes items that measure nervousness, depressed mood, and role functioning limited by emotional problems, all of which are closely related to the known mental and emotional predictors of subjective cognitive function.1–3,4,12 The positive association of future uncertainty with subjective cognitive functioning appeared to be mediated by other indicators of mental well-being.
The association between physical well-being and subjective cognitive function has been reported previously.2,41,42 In one of these publications, it was suggested that cognitive deficits reflect a more general state of feelings of diminished well-being.42
Remarkably, no significant association was observed between earlier radiotherapy and objective cognitive deficits. Radiotherapy in patients with low-grade glioma may result in cognitive deterioration over time, but recent studies have shown that this holds true primarily for patients with a median survival that is longer than that in our study.43
Finally, similar sets of predictors of objective and subjective cognitive functioning were found in a recent study of 50 patients with multiple sclerosis.44 Regression analyses were conducted that were comparable with those described here, with the addition of neuroimaging data. The results indicated that worse cognitive performance was associated with male sex, lower education, and lower gray-matter volume. Subjective cognitive complaints were associated with fatigue and less hippocampal atrophy. Unfortunately, such volumetric data were not available for our current study.
Several other limitations of our study should be noted. First, patients who did not report at least minimal levels of cognitive symptoms were not included in the study. This may explain why the correlations observed in our study were somewhat lower than those reported in other studies conducted on subjective and objective cognitive function in patients with glioma.15,16 Although the cutoff used to establish minimal levels of cognitive symptoms was low (ie, experiencing 1 of 6 symptoms at least some of the time), strictly speaking, the results of the current analysis can only be generalized to glioma patients with at least a minimal level of self-reported cognitive symptoms. In addition, it should be noted that a small subset of patients (7%) without histopathological diagnosis (presumed to have a grade II glioma based on clinical and MRI features) was also included in the analyses. Finally, the inclusion criteria for age and KPS in grade III glioma and stable disease for both grades of malignancy may have resulted in a sample with more favorable characteristics than the grades II and III glioma populations as a whole in clinical practice.
Second, we chose to use principal component analysis to reduce the large set of neuropsychological test variables to a more manageable number of factors for use in further analyses. We opted for this empirical method in order to obtain a grouping of variables in the most objective way. We acknowledge that the attention and executive functioning factor included a wide variety of test variables measuring simple and complex attention, processing speed, working memory, and executive functioning. This global factor of diffuse attention, speed of information processing, and executive function may in fact reflect the typical cognitive profile of patients with slowly growing glioma.
In addition, generating indirect measures for cognitive performance may have resulted in some data loss, which in turn may have attenuated the strength of the correlations observed. However, the attention and executive factor correlated more highly with subjective cognitive functioning than any of the other single test variables. Taken together, we think that the current empirical approach was appropriate for the main purpose of this study, namely to systematically examine the associations of objective (and subjective) indicators of cognitive function with a range of explanatory factors.
Third, we cannot draw any conclusions about causality because of the cross-sectional nature of our data. However, we gave higher priority in our statistical approach to clusters of variables that were presumed to take causal precedence over other variables (eg, clinical factors were assumed to more likely to cause mental health symptoms, such as fatigue, than vice versa).
Finally, the predictor variables included in our statistical models explained a fair amount, but not all, of the variance in subjective cognitive symptoms and objective cognitive test results. Thus, there may be other factors that contribute significantly to explaining variance in these outcomes, which still need to be identified.44 It would be interesting to investigate whether imaging data (eg, lesion volume/location) represent additional significant explanatory variables.
We do not believe that the limitations of our study can explain the fact that we (and others) have consistently observed a weak association between subjective and objective measures of cognitive function. Other factors, including validity issues, may also play a role here. For example, it has been frequently reported that ratings of patients' cognitive function by informants (both formal and informal caregivers) are more strongly associated with patients' neuropsychological test performance and everyday cognitive skills than the patients' self-rated cognitive symptoms and functioning.3,45,46
Both underreporting and overreporting may play a role here.3 Underreporting has been found to occur more often in patients with more severe objective cognitive impairment.3,47 In fact, a lack of insight has been related to frontal lobe dysfunction and can thus be part of the cognitive deficits.48,49 Therefore, clinicians should be aware that patients who do not report any cognitive symptoms may still have cognitive deficits. Conversely, overreporting of cognitive problems may be associated with depression and other mental symptoms.1–4,6–10 In fact, Kinsinger et al50 demonstrated that treatment of depression may ameliorate subjective cognitive symptoms in patients with multiple sclerosis. In that study, participants underwent either cognitive behavioral therapy or supportive emotion-focused therapy. Improvements in depression and fatigue were related to decreased levels of perceived cognitive symptoms but not to changes in objective neuropsychological test performance.
Neuropsychological tests have their limitations as well. For example, in a study of 88 community-dwelling, cognitively healthy older adults, cognitive tests did not predict quality of everyday activity completion. Instead, self-reported instrumental activities of daily living and a performance-based test of everyday competence were unique predictors of direct observations of everyday activities.51
In addition to the evidence from clinical studies for a weak association between subjective rating of cognitive function and actual cognitive performance, there is also preliminary functional-anatomical evidence for this association. In an experimental study in 20 young, healthy male volunteers, functional MRI was used to identify distinct neural correlates of a subjective sense of memory strength and retrieval accuracy after actual encoding of information. Activity in the left ventrolateral prefrontal cortex and temporoparietal junction was associated with subjective ratings of memory strength, whereas parahippocampal and hippocampal activity was associated with the number of details recalled accurately.52
In conclusion, our findings suggest that distinct mechanisms may underlie subjective cognitive symptoms versus objective cognitive impairment in patients with glioma. Comprehensive neuropsychological assessments should include both neuropsychological tests and measures of subjective cognitive function, psychological symptoms such as depression, anxiety, and fatigue, and possibly a caregivers' evaluation of the patient's daily cognitive performance in order to determine which areas are most affected and which specific intervention strategies are most appropriate.
It may also be important to target nonpharmacological intervention strategies for objective and subjective cognitive problems, either separately or in combination. For example, subjective cognitive symptoms (without clear objective cognitive impairment and potentially accompanied by affective symptoms and fatigue) may benefit most from psychotherapy,50 whereas objectively verified cognitive deficits may be treated more successfully with cognitive rehabilitation.17,53–55
Funding
Supported by the Dutch Cancer Society (UU 2003–2783; UvT2010-4642).
Acknowledgments
We thank Dr Wobbe P. Zijlstra for his statistical advice.
Conflict of interest statement. There are no conflicts of interest.
References
- 1.Duits A, Munnecom T, van Heugten C, et al. Cognitive complaints in the early phase after stroke are not indicative of cognitive impairment. J Neurol Neurosurg Psychiatry. 2008;79(2):143–146. doi: 10.1136/jnnp.2007.114595. [DOI] [PubMed] [Google Scholar]
- 2.Stulemeijer M, Vos PE, Bleijenberg G, et al. Cognitive complaints after mild traumatic brain injury: things are not always what they seem. J Psychosom Res. 2007;63(6):637–645. doi: 10.1016/j.jpsychores.2007.06.023. [DOI] [PubMed] [Google Scholar]
- 3.Benedict RH, Cox D, Thompson LL, et al. Reliable screening for neuropsychological impairment in multiple sclerosis. Mult Scler. 2004;10(6):675–678. doi: 10.1191/1352458504ms1098oa. [DOI] [PubMed] [Google Scholar]
- 4.Hall KE, Isaac CL, Harris P. Memory complaints in epilepsy: an accurate reflection of memory impairment or an indicator of poor adjustment? A review of the literature. Clin Psychol Rev. 2009;29(4):354–367. doi: 10.1016/j.cpr.2009.03.001. [DOI] [PubMed] [Google Scholar]
- 5.van der Werf-Eldering MJ, Burger H, Jabben N, et al. Is the lack of association between cognitive complaints and objective cognitive functioning in patients with bipolar disorder moderated by depressive symptoms? J Affect Disord. 2011;130(1–2):306–311. doi: 10.1016/j.jad.2010.10.005. [DOI] [PubMed] [Google Scholar]
- 6.Cull A, Hay C, Love SB, et al. What do cancer patients mean when they complain of concentration and memory problems? Br J Cancer. 1996;74(10):1674–1679. doi: 10.1038/bjc.1996.608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Poppelreuter M, Weis J, Kulz AK, et al. Cognitive dysfunction and subjective complaints of cancer patients. A cross-sectional study in a cancer rehabilitation centre. Eur J Cancer. 2004;40(1):43–49. doi: 10.1016/j.ejca.2003.08.001. [DOI] [PubMed] [Google Scholar]
- 8.Schagen SB, Boogerd W, Muller MJ, et al. Cognitive complaints and cognitive impairment following BEP chemotherapy in patients with testicular cancer. Acta Oncol. 2008;47(1):63–70. doi: 10.1080/02841860701518058. [DOI] [PubMed] [Google Scholar]
- 9.Pullens MJ, De Vries J, Roukema JA. Subjective cognitive dysfunction in breast cancer patients: a systematic review. Psychooncology. 2010;19(11):1127–1138. doi: 10.1002/pon.1673. [DOI] [PubMed] [Google Scholar]
- 10.Hutchinson AD, Hosking JR, Kichenadasse G, et al. Objective and subjective cognitive impairment following chemotherapy for cancer: A systematic review. Cancer Treat Rev. 2012;38(7):926–934. doi: 10.1016/j.ctrv.2012.05.002. [DOI] [PubMed] [Google Scholar]
- 11.Fox SW, Lyon D, Farace E. Symptom clusters in patients with high-grade glioma. J Nurs Scholarsh. 2007;39(1):61–67. doi: 10.1111/j.1547-5069.2007.00144.x. [DOI] [PubMed] [Google Scholar]
- 12.Gustafsson M, Edvardsson T, Ahlstrom G. The relationship between function, quality of life and coping in patients with low-grade gliomas. Support Care Cancer. 2006;14(12):1205–1212. doi: 10.1007/s00520-006-0080-3. [DOI] [PubMed] [Google Scholar]
- 13.Taphoorn MJ, Heimans JJ, Snoek FJ, et al. Assessment of quality of life in patients treated for low-grade glioma: a preliminary report. J Neurol Neurosurg Psychiatry. 1992;55(5):372–376. doi: 10.1136/jnnp.55.5.372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Klein M, Taphoorn MJ, Heimans JJ, et al. Neurobehavioral status and health-related quality of life in newly diagnosed high-grade glioma patients. J Clin Oncol. 2001;19(20):4037–4047. doi: 10.1200/JCO.2001.19.20.4037. [DOI] [PubMed] [Google Scholar]
- 15.Klein M, Heimans JJ, Aaronson NK, et al. Effect of radiotherapy and other treatment-related factors on mid-term to long-term cognitive sequelae in low-grade gliomas: a comparative study. Lancet. 2002;360(9343):1361–1368. doi: 10.1016/s0140-6736(02)11398-5. [DOI] [PubMed] [Google Scholar]
- 16.Aaronson NK, Taphoorn MJ, Heimans JJ, et al. Compromised health-related quality of life in patients with low-grade glioma. J Clin Oncol. 2011;29(33):4430–4435. doi: 10.1200/JCO.2011.35.5750. [DOI] [PubMed] [Google Scholar]
- 17.Gehring K, Sitskoorn MM, Gundy CM, et al. Cognitive rehabilitation in patients with gliomas: a randomized, controlled trial. J Clin Oncol. 2009;27(22):3712–3722. doi: 10.1200/JCO.2008.20.5765. [DOI] [PubMed] [Google Scholar]
- 18.Gehring K, Aaronson N, Taphoorn M, et al. A description of a cognitive rehabilitation programme evaluated in brain tumour patients with mild to moderate cognitive deficits. Clin Rehabil. 2011;25(8):675–692. doi: 10.1177/0269215510395791. [DOI] [PubMed] [Google Scholar]
- 19.Gehring K, Aaronson NK, Gundy CM, et al. Predictors of neuropsychological improvement following cognitive rehabilitation in patients with gliomas. J Int Neuropsychol Soc. 2011;17(2):256–266. doi: 10.1017/S1355617710001530. [DOI] [PubMed] [Google Scholar]
- 20.Stewart AL, Ware JE, Sherbourne CD, et al. Psychological distress/well-being and cognitive functioning measures. In: Stewart AL, Ware JE, editors. Measuring Functioning and Well-being: The Medical Outcomes Study Approach. Durham, NC: Duke University Press; 1992. pp. 102–142. [Google Scholar]
- 21.Ware JE, Jr., Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care. 1992;30(6):473–483. [PubMed] [Google Scholar]
- 22.Aaronson NK, Muller M, Cohen PD, et al. Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease populations. J Clin Epidemiol. 1998;51(11):1055–1068. doi: 10.1016/s0895-4356(98)00097-3. [DOI] [PubMed] [Google Scholar]
- 23.Gandek B, Ware JE, Jr., Aaronson NK, et al. Tests of data quality, scaling assumptions, and reliability of the SF-36 in eleven countries: results from the IQOLA Project. International Quality of Life Assessment. J Clin Epidemiol. 1998;51(11):1149–1158. doi: 10.1016/s0895-4356(98)00106-1. [DOI] [PubMed] [Google Scholar]
- 24.Osoba D, Aaronson NK, Muller M, et al. The development and psychometric validation of a brain cancer quality-of-life questionnaire for use in combination with general cancer-specific questionnaires. Qual Life Res. 1996;5(1):139–150. doi: 10.1007/BF00435979. [DOI] [PubMed] [Google Scholar]
- 25.Smets EM, Garssen B, Bonke B, et al. The Multidimensional Fatigue Inventory (MFI) psychometric qualities of an instrument to assess fatigue. J Psychosom Res. 1995;39(3):315–325. doi: 10.1016/0022-3999(94)00125-o. [DOI] [PubMed] [Google Scholar]
- 26.Taphoorn MJ, Claassens L, Aaronson NK, et al. An international validation study of the EORTC brain cancer module (EORTC QLQ-BN20) for assessing health-related quality of life and symptoms in brain cancer patients. Eur J Cancer. 2010;46(6):1033–1040. doi: 10.1016/j.ejca.2010.01.012. [DOI] [PubMed] [Google Scholar]
- 27.Jolles J, Houx PJ, Van Boxtel MPJ, et al. Maastricht Aging Study: Determinants of Cognitive Aging. Maastricht, The Netherlands: Neuropsych Publishers; 1995. [Google Scholar]
- 28.Brand N, Jolles J. Information processing in depression and anxiety. Psychol Med. 1987;17(1):145–153. doi: 10.1017/s0033291700013040. [DOI] [PubMed] [Google Scholar]
- 29.Vink M, Jolles J. A new version of the trail-making test as an information processing task. J Clin Neuropsych. 1985;7:162. [Google Scholar]
- 30.Stroop JR. Studies of interference in serial verbal reactions. J Exp Psychol. 1935;18(6):643–662. [Google Scholar]
- 31.Hammes J. De Stroop Kleur-Woord Test: Handleiding [The Stroop Color-Word Test: Manual] Amsterdam, The Netherlands: Harcourt; 1971. [Google Scholar]
- 32.Wechsler D. Wechsler Adult Intelligence Scale—Revised. New York, NY: Psychological Corporation; 1981. [Google Scholar]
- 33.Lezak MD. Neuropsychological Assessment. 3rd ed. New York, NY: Oxford University Press; 1995. [Google Scholar]
- 34.Luteijn F, Van der Ploeg FAE. Handleiding Groninger Intelligentie Test. Lisse, The Netherlands: Swets & Zeitlinger; 1983. [Google Scholar]
- 35.Robertson IH, Ward T, Ridgeway V, et al. The Test of Everyday Attention. Bury St Edmunds, UK: Thames Valley Test Company; 1994. [Google Scholar]
- 36.Brand N, Jolles J. Learning and retrieval rate of words presented auditorily and visually. J Gen Psychol. 1985;112(2):201–210. doi: 10.1080/00221309.1985.9711004. [DOI] [PubMed] [Google Scholar]
- 37.Lezak MD, Howieson DB, Loring DW, et al. Neuropsychological Assessment. 4th ed. New York, NY: Oxford University Press; 2004. [Google Scholar]
- 38.Roth RS, Geisser ME, Theisen-Goodvich M, et al. Cognitive complaints are associated with depression, fatigue, female sex, and pain catastrophizing in patients with chronic pain. Arch Phys Med Rehabil. 2005;86(6):1147–1154. doi: 10.1016/j.apmr.2004.10.041. [DOI] [PubMed] [Google Scholar]
- 39.Covassin T, Bay E. Are there gender differences in cognitive function, chronic stress, and neurobehavioral symptoms after mild-to-moderate traumatic brain injury? J Neurosci Nurs. 2012;44(3):124–133. doi: 10.1097/JNN.0b013e318252737d. [DOI] [PubMed] [Google Scholar]
- 40.Slavin MJ, Brodaty H, Kochan NA, et al. Prevalence and predictors of “subjective cognitive complaints” in the Sydney Memory and Ageing Study. Am J Geriatr Psychiatry. 2010;18(8):701–710. doi: 10.1097/jgp.0b013e3181df49fb. [DOI] [PubMed] [Google Scholar]
- 41.Roding J, Glader EL, Malm J, et al. Perceived impaired physical and cognitive functions after stroke in men and women between 18 and 55 years of age—a national survey. Disabil Rehabil. 2009;31(13):1092–1099. doi: 10.1080/09638280802510965. [DOI] [PubMed] [Google Scholar]
- 42.Comijs HC, Deeg DJ, Dik MG, et al. Memory complaints; the association with psycho-affective and health problems and the role of personality characteristics. A 6-year follow-up study. J Affect Disord. 2002;72(2):157–165. doi: 10.1016/s0165-0327(01)00453-0. [DOI] [PubMed] [Google Scholar]
- 43.Douw L, Klein M, Fagel SS, et al. Cognitive and radiological effects of radiotherapy in patients with low-grade glioma: long-term follow-up. Lancet Neurol. 2009;8(9):810–818. doi: 10.1016/S1474-4422(09)70204-2. [DOI] [PubMed] [Google Scholar]
- 44.Hulst HE, Gehring K, Uitdehaag BM, et al. Indicators for cognitive performance and subjective cognitive complaints in multiple sclerosis: a role for advanced MRI? Mult Scler. 2013;20(8):1131–1134. doi: 10.1177/1352458513513969. [DOI] [PubMed] [Google Scholar]
- 45.Benedict RH, Munschauer F, Linn R, et al. Screening for multiple sclerosis cognitive impairment using a self-administered 15-item questionnaire. Mult Scler. 2003;9(1):95–101. doi: 10.1191/1352458503ms861oa. [DOI] [PubMed] [Google Scholar]
- 46.Chaytor N, Schmitter-Edgecombe M. The ecological validity of neuropsychological tests: a review of the literature on everyday cognitive skills. Neuropsychol Rev. 2003;13(4):181–197. doi: 10.1023/b:nerv.0000009483.91468.fb. [DOI] [PubMed] [Google Scholar]
- 47.Demers M, Rouleau I, Scherzer P, et al. Impact of the cognitive status on the memory complaints in MS patients. Can J Neurol Sci. 2011;38(5):728–733. doi: 10.1017/s031716710005410x. [DOI] [PubMed] [Google Scholar]
- 48.Pannu JK, Kaszniak AW. Metamemory experiments in neurological populations: a review. Neuropsychol Rev. 2005;15(3):105–130. doi: 10.1007/s11065-005-7091-6. [DOI] [PubMed] [Google Scholar]
- 49.Morton N, Barker L. The contribution of injury severity, executive and implicit functions to awareness of deficits after traumatic brain injury (TBI) J Int Neuropsychol Soc. 2010;16(6):1089–1098. doi: 10.1017/S1355617710000925. [DOI] [PubMed] [Google Scholar]
- 50.Kinsinger SW, Lattie E, Mohr DC. Relationship between depression, fatigue, subjective cognitive impairment, and objective neuropsychological functioning in patients with multiple sclerosis. Neuropsychology. 2010;24(5):573–580. doi: 10.1037/a0019222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Schmitter-Edgecombe M, Parsey C, Cook DJ. Cognitive correlates of functional performance in older adults: comparison of self-report, direct observation, and performance-based measures. J Int Neuropsychol Soc. 2011;17(5):853–864. doi: 10.1017/S1355617711000865. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Qin S, van Marle HJ, Hermans EJ, et al. Subjective sense of memory strength and the objective amount of information accurately remembered are related to distinct neural correlates at encoding. J Neurosci. 2011;31(24):8920–8927. doi: 10.1523/JNEUROSCI.2587-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Locke DE, Cerhan JH, Wu W, et al. Cognitive rehabilitation and problem-solving to improve quality of life of patients with primary brain tumors: a pilot study. J Support Oncol. 2008;6(8):383–391. [PubMed] [Google Scholar]
- 54.Hassler MR, Elandt K, Preusser M, et al. Neurocognitive training in patients with high-grade glioma: a pilot study. J Neurooncol. 2010;97(1):109–115. doi: 10.1007/s11060-009-0006-2. [DOI] [PubMed] [Google Scholar]
- 55.Zucchella C, Capone A, Codella V, et al. Cognitive rehabilitation for early post-surgery inpatients affected by primary brain tumor: a randomized, controlled trial. J Neurooncol. 2013;114(1):93–100. doi: 10.1007/s11060-013-1153-z. [DOI] [PubMed] [Google Scholar]