Abstract
Background
Patients with glioblastoma (GB) are more likely to suffer cognitive deficits with poor quality of life as compared with lower-grade glioma patient groups, for whom cognition research is plentiful. The objective of this systematic review is to evaluate the cognitive function of patients with GB before and after surgery.
Methods
This review was prospectively registered with PROSPERO. PubMed and EMBASE searches were performed, most recently March 15, 2018. Inclusion criteria were adult patients, histologically confirmed GB, and cognitive tests conducted before and/or after surgery. Screening and data extraction were carried out independently by 2 authors.
Results
A total of 512 abstracts were screened. Nineteen studies were included with 902 participants, of whom only 423 had histologically confirmed GB. Only 11 studies tested cognitive function both before and after surgery. A total of 114 different cognitive tests were used. The most common test was used in only 9 studies; 82 tests were used only once. Follow-up time ranged from 1 week to 16 months with extremely high dropout rates. Eighteen of 19 studies reported cognitive deficits in their samples, with prevalence ranging from 22% to 100% (median 64%, interquartile range 42%). Only 1/11 longitudinal studies reported normal cognitive function, 3/11 reported initial deficits with improvement after surgery, 3/11 reported static deficits, and 4/11 reported deterioration.
Conclusion
There is a consistently high risk of cognitive deficit for patients with GB undergoing surgery. The included studies showed marked heterogeneity in study design, case-mix of included diagnoses, and the type and timing of cognitive tests used. We highlight considerations for the design of future studies to avoid such bias.
Keywords: cognition, glioblastoma, quality of life, surgery
Glioblastoma (GB) is the most aggressive subtype of the most common malignant primary brain tumors, gliomas. However, it still carries a poor prognosis, with a median survival of 15 months.1 This is despite maximal combined therapy with surgical resection of the enhancing tumor confirmed by imaging and adjuvant chemotherapy and radiotherapy.1 It accounts for more years of life lost in adults than all other more common cancers combined.2
A report by The Brain Tumor Charity reflects the high likelihood of cognitive deficits faced by these patients following surgery and adjuvant therapy, with numerous accounts of the negative impact on quality of life for these patients.3 Objectively, cognitive deficits have been repeatedly associated with lower quality of life measures and difficulties in social participation for patients with stroke, traumatic brain injury, and cancer.4–6
Especially in the context of patients with GB, for whom there has been little progress in extending patient survival with current combined treatments, the shift in focus to preserve their quality of life posttreatment necessitates careful consideration of protecting their cognitive function from the risks of treatment. In the event of treatment advances for GB substantially increasing survival, addressing disabling cognitive deficits at that stage would be too late. Additionally, there are studies that have employed computerized cognition rehabilitation interventions in patient groups such as pediatric oncology7 and severe traumatic brain injury,8 for which traditional inpatient rehabilitation was deemed unfeasible and burdensome. Such strategies may be useful in the context of GB as early in the management pathway as deficits are detected, to preserve quality of life despite patients’ currently limited survival.
Despite this patient group being at high risk of cognitive deficits as they progress through treatment with surgery, chemotherapy, and radiotherapy for the brain, there seems to be a bias in the literature toward assessing cognitive function in patients with low-grade gliomas (LGGs), which are much rarer and tend to cause less-severe deficits by comparison.9 There are even systematic reviews focusing on cognitive function in patients with LGG10 and meningioma patients,11 but no such review exists specifically for patients with GB, which may reflect such a bias against using cognitive assessments in this patient group.
The current systematic review primarily aims to evaluate the available data for the assessment of cognition in patients with GB before surgery and/or after surgery. We have chosen this perioperative phase before adjuvant chemotherapy and radiotherapy to understand the impact of tumors on cognition before any treatment and then subsequently how this is associated with surgery longitudinally, because of the inherent risks of surgery in damaging brain tissue. This phase seems underreported in the literature as studies evaluating cognition are often conducted alongside radiation treatments and start assessments after surgery but before radiotherapy.12–14 Focusing on this phase may yield knowledge on which surgical strategies could be based and modified to protect cognitive function, although the longer-term implications for cognitive function are also considerably acted on by adjuvant treatment with chemotherapy and radiotherapy starting within a few weeks of surgery.
Our secondary aims are to investigate the reported prevalence of cognitive dysfunction in this population and highlight considerations for future studies based on the evaluations made of the current literature. Investigating the literature in this regard may reveal consistency across studies in assessing cognition in patients with GB or whether inconsistencies among studies may be a source of bias. Focusing on the cognition data in this perioperative phase will also have implications for the presurgical counseling of patients, and our highlighted considerations may help to guide future cognition research in patients with GB to provide a reliable base for assessing interventions such as targeted rehabilitation.
Methods
Search Strategy
Our objective was to identify all peer-reviewed research articles reporting scores and outcomes of cognitive function tests before and after surgical intervention in adult patients with GB. Our search was carried out September 2, 2017, and subsequently March 15, 2018, to identify any publications in the interim during completion of the current study. Our systematic review was registered with PROSPERO prior to data extraction. We created a search strategy in PubMed that was adapted for Embase. Search terms included derivations of glioblastoma, surgery, cognition, attention, and memory (Table 1). Results were filtered to exclude nonhuman participants and an age filter was applied to capture adult patients. No restriction was placed on the date of publication.
Table 1.
PubMed |
(((((surge*[Title/Abstract]) OR surgi*[Title/Abstract]) OR operate*[Title/Abstract]) OR operati*[Title/Abstract])) AND (((((((((((((““executive function”“[mh]) OR ““executive function”“[Title/Abstract]) OR ““attention”“[mh]) OR attention*[Title/Abstract]) OR ““memory”“[mh]) OR memor*[Title/Abstract]) OR memory[Title/Abstract]) OR cogniti*[Title/Abstract]) OR cognitive[Title/Abstract]) OR cognition[Title/Abstract]) OR ““cognition disorders”“[mh])) AND ((((((HGG*[Title/Abstract]) OR GBM*[Title/Abstract]) OR ““high grade glioma”“[Title/Abstract]) OR glioma*[Title/Abstract]) OR ““glioma”“[mh]) OR glioblastoma*[Title/Abstract])) Filters: Humans; Adult: 19+ years |
Embase |
(((exp *glioma/ OR exp *glioblastoma/ OR glioma*.ti,ab. OR GBM*.ti,ab. OR “high grade glioma”.ti,ab. OR HGG*.ti,ab. OR glioblastoma*.ti,ab.) AND (exp *cognition/ OR cognition*.ti,ab. OR cognitive*.ti,ab. OR cogniti*.ti,ab. OR exp *memory/ OR memory*.ti,ab. OR memor*.ti,ab. OR attention*.ti,ab. OR exp *attention/ OR exp *executive function/ OR “executive function”.ti,ab.) AND (surge*.ti,ab. OR surgi*.ti,ab. OR operati*.ti,ab. OR operate*.ti,ab.)) Limit search (human and (adult <18 to 64 years> or aged <65+ years>))) |
Study Selection
Titles and abstracts were initially screened for inclusion in this study using the following criteria: (i): adult patients aged 18 years or older; (ii): cohort included patients with histologically confirmed GB; (iii): results of GB patients were reported as a stratified group or with other histology types; (v): cognitive function was reported using standardized test(s) either pre-, postoperatively, or both. Publications were excluded if: (i): the study was a review or case study; (ii): cognitive function was reported postradiotherapy/postchemotherapy only; (iii): no full text was available, abstract only; (iv): cognitive testing included MMSE only. We did not exclude studies with mixed populations of patients, as data may be stratified by tumor histology in these studies. Where studies did not stratify results, we still extracted data to record how study design was implemented in these mixed populations.
Selection of relevant publications for this study was performed independently by 2 authors (RS and JMS). Titles and abstracts were initially screened using the inclusion and exclusion criteria above. Full-text papers were retrieved of those applicable and were further scrutinized for relevance. The final list of papers for inclusion was agreed on by the 2 authors with any discrepancies discussed and resolved. The reference lists of the selected articles were checked by the authors to identify additional relevant publications for inclusion; 2 such articles were included. In case of articles with overlapping study populations, the study with the larger sample of patients was included.
Data Extraction
Two authors (RS and JMS) independently carried out data extraction of relevant publications using a standardized data extraction template. Data were collected regarding study characteristics (study design, location, selection criteria of patients, and follow-up time); patient cohort characteristics (number of patients, mean age, male:female ratio, number diagnosed with GB, and number of patients lost to follow-up); cognitive dysfunction (definition and z-score cut-offs used); cognitive domains tested (assessments of attention, memory, executive function, and language); assessment tools used (standardized tests); and results reported (results reported pre-/postoperatively or both; stratification of results according to histology and any statistical tests used).
Data Synthesis
Qualitative and quantitative data were extracted and we have presented our results using descriptive analysis. Using the framework for data extraction detailed above, we have presented the range of cognitive tests used in the included articles, the prevalence of cognitive dysfunction as reported in the respective patient populations and, where longitudinal design permitted, the cognitive outcomes as associated with surgery. Where prevalence of cognitive deficits has not been explicitly stated in the articles, we have recorded it as largest proportion of patients to have a score consistent with a deficit on any of the cognitive tests used to avoid underestimating the risk of deficit. To encompass the greatest risk of cognitive deficit in the perioperative phase, either before or after surgery, we recorded the highest prevalence that could be determined between the available assessment time points.
Results
Using the search strategy detailed above, 711 titles were identified and downloaded into Mendeley. Removal of duplicates left 512 titles, which were screened by the criteria detailed above resulting in 19 studies included in the review15–33 for data extraction from the full-text articles (Fig. 1).
Study Population and Design
The total number pooled from the 19 studies was 902 patients. However, of this pooled sample only 423 patients had a histologically confirmed diagnosis of GB. Supplementary Figure 1 and Table 2 present further details of case mix and demographics, respectively.
Table 2.
First Author | Year | No. of Patients | Mean Age (Years) | Male:Female | Patient Selection | Result Stratification | Assessment Pre-/Postoperatively |
---|---|---|---|---|---|---|---|
Bello 15 | 2007 | 88 | Not given | 43:45 | Mixed glioma | No | Both |
Bosma 16 | 2007 | 68 | 60.1 | 27:9 | HGG | No | Post |
Campanella 17 | 2015 | 66 | 55.76 | Not given | Mixed lesions | No | Both |
Dallabona 18 | 2017 | 30 | 59.3 | 19:11 | HGG | No | Both |
Raysi Dehcordi 19 | 2013 | 42 | 53.43 | 26:16 | Mixed glioma | No | Both |
Fang 20 | 2014 | 45 | 44.15 | 28:17 | Mixed glioma | No | Post |
Froklage 21 | 2013 | 39 | 52 | 28:11 | Mixed glioma | No | Post |
Giussani 22 | 2010 | 18 | 47 | 11:7 | Mixed lesions | No | Both |
Habets 23 | 2014 | 62 | 60.6 | 38:24 | HGG | No | Both |
Hilverda 24 | 2010 | 13 | 53 | 11:2 | GB | Yes | Post |
Johnson 25 | 2012 | 91 | 53.9 | 55:36 | GB | Yes | Post |
Lang 26 | 2017 | 18 | 39.5 | 13:5 | HGG | No | Both |
Lee 27 | 2015 | 55 | 58.8 | 34:16 | GB | Yes | Pre |
Mandonnet 28 | 2015 | 25 | 43 | Not given | Mixed glioma | Yes | Both |
Miotto 29 | 2011 | 27 | 54.63 | Not given | Mixed glioma | No | Pre |
Santini 30 | 2012 | 22 | Not given | 10:12 | Mixed glioma | No | Both |
Satoer 31 | 2014 | 45 | 39.09 | 28:17 | Mixed glioma | No | Both |
Talacchi 32 | 2011 | 29 | Not given | 18:11 | Mixed glioma | No | Both |
Wefel 33 | 2016 | 119 | 46.05 | 67:52 | HGG | Yes | Pre |
Abbreviations: GB, glioblastoma; HGG, high-grade glioma.
Five of 19 studies were retrospective; the remaining 14 were prospectively undertaken. Only 3/19 studies focused exclusively on patients with GB. Another 5 studies pooled all high-grade glioma patients together, and the remaining 11 studies had considerably mixed patient groups, including patients with LGG, meningioma, pituitary lesions, and even arteriovenous malformations.
Eight of 19 studies were cross-sectional in relation to the timing of surgery (3 studies assessing cognition preoperatively only and 5 postoperatively only). The remaining 11 studies employed a longitudinal design to assess cognition both before and after surgery.
Planned time-points for follow-up ranged from 1 week to 16 months. Ten of 19 studies reported patient drop-out in the data collection, rates of which ranged from 9% up to 100%. A common reason cited across studies for this high loss in follow-up data collection was that the patients were finding the cognitive testing batteries too burdensome. Table 3 presents further details of the findings described here.
Table 3.
First Author | Year | No. of Patients | Follow-Up Time | Number Lost to Follow-Up/ Incomplete Datasets (%) | Definition of Cognitive Deficit | Cognitive Deficit Prevalence; GB/ Mixed Group | Group Cognitive Outcome (Cross-Sectional ± Longitudinal) |
---|---|---|---|---|---|---|---|
Bello 15 | 2007 | 88 | 3 months | 0/88 (0) | Cut-off given per test | 28.4%; mixed group | Impaired and deteriorated |
Bosma 16 | 2007 | 68 | 16 months | 50/68 (74) | z-score relative to control | 100%; mixed group | Impaired |
Campanella 17 | 2015 | 66 | 1 week | 66/66 (100) | Cut-off given per test | 54.54%; mixed group | Impaired but static |
Dallabona 18 | 2017 | 30 | 40 days | 0/30 (0) | Cut-off given per test | 73.3%; mixed group | Impaired but improved |
Raysi Dehcordi 19 | 2013 | 42 | 12 months | 0/42 (0) | Not given | Not given | Impaired but improved |
Fang 20 | 2014 | 45 | 12 months | 0/45 (0) | z-score >2 SDs below control mean | Not given | Impaired |
Froklage 21 | 2013 | 39 | 7 months | 29/39 (74) | z-score >1.5 SDs below control mean | 63.6%; mixed group | Impaired |
Giussani 22 | 2010 | 18 | 3 months | 0/18 (0) | Not given | 22%; mixed group | Normal function |
Habets 23 | 2014 | 62 | 5 weeks | 26/62 (42) | z-score >1.5 SDs below control mean | 79%; mixed group | Impaired and deteriorated |
Hilverda 24 | 2010 | 13 | 6 months | 2/13 (15) | z-score >1.5 SDs below control mean | 84.6%; GB group | Impaired |
Johnson 25 | 2012 | 91 | 3 weeks | 0/91 (0) | z-score >1.5 SDs below control mean | 60%; GB group | Impaired |
Lang 26 | 2017 | 18 | 1 month | 5/18 (28) | z-score >1 SD below control mean | 92%; mixed group | Impaired and deteriorated |
Lee 27 | 2015 | 55 | 6 months | 5/55 (9) | z-score >1.5 SDs below control mean | 64%; GB group | Impaired |
Mandonnet 28 | 2015 | 25 | 3 months | 0/25 (0) | Not given | 30%; mixed group | Impaired but static |
Miotto 29 | 2011 | 27 | N/A | N/A | Not given | 88%; mixed group | Impaired |
Santini 30 | 2012 | 22 | 6 months | 11/22 (50) | Not given | 59%; mixed group | Impaired and deteriorated |
Satoer 31 | 2014 | 45 | 12 months | 5/45 (11) | Not given | Not given | Impaired but improved |
Talacchi 32 | 2011 | 29 | 3 months | 0/29 (0) | z-score >2 SDs below control mean | 38%; mixed group | Impaired but static |
Wefel 33 | 2016 | 119 | N/A | 87/119 (73) | z-score >1.5 SDs below control mean | 83%; GB group | Impaired |
Abbreviations: GB, glioblastoma; N/A: not available.
Cognitive Assessments and Healthy Performance Cut-Offs
In 7/19 studies there was no definition given for cognitive performances consistent with deficit. Six of 19 studies defined z-scores lower than 1.5 SDs from the healthy population mean as being consistent with deficit. Another 2 studies set this threshold at 2 SD below the healthy population mean, and a single study set the threshold for deficit at 1 SD; 92% of patients in this study were found to have a cognitive deficit. The remaining 3 studies cited individual cut-off scores for deficits per test used. Table 3 presents these results in more detail.
For reporting the cognitive outcomes, the majority (14/19) did not stratify their results per diagnostic category and instead presented data for a heterogeneous patient group with remarkably different pathologies. The prevalence of cognitive deficits in these mixed sample populations ranged from 22% to 100%. For 3 studies there was no prevalence figure quoted for cognitive deficits in the patients tested. Only 4 studies provided cognitive deficit prevalence specifically for patients with GB, ranging from 60% to 85%. Of particular note, 1 study assessed cognition only for the LGG patients in the sample while offering no cognitive testing whatsoever to the patients with GB.
In the 19 included articles, 114 different cognitive tests were used in varying combination batteries. The most commonly used tests, Trail-Making Test A and B, were used in only 9/19 studies. Eighty-two different cognitive tests were used only once across all the included studies. Supplementary Table 1 presents the frequency of usage of the cognitive tests identified.
Overall, 18/19 studies detected cognitive deficits in their sample populations. Of the 11 studies using a longitudinal study design, 1 study reported normal cognitive function in its heterogeneous sample population before and after surgery. Three of 11 studies reported cognitive impairment before surgery that improved postoperatively, another 3/11 studies reported impairments before surgery that remained static after surgery, and 4/11 reported impairments before surgery and deterioration after surgery.
Discussion
This study aimed to systematically review the literature for data regarding assessments of cognitive function in patients with GB before and after surgery. Our search strategy enabled us to identify a large volume of potentially relevant articles (n = 512). In the 19 articles that met the inclusion criteria, 423 patients with GB were assessed with 114 different tests. In total these studies recruited 902 patients for testing. However, because of the inclusion of patients with other pathologies (such as LGG, meningioma, metastases, pituitary tumors, and even arteriovenous malformations) and high drop-out of initially recruited patients, the proportion of GB patients within the pooled sample is less than half.
Of the 19 studies, only 4 had cognitive function data stratified specifically for GB patients. Consequently, of the total pool of 423 GB patients, only 211 patients had cognitive assessment results that were not merged with patients with other less-malignant disease processes. The 4 studies that reported stratified results for the pooled 211 GB patients were all cross-sectional, 2 testing only after surgery and 2 only before surgery, yet they all reported a consistently high prevalence of cognitive dysfunction ranging between 60% and 85%. When considering the entire pooled population in the included studies, of which 423 of the pooled sample of 903 had GB, the prevalence of cognitive dysfunction ranged from 22% to 100% (median 64%, interquartile range 42%). The lower prevalence in this range is likely a result of the heterogeneous populations tested, of which small numbers of patients with GB were assessed alongside patients with less-aggressive and less-infiltrative lesions. However, the data from the combined pool of patients across the included studies were subject to considerable biases, as we will describe next and that are also highlighted in Table 4.
Table 4.
First Author | Tumor Location | Tumor/Resection volume | Previous Adjuvant Rx | Antiepileptic Drugs | Steroids | Functional Status | Age | Education level | Multidomain Assessment | Notes for addressing Confound | Notes for Additional Sources of Bias | Statistical Notes |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Bello 15 | Yes | Yes | N/A | No | No | No | No | Yes | No | Intraoperative awake brain stimulation | Predominantly language tests | Penalized maximum likelihood estimate |
Bosma 16 | No | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Tumor recurrence analysis | Missing data at baseline and follow-up | ANCOVA |
Campanella 17 | Yes | Yes | No | No | No | No | Yes | Yes | Yes | N/A | High dropout rate | ANCOVA with Tukey test and Bonferroni correction |
Dallabona 18 | Yes | Yes | Yes | No | Yes | No | Yes | Yes | Yes | Included analysis of mass effect from surrounding edema | Categorical and skewed analysis of age with cut-off at age 65 years | Wilcoxon rank sum test, paired Wilcoxon test, Spearman rank correlation, and linear modeling |
Raysi Dehcordi 19 | Yes | Yes | No | No | No | No | Yes | No | Yes | Analysis of edema | No comparison to normative data, ipsative improvements may be due to test-wise | Mann-Whitney U Test, Kruskal Wallis test, Wilcoxon signed rank test |
Fang 20 | Yes | No | No | No | No | Yes | Yes | Yes | No | Healthy controls recruited | Gross total resection and MMSE > 24 patients only | Mann-Whitney U test |
Froklage 21 | No | No | Yes | No | Yes | No | Yes | No | Yes | White-matter hyperintensity and atrophy analysis | High dropout rate (burdensome tests) | Kaplan-Meier |
Giussani 22 | Yes | No | No | No | No | No | Yes | No | No | Brain mapping with cortical stimulation | Only intact patients at baseline included | No statistical analysis, small sample |
Habets 23 | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | Yes | Epilepsy and headaches assessed | High dropout rate | Pearson χ2 with Bonferroni correction, Student t-test, Wilcoxon signed-rank test |
Hilverda 24 | Yes | No | Yes | Yes | Yes | Yes | Yes | No | Yes | Seizure frequency included | Only changes in z-scores, test-wiseness not addressed | No statistical analysis, small sample |
Johnson 25 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Included motor grip assessment, class RPA | Selection bias of patients suitable for resection | Cox proportional hazards models, Kaplan-Meier curves, P < .01 |
Lang 26 | Yes | Yes | No | No | No | No | Yes | Yes | Yes | Fully corrected T-scores for age and education, motor, and emotional function assessed | Mixed tumor grades and 3 patients with recurrent tumors | Paired sample T-tests and multivariate regression |
Lee 27 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Included molecular marker subtype data, depression data, and authors blinded to scores | Arbitrary performance dichotomy used | Student t-test, chi-squared, Pearson and Spearman correlations, logistic regression and Kaplan-Meier |
Mandonnet 28 | Yes | Yes | Yes | No | No | Yes | Yes | No | Yes | Return to work analysis and intraoperative mapping | Did not test cognition of GB patients | No statistical analysis |
Miotto 29 | Yes | No | No | No | No | No | No | No | Yes | N/A | Most confounding factors not addressed | No statistical analysis |
Santini 30 | Yes | Yes | Yes | Yes | Yes | No | Yes | No | Yes | Intraoperative brain mapping, affect, and edema measures included | Only left hemisphere lesions; skew toward total resection 77% vs 23% | Chi-squared tests, ANOVA,Wilcoxon sum rank and McNemar tests |
Satoer 31 | Yes | Yes | Yes | No | No | No | Yes | Yes | Yes | N/A | Skew toward partial resection surgery (58%) | T-tests and bootstrapping |
Talacchi 32 | Yes | Yes | Yes | Yes | No | Yes | Yes | No | Yes | Anxiety and depression data analysis | Steroids use not standardized | Chi-squared, Wilcoxon sum rank tests, and multiple regressions |
Wefel 33 | Yes | Yes | Yes | No | No | Yes | Yes | Yes | Yes | Included motor function and IDH status in analysis | Heterogeneous sample with skew toward GB | T-tests, chi-squared tests, Cohen dx statistic, Pearson correlations |
Abbreviations: GB, glioblastoma; IDH, isocitrate dehydrogenase; RPA, recursive partitioning analysis; Rx, prescription.
Bias in the Literature for Testing Cognition in LGG Patients but Not GB Patients
A recurrent theme in the literature was a bias toward testing in cognition in patients with LGG, consistently more so than the cognitive testing undertaken for patients with GB.
At the abstract screening stage, 78 studies were excluded because their sample populations contained patients with LGG but not GB patients. Of the included 20 studies, only 3 assessed exclusively GB patients. This 78:3 ratio is in direct opposition to the prevalence of the diseases, with GB being by far the most common form, accounting for 50% to 65%34 of all gliomas.
The study by Mandonnet et al28 serves as a good example of this bias in practice, whereby the GB patients in the sample had their tumor volume, extent of resection, and neurological status reported but only the LGG patients in the study underwent cognitive testing. The reason cited was that the time between radiological diagnosis and surgery was too short to accommodate systematic cognitive testing. A preliminary study9 with a smaller but overlapping population to the included study by Wefel and colleagues33 stratified the cognitive scores by tumor grade separately for patients with LGG and GBs specifically. They concluded that by direct comparison GBs “disrupt memory, language, processing speed, and executive functions to a greater extent than lower-grade tumors.”
Given the scarcity of neuropsychological services in our health care system,35 the GB patient population is often left out when considering the prioritization of such functionally valuable assessments, irrespective of the high risk posed to their cognitive function by such an infiltrative cancer and attempts at maximal resection surgery. Even if their poor prognosis at this point in time limits the potential for available cognitive rehabilitation, the scarce data available show rates of cognitive dysfunction that are concerningly high.
Marked Study Design Heterogeneity in the Literature Prevents Meta-Analysis and Limits Reliable Clinical Inference
As detailed in Table 3, there is marked variability across the included studies. Only 11/19 studies employed a longitudinal design to test cognition before and after surgery; 14/19 were prospectively performed; planned follow-up interval ranged from 1 week to 16 months; cut-offs for defining cognitively abnormal scores varied; and the choice of cognitive tests employed among studies was also greatly heterogeneous, with some studies assessing broadly across multiple domains of cognition and other studies focusing on only 1 cognitive faculty such as face processing.17 It follows that with so much heterogeneity in the different study designs, the prevalence of cognitive dysfunction, where reported, for GB patients undergoing surgery ranges vastly from 22% to 100%.
To further illustrate this point, the study in which 22% of patients had cognitive deficits17 was focused on emotional expression recognition but did not use memory or executive function tests. Furthermore, any patient who had preoperative visual neglect or emotion recognition deficits was excluded from the study. Given the propensity of patients to specifically report problems with executive functions and memory during the diagnostic phase36 and this preoperative exclusion criteria confounding the actual incidence of emotional recognition deficit, it is clear to see how such factors would lead to unreliable inference about the real prevalence of cognitive dysfunction for GB patients before or after surgery. This was the only included study that concluded that the population had normal cognitive function.
Only 3/19 studies reported improvement following surgery using a longitudinal design, but they did not stratify the results for their GB patients and instead reported on a heterogeneous group as a whole. One study of these studies reported that the cognitive “ipsative” score results showed improvement after surgery14; however, it did not report cut-off definitions for cognitive deficit or z-scores and neither was a prevalence rate of cognitive deficit reported. This selection of reporting changes in raw scores limits the inference we can make clinically and may be attributable to developing “test-wiseness,” especially as no control group was tested to account for learning effects or longitudinal variation.37–39 The remaining 16 studies reported variable rates of cognitive deficits that either persisted postoperatively or deteriorated. If we are to infer anything from the results, it is likely to be that the reported prevalence is likely a conservative approximation of the reality for patients with GB.
Presurgical Baseline Assessments Are Essential for Understanding the Risks to Cognitive Function
Ideally, premorbid baseline assessments of cognition would help in understanding how developing a GB affects cognitive function, but this is not currently feasible for the healthy population as a whole. Nonetheless, once these patients present with a radiologically confirmed lesion compatible with a diagnosis of GB, it is imperative that this point be used as a baseline to assess cognitive function before starting any treatment. This is because the most common first treatment modality used is surgery to either resect or biopsy the tumor, both of which carry significant risk of injuring the adjacent functional brain tissue. Although treatment of GB is multimodal, with chemotherapy and radiotherapy often starting within 2 to 6 weeks of surgery, our rationale to focus on the perioperative period up until adjuvant therapies is to simplify our understanding of deficit aggregation from the tumor at diagnosis followed by the impact of surgery without the confound of other treatments.
By investigating the preoperative and postoperative cognition data alike we can understand the additional risk surgery carries to patients who may already have a range of cognitive deficits at presentation from tumor effects alone.
Five of 19 studies used the postoperative surgical time-point as a baseline. In addressing the risk of adjuvant treatments such as radiotherapy and chemotherapy, this can give a misleading impression of patients’ cognitive function, which may already have been significantly impaired by surgery rather than direct tumor effects. Understanding how surgery affects cognitive function by comparing against a preoperative baseline is vital to protecting patients in the future, as surgical strategies and approaches may be modified for this purpose, or indeed early targeted rehabilitation options could be considered alongside these treatments. Our goal as clinicians should be to maintain or improve cognitive function at every treatment stage, with every modality used.
Collection of Additional Confounding Factor Data Is Vital to Minimize Bias in Data Interpretation
While the included studies collected confounding data for analysis of additional effects, as presented in Table 4, this was again highly variable. For a reliable understanding of cognitive function in the GB patient group, all studies should ideally collect data about tumor size and location, use of antiepileptic and steroid medications, other preexisting neurological diagnoses, years of education, first language, language/hand dominance, and preexisting deficits such as visual neglect. Twelve of 19 included studies analyzed for relationships between tumor location or tumor volume/extent of resection with cognitive function. Of these, 6/12 found tumor location to have a significant relationship with cognitive function and 5/12 found lesion size/extent of resection to have a significant relationship with cognitive function. This variance may be better explained by including molecular subtyping data in the analysis.
Importantly, only 2 studies included the molecular subclassification of the tumor diagnoses. The field of glioma neuro-oncology is being radically reappraised in view of molecular subtyping with gene mutations for isocitrate dehydrogenase (IDH), 1p/19q codeletions, Telomerase Reverse Transcriptase (TERT) promoter mutation, and hypermethylation of the O6-methylguanine-DNA-methyltransferase (MGMT) gene. By adding these subtypes to the diagnostic process, the variations in survival times and response to treatments are far better explained than with the WHO grade 1-4 classification of glioma alone.40 Since cognitive function has been shown to also be a key prognostic factor41 and early sign of progression after treatment,42 the inclusion of such “molecular marker data” would also help to explain the variance in patient outcomes including cognition as well as survival. One of the included studies33 reported an interesting finding regarding the association of IDH molecular subtype, cognitive function, and lesion size in particular. The authors postulated an effect of “lesion momentum,” such that the IDH–wild-type subgroup had worse cognitive function than the IDH-mutant subgroup, despite having smaller lesion sizes overall. Their interpretation was that IDH wild-type was associated with faster growth and less time for plastic reorganization before cognitive deficits were detected at diagnosis as compared with IDH mutation, which was associated with larger tumor volumes that may have grown more slowly to allow plasticity and cognitive compensation.
Furthermore, none of the included studies measured patient-reported quality of life measures. Cognitive function has repeatedly been associated with quality of life outcomes for patients with GB, with implications of the objective former being a surrogate marker for the subjective latter43,44; however, such studies have relied on the MMSE screening test, which is not a sufficiently sensitive tool for use in the GB population.45 Once again, given the poor overall survival achievable with maximal combined treatment for patients with GB, quality of life and in turn cognition should be prioritized as key outcomes with consequent longitudinal measurement as patients undergo treatment.
Drop-Out Rates in the Literature Were Consistently High
Eight of 19 studies reported collecting complete datasets for all selected cognitive tests as per their scheduled assessment time-points, including follow-up time points, although these varied considerably among studies. The median drop-out rate reported among the remaining studies was 46% (interquartile range 60%), with 1 study having 100% loss of follow-up cognition data. Although for the few studies with very long planned follow-up times, disease progression and death were a factor in this, the consistently cited reason was that participants found the paper-based testing batteries too burdensome and fatigue inducing. Combined with the scarcity of trained neuropsychologists available in health care settings35 to provide such assessments for patients with GB, it may be the case that this mode of administration is not suitably efficient enough in this context.
A potential solution to these barriers may be the use of computerized cognition testing batteries that are being successfully employed to assess patients in a wide variety of clinical contexts, including traumatic brain injury, pediatric oncology, and psychiatric inpatient management.46–49 These tools have the benefits of incorporating well-validated tests into a faster format with less interobserver variability, fewer recording errors, and options to assess additional metrics such as reaction times and rapidly calculated performance indices that are otherwise resource intensive.50 These tools also allow for nonspecialists without a background in clinical psychology to administer the tests, and this broadening of access via other health care professionals may serve to widen availability of cognitive tests to patients with GB and improve follow-up compliance because of their efficiency. Unpublished data from our unit demonstrate that 35 patients with GB have been cognitively assessed longitudinally in the perioperative phase using such a computerized battery of tests, with only 1 patient refusing at follow-up because of the burden of testing.
Limitations of the Current Study
Owing to the largely absent reporting of effect sizes and the heterogeneous study designs of the included articles, it has not been possible to undertake a meaningful meta-analysis of cognitive function in this patient group, which would have provided more robust understanding of the risk to patients with GB. The search strategy we have employed attempted to capture cognition data in patients with GB broadly by using multiple derivations of terms such as “glioma,” “memory,” “attention,” and “executive.” However, this is not exhaustive and may have limited our detection of relevant articles. This limitation is evidenced by our identification of 2 studies from reference lists that were not detected with our search strategy. Our gray literature search did not reveal additional data to include, although it is possible that such data exist.
Another major limitation in our methodology is our inclusion of studies that have mixed populations of patients with differing histological diagnoses without stratification of the cognitive test data by diagnosis. A more robust and strict methodology would have excluded these studies and also studies that were cross-sectional rather than longitudinal because of the bias introduced from less-malignant diagnoses and differing preoperative and postoperative populations. However, all 4 studies that had data stratified for GB patients were cross-sectional, hence the most robust application of exclusion criteria would have excluded all studies. We have chosen to include studies with unstratified mixed populations and cross-sectional data, acknowledging and highlighting the bias they introduce to prevent similar difficulties in future research.
Our data extraction presents the trends in these patient populations at a group level, where often these were heterogeneous patient groups. Some of the studies presented the data at the individual patient level and, by not extracting these details, we have limited our appreciation of the granularity in the data available.
We have not included cognitive function in GB patients longitudinally as they undergo adjuvant treatment with chemotherapy and radiotherapy. Although there is considerable literature about the risk to cognition from “chemo-brain” and radiotherapy, even in the treatment of non-neurological cancers, such investigation has been outside the scope of the current study, which aimed to focus on the initial presentation at diagnosis with GB and surgery-related risks to cognition. Furthermore, our focus has been on the objective measures of cognition; however, it is ultimately the resultant impact on patient-reported quality of life that is the outcome measure that also requires prioritizing.
Future Directions
Given the current redefining of GB and other less-aggressive gliomas on the basis of molecular subtyping, now is an opportune time to combine this change in perspective with a change in how future research into the cognitive function of patients with GB is carried out. Table 5 contains our suggested considerations for future studies on the basis of findings of this systematic review, which build on earlier recommendations made by the International Cognition and Cancer Task Force for cognition research in the context of chemotherapy for non-CNS tumors.51
Table 5.
• Longitudinal prospective design (including assessments before and after surgery) |
• Postoperative follow up time-points prior to starting adjuvant chemoradiotherapy |
• Use of validated cognitive tests spanning across all domains, with options for retesting |
• Use of computerized testing batteries may increase access for patients and may improve follow-up compliance |
• 1.5 SD cut-off from normative population may aid in detecting mild deficits |
• “Reliability change indices” and “Minimal Clinically Important Differences” may better detect longitudinal change in cognitive function |
• Reporting effect sizes as z-scores may facilitate interpretation of longitudinal changes and meta-analysis |
• Stratifying all results by tumor type and molecular subtype may aid analysis of tumor metabolism alongside cognitive function |
• Include confounding factor data in analysis such as years of education, lobar location of tumor, tumor volume, extent of resection, steroid, and anticonvulsant medication dosages |
• Include patient-reported quality of life outcome measures |
In conclusion, based on this systematic review of the literature, there is a consistently high risk of cognitive dysfunction and further deterioration to patients with GB undergoing treatment with surgery. The data suffer biases by being combined with results of patients with less-aggressive gliomas and considerable variability in study designs among articles. Consequently, the reported prevalence of cognitive dysfunction is likely to be a gross underestimation. Future studies should consider the benefits of employing a longitudinal study design; applying an efficient battery of tests across all domains of cognition to retain participants at follow-up; stratifying results by tumor type and molecular marker subtype; and collecting relevant confounding factor data for analysis, including lesion volume, extent of resection, steroid, and antiseizure medication history. Such considerations may allow for a more reliable understanding of cognitive dysfunction in this population so we can make meaningful inferences with which patients can make informed decisions, and we as clinicians can start to address how to reduce additional iatrogenic injury with our treatments, especially surgery.
Funding
This work was supported by the Royal College of Surgeons of England [RRAG/093 to RS] and Cancer Research UK [RRZB/040 to RS].
Conflict of interest statement
None declared
Supplementary Material
References
- 1. Stupp R, Mason W, van den Bent MJ, et al. . Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. N Engl J Med. 2005;352(10):987–996. [DOI] [PubMed] [Google Scholar]
- 2. Burnet NG, Jefferies SJ, Benson RJ, Hunt DP, Treasure FP. Years of life lost (YLL) from cancer is an important measure of population burden—and should be considered when allocating research funds. Br J Cancer. 2005;92(2):241–245. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. The Brain Tumour Charity. Losing myself: the reality of life with a brain tumour 2015. https://assets.thebraintumourcharity.org/live/media/filer_public/94/6d/946d8473-2e1a-4f36-ac2a-0a989a176226/losing_myself_the_reality_of_life_with_a_brain_tumour.pdf. Accessed March 7, 2019.
- 4. Schagen SB, Klein M, Reijneveld JC, et al. . Monitoring and optimising cognitive function in cancer patients: present knowledge and future directions. EJC Suppl. 2014;12(1):29–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Sashika H, Takada K, Kikuchi N. Rehabilitation needs and participation restriction in patients with cognitive disorder in the chronic phase of traumatic brain injury. Medicine (Baltimore). 2017;96(4):e5968. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Kapoor A, Lanctôt KL, Bayley M, et al. . “Good outcome” isn’t good enough: cognitive impairment, depressive symptoms, and social restrictions in physically recovered stroke patients. Stroke. 2017;48(6):1688–1690. [DOI] [PubMed] [Google Scholar]
- 7. Kesler SR, Lacayo NJ, Jo B. A pilot study of an online cognitive rehabilitation program for executive function skills in children with cancer-related brain injury. Brain Inj. 2011;25(1):101–112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Bergquist T, Gehl C, Mandrekar J, et al. . The effect of internet-based cognitive rehabilitation in persons with memory impairments after severe traumatic brain injury. Brain Inj. 2009;23(10):790–799. [DOI] [PubMed] [Google Scholar]
- 9. Noll KR, Sullaway C, Ziu M, Weinberg JS, Wefel JS. Relationships between tumor grade and neurocognitive functioning in patients with glioma of the left temporal lobe prior to surgical resection. Neuro Oncol. 2015;17(4):580–587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. van Loon EM, Heijenbrok-Kal MH, van Loon WS, et al. . Assessment methods and prevalence of cognitive dysfunction in patients with low-grade glioma: a systematic review. J Rehabil Med. 2015;47(6):481–488. [DOI] [PubMed] [Google Scholar]
- 11. Meskal I, Gehring K, Rutten GJ, Sitskoorn MM. Cognitive functioning in meningioma patients: a systematic review. J Neurooncol. 2016;128(2):195–205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Flechl B, Sax C, Ackerl M, et al. . The course of quality of life and neurocognition in newly diagnosed patients with glioblastoma. Radiother Oncol. 2017;125(2):228–233. [DOI] [PubMed] [Google Scholar]
- 13. Zhu JJ, Demireva P, Kanner AA, et al. . Health-related quality of life, cognitive screening, and functional status in a randomized phase III trial (EF-14) of tumor treating fields with temozolomide compared to temozolomide alone in newly diagnosed glioblastoma. J Neurooncol. 2017;135(3):545–552. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Correa DD, Baser R, Beal K, et al. NC-08. Longitudinal cognitive follow-up in newly diagnosed glioblastoma patients treated with bevacizumab, temozolomide and hypofractionated stereotactic radiotherapy. Neuro Oncol. 2011;13(suppl 3):iii73–iii75. [Google Scholar]
- 15. Bello L, Gallucci M, Fava M, et al. . Intraoperative subcortical language tract mapping guides surgical removal of gliomas involving speech areas. Neurosurgery. 2007;60(1):67–80; discussion 80–82. [DOI] [PubMed] [Google Scholar]
- 16. Bosma I, Vos MJ, Heimans JJ, et al. . The course of neurocognitive functioning in high-grade glioma patients. Neuro Oncol. 2007;9(1):53–62. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Campanella F, Fabbro F, Ius T, Shallice T, Skrap M. Acute effects of surgery on emotion and personality of brain tumor patients: surgery impact, histological aspects, and recovery. Neuro Oncol. 2015;17(8):1121–1131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Dallabona M, Sarubbo S, Merler S, et al. . Impact of mass effect, tumor location, age, and surgery on the cognitive outcome of patients with high-grade gliomas: a longitudinal study. Neuro-Oncology Pract. 2017;4(4): 229–240. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Raysi Dehcordi S, Mariano M, Mazza M, Galzio RJ. Cognitive deficits in patients with low and high grade gliomas. J Neurosurg Sci. 2013;57(3):259–266. [PubMed] [Google Scholar]
- 20. Fang D, Jiang J, Sun X, et al. . Attention dysfunction of postoperative patients with glioma. World J Surg Oncol. 2014;12:317. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Froklage FE, Oosterbaan LJ, Sizoo EM, et al. . Central neurotoxicity of standard treatment in patients with newly-diagnosed high-grade glioma: a prospective longitudinal study. J Neurooncol. 2014;116(2):387–394. [DOI] [PubMed] [Google Scholar]
- 22. Giussani C, Pirillo D, Roux FE. Mirror of the soul: a cortical stimulation study on recognition of facial emotions. J Neurosurg. 2010;112(3):520–527. [DOI] [PubMed] [Google Scholar]
- 23. Habets EJ, Kloet A, Walchenbach R, Vecht CJ, Klein M, Taphoorn MJ. Tumour and surgery effects on cognitive functioning in high-grade glioma patients. Acta Neurochir (Wien). 2014;156(8):1451–1459. [DOI] [PubMed] [Google Scholar]
- 24. Hilverda K, Bosma I, Heimans JJ, et al. . Cognitive functioning in glioblastoma patients during radiotherapy and temozolomide treatment: initial findings. J Neurooncol. 2010;97(1):89–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Johnson DR, Sawyer AM, Meyers CA, O’Neill BP, Wefel JS. Early measures of cognitive function predict survival in patients with newly diagnosed glioblastoma. Neuro Oncol. 2012;14(6):808–816. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Lang S, Cadeaux M, Opoku-Darko M, et al. . Assessment of cognitive, emotional, and motor domains in patients with diffuse gliomas using the National Institutes of Health Toolbox Battery. World Neurosurg. 2017;99:448–456. [DOI] [PubMed] [Google Scholar]
- 27. Lee ST, Park CK, Kim JW, et al. . Early cognitive function tests predict early progression in glioblastoma. Neuro-Oncology Pract. 2015;2(3): 137–143. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Mandonnet E, De Witt Hamer P, Poisson I, et al. . Initial experience using awake surgery for glioma: oncological, functional, and employment outcomes in a consecutive series of 25 cases. Neurosurgery. 2015;76(4):382–389; discussion 389. [DOI] [PubMed] [Google Scholar]
- 29. Miotto EC, Junior AS, Silva CC, et al. . Cognitive impairments in patients with low grade gliomas and high grade gliomas. Arq Neuropsiquiatr. 2011;69(4):596–601. [DOI] [PubMed] [Google Scholar]
- 30. Santini B, Talacchi A, Squintani G, Casagrande F, Capasso R, Miceli G. Cognitive outcome after awake surgery for tumors in language areas. J Neurooncol. 2012;108(2):319–326. [DOI] [PubMed] [Google Scholar]
- 31. Satoer D, Visch-Brink E, Smits M, et al. . Long-term evaluation of cognition after glioma surgery in eloquent areas. J Neurooncol. 2014;116(1):153–160. [DOI] [PubMed] [Google Scholar]
- 32. Talacchi A, Santini B, Savazzi S, Gerosa M. Cognitive effects of tumour and surgical treatment in glioma patients. J Neurooncol. 2011;103(3):541–549. [DOI] [PubMed] [Google Scholar]
- 33. Wefel JS, Noll KR, Rao G, Cahill DP. Neurocognitive function varies by IDH1 genetic mutation status in patients with malignant glioma prior to surgical resection. Neuro Oncol. 2016;18(12):1656–1663. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Ohgaki H, Kleihues P. Epidemiology and etiology of gliomas. Acta Neuropathol. 2005;109(1):93–108. [DOI] [PubMed] [Google Scholar]
- 35. Public Health England. Improving people’s health: applying behavioural and social sciences to improve population health and wellbeing in England 2018. https://www.gov.uk/government/publications/improving-peoples-health-applying-behavioural-and-social-sciences. Accessed March 7, 2019.
- 36. Tucha O, Smely C, Preier M, Lange KW. Cognitive deficits before treatment among patients with brain tumors. Neurosurgery. 2000;47(2):324–3 33; discussion 333–334. [DOI] [PubMed] [Google Scholar]
- 37. Temkin NR, Heaton RK, Grant I, Dikmen SS. Detecting significant change in neuropsychological test performance: a comparison of four models. J Int Neuropsychol Soc. 1999;5(4):357–369. [DOI] [PubMed] [Google Scholar]
- 38. Heaton RK, Temkin N, Dikmen S, et al. . Detecting change: a comparison of three neuropsychological methods, using normal and clinical samples. Arch Clin Neuropsychol. 2001;16(1):75–91. [PubMed] [Google Scholar]
- 39. Woods SP, Childers M, Ellis RJ, et al. . A battery approach for measuring neuropsychological change. Arch Clin Neuropsychol. 2006;21(1):83–89 . [DOI] [PubMed] [Google Scholar]
- 40. Eckel-Passow JE, Lachance DH, Molinaro AM, et al. . Glioma groups based on 1p/19q, IDH, and TERT promoter mutations in tumors. N Engl J Med. 2015;372(26):2499–2508. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Klein M, Postma TJ, Taphoorn MJ, et al. . The prognostic value of cognitive functioning in the survival of patients with high-grade glioma. Neurology. 2003;61(12):1796–1798. [DOI] [PubMed] [Google Scholar]
- 42. Armstrong CL, Goldstein B, Shera D, Ledakis GE, Tallent EM. The predictive value of longitudinal neuropsychologic assessment in the early detection of brain tumor recurrence. Cancer. 2003;97(3):649–656. [DOI] [PubMed] [Google Scholar]
- 43. Bunevicius A, Tamasauskas S, Deltuva V, Tamasauskas A, Radziunas A, Bunevicius R. Predictors of health-related quality of life in neurosurgical brain tumor patients: focus on patient-centered perspective. Acta Neurochir (Wien). 2014;156(2):367–374. [DOI] [PubMed] [Google Scholar]
- 44. Yavas C, Zorlu F, Ozyigit G, et al. . Health-related quality of life in high-grade glioma patients: a prospective single-center study. Support Care Cancer. 2012;20(10):2315–2325. [DOI] [PubMed] [Google Scholar]
- 45. van Kessel E, Baumfalk AE, van Zandvoort MJE, Robe PA, Snijders TJ. Tumor-related neurocognitive dysfunction in patients with diffuse glioma: a systematic review of neurocognitive functioning prior to anti-tumor treatment. J Neurooncol. 2017;134(1):9–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Keith MS, Stanislav SW, Wesnes KA. Validity of a cognitive computerized assessment system in brain-injured patients. Brain Inj. 1998;12(12):1037–1043. [DOI] [PubMed] [Google Scholar]
- 47. Conklin HM, Ashford JM, Di Pinto M, et al. . Computerized assessment of cognitive late effects among adolescent brain tumor survivors. J Neurooncol. 2013;113(2):333–340. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Weber B, Fritze J, Schneider B, Simminger D, Maurer K. Computerized self-assessment in psychiatric in-patients: acceptability, feasibility and influence of computer attitude. Acta Psychiatr Scand. 1998;98(2):140–145. [DOI] [PubMed] [Google Scholar]
- 49. Meskal I, Gehring K, van der Linden SD, Rutten GJ, Sitskoorn MM. Cognitive improvement in meningioma patients after surgery: clinical relevance of computerized testing. J Neurooncol. 2015;121(3):617–625. [DOI] [PubMed] [Google Scholar]
- 50. Manly T, Dove A, Blows S, et al. . Assessment of unilateral spatial neglect: scoring star cancellation performance from video recordings—method, reliability, benefits, and normative data. Neuropsychology. 2009;23(4):519–528. [DOI] [PubMed] [Google Scholar]
- 51. Wefel JS, Vardy J, Ahles T, Schagen SB. International Cognition and Cancer Task Force recommendations to harmonise studies of cognitive function in patients with cancer. Lancet Oncol. 2011;12(7):703–708. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.