Computerized assessment of cognitive impairment among children undergoing radiation therapy for medulloblastoma

Abstract

Purpose:

Advantages to computerized cognitive assessment include increased precision of response time measurement and greater availability of alternate forms. Cogstate is a computerized cognitive battery developed to monitor attention, memory, and processing speed. Although the literature suggests the domains assessed by Cogstate are areas of deficit in children undergoing treatment for medulloblastoma, the validity of Cogstate in this population has not been previously investigated.

Methods:

Children participating in an ongoing prospective trial of risk-adapted therapy for newly diagnosed medulloblastoma (n = 73; mean age at baseline = 12.1 years) were administered Cogstate at baseline (after surgery, prior to adjuvant therapy) and three months later (six weeks after completion of radiation therapy). Gold-standard neuropsychological measures of similar functions were administered at baseline.

Results:

Linear mixed models revealed performance within age expectations at baseline across Cogstate tasks. Following radiation therapy, there was a decline in performance on Cogstate measures of reaction time (Identification and One Back). Females exhibited slower reaction time on One Back and Detection tasks at baseline. Higher-dose radiation therapy and younger age were associated with greater declines in performance. Pearson correlations revealed small-to-moderate correlations between Cogstate reaction time and working memory tasks with well-validated neuropsychological measures.

Conclusions:

Cogstate is sensitive to acute cognitive effects experienced by some children with medulloblastoma and demonstrates associations with clinical predictors established in the literature. Correlations with neuropsychological measures of similar constructs offer additional evidence of validity. The findings provide support for the utility of Cogstate in monitoring acute cognitive effects in pediatric cancer.

Introduction

Medulloblastoma, a tumor arising in the brain’s posterior fossa, is the most common form of malignant brain tumor diagnosed in children [1]. Survivors of medulloblastoma are at an increased risk for deficits in cognitive functioning [2], which arise from both disease- and treatment-related risk factors [3]. Current medulloblastoma treatment typically involves surgical resection, postsurgical risk-adapted craniospinal irradiation, and adjuvant chemotherapy [4]. With survival rates now greater than 70% [5], there is increased focus on monitoring for cognitive effects and identifying vulnerable cognitive processes for tailored intervention [6].

Longitudinal studies of children with medulloblastoma have consistently revealed declines on global cognitive measures, which result not from a loss of skills, but a failure to gain new skills at the rate of peers [7]. More recent findings suggest that impairments in attention [8], working memory [9], and processing speed [10] may be proximal contributors to global decline [11]. Common risk factors associated with cognitive decline include female gender, younger age at treatment, longer duration since treatment, increased treatment intensity, and complicating medical factors such as hydrocephalus and posterior fossa syndrome (PFS) following surgical resection [12–14].

Due to the risk of cognitive deficits among childhood medulloblastoma survivors and the potential to significantly impact quality of life, serial monitoring of cognitive skills has become standard of care [15]. Computerized assessment has several advantages over traditional neuropsychological tests, including improved standardized administration across examiners and settings, increased precision and reliability in measuring response time, greater ease of development of alternate forms, decreased testing time, and increased portability.

Cogstate is a flexible battery of computer-based tests assessing domains such as attention, memory, and processing speed (see https://www.cogstate.com). In adults, Cogstate has demonstrated the ability to detect cognitive impairment consistent with clinical conditions (e.g., Alzheimer’s disease, traumatic brain injury, schizophrenia) [16–19], and it is sensitive to changes in cognitive functioning following brain tumor surgery [20] and radiation therapy [21]. The battery has demonstrated associations with other traditional neuropsychological tests measuring similar neuropsychological constructs [17, 18], has minimal cultural bias [22, 23], and limited practice effects [18, 24]. Among children, Cogstate has demonstrated utility identifying changes following cognitive rehabilitation [25, 26], and is sensitive to the cognitive effects of stimulant medication [27].

Cogstate has demonstrated feasibility as a neurocognitive assessment for children with cancer [28, 29]. However, studies have yet to utilize Cogstate pre- and post-irradiation in children with medulloblastoma, or to explore the associations between well-established predictors of cognitive effects and Cogstate performance. Additionally, studies assessing Cogstate in children have not established concordance with gold standard neuropsychological measures in this population. Thus, the primary goal of this study was to assess the sensitivity of Cogstate in detecting therapy-related changes in children undergoing medulloblastoma treatment. Secondary goals sought to further establish the validity of the Cogstate tasks by investigating clinical predictors of performance and correlations with established neuropsychological measures.

Materials and Methods

Participants

Participants in this study were enrolled on an ongoing, multi-site clinical trial (SJMB12; NCT 01878617) for patients with newly diagnosed medulloblastoma or medulloblastoma variants (e.g. melanotic medulloblastoma; medullomyoblastoma), between 3 and 22 years of age at the time of diagnosis. Participants had no previous radiotherapy, chemotherapy, or other brain-directed therapy other than surgery and corticosteroid therapy. Briefly, patients were assigned to treatment strata based on three molecular subgroups that display similarities in histology, chromosomal changes, demographics, and clinical outcomes [30]. Patients were then risk stratified by clinical factors, including degree of metastatic disease, extent of surgical resection, residual disease, histological variant, and molecular features. All participants received post-surgical radiation therapy, with craniospinal doses ranging from 15 Gy to 39.6 Gy. The cumulative primary site doses ranged from 51.0–54 Gy. Additionally, participants received 4–7 cycles of chemotherapy, with some receiving maintenance chemotherapy based on risk strata. During radiation therapy, a subset of participants was randomized to aerobic exercise intervention or standard of care as part of the larger study protocol.

Those with limited English proficiency, pre-morbid neurological or neurodevelopmental conditions, or significant sensory or motor impairment that precluded valid assessment did not receive protocol-based cognitive evaluations. The Institutional Review Board approved the study; written informed consent was required prior to participation.

Socioeconomic status was measured using the Barratt Simplified Measure of Social Status (BSMSS), which amalgamates parental educational attainment and occupational prestige [31].

Cognitive Assessment

Participants completed cognitive assessments at baseline (post-surgery; prior to adjuvant therapy) and 3 months later (6 weeks post completion of radiation therapy). The baseline assessment included Cogstate tasks and gold standard neuropsychological tests, whereas the post-radiation assessment only included Cogstate tasks and the Behavior Rating Inventory of Executive Function (BRIEF).

The Cogstate test battery was administered in a quiet room by a master’s level psychological examiner, under the supervision of a neuropsychologist. Two different versions were used, one for children 5 to 9 years and one for children 10 years and older. Tasks were administered on a standard laptop using the keyboard and mouse to capture responses. Task instructions were displayed on the computer screen and read by the examiner; comprehension of instructions was verified before proceeding to test items. All responses were captured by the computer. Age-standardized z scores were derived (based on normative data) with a mean of 0 and standard deviation of 1. For all Cogstate measures except One Back Accuracy a lower z score indicated better performance (e.g., faster reaction time or fewer errors). For consistency and ease of viewing, Cogstate z scores were reverse coded in Figures 1 and 2 so that a lower z score indicated poorer performance across all measures. The test battery took approximately 25 minutes to administer. See Table 1 for subtest descriptions.

Fig. 1.

Cogstate performance at baseline and post radiation. ** < .01, * < .05, † p < .10 indicating significant change from baseline to post radiation therapy in linear mixed model analyses. GMLT Groton Maze Learning Task. Z scores for Detection Reaction Time, Identification Reaction Time, One Back Reaction Time, and GMLT Errors were reverse coded so that a lower z score represented poorer performance on the task. No baseline measure differed from normative expectations (Z = 0) using one sample t-tests (all p values > .05)

Fig. 2.

Estimated Cogstate performance by radiation exposure Z scores for Identification Reaction Time and One Back Reaction Time were reverse coded so that a lower z score represented poorer performance on the task. Low radiation exposure involved 15Gy craniospinal radiation/51 Gy boost, standard was 23.4 Gy craniospinal radiation/54 Gy boost, and high was 36 Gy craniospinal radiation/54 Gy boost

Table 1.

Cogstate Task Descriptions

Cogstate Task	Cognitive Domain	Task Description	Dependent Variable	Duration (min)
Detection	Processing speed / response time	Press “yes” as soon as a card turns face up	Reaction time	3
Identification	Attention / choice response time	Press “yes” if a flipped card is red and “no” if it is black	Reaction time	3
One Back	Working memory	Press “yes” when a flipped card is the same as the card you just saw and “no” when it is not	Reaction time / accuracy	4
Continuous Paired Association Learning (CPAL)	Learning & memory	After a learning period, one shape appears in a central location and participant indicates where the matching shape is hiding	Errors	7
Groton Maze Learning Task (GMLT)	Executive function / implicit learning	Learn a hidden path through a grid, initially trial and error, followed by repeat exposure	Errors	7

To evaluate convergent validity, gold standard neuropsychological measures were administered. Measures of working memory included Digit Span and Letter-Number Sequencing subtests from the Wechsler Intelligence Scales [32, 33] as well as Woodcock Johnson – Third Edition (WJ-III) [34] Auditory Working Memory. Digit Span includes Digit Span Forward (participant repeats digits verbatim) and Digit Span Backward (participant repeats digits in reverse order). For Letter-Number Sequencing, the examiner presents sequences of numbers and letters, after which the participant repeats the numbers in ascending order followed by the letters in alphabetic order. During Auditory Working Memory, the examinee listens to a series of numbers and objects, then the participant repeats the objects in sequential order followed by the digits in sequential order.

Additional attention and executive function measures included the Omissions, Commissions, Detectability, and Response Style indices of the Conners’ Continuous Performance Test – Kiddie or Second Edition (CPT) [35, 36], as well as the Retrieval Fluency subtest of the WJ-III. The CPT is a computerized measure of sustained attention. Letters are presented on a computer screen, and children press the space bar as quickly and accurately as possible for any letter except the letter X. The Retrieval Fluency subtest involves naming as many examples as possible from a given category (e.g., colors, names).

Processing speed measures included Wechsler Coding and Symbol Search [32, 33] and Hit Reaction Time of the CPT [35, 36]. For Coding, the child transcribes a digit-symbol code as quickly as possible. Symbol Search involves scanning and matching symbols at a quick pace. The Coding and Symbol Search subtests comprise the Wechsler Processing Speed Index (PSI). The Hit Reaction Time Index of the CPT measures how quickly the examinee responds to presented target stimuli.

Learning and memory were assessed with the child and adult versions of the California Verbal Learning Test (CVLT) [37, 38]. The CVLT is a serial list learning task.

In addition to performance measures, the BRIEF and Behavior Assessment System for Children – Second Edition (BASC-2) parent report forms were administered [39, 40]. BRIEF scores of interest included the Working Memory Scale, Metacognitive Index, and Global Executive Composite. The BASC-2 Attention Problems Scale was used.

Statistical Analyses

Descriptive statistics of demographic and clinical variables were calculated to characterize the sample. Linear mixed models (LMM) were used to estimate how the sample differed from the normative population at baseline and to assess Cogstate sensitivity to neurocognitive changes pre- to post-irradiation. Potential demographic and clinical contributors were individually added to the models to investigate their effect on Cogstate performance at baseline as well as the change over time. Convergent validity was assessed using Pearson correlations, comparing Cogstate tasks with gold standard neuropsychological measures of similar constructs. There was a large amount of missing data for the Cogstate Continuous Paired Associate Learning (CPAL) task due to inadequate norms (baseline n = 24; post-radiation n = 22), thus the CPAL task was removed from all analyses. The exercise intervention study is ongoing and has not completed enrollment. Therefore, there was not adequate power to detect changes in performance on Cogstate variables from pre- to post-irradiation for the subsample randomized to exercise versus those randomized to standard of care; data were combined for these two groups for analyses.

Results

Demographic and clinical characteristics

Demographic and clinical characteristics of participants are listed in Table 2. Participants were 75.3% white and 65.8% male. At baseline, the mean age of study participants was 12.1 years (SD = 4.8). On average, children had 1.3 brain surgeries (SD = 0.9) prior to initial testing. Most participants underwent a gross total resection (83.6%), with 6.9% of the sample developing PFS. Just over half of the sample received high dose (≥36 Gy craniospinal radiation/54 Gy boost) radiation therapy (53.4%). The sample was predominately middle class (BSMSS = 36.4 ± 13.5). Radiation treatment groups did not differ on age, race, gender, BSMSS, number of surgeries, or PFS (ps = .167–1.000). However, significantly more children in the high-dose radiation group received a STR or NTR compared to the low-dose or standard-dose groups (p = 0.014).

Table 2.

Demographic and Clinical Characteristics

	n	(%)
Gender  Male  Female	48 25	65.8 34.3
Race  White  Other	55 18	75.3 24.7
Radiation Exposurea  Low  Standard  High	8 26 39	11.0 35.6 53.4
Extent of Surgical Resection  STR  NTR  GTR	7 5 61	9.6 6.8 83.6
PFS  No  Yes	67 5	93.1 6.9
	Mean ± SD	Range
Age at Baseline	12.1 ± 4.8	5.1 – 22.9
SES (BSMSS)b	36.4 ± 13.5	9.0 – 58.0
Number of surgeriesc	1.3 ± 0.9	1 – 8

STR subtotal resection, incomplete tumor resection with gross residualdisease present on neuroimaging, NTR near total resection, incompletetumor resection with minimal residual disease present on post-operativeneuroimaging, GTR gross total resection, resection of tumor without apparent gross residual disease observed by the operating neurosurgeonand confirmed on operative neuroimaging, SES socioeconomic status,PFS posterior fossa syndrome

^aLow radiation exposure involved 15Gy craniospinal radiation/51 Gy, boost, standard was 23.4 Gy craniospinal radiation/54 Gy boost, and high was 36 Gy craniospinal radiation/54 Gy boost

^bSES is based on the Barratt Simplified Measure of Social Status (BSMSS), which takes into account maternal education and occupation and paternal education and occupation. Scores can range from 8 to 66 with higher scores indicative of higher SES

^cThe distribution for number of surgeries included: 1 surgery (n=59), 2 surgeries (n=10), 3 surgeries (n=3), 8 surgeries (n=1)

Cogstate sensitivity to radiation therapy

See Figure 1 for Cogstate performance pre- to post-irradiation. At baseline, LMMs indicated that mean performance on all Cogstate measures did not statistically differ from normative expectations (ps = 0.144 – 0.993). Participants displayed significantly slower Identification reaction time (t = 2.92, p = 0.007) and One Back reaction time (t = 2.06, p = 0.049) at post-irradiation assessment relative to baseline. There was no difference from baseline on One Back Accuracy or Groton Maze Learning Task (GMLT) Total Errors following radiation therapy (t = 0.20, p = 0.840 and t = 1.28, p = 0.212, respectively).

Risk factors

Due to the small number of participants with PFS who were able to complete Cogstate testing (baseline n = 2, post-irradiation n = 3) this risk factor was removed from LMM analyses. Additionally, due to the lack of variability in extent of surgical resection (only 7 patients with less than NTR or GTR) we chose not to explore the effects of this predictor. Main effects and interactions for socioeconomic status (SES) and number of surgeries were explored with no significant findings.

Gender.

LMMs suggested that females exhibited a significantly slower reaction time on Detection and One Back tasks at baseline (p = 0.038 and p = 0.022, respectively); however, there was no effect of gender post-irradiation (p = 0.055 and p = 0.344, respectively). One Back reaction time significantly slowed for males pre- to post-irradiation (p = 0.026); whereas, female reaction time did not change (p = 0.984). However, there was no significant gender by time interaction or difference in slopes across genders (p = 0.187).

Age.

Model fitting suggested that there was a significant interaction between age at baseline testing (treated as a continuous variable) and GMLT performance from pre- to post-irradiation (p = 0.020). Performance on the GMLT worsened for younger participants after radiation therapy but improved for older participants.

Radiation Exposure.

To assess the impact of radiation dose on Cogstate performance, LMMs were fitted to three treatment groups (low-dose, standard-dose, and high-dose radiation) corresponding to craniospinal and primary site dose combinations of 15Gy/51Gy, 23.4Gy/54Gy, and ≥36Gy/54Gy, respectively. The models revealed that participants receiving high-dose irradiation displayed significantly slower reaction time on the Identification task post-irradiation compared to baseline assessment (p = 0.004); whereas, participants receiving low- or standard-dose radiation therapy did not (p = 0.313 and 0.952, respectively; see Figure 2). Consistently, One Back reaction time slowed from pre- to post-irradiation assessments only for the children receiving high-dose radiation (p = 0.021). Children receiving standard-dose (p = 0.399) or low-dose (p = 0.451) radiation did not significantly slow from baseline to post-irradiation on One Back reaction time. Despite the differences in dose-response, there was no significant interaction between degree of radiation exposure and time for either Identification or One Back reaction time (p = 0.345 and p = 0.443, respectively). At baseline, the high-dose and standard dose groups displayed significantly quicker reaction time on One Back than the low-dose group (p = 0.043 and p = 0.036, respectively), but the groups did not significantly differ post-irradiation (p = 0.815 and p = 0.375, respectively; see Figure 2).

Correlations between Cogstate and gold standard neuropsychological measures

Table 3 displays Pearson correlations between Cogstate measures and gold standard neuropsychological measures of similar constructs. Cogstate measures of decision making speed on One Back, Detection, and Identification tasks displayed small-to-moderate correlations with the computerized CPT reaction time (r = 0.33, p = 0.036; r = 0.50, p < 0.001; r = 0.50, p < 0.001, respectively) and Wechsler PSI (r = −0.44, p = 0.002; r = −0.37, p = 0.011; and r = −0.48, p < 0.001, respectively). Consistently, Detection and Identification reaction time measures were significantly associated with CPT Omissions and Response Style (rs = 0.39 – 0.43, ps = 0.005 – 0.012). Further, the One Back reaction time measure displayed significant associations with Wechsler DSB and WJ Retrieval Fluency (r = −0.39, p = 0.006 and r = −0.38, p = 0.008, respectively). There was a small correlation between One Back Accuracy and Auditory Working Memory on the WJ (r = 0.35, p = 0.013). Errors on the GMLT did not significantly correlate with any neuropsychological measures.

Table 3.

Correlations between Cogstate and Gold Standard Neuropsychological Measures

	Detection Reaction Time	Identification Reaction Time	One Back Reaction Time	One Back Accuracy	GMLT Errors
Wechsler DSF	NS	NS	--	--	--
Wechsler PSI	−.37*	−.48***	−.44**	--	--
CPT Omissions	.42**	.43**	--	--	NS
CPT Commissions	NS	NS	--	--	NS
CPT Hit Reaction Time	.50***	.50***	.33*	--	--
CPT Detectability	NS	NS	--	--	--
CPT Response Style	.44**	.39*	--	--	--
BASC Attention Problems	NS	NS	--	--	--
Wechsler DSB	--	--	−.39**	NS	NS
WJ Retrieval Fluency	--	--	−.38**	NS	NS
WJ-III Auditory WM	--	--	NS	.35*	NS
CVLT Total	--	--	--	.27†	−.30†
BRIEF WM	--	--	NS	NS	NS
BRIEF MCI	--	--	NS	NS	NS
BRIEF GEC	--	--	NS	NS	NS

^***p < .001,

^**p < .01,

^*p < .05,

^†p < .10

GMLT Groton Maze Learning Test, DSF Digit Span Forward, PSI Processing Speed Index, CPT Continuous Performance Test, BASC Behavior Assessment Scale for Children, DSB Digit Span Backwards, WMI Working Memory Index, WJ Woodcock Johnson Tests of Cognitive Abilities, CVLT California Verbal Learning Test, BRIEF Behavior Rating Inventory of Executive Function, WM Working Memory, MCI Metacognitive Index, GEC Global Executive Composite

Dashes indicate that we did not run a correlation as, rationally, we did not expect the two measures to correlate (e.g. simple reaction time with higher order executive tasks or a measure of working memory with response time)

Discussion

Among the Cogstate measures, response time variables were more sensitive to treatment effects than measures of accuracy. This finding is consistent with previous research demonstrating that processing speed is a core cognitive deficit among medulloblastoma survivors [11]. White matter (myelinated brain tissue) is central to the development of information processing speed due to its role in enabling the rate of neuronal transmission [41], and these white matter pathways are highly vulnerable to damage from cranial radiation therapy [42, 43]. There is a dose-response pattern seen in neurocognitive testing, where higher-dose radiation is associated with poorer overall cognitive functioning post-treatment [44, 45]. In the present study, we found a similar pattern on two Cogstate measures (Identification and One Back), where the high-dose radiation group significantly slowed between pre- to post-radiation assessments, but the low and standard dose groups did not. Notably, the low-dose group was significantly slower on these tasks at baseline than the standard- and high-dose groups. The only significant difference between the radiation treatment groups on demographic and medical factors was degree of resection. The high-dose group possessed a disproportionate number of children receiving a STR compared to the standard-dose and low-dose groups, both of which only had children receiving a GTR. Previous research has demonstrated that STR is actually associated with poorer baseline functioning [46]. Thus, degree of resection is unlikely to explain the slower baseline functioning in the low-dose group.

In addition to processing speed, the Identification and One-Back reaction time measures further employ attention and working memory resources. These skills are known to be core cognitive deficits in medulloblastoma and arise independently of processing speed following radiation exposure[11, 47]. Detection, a measure of basic processing speed, only showed a trend towards decline from pre- to post-irradiation. This suggests that at an acute phase following radiation exposure, attention and working memory skills may show differential impairment.

Demographic predictors of Cogstate performance were broadly consistent with the pediatric brain tumor literature. Females exhibited the greatest performance deficits at baseline, and younger children appeared more susceptible to changes in neurocognitive functioning following radiation therapy. Cogstate performance was not associated with SES, consistent with previous studies demonstrating no impact of SES on Cogstate [22, 26].

Correlations with well-validated neuropsychological measures offer some support for the validity of Cogstate with this population. Cogstate tasks assessing reaction time demonstrated small-to-moderate correlations with computerized and traditional neuropsychological tasks of processing speed. Weaker associations were observed between a Cogstate visual working memory task and widely used auditory working memory measures. The GMLT, a measure of executive functioning and implicit learning, did not correlate significantly with attention or working memory tasks. The wide range of executive functions assessed by the GMLT may reduce correlations with traditional working memory and attention tasks focused on a narrower band of cognition. Across Cogstate measures, parent ratings of attention and executive functioning were not associated with performance, consistent with the literature [48].

These findings add to the limited body of literature demonstrating the utility of Cogstate in monitoring for acute cognitive effects following cancer treatment. Specifically, measures of response time were sensitive to treatment effects and demonstrated moderate correlations with well-validated neuropsychological measures of processing speed. Cogstate’s short administration time (approximately 25 minutes) lends itself to use as a screener for difficulties that can be further assessed with a comprehensive evaluation, although more information on sensitivity and specificity of the tasks is needed at this time. The highly standardized administration procedures of Cogstate add to its utility as a tool to be used within multi-site trials common in pediatric oncology research. The lack of SES effects on Cogstate performance add to its utility for assessing diverse and geographically isolated patients [23, 25].

Several study limitations exist. Many of the participants in our sample with PFS were unable to complete Cogstate testing preventing exploration of this clinical risk factor known to impact neuropsychological functioning in patients with medulloblastoma. Tasks requiring less sustained attention and motor control typically used in rehabilitation settings are likely better equipped to assess functioning than Cogstate in children with PFS. Additionally, due to a lack of adequate childhood norms we lacked the power to assess the validity of the CPAL task in this population. The development of norms for younger children is essential to assessing the validity and utility of the CPAL task. The current findings display Cogstate’s sensitivity to treatment effects six weeks after radiation therapy; yet, we did not explore the impact of chemotherapy or longer-term outcomes. Future research should assess Cogstate’s ability to detect cognitive effects longitudinally.