Computerized Assessment of Cognitive Late Effects among Adolescent Brain Tumor Survivors

Abstract

BACKGROUND

Advantages of computerized assessment of neuropsychological functions include improved standardization and increased reliability of response time variables. ImPACT (Immediate Post-Concussion Assessment and Cognitive Testing) is a computerized battery developed for monitoring recovery following mild brain injuries that assesses attention, memory and processing speed. Despite evidence that core areas of deficit among cancer survivors are those assessed by ImPACT, it has not previously been used with this population.

PROCEDURE

Twenty four childhood brain tumor (BT) survivors treated with conformal radiation therapy (mean age= 15.7±1.6; mean age at irradiation= 9.8±2.5), twenty solid tumor (ST) survivors treated without CNS-directed therapy (mean age= 16.2±1.8) and twenty healthy siblings (mean age= 15.1± 1.6 years) were administered an age modified version of ImPACT. Additional computerized measures of working memory and recognition memory were administered.

RESULTS

Univariate ANOVAs revealed group differences (p< .05) on measures of recognition memory, spatial working memory, processing speed and reaction time, with BT survivors performing significantly worse than ST survivors and siblings. Pearson correlation coefficients revealed significant associations between ImPACT memory tasks and computerized forced choice recognition tasks (rs= .30-.33, p< .05). Multiple surgical resections, hydrocephalus and CSF shunt placement most consistently predicted worse ImPACT performance using linear mixed models (p< .05).

CONCLUSIONS

The ImPACT test battery demonstrated sensitivity to cognitive late effects experienced by some BT survivors with clinical predictors of performance consistent with the pediatric oncology literature. Correlations with measures of similar constructs provide evidence for convergent validity. Findings offer initial support for the utility of ImPACT for monitoring of cognitive late effects.

INTRODUCTION

Childhood brain tumor (BT) survivors are at increased risk for cognitive impairments secondary to disease- and treatment-related factors [1, 2]. These impairments take a significant toll on quality of life resulting in negative long-term social, academic and vocational consequences [3-5]. As survival rates continue to improve, there is a heightened focus on monitoring for cognitive late effects and designing interventions to improve cognitive outcomes.

Longitudinal studies of children treated for BTs reveal vulnerability to declines in intellectual functioning [2]. Risk factors associated with decline most reliably include younger age at treatment, longer time since treatment, female gender, treatment intensity and complicating medical factors such as hydrocephalus or multiple surgical interventions [6]. Historically, there has been an over reliance on global cognitive measures that are not specific enough to isolate vulnerable neural systems or develop targeted interventions. More recently, core deficits in attention [7, 8], working memory (WM) [7, 9] and processing speed [10] have been identified that likely underlie IQ and academic declines.

Radiation therapy is perhaps the greatest risk factor for cognitive deficits, with risk significantly impacted by radiation dose and volume of irradiated brain [11]. Conformal radiation therapy (CRT) involves sophisticated planning and delivery techniques designed to limit the highest radiation doses to volumes at risk while sparing surrounding normal tissues [12]. Preliminary research findings indicate CRT is associated with a high rate of disease control and better preservation of cognitive abilities [13-15]. Finer characterization of cognitive outcomes following CRT is warranted to delineate risks and benefits of this increasingly popular treatment approach.

Due to significant cognitive, educational and social changes that occur during adolescence [16], survivors may be particularly vulnerable to challenges during this time period; research indicates adolescent cancer survivors exhibit elevated levels of attention deficits, social difficulties, and symptoms of anxiety and depression [17-19]. Often during adolescence, the achievement gap between survivors and their peers widens [20] presumably due to increased expectations for independence revealing already existing cognitive problems, but also possibly due to an unveiling of frontal lobe dysfunction secondary to typical neurodevelopmental processes [21]. Accordingly, careful monitoring of cognitive abilities during adolescence has important implications for prognosticating, selecting and timing interventions, and monitoring intervention outcome.

Given the risk for cognitive late effects among childhood BT survivors and the potential to significantly impact quality of life, serial monitoring of cognitive skills has become standard of care. While these neuropsychological assessments have traditionally been conducted by a trained examiner, there has been an increase in the use of computerized measures to assess cognitive abilities. There are a number of advantages to computerized assessment including improved standardization of administration across examiners and settings, increased precision and reliability in measuring response time, greater ease in development of alternate forms for following performance longitudinally, decreased testing time and increased portability. ImPACT (Immediate Post-Concussion Assessment and Cognitive Testing) is a computerized assessment battery that assesses attention, memory, processing speed and reaction time [22]. ImPACT was originally developed for monitoring recovery in individuals sustaining mild brain injuries; there are over 100 peer-reviewed publications demonstrating reliability and validity of this measure for assessing brain injury related change (http://impacttest.com/publications) including evidence for minimal practice effects in the context of alternate forms.

Despite evidence that core areas of impairment among childhood BT survivors are among those assessed by ImPACT, it has not previously been used with this population. Accordingly, the primary goal of this study was to assess the sensitivity of an age modified ImPACT to cognitive late effects among adolescent BT survivors treated with CRT. We predicted BT survivors would perform worse than both healthy sibling controls and solid tumor (ST) controls that did not receive CNS-directed therapy. The second study aim was to investigate the initial validity of the age modified ImPACT with the adolescent BT population. We hypothesized performance on ImPACT tasks would correlate with measures of similar cognitive constructs and well-established risk factors for cognitive late effects would predict ImPACT performance.

METHODS

Participants

The study sample was comprised of three groups: BT survivors treated with CRT, healthy siblings of children treated for a BT, and ST survivors. BT survivors were treated for a primary CNS tumor (ependymoma, low grade glioma or craniopharyngioma) on an institutional phase II trial of CRT. CRT was initiated at least two years prior to enrollment, without evidence of recurrent disease. CRT, including intensity-modulated radiation therapy, was delivered over six to seven weeks with a prescribed dose of 54-59.4 Gy. The irradiated clinical target volume included a 10 mm margin surrounding the tumor, tumor bed or both, in order to treat microscopic disease. An additional 3 to 5 mm, expanded in three dimensions, was included to account for uncertainty in patient positioning and image registration [23]. ST survivors were treated for a primary ST (Ewing sarcoma, osteosarcoma, soft tissue/rhabdomyosarcoma, neuroblastoma or Wilms’ tumor) without CNS-directed therapy (i.e., cranial radiation therapy, intrathecal chemotherapy or high dose methotrexate), and were diagnosed at least two years prior to study enrollment. Sibling control participants were healthy siblings of St. Jude BT patients (two of which participated in this BT group).

For this study, we included participants from a larger study [24] that were between 13 and 18 years of age, consistent with the target age range of the age modified ImPACT test battery. All participants were primary English speakers. Individuals were excluded from participation for impaired intellectual function (IQ less than 70 for BT survivors or a history of pull-out special education services for ST and sibling control participants for whom IQ data was not available at enrollment), history of CNS injury/disease or documented Attention Deficit Hyperactivity Disorder (predating cancer diagnosis for BT survivors), treatment with psychostimulant or psychotropic medication within two weeks of study participation, or a major sensory or motor impairment that would preclude valid testing. The study was approved by the Institutional Review Board; written informed consent was required prior to participation. Eligible participants were contacted in the order of upcoming hospital visits.

Measures

ImPACT is a computerized assessment battery developed for monitoring recovery following mild brain injuries that assesses attention, memory and processing speed [22]. The version used in this study was modified by researchers at the Children's National Medical Center and University of Pittsburgh for use with adolescents; task parameters and stimuli are identical to the adult version with modified task instructions (simplified and elaborated) to be developmentally appropriate. The test battery was administered in a quiet room in our Psychology Clinic by a master's level research assistant, under the supervision of a neuropsychologist (HMC). Tasks were administered on a standard laptop or desk top computer using the keyboard and mouse for responding; the only computer adaptation was placement of colored stickers on two keys for responding during a reaction time task. Task instructions were written on the computer screen and read by the examiner; comprehension of instructions was verified before proceeding to test items. All responses were captured by the computer, including time in milliseconds. See Table 1 for subtest descriptions.

Table 1.

ImPACT Subtest Descriptions

ImPACT Subtest	Cognitive Domain	Task Description	Dependent Variable
Speed Click	Reaction Time	Presented with a yellow and black circle simultaneously. Click as fast as you can on the yellow circle.	Reaction Time
Word Memory	Verbal Learning & Memory	Presented with 12 individual words to memorize. Immediately following presentation, indicate which words were previously viewed among presented targets and distractors.	Percentage Correct
Design Memory	Visual Learning & Memory	Presented with 12 abstract designs to memorize. Immediately following presentation, indicate which designs were previously viewed among presented targets and distractors.	Percentage Correct
X's & O's	Spatial Working Memory with Interference	Three X's and O's are highlighted among several X's and O's. Participant must remember their location while completing a reaction time task whereby they press a red key when they see a red circle and a blue key when they see a blue square.	Percentage Correct (X's & O's)
Symbol Match	Processing Speed & Associative Memory	A symbol is presented and the participant must quickly click on the corresponding number from a provided key. After multiple trials of each symbol, the key disappears and the participant must indicate, from memory, the number corresponding to presented symbols.	Percentage Correct (Incidental Learning) Reaction Time (Key Visible and Incidental Learning)
Color Match	Choice Reaction Time	Presented with a color word (e.g., RED). Participant must quickly indicate whether the written word and ink color match.	Percentage Correct & Reaction Time
Four Letters	Spatial Working Memory with Interference/Processing Speed	Four letters are presented simultaneously. Participant must remember them, in order, while completing a processing speed task whereby they click on number tiles in reverse order.	Percentage Correct (Four Letters)
Word Memory Delay	Delayed Recognition Memory	Participant indicates which words among presented targets and distractors were on the list presented 20 minutes earlier.	Percentage Correct
Design Memory Delay	Delayed Recognition Memory	Participant indicates which designs among presented targets and distractors were among designs presented 20 minutes earlier.	Percentage Correct

To evaluate convergent validity, participants were administered additional computerized measures of recognition memory and WM that have been described previously by our group [21, 24]. Briefly, analogous Verbal and Face Recognition Memory tasks were administered; each task requires recognition of previously presented stimuli using a forced-choice matching procedure. Twelve words or faces are presented one at a time on a computer monitor. Subsequently, the participant is presented with two words or two faces and must indicate which one was previously viewed. Two 12 item trial lists were administered for both words and faces. The dependent variable of interest is percent of correct responses.

Participants were also administered Self-Ordered Search Verbal (SOS-V) and Self-Ordered Search-Object (SOS-O) WM tasks. For these tasks, words or objects are presented on a computer screen in different array lengths to parametrically vary difficulty level [SOS-V- 2 × 2 (four words), 3 × 2 (six words), 3 × 3 (nine words) and 4 × 3 (twelve words); SOS-O- last selected location is blacked out resulting in 2 × 2 (three objects), 3 × 2 (five objects), 3 × 3 (eight objects) and 4 × 3 (eleven objects)]. Participants are instructed to select one stimulus (word or object) for each trial with stimuli re-arranging after each selection. The goal is to select all stimuli in as few trials as possible thus requiring the participant to hold selected stimuli within their WM. The dependent variable of interest is the Error score, or number of erroneous selections per stimuli [E = (R-N)/N where R is the number of responses and N the number of stimuli in a given trial].

To characterize general cognitive ability of all three groups, the two subtest version (Vocabulary and Matrix Reasoning) of the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999) was administered. This estimated measure of intelligence is highly correlated with full scale IQ obtained from Wechsler scales (correlations .81 to .87) [25].

Statistical Analyses

Descriptive statistics of demographic and clinical variables were calculated to characterize and compare participant groups. To investigate the ability of an age modified ImPACT to differentiate among groups, univariate ANOVAs with appropriate post-hoc tests were conducted. To assess convergent validity, Pearson partial correlations (accounting for group membership) were completed comparing ImPACT subtests with previously established measures of similar constructs. Linear mixed models were used to examine potential demographic and clinical contributors to ImPACT performance.

RESULTS

Demographic and Clinical Characteristics

The three groups were balanced with respect to age at participation, gender, and race (Table 2). The ST group was significantly younger than the BT group at time of diagnosis and was significantly further out from diagnosis than the BT group at time of study participation. All three groups had current mean IQs within the average range; however, the BT group had a significantly lower IQ than both the ST and sibling control groups. No individuals were removed from data analysis for an IQ less than 70. The BT group had significantly lower SES compared to the sibling group. The BT group was balanced by diagnosis; a majority of patients had supratentorial tumor location (Table 3). Two thirds of the patients experienced hydrocephalus with half receiving CSF shunting. Almost all underwent surgical resection prior to CRT with the majority receiving a biopsy or subtotal resection. Only two patients received chemotherapy prior to CRT.

Table 2.

Demographic Characteristics by Group

	Brain Tumor n = 24	Sibling n = 20	Solid Tumor n = 20	pa
Gender (% male)	46	50	50	0.94
Race (% Caucasian)	91.7	95.0	95.0	0.87
SESb,c	35.3 ± 12.1	45.5 ± 13.2	42.4 ± 14.4	0.04
Age at participation (years)	15.7 ± 1.6	15.1 ± 1.6	16.2 ±1.8	0.14
Age at diagnosis (years)	8.5 ±3.1	NA	5.8 ± 5.1	0.04
Time since diagnosis (years)	7.2 ± 3.0	NA	10.4 ± 4.6	0.01
WASI IQ (standard score)d	96.4 ± 11.5	110.8 ± 10.5	105.6 ± 10.5	<.01

^aP-values are from ANOVAs for analyses with three groups and independent t-tests for analyses with two groups.

^bSES is based on the Barratt Simplified Measure of Social Status which takes into account maternal education and occupation, and paternal education and occupation. Scores can range from 8 to 66 with higher score indicative of higher SES.

^cAll values are presented as mean ± standard error unless otherwise specified.

^dWASI= Wechsler Abbreviated Scale of Intelligence; standard scores have a mean of 100 and standard deviation of 15.

Table 3.

Clinical Characteristics of Brain Tumor Group

	n	%	pa
Tumor Diagnosis
Ependymoma	8	33.3	1.00
Low Grade Glioma	8	33.3
Craniopharyngioma	8	33.3
Tumor Location
Infratentorial	9	37.5	0.22
Supratentorial	15	62.5
Hydrocephalus
No	9	37.5	0.22
Yes	15	62.5
CSFb Shunting
No	16	66.7	0.10
Yes	8	33.3
Pre-CRTc Chemotherapy
No	22	91.7	<.01
Yes	2	8.3
Extent of Surgical Resectiond
Biopsy/STR	16	66.7	0.10
NTR/GTR	8	33.3
Pre-CRT Surgerye
n = 1	13	59.1%	0.39
n = 2-4	9	40.9%

^aP-value indicates whether the group is equally distributed across subcategories using Chi-square for three groups and binomial tests for two groups.

^bCSF= cerebrospinal fluid.

^cCRT= conformal radiation therapy.

^dBiopsy= tumor sampling to establish histologic diagnosis without intent to resect; STR= subtotal resection, incomplete tumor resection with gross residual disease present on post-operative neuroimaging; NTR= near total resection, incomplete tumor resection with minimal residual disease present on post-operative neuroimaging; GTR= gross total resection, resection of tumor without apparent gross residual disease observed by the operating neurosurgeon and confirmed on post-operative neuroimaging.

^eTwo patients underwent a needle biopsy only; denominator for calculated percents is 22 for this variable.

Cognitive Outcomes

With respect to the first aim to evaluate the ability of the age modified ImPACT to distinguish among groups, univariate ANOVAs revealed significant group differences in accuracy for the Word Memory Delay, Design Memory, Design Memory Delay, and X's and O's subtests (p < .05; Figure 1). For each of these subtests, post hoc comparisons indicated the BT group performed worse than the ST control group (ps= .004 - .038) and the sibling control group (ps= .002 - .041), who did not differ from one another (ps = .455 - .842). There were also trends for group differences in accuracy for the Word Memory and Symbol Match (incidental learning; p < .10). Further, univariate ANOVAs revealed significant group differences in reaction time across the Speed Click, Color Match, Symbol Match (key visible) and Symbol Match (incidental learning; p < .005; Figure 2). For each of these subtests, post hoc comparisons indicated the BT group performed worse than the ST control group (ps= .0003 - .0067) and the sibling control group (ps < .0001 - .0057), who did not differ from one another (ps= .0987 -.8690).

Figure 1.

ImPACT Subtest Accuracy by Group

Figure 2.

ImPACT Subtest Speed by Group

With respect to convergent validity, Pearson partial correlations (accounting for group membership) revealed a significant correlation between the Verbal Recognition Memory Task and ImPACT Word Memory (r= .29, p= .025) and ImPACT Word Memory Delay (r=.30, p= .021). Pearson partial correlations were also significant for the Face Recognition Memory Task and ImPACT Design Memory (r= .26, p= .039) and ImPACT Design Memory Delay (r= .25, p= .047). It should be noted that during the Verbal and Face Recognition Memory Tasks examiner observations revealed a small subset of participants who misunderstood task directions to select previously encountered stimuli and instead selected the novel stimuli not previously encountered [24]. When observed, instructions were carefully re-explained for the second trial; only data from the second trial were used for all participants. Given this finding, data were excluded for participants with percent accuracy scores < 25%, which is significantly less than chance (50%). This resulted in removal of data for one participant in the BT group and two participants in the sibling group from the Verbal Recognition Memory Task but none from the Face Recognition Memory Task. In contrast to recognition memory tasks, Pearson partial correlations for WM tasks were not significant between SOS-V and Four Letters (r= -.08, p= .52) or SOS-O and X's and O's (r= .06, p= .66).

Univariate linear mixed models were used to examine potential demographic and clinical contributors (variables listed in Table 2 and 3) to ImPACT performance (dependent variables included in Figures 1 and 2). Hydrocephalus at the time of diagnosis, CSF shunt insertion, and greater number of pre-CRT surgeries were significantly predictive of worse performance across many of the ImPACT variables, particularly processing speed variables (p< .05). Age, SES, tumor diagnosis, and tumor location were predictive of one or two of the twelve ImPACT dependent variables but not always in the expected direction. Gender, race, age at CRT, IQ and extent of resection were not predictive of cognitive performance.

DISCUSSION

In general, findings were consistent with a priori hypotheses. Study results indicate the age modified ImPACT battery is sensitive to cognitive late effects experienced by some BT survivors. As a group, BT survivors were less accurate than both ST survivors and healthy siblings on measures of verbal and nonverbal recognition memory as well as a measure of spatial WM. Further, BT survivors exhibited a significantly slower response time relative to both control groups across all tasks. Findings offer some support for the convergent validity of the ImPACT battery with this population. Correlations among ImPACT subtests assessing verbal and nonverbal recognition memory correlated moderately with previously studied experimental tasks assessing the same constructs; however, ImPACT subtests assessing verbal and spatial WM did not correlate significantly with other computerized WM tasks. Finally, clinical predictors of ImPACT performance were generally consistent with the pediatric oncology literature. Those children potentially subject to greater neurodevelopmental disruption due to hydrocephalus, shunt insertion and multiple surgical resections showed the greatest performance deficits.

Results indicate that response time variables were more sensitive to group differences than performance accuracy measures. This finding is consistent with the extant literature where processing speed is emerging as a core cognitive late effect among childhood BT survivors [10, 26, 27]. Processing speed deficits are in accordance with prominent conceptual models that suggest reductions in cerebral white matter (myelin) may underlie the emergence of cognitive late effects. Cranial radiation therapy is a well-established cause of change in cerebral white matter [28] and there is accumulating evidence indicating reduced cerebral white matter accounts for a significant proportion of the observed decline in IQ among childhood BT survivors [29, 30]. Cerebral white matter is particularly relevant to the development of information processing speed because of its role in facilitating the rate of neuronal transmission [31]. In fact, fractional anisotropy (a Diffusion Tensor Imaging derived index of white matter integrity) has been shown to be reduced in childhood cancer survivors and associated with speed of processing and motor speed [32].

Recent research indicates that CRT for childhood BTs results in high survival rates with better preservation of intellectual functioning [33], academic skills [13], learning and memory [14], and adaptive functioning [15]. Current results indicate remaining risk for cognitive disruption with this treatment approach and highlight the need to incorporate measures sensitive to specific cognitive processes in clinical and research assessment batteries. These measures hold the greatest promise for revealing vulnerable neural processes and identifying areas for targeted interventions. In contrast to the BT group, ST survivors performed similarly to healthy sibling controls across measures suggesting minimal cognitive risk related to systemic chemotherapy received by these children. This finding also indicates that performance deficits on ImPACT were specific to CNS disease and/or treatment rather than reflective of the cancer experience in general (e.g., school absence).

Potential study limitations exist. First, the current sample size did not allow for a full exploration of demographic and clinical predictors of ImPACT performance. It is possible that some of the negative predictive findings were due to reduced power to detect significance. Second, while an adolescent sample is of particular interest with respect to cognitive late effects given heightened developmental vulnerability, the truncated age range for this study likely limited variation needed to detect relationships among ImPACT performance and variables such as age at assessment or time since treatment. Finally, there were some issues with comprehension of instructions for the non-ImPACT computerized recognition memory tasks. While this issue was addressed retrospectively through data analysis, it will be important to confirm real-time participant understanding of all directions in future studies. Understanding of directions was not problematic for the ImPACT battery suggesting appropriateness of age-related modifications.

Current findings offer initial support for the utility of the age modified ImPACT in monitoring for cognitive late effects following cancer treatment. This battery may serve as a useful adjunct or alternative to traditional neuropsychological assessments, particularly for the assessment of processing speed given increased reliability in measuring response time. The highly standardized administration of ImPACT also lends itself well to multi-site and cooperative group trials, which are common place for pediatric oncology research. The potential to train non-doctoral level administrators may reduce financial burden and extend reach to additional administration sites. Given the relatively short administration time (approximately 45 minutes), this battery may serve as a useful screener for deficits prompting a more comprehensive neuropsychological evaluation if warranted; additional research thoroughly documenting sensitivity and specificity would first be required. Additional research applications include longitudinal investigations to evaluate sensitivity to emergence of late effects, assessment in the neuroimaging setting to increase understanding of brain-behavior relationships following cancer treatment, or possible remote assessment to better serve geographically isolated patients.