Abstract
While longitudinal studies of children treated for brain tumors have consistently revealed declines on measures of intellectual functioning, greater specification of cognitive changes following treatment is imperative for isolating vulnerable neural systems and developing targeted interventions. Accordingly, this cross-sectional study evaluated the performance of childhood brain tumor survivors (n= 50) treated with conformal radiation therapy, solid tumor survivors (n= 40) who had not received CNS-directed therapy, and healthy sibling controls (n= 40) on measures of working memory [Digit Span and computerized self-ordered search (SOS) tasks]. Findings revealed childhood brain tumor survivors were impaired on both traditional [Digit Span Backward- F(2, 127)= 5.98, p< .01] and experimental [SOS-Verbal- F(2, 124)= 4.18, p< .05; SOS-Object- F(2, 126)= 5.29, p< .01] measures of working memory, and performance on working memory measures correlated with intellectual functioning (Digit Span Backward- r= .45, p< .0001; SOS- r= −.32 − −.26, p< .01). Comparison of performance on working memory tasks to recognition memory tasks (computerized delayed match-to-sample) offered some support for greater working memory impairment. This pattern of findings is consistent with vulnerability in functional networks that include prefrontal brain regions and has implications for the clinical management of children with brain tumors.
Keywords: cancer, radiation therapy, cognitive late effects, prefrontal cortex, pediatric, neurodevelopment
Survivors of brain tumors (BTs) diagnosed in childhood are at significant risk for cognitive impairments secondary to disease- and treatment-related factors. As survival rates improve, efforts to optimize long-term cognitive outcomes take on added importance. Longitudinal studies of children treated for BTs most consistently reveal declines in intellectual functioning (Mulhern et al., 2004). Progressive IQ loss likely reflects a decreased rate of learning compared to peers rather than a loss of previously acquired knowledge (Palmer et al., 2001). Risk factors associated with IQ decline most reliably include younger age at treatment, longer time since treatment, female gender, treatment intensity, and complicating medical factors (as reviewed in Mulhern & Butler, 2004).
Historically, the cancer survivorship literature has been limited by over reliance on global cognitive measures not functionally specific enough to facilitate identification of vulnerable neural systems or development of targeted cognitive interventions. In more recent years, investigations have begun to identify specific areas of cognitive impairment including attention (Dennis et al., 1998; Reeves et al., 2006), working memory (WM; Dennis et al., 1992; Dennis et al., 1998; Kirschen et al., 2008 but not Mabbott et al., 2008) and processing efficiency (Waber et al., 2006; Mabbott et al., 2008) that may be proximal contributors to global IQ declines. Emerging areas of core deficit are informative as nearly half of age-related improvements in IQ can be accounted for by developmental improvements in WM and processing speed (Fry & Hale, 1996). WM may be particularly vulnerable to radiation effects as indicated by findings revealing WM impairment larger than predicted based on reduced processing speed (Schatz et al., 2000).
A primary method of treatment for childhood BTs is radiation therapy, which is a well-established cause of change in cerebral white matter (Filley & Kleinschmit-DeMasters, 2001). There is accumulating evidence to suggest reduced cerebral white matter accounts for a notable proportion of the observed decline in IQ among BT survivors (Mulhern et al., 1999; Reddick et al., 2000). Total dose of irradiation plays a significant role in cognitive outcomes following radiation therapy (Grill et al., 1999). Thus, there is strong rationale for reduction in radiation dose and volume when appropriate tumor control can be maintained. Accordingly, conformal radiation therapy (CRT) encompasses sophisticated planning and delivery techniques developed to limit highest radiation doses to volumes at risk while sparing surrounding normal tissues (Merchant et al., 2004). Preliminary evidence suggests CRT results in a high rate of disease control and better preservation of cognitive abilities (Conklin et al., 2008; Merchant et al., 2004). Further characterization of cognitive outcomes following therapy, including exploration of specific processes that may be more sensitive to radiation effects, is warranted to understand risks and benefits of this treatment approach.
WM is an ideal system to study in this regard as it is well defined behaviorally, in keeping with Baddeley and Hitch's tripartite model (Baddeley, 1998), and neuroanatomically, with convergent evidence for involvement of the dorsolateral pre-frontal cortex from both primate (Goldman-Rakic, 1995) and functional neuroimaging (Petrides, 1995; Smith & Jonides, 1998) studies. WM is a limited capacity system that facilitates online maintenance and manipulation of information used to guide cognition and behavior (Baddeley, 1998; Goldman-Rakic, 1995). It is an important cognitive system that has been shown to subserve many cognitive and academic skills including language comprehension (Hanten, Levin & Song, 1999), mathematical computation (Ayr, Yeates & Enrile, 2005), reading and writing (De Jonge & de Jong, 1996; Swanson, 1999). Given protracted myelination of the prefrontal cortex (Giedd, 2004; Sowell, et al., 2001), and radiation induced cerebral white matter changes, WM may be particularly vulnerable to radiation-related neurotoxicity.
Studies conducted by Dennis and colleagues in the 1990s provided the first evidence for WM impairment among childhood BT survivors. Using clinical measures, they identified WM deficits associated with a history of radiation therapy and principal tumor site involving thalamic/epithalamic brain regions (Dennis et al., 1992). Subsequently they found risk for WM impairment depended on the interaction between tumor location and treatment, with worst outcomes for patients receiving radiation therapy for a posterior third-ventricle tumor (Dennis et al., 1998). More recently, investigators have found WM impairment in patients with tumors in other locations, including cerebellar tumors (Kirschen et al., 2008). However, these findings have not been ubiquitous, with other investigators failing to find WM impairment in children treated for posterior fossa tumors (Mabbott et al., 2008). Further investigation is warranted using experimental measures that may better isolate specific cognitive processes and offer clues regarding neurological processes based on prior empirical studies.
Whereas clinical WM tasks typically require free recall of information, recognition memory tasks involve identification of previously encountered information using a forced-choice matching procedure. Thus, recognition memory tasks exert a relatively low level of WM executive control dependent on frontal brain regions (Luciana et al., 2005). Recognition memory has traditionally been thought to rely on more posterior, medial temporal lobe, memory systems (Nelson, 1995), which is supported by recent functional neuroimaging studies (Olsen et al., 2009; Schon et al., 2004). Research with typically developing children has revealed a dissociation in findings whereby performance improves during adolescence on WM tasks but not recognition tasks, in keeping with protracted frontal lobe development (Conklin et al., 2007). These findings suggest WM, not fully matured by adolescence, may be more vulnerable than recognition memory in childhood BT survivors. To our knowledge, this is the first study that assesses BT survivors using a battery of memory measures, including traditional and experimental measures, WM and recognition memory, grounded in the prevailing cognitive and neuroscience models.
For the current study, we used a controlled, cross-sectional experimental design to test our hypotheses about WM performance. The primary objective was to assess childhood BT survivors, treated homogeneously with CRT, on experimental measures of WM and recognition memory in order to identify cognitive processes that may contribute to the well-established decline on global cognitive measures. We predicted: 1) childhood BT survivors would perform significantly worse than solid tumor (ST) survivors who had not received CNS-directed therapy and healthy siblings on measures of WM, 2) performance on measures of WM would have a modest but significant association with performance on a measure of intellectual functioning and 3) childhood BT survivors would demonstrate a significantly higher incidence of WM impairment than recognition memory impairment. In the event WM deficits were identified among BT survivors, we planned exploratory analyses to identify demographic or clinical factors predictive of WM performance.
Method
Participants
Participants were recruited to form three groups: BT survivors treated with CRT, healthy siblings of children treated for a BT, and ST survivors. A sibling control group was included given the use of experimental measures without published norms, and to increase group similarity on potentially contributory variables. A ST control group was included to facilitate identification of cognitive effects specific to CNS-directed therapy while accounting for the childhood cancer experience including frequent school absences. Participants were required to be English speaking and between 8 and 18 years of age. All three groups were stratified by gender and age (8–12; 13–18). The BT group was further stratified by tumor location (infratentorial; supratentorial). The study was approved by the Institutional Review Board; written informed consent was required prior to participation. Eligible participants were contacted in the order of upcoming hospital visits for routine medical care.
All BT patients were treated for a primary CNS tumor (ependymoma, low grade glioma or craniopharyngioma) on an institutional phase II trial of CRT (RT-1). They were required to have initiated radiation therapy at least two years prior to enrollment, with no evidence of recurrent disease, to characterize late rather than acute effects of treatment. Patients received CRT, including intensity-modulated radiation therapy, over six to seven weeks with a prescribed dose of 54–59.4 Gy. The irradiated clinical target volume included a 10 mm margin surrounding the tumor, tumor bed or both, in order to treat microscopic disease. An additional 3 to 5 mm was included to account for uncertainty in patient positioning and image registration. Target volume definitions and treatment parameters have been previously reported (Merchant, 2002). All ST survivors were treated for a primary ST (Ewing sarcoma, osteosarcoma, soft tissue/rhabdomyosarcoma, neuroblastoma or Wilms' tumor), without CNS-directed therapy (i.e., cranial radiation therapy, intrathecal chemotherapy or high dose methotrexate), and were diagnosed at least two years prior to study enrollment. Sibling control participants were healthy siblings of St. Jude BT patients (15 of which participated in this study).
Individuals were excluded from participation for significant impairment in global intellectual functioning to maximize the ability to complete cognitive tasks. All BT patients were excluded for an IQ less than 70 as indicated by prior RT-1 testing. ST and sibling control participants were excluded if they had a history of pull-out special education services. In the unlikely event current study testing revealed an IQ less than 70, data from that participant was removed from analysis. Participants were also excluded for a history of CNS injury or disease (predating cancer diagnosis in BT patients), documented Attention Deficit Hyperactivity Disorder (predating cancer diagnosis for BT patients), treatment with psychostimulant or psychotropic medication within two weeks of study participation, or a major sensory or motor impairment that would preclude valid testing.
Procedure
Assessment of Working Memory
All participants were administered the Digit Span Task from the age appropriate Wechsler Scale [Wechsler Intelligence Scale for Children- Fourth Edition, Integrated (WISC-IV Integrated; Wechsler, 2003) or Wechsler Adult Intelligence Scale-Third Edition (WAIS-III; Wechsler, 1999)]. Internal consistency reliability for this task is high (WISC-IV Integrated, r= .87). Z-scores were computed separately for longest Digit Span Forward and Digit Span Backward using published normative data so WISC-IV and WAIS-III scores could be combined.
Two experimental WM measures were also administered: Self-Ordered Search-Verbal (SOS-V) and Self-Ordered Search-Object (SOS-O), modeled after examiner administered tasks by Petrides and Milner (1982). They have been used previously in behavioral studies of typically developing children (Conklin et al., 2007; Luciana et al., 2005) and patients with schizophrenia (Conklin et al., 2005), as well as neuroimaging studies with healthy adults (Curtis, Zald & Pardo, 2000). Positron emission tomography indicates performance on SOS-O is associated with an increase in regional cerebral blood flow in the dorsolateral prefrontal cortex and the frontopolar region, with better performing participants demonstrating a greater amount of blood flow to the prefrontal cortex (Curtis et al., 2000).
For the SOS-V task, words are presented in an array on a computer monitor. Four array size trials were used: 2 × 2 for four words, 3 × 2 for six words, 3 × 3 for nine words and 4 × 3 for 12 words (see Figure 1). Participants select each word once, and only once, in any order. After each response, the words randomly rearrange cueing initiation of the next response. The goal is to complete each trial with as few responses as possible, thus requiring the participant to hold previously selected words in WM. All words have an age of acquisition of ≤ eight years of age (Gilhooly & Logie, 1980) and were chosen to be low in visual imagery to encourage verbal mnemonic strategies (Gilhooly & Logie, 1980). The computer program precludes an alphabetization strategy or selecting the same location multiple times in a row, thus capitalizing on randomized spatial presentation of words. For each difficulty level, the trial ends when the participant selects all words or after 3N responses (where N is the number of words in a trial), whichever occurs first. The dependent variable of interest is the error score (E) which is the number of erroneous attempts per word. E= (R−N)/N where R is the number of responses.
The SOS-O task is parallel in format to the SOS-V task but includes geometric objects rather than words as stimuli. Four array size trials were used: 2 × 2 for three objects, 3 × 2 for five objects, 3 × 3 for eight objects and 4 × 3 for eleven objects (see Figure 2). During this task, the location of the most recently selected object is muted with a black square that does not accept a response for the subsequent response. Random rearrangement of objects and muting of previous responses limits the use of spatial mnemonic strategies. Objects were selected that are not readily nameable in order to limit verbal mnemonic strategies.
Assessment of Recognition Memory
To assess recognition memory, analogous verbal and face recognition tasks were administered. Both tasks require recognition of previously presented stimuli using a forced-choice, delayed match-to-sample, procedure. For the Verbal Recognition Memory Task, a series of twelve words is presented one-by-one on a computer monitor. Subsequently, the participant is presented with twelve word pairs and must indicate which of two presented words was on the list of twelve previously viewed words. Two trials of twelve words were administered. The dependent variable of interest is the percent of correct responses.
The Face Recognition Memory Task is the same as the Verbal Recognition Memory Task but includes twelve faces rather than words. The faces are neutral in facial expression and derived from the MacBrain Stimulus Set (developed by Nim Tottenham at the University of Minnesota). Two trials of twelve faces were administered.
Assessment of General Cognitive Ability
All participants were administered the Vocabulary and Matrix Reasoning subtests of the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999). This abbreviated IQ is highly correlated with full-scale IQ (correlation with WISC-IV is .82; Wechsler, 2003). The Reading subtest of the Wide Range Achievement Test, Third Edition (WRAT-3; Wilkinson, 1994) was administered to estimate reading level of participants and verify adequate ability for completing verbal tasks.
Assessment of Demographic Characteristics
Caregivers completed a demographic and developmental questionnaire that includes questions for derivation of an index of socioeconomic status (SES). The Barratt Simplified Measure of Social Status was used (Barratt, 2006). Scores can range from 8 to 66 with higher scores indicative of higher SES.
Identification of Clinical Variables
Clinical variables were extracted from the medical record. Extent of surgical resection was categorized as biopsy, subtotal resection, near total resection or gross total resection based on gross residual disease on post-operative neuroimaging. Hydrocephalus was categorized as present/not present and tumor location as infratentorial/supratentorial based on neuroimaging scans at diagnosis. Medical records were also reviewed to identify BT patients who received a ventriculo-peritoneal shunt for hydrocephalus management.
Data Analytic Plan
Descriptive statistics of demographic and clinical variables were calculated to characterize participant groups. Groups were compared using binomial tests, analyses of variance (ANOVA) or chi-square to establish group similarity. ANOVAs were used to investigate group differences on the Digit Span and Recognition Memory tasks, with appropriate post hoc comparisons to evaluate significant findings. For the SOS tasks, linear mixed models were used to examine main effects of age, task difficulty (array size) and group, as well as their interactions. When significant main effects were identified, post hoc comparisons were conducted. To address multiple comparisons and risk of Type I error, the Adaptive Holm method was used to ensure statistically significant findings remained significant after adjusting for number of group comparisons for each outcome measure.
The association between WM performance and intellectual functioning was characterized by Pearson correlation coefficients. To compare the rate of WM impairment to recognition memory impairment in BT survivors, each BT patient was categorized as impaired/not impaired based on the sibling control group (performance more than one standard deviation worse than siblings). A McNemar test was then used to compare the percentage impairment on WM tasks to recognition memory tasks. This process was repeated using the ST group to define impairment. Further, linear mixed models were used to investigate whether group differences on the SOS tasks remained after adjusting for performance on recognition memory tasks. Finally, univariate linear models were used to explore clinical predictors (variables listed in Tables 1 and 2) of WM performance on the Digit Span Task and SOS tasks for the BT group.
Table 1.
Brain Tumor n = 50 | Sibling n = 40 | Solid Tumor n = 40 | p a | |
---|---|---|---|---|
Gender (% male) | 50 | 50 | 50 | 1.00 |
Race (% Caucasian) | 92 | 95 | 95 | 0.81 |
SESb,c | 37.61 ± 1.72 | 42.63 ± 1.90 | 42.09 ± 2.11 | 0.11 |
Age at participation (years) | 13.18 ± 0.41 | 12.91 ± 0.41 | 13.21 ± 0.55 | 0.88 |
Age at diagnosis (years) | 6.38 ± 0.48 | NA | 4.50 ± 0.66 | 0.02 |
Time since diagnosis (years) | 6.80 ± 0.37 | NA | 8.71 ± 0.62 | <.01 |
WASI IQ (standard score)d | 98.20 ± 1.97 | 109.00 ± 2.02 | 107.88 ± 1.97 | <.01 |
WRAT-3 (standard score)e | 100.08 ± 2.37 | 109.65 ± 1.62 | 106.88 ± 1.42 | <.01 |
P-values are from ANOVAs for analyses with three groups and independent t-tests for analyses with two groups.
SES is based on the Barratt Simplified Measure of Social Status which takes into account maternal education and occupation, and paternal education and occupation. Scores can range from 8 to 66 with higher score indicative of higher SES.
All values are presented as mean ± standard error unless otherwise specified.
WASI= Wechsler Abbreviated Scale of Intelligence; standard scores have a mean of 100 and standard deviation of 15.
WRAT-3= Wide Range Achievement Test, Third Edition; standard scores have a mean of 100 and standard deviation of 15.
Table 2.
n | % | p a | |
---|---|---|---|
Tumor Diagnosis | |||
Ependymoma | 22 | 44 | 0.22 |
Low Grade Glioma | 12 | 24 | |
Craniopharyngioma | 16 | 32 | |
Tumor Location | |||
Infratentorial | 22 | 44 | 0.40 |
Supratentorial | 28 | 56 | |
Hydrocephalus | |||
No | 21 | 42 | 0.26 |
Yes | 29 | 58 | |
CSFb Shunting | |||
No | 29 | 58 | 0.26 |
Yes | 21 | 42 | |
Pre-CRTc Chemotherapy | |||
No | 44 | 88 | <0.01 |
Yes | 6 | 12 | |
Extent of Surgical Resectiond | |||
Biopsy/STR | 25 | 50 | 1.00 |
NTR/GTR | 25 | 50 | |
Pre-CRT Surgerye | |||
n = 1 | 31 | 65 | 0.04 |
n = 2–4 | 17 | 35 |
P-value indicates whether the group is equally distributed across subcategories using Chi-square for three groups and binomial tests for two groups.
CSF= cerebrospinal fluid.
CRT= conformal radiation therapy.
Biopsy= tumor sampling to establish histologic diagnosis without intent to resect; STR= subtotal resection, incomplete tumor resection with gross residual disease present on post-operative neuroimaging; NTR= near total resection, incomplete tumor resection with minimal residual disease present on post-operative neuroimaging; GTR= gross total resection, resection of tumor without apparent gross residual disease observed by the operating neurosurgeon and confirmed on post-operative neuroimaging.
Two patients underwent a needle biopsy only; denominator for calculated percents is 48 for this variable.
Results
Demographic and Clinical Characteristics
The three groups were balanced with respect to gender, race, SES and age at participation (Table 1). The ST group was significantly younger than the BT group at time of diagnosis and was significantly further out from diagnosis than the BT group at time of study participation. While all three groups had estimated IQ means within the average range, the BT group had a significantly lower IQ than both the ST and sibling control groups. No individuals, in any group, were removed from data analysis for an IQ less than 70.
The BT group was balanced by diagnosis and tumor location (Table 2). Only six patients received chemotherapy prior to radiation therapy; whereas, all but two patients had undergone surgical resection(s). Most patients undergoing chemotherapy received multiagent chemotherapy including cyclophosphamide, cisplatin or carboplatin, etoposide and vincrinstine.
Cognitive Outcomes
Working Memory
There were no significant group differences on the Digit Span Forward Task (p= .07). There were significant group differences on the Digit Span Backward Task. Post hoc comparisons indicated BT survivors performed significantly worse than both ST and healthy controls (p< .01), who did not differ from one another (p= .61). See Figure 3.
For the SOS-V task, two participants in the BT group were excluded from data analysis due to reading ability below the 8 year-old level. Linear mixed models did not reveal any significant interactions among array size, group and age. There were statistically significant main effects for array size (p< .0001) and group (p< .05), but not age (p= .50). Post hoc comparisons indicated performance across groups worsened with increasing array size. Post hoc comparisons also indicated, after accounting for age and array size, the BT group performed significantly worse than the ST group (p< .01) and worse than the sibling group (p< .05), which did not differ from each other (p= .49).
Similar to the SOS-V task, linear mixed models did not reveal any significant interaction among array size, group and age for the SOS-O task. There were statistically significant main effects for array size (p< .0001) and group (p< .01), but not age (p= .20). Post hoc comparisons indicated across groups performance worsened with increasing array size. Post hoc comparisons also indicated, after accounting for age and array size, the BT group performed significantly worse than the ST group (p< .01) and worse than the sibling group (p< .05), which did not differ from each other (p= .70). See Figure 4.
Working Memory and Intellectual Functioning
For data reduction purposes, a mean score for SOS-V and SOS-O was computed. There was a small, but significant, correlation between the SOS-V mean error score and estimated IQ for the combined sample (r= −.26, p< .01). Likewise, there was a significant correlation between SOS-O mean error score and estimated IQ (r= −.32, p< .001). For both Digit Span Tasks, there was also a significant correlation between estimated IQ and span length (Digit Span Forward- r= .35, p< .0001; Digit Span Backward- r= .45, p< .0001). When looking within groups, the correlations were all in the expected directions and generally of similar magnitude across groups. Despite a reduction in power to detect significance, correlations reached at least a trend for significance (p < .10) for the SOS-V task in the ST group, the SOS-O task in the ST and sibling groups, Digit Span Forward in the BT group and Digit Span Backward in the BT and sibling groups.
Recognition Memory
The same two participants excluded from data analysis for the SOS-V task were excluded for the Verbal Recognition Memory Task due to inadequate reading ability. For both Verbal and Face Recognition Memory Tasks participants completed two trials. Examiner observations revealed a small subset of participants misunderstood the directions to select the stimuli previously encountered and instead selected novel stimuli. When observed, the instructions were carefully re-explained for the second trial. Given this finding, only data from the second trial was used for all participants. Post hoc data inspection revealed there were still a few participants that appeared to perform the opposite task of the instructions resulting in performance levels of 0% or near 0% accuracy. Given this finding, data was excluded for any pariticipants with percent accuracy scores < 25%, which is significantly less than chance performance (50%) and suggestive of a failure to understand task instructions. This resulted in removal of data for two participants in the BT group and two in the sibling group from the Verbal Recognition Memory Task but none from the Face Recognition Memory Task.
There were significant group differences on the Verbal Recognition Memory Task (p< .05). Post hoc comparisons indicated the BT group performed significantly worse than the sibling group (p< .05), with a trend for worse performance than the ST group (p= .08); there was no significant difference between the ST and sibling groups (p= .57). There were no significant group differences on the Face Recognition Memory Task (p= .30). See Figure 5.
Working Memory vs. Recognition Memory
When defining impairment based on the sibling group (worse than 1 SD from their mean), rate of impairment among the BT group on the SOS-V Task (30%) was not greater than on the Verbal Recognition Memory Task (35%; p= .79). Likewise, rate of impairment among the BT group on the SOS-O Task (34%) was not greater than on the Face Recognition Memory Task (26%; p= .50). In contrast, when defining impairment based on the ST group (worse than 1 SD from their mean), rate of impairment among the BT group on the SOS-V Task (37%) was significantly greater than on the Verbal Recognition Memory Task (15%; p< .05). Likewise, rate of impairment among the BT group on the SOS-O Task (44%) was greater than on the Face Recognition Memory Task (14%, p< .01). Linear mixed models investigating group differences on the SOS-V and SOS-O tasks after adjusting for Verbal and Face Recognition Memory, respectively, produced convergent findings. On the SOS-V task, the BT group performed significantly worse than the ST group (p<.05) with a trend for worse performance compared to the sibling group (p= .07); whereas, on the SOS-O task the BT group performed significantly worse than both the ST and sibling groups (p< .05).
Clinical Predictors
Exploratory univariate linear models were used to investigate demographic and clinical variables predictive of WM performance for the BT group on the Digit Span Task, SOS-V or SOS-O. Mean error scores for SOS-V and SOS-O continued to be used for data reduction purposes. No significant predictive relationships were identified.
Discussion
Study findings were largely consistent with a priori hypotheses. Children treated for a BT with CRT performed significantly worse than control participants on traditional, clinical measures of WM as well as experimental, computerized measures of WM. Further, deficits were identified in comparison to healthy siblings and children who had been treated for a ST without CNS-directed therapy. Performance on WM tasks tended to correlate with estimated IQ suggesting WM problems may be a contributor to previously established declines in intellectual functioning. Given a significant proportion of age-related improvements in IQ can be accounted for by developmental improvements in WM, WM deficits may help explain the time-since-treatment effect whereby the gap in intellectual abilities between BT survivors and peers widens over time. There was mixed support for the hypothesis that BT survivors would demonstrate a significantly higher rate of WM impairment relative to recognition memory impairment, with more support for this prediction when using ST survivors as the control group.
Impaired WM performance on experimental measures, for which performance has been shown to be mediated by activation of the prefrontal cortex (Curtis et al., 2000), adds behavioral support to the theory frontal brain regions are particularly vulnerable to radiation effects. Diffusion tensor imaging (DTI) has been used to investigate regional white matter vulnerabilities following CNS-directed therapy for childhood BTs. For example, Qiu and colleagues (2007) used DTI to study survivors of medulloblastoma treated with whole-brain irradiation. Fractional anisotropy (FA- a DTI-derived index of white matter integrity) of the frontal and parietal lobes was found to be significantly less compared with controls. Moreover, the frontal lobe was found to have a significantly larger difference in FA compared with the parietal lobe despite the same irradiation dose, suggestive of radiosensitivity of frontal brain regions. In another study comparing children treated for medulloblastoma with surgery, adjuvant chemotherapy and radiation therapy to children treated with surgery only for pilocytic astrocytoma, both groups were found to have significantly decreased FA values in cerebellar midline structures and the frontal lobes; however, the amount of decreased FA was greater in medulloblastoma survivors thus revealing distal effects of local cerebellar treatment and additional neurotoxic effects of adjuvant treatment (Rueckriegel et al., 2010). Distal treatment effects are consistent with reports that WM performance is related to integrity of cerebello-thalamo-cerebral connections such that break down in myelin anywhere in this pathway may interrupt communication with the frontal lobes (Law et al., 2011). Taken together these findings offer important clues to neuropathology underlying cognitive late effects that may inform further refinement of cancer-directed therapy.
CRT represents a significant advancement in the treatment of childhood BTs. Recent research suggests CRT results in a high rate of disease control and better preservation of intellectual functioning (Merchant et al., 2009), academic skills (Conklin et al., 2008), and learning and memory (Di Pinto et al., 2010). However, current findings suggest risk remains for cognitive disruption that can be detected by WM measures. It is important that researchers and clinicians look beyond global measures to prevent missing specific difficulties that may impact functional outcomes. Measures with greater sensitivity to specific cognitive processes also provide better insight to underlying neurological processes and suggest areas for targeted intervention. It is likely we are approaching a plateau in the ability to limit neurotoxicity associated with cancer-directed therapies while maintaining high survival rates, highlighting the need to develop focused interventions that mitigate potentially unavoidable cognitive late effects.
The same pattern of difficulties in BT survivors was generally revealed irrespective of control group with BT survivors performing significantly worse than healthy and cancer control groups on IQ, Digit Span Backward, both experimental WM tasks, and the verbal forced-choice recognition task. These findings indicate WM problems are not simply the result of the childhood cancer experience but rather appear specific to CNS disease and/or treatment. There has been recent interest in studying cognitive late effects of systemic chemotherapy used to treat STs (Minisini et al., 2004; Vardy & Tannock, 2007); the current findings are not suggestive of risk. The current study also highlights the importance of including control groups in study design. By relying on published normative data, group differences in IQ and Digit Span would have been missed as control groups performed better than published norms. This phenomenon has previously been reported in the leukemia literature (e.g., Janzen & Speigler, 2008).
While WM problems were prevalent among BT survivors, and independent of type of measure or control group, evidence for WM as a specific deficit for BT survivors was mixed. BT survivors performed similarly to both control groups on the nonverbal forced-choice recognition task, suggesting an area of spared functioning. However there were group differences on the verbal forced-choice recognition task, and there were no greater differences in WM performance relative to recognition memory performance when using sibling controls. This finding might indicate performance on the verbal forced-choice recognition task is mediated by frontal brain areas in addition to medial temporal lobe areas (Ranganath, Johnson & D'Esposito, 2003) and/or reflect additional adverse impact of radiation on the hippocampus (Nageswara Rao et al., 2011).
Potential study limitations exist. First, the experimental SOS tasks used in this study demonstrated good psychometric properties with respect to revealing group differences and parametric varying of task difficulty. However, in contrast to earlier studies with typically developing children (Conklin et al., 2007; Luciana et al., 2005), improvement in WM performance with age was not identified. Age effects may have been obscured by treatment effects in the patient groups. Retrospective exploratory analyses revealed some age related trends among the sibling group that may have been difficult to detect with a small sample size. Second, it came to the study team's attention that some participants misunderstood directions for the forced-choice recognition tasks. While this issue was addressed retrospectively through data analysis, in future studies it will be important to confirm participant understanding of directions in real-time. Third, a stricter comparison of nonverbal working memory to recognition memory would include tasks using the same stimuli type (i.e., objects or faces). Fourth, performance on the Verbal Recognition Memory Task may have been associated with ceiling effects thus limiting comparison of verbal recognition and working memory tasks; however, this was in part countered by using the same reference group (healthy siblings or ST patients) for task comparisons and was not a factor in comparing nonverbal recognition and working memory tasks. Finally, cross-sectional study design precluded investigation of development of deficits over time. Prospective, longitudinal studies would afford the opportunity to investigate whether WM deficits precede the emergence of IQ deficits thus revealing a window of time during which intervention might prevent a sequential cascade of cognitive issues.
Current findings have clinical implications and suggest areas for investigation. Identification of WM as an area of difficulty following CRT for childhood BTs suggests vulnerability of underlying neural substrates, most notably a well-established frontal-parietal network including the dorsolateral prefrontal cortex. In evaluating newer treatment approaches such as proton beam therapy, which promise greater sparing of healthy brain tissue, it will be important to assess WM as a test of decreased neurotoxicity. Future work more directly assessing brain function with functional neuroimaging (fMRI and DTI) is needed to examine when and how development of anterior brain regions deviates from the typical trajectory, which may inform treatment delivery. Current findings also suggest WM may be a core deficit underlying the well documented decline in IQ. This finding may direct closer monitoring to facilitate earlier intervention. There is emerging support for interventions that mitigate the impact of cognitive late effects among cancer survivors (Butler et al., 2008; Conklin et al., 2010). Current findings identify WM as a target for future intervention trials.
Acknowledgments
This work was supported in part by the National Cancer Institute (St. Jude Cancer Center Support [CORE] Grant number P30 CA21765), (H.C., grant number R21 CA131616); the International Neuropsychological Society (H.C., Rita Rudel Award); and the American Lebanese Syrian Associated Charities (ALSAC). The authors wish to thank Clay Curtis, Catalina Hooper and Matt Scoggins for their contributions to developing the experimental computerized tasks. We also thank the patients and their families who volunteered their time to participate.
Footnotes
Portions of this paper were presented at the annual meetings of the International Neuropsychological Society in Acapulco, Mexico, February 3–6, 2010 and the International Society for Pediatric Oncology in Boston, Massachusetts, October 21–25, 2010.
The authors have no conflicts of interest to disclose.
References
- Ayr LK, Yeates K, Enrile BG. Arithmetic skills and their cognitive correlates in children with acquired and congenital brain disorder. Journal of the International Neuropsychology Society. 2005;11:249–262. doi: 10.1017/S1355617705050307. doi: 10.1017/S1355617705050307. [DOI] [PubMed] [Google Scholar]
- Baddeley A. Working memory. Comptes Rendus de l Academie des Sciences- Series iii, Sciences d la Vie. 1998;321:167–173. doi: 10.1016/s0764-4469(97)89817-4. doi:10.1016/S0764-4469(97)89817-4. [DOI] [PubMed] [Google Scholar]
- Barratt WR. The Barratt Simplified Measure of Social Status. 2006 Retrieved from the Indiana State University website: http://wbarratt.indstate.edu/socialclass/Barratt_Simplifed_Measure_of_Social_Status.pdf.
- Butler RW, Copeland DR, Fairclough DL, Mulhern RK, Katz ER, Kazak AE, Sahler OJ. A multi-center, randomized clinical trial of a cognitive remediation program for childhood survivors of a pediatric malignancy. Journal of Consulting and Clinical Psychology. 2008;76:367–378. doi: 10.1037/0022-006X.76.3.367. doi:10.1037/0022-006X.76.3.367. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Conklin HM, Curtis CE, Calkins ME, Iacono WG. Working memory impairment in schizophrenia patients and their first-degree relatives: Cognitive functioning shedding light on etiology. Neuropsychologia. 2005;43:930–942. doi: 10.1016/j.neuropsychologia.2004.09.013. doi:10.1016/j.neuropsychologia.2004.09.013. [DOI] [PubMed] [Google Scholar]
- Conklin HM, Li C, Xiong X, Ogg RJ, Merchant TE. Predicting change in academic abilities after conformal radiation therapy for localized ependymoma. Journal of Clinical Oncology. 2008;26:3965–3970. doi: 10.1200/JCO.2007.15.9970. doi: 10.1200/JCO.2007.15.9970. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Conklin HM, Luciana M, Hooper CJ, Yarger RS. Working memory performance in typically developing children and adolescents: behavioral evidence of protracted frontal lobe development. Developmental Neuropsychology. 2007;31:103–128. doi: 10.1207/s15326942dn3101_6. doi:10.1207/s15326942dn3101_6. [DOI] [PubMed] [Google Scholar]
- Conklin HM, Reddick WE, Ashford J, Ogg S, Howard SC, Morris EB, Khan RB. Long-term efficacy of methylphenidate in enhancing attention regulation, social skills, and academic abilities of childhood cancer survivors. Journal of Clinical Oncology. 2010;28:4465–4472. doi: 10.1200/JCO.2010.28.4026. doi:10.1200/JCO.2010.28.4026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Curtis CE, Zald DH, Pardo JV. Organization of working memory within the human prefrontal cortex: a PET study of self-ordered object working memory. Neuropsychologia. 2000;38:1503–1510. doi: 10.1016/s0028-3932(00)00062-2. doi:10.1016/S0028-3932(00)00062-2. [DOI] [PubMed] [Google Scholar]
- De Jonge P, de Jong PF. Working memory, intelligence and reading ability in children. Personality and Individual Differences. 1996;21:1007–1020. doi:10.1016/S0191-8869(96)00161-4. [Google Scholar]
- Dennis M, Speigler BJ, Obonsawin MC, Maria BL, Cowell C, Hoffman HJ. Brain tumors in children and adolescents- III. Effects of radiation and hormone status on intelligence and on working, associative and serial order memory. Neuropsychology. 1992;30:257–275. doi: 10.1016/0028-3932(92)90004-6. doi:10.1016/0028-3932(92)90004-6. [DOI] [PubMed] [Google Scholar]
- Dennis M, Hetherington CR, Spiegler BJ. Memory and attention after childhood brain tumors. Medical and Pediatric Oncology Supplement. 1998;1 doi: 10.1002/(sici)1096-911x(1998)30:1+<25::aid-mpo4>3.0.co;2-a. doi:10.1002/(SICI)1096-911X(1998)30:1+<25‷AID-MPO4>3.3.CO;2-1. [DOI] [PubMed] [Google Scholar]
- Di Pinto M, Conklin HM, Li C, Xiong X, Merchant TE. Investigating verbal and visual auditory learning after conformal radiation therapy for childhood ependymoma. International Journal of Radiation, Oncology, Biology, Physics. 2010;77:1002–1008. doi: 10.1016/j.ijrobp.2009.06.003. doi:10.1016/j.ijrobp.2009.06.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Filley CM, Kleinschmit-DeMasters BK. Toxic leukoencephalopathy. New England Journal of Medicine. 2001;345:425–432. doi: 10.1056/NEJM200108093450606. doi:10.1056/NEJM200108093450606. [DOI] [PubMed] [Google Scholar]
- Fry AS, Hale S. Processing speed, working memory, and fluid intelligence: evidence for a developmental cascade. Psychological Science. 1996;4:237–241. doi:10.1111/j.1467-9280.1996.tb00366.x. [Google Scholar]
- Giedd JN. Structural magnetic resonance imaging of the adolescent brain. Annals of the New York Academy of Sciences. 2004;1021:105–109. doi: 10.1196/annals.1308.009. doi:10.1196/annals.1308.009. [DOI] [PubMed] [Google Scholar]
- Gilhooly KJ, Logie RH. Age of acquisition, imagery, concreteness, familiarity and ambiguity measures for 1944 words. Behavioral Research Methods and Instrumentation. 1980;12:395–427. doi:10.3758/BF03201693. [Google Scholar]
- Goldman-Rakic PS. Architecture of the prefrontal cortex and the central executive. Annals of the New York Academy of Sciences. 1995;769:71–83. doi: 10.1111/j.1749-6632.1995.tb38132.x. doi:10.1111/j.1749-6632.1995.tb38132.x. [DOI] [PubMed] [Google Scholar]
- Grill J, Renaux BK, Bultau C, Viguier D, Levy-Piebois C, Sainte-Rose C, Kalifa C. Long-term intellectual outcome in children with posterior fossa tumors according to radiation doses and volumes. International Journal of Radiation Oncology, Biology, and Physics. 1999;45:137–145. doi: 10.1016/s0360-3016(99)00177-7. doi:10.1016/S0360-3016(99)00177-7. [DOI] [PubMed] [Google Scholar]
- Hanten G, Levin HS, Song JX. Working memory and metacognition in sentence comprehension by severely head injured children: A preliminary study with implications for rehabilitation. Developmental Neuropsychology. 1999;16:393–414. doi:10.1207/S15326942DN1603_23. [Google Scholar]
- Janzen LA, Spiegler BJ. Neurodevelopmental sequelae of pediatric acute lymphoblastic leukemia and its treatment. Developmental Disabilities Research Review. 2008;14:185–195. doi: 10.1002/ddrr.24. doi:10.1002/ddrr.24. [DOI] [PubMed] [Google Scholar]
- Kirschen MP, Davis-Ratner MS, Milner MW, Chen S, H., Schraedley-Desmond P, Fisher PG, Desmond JE. Verbal memory impairments in children after cerebellar tumor resection. Behavioral Neurology. 2008;20:39–53. doi: 10.3233/BEN-2008-0216. doi: 10.3233/BEN-2008-0216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Law N, Bouffet E, Laughlin S, Laperrier N, Briere M, Strother D, Mabbott D. Cerebello-thalamo-cerebral connections in pediatric brain tumor patients: Impact on working memory. NeuroImage. 2011;56:2238–2248. doi: 10.1016/j.neuroimage.2011.03.065. doi:10.1016/j.neuroimage.2011.03.065. [DOI] [PubMed] [Google Scholar]
- Luciana M, Conklin HM, Hooper CJ, Yarger RS. The development of nonverbal working memory and executive control processes in adolescents. Child Development. 2005;76:697–712. doi: 10.1111/j.1467-8624.2005.00872.x. doi:10.1111/j.1467-8624.2005.00872.x. [DOI] [PubMed] [Google Scholar]
- Mabbott DJ, Penkman L, Witol A, Strother D, Bouffet E. Core neurocognitive functions in children treated for posterior fossa tumors. Neuropsychology. 2008;22:159–168. doi: 10.1037/0894-4105.22.2.159. doi:10.1037/0894-4105.22.2.159. [DOI] [PubMed] [Google Scholar]
- Merchant TE. Current management of childhood ependymoma. Oncology. 2002;16:629–644. Retrieved from http://www.rosalina.info/links/Ependymoma.pdf. [PubMed] [Google Scholar]
- Merchant TE, Conklin HM, Wu S, Lustig RH, Xiong X. Late effects of conformal radiation therapy for pediatric patients with low-grade glioma: prospective evaluation of cognitive, endocrine, and hearing deficits. Journal of Clinical Oncology. 2009;27:3691–3697. doi: 10.1200/JCO.2008.21.2738. doi:10.1200/JCO.2008.21.2738. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Merchant TE, Mulhern RK, Krasin MJ, Kun LE, Williams T, Li C, Sandford RA. Preliminary results from a phase II trial of conformal radiation therapy and evaluations of radiation-related CNS effects for pediatric patients with localized ependymoma. Journal of Clinical Oncology. 2004;22:3156–3162. doi: 10.1200/JCO.2004.11.142. doi:10.1200/JCO.2004.11.142. [DOI] [PubMed] [Google Scholar]
- Minisini A, Atalay G, Bottomley A, Puglisi F, Piccart M, Biganzoli L. What is the effect of systemic anticancer treatment on cognitive function? Lancet Oncology. 2004;5:774–779. doi: 10.1016/S1470-2045(04)01465-2. doi:10.1016/S1470-2045(04)01465-2. [DOI] [PubMed] [Google Scholar]
- Mulhern RK, Butler RW. Neurocognitive sequelae of childhood cancers and their treatment. Pediatric Rehabilitation. 2004;7:1–14. doi: 10.1080/13638490310001655528. doi:10.1080/13638490310001655528. [DOI] [PubMed] [Google Scholar]
- Mulhern RK, Merchant TE, Gajjar A, Reddick WE, Kun LE. Late neurocognitive sequelae in survivors of brain tumours in childhood. The Lancet. 2004;5:399–408. doi: 10.1016/S1470-2045(04)01507-4. doi:10.1016/S1470-2045(04)01507-4. [DOI] [PubMed] [Google Scholar]
- Mulhern RK, Reddick WE, Palmer SL, Glass JO, Elkin TD, Kun LE, Gajjar A. Neurocognitive deficits in medulloblastoma survivors and white matter loss. Annuals of Neurology. 1999;46:834–841. doi: 10.1002/1531-8249(199912)46:6<834::aid-ana5>3.0.co;2-m. doi:10.1002/1531-8249(199912)46:6<834∷AID-ANA5>3.0.CO;2-M. [DOI] [PubMed] [Google Scholar]
- Nageswara Rao AA, Ye H, Decker PA, Howe CL, Wetmore C. Therapeutic doses of cranial irradiation induce hippocampus-dependent cognitive deficits in young mice. Journal of Neurooncology. 2011;105:191–198. doi: 10.1007/s11060-011-0582-9. doi:10.1007/s11060-011-0582-9. [DOI] [PubMed] [Google Scholar]
- Nelson CA. The ontogeny of human memory: a cognitive neuroscience perspective. Developmental Psychology. 1995;31:723–738. doi:10.1037//0012-1649.31.5.723. [Google Scholar]
- Olsen RK, Nichols EA, Chen J, Hunt JF, Glover GH, Gabrieli JDE, Wagner AD. Performance-related sustained and anticipatory activity in human medial temporal lobe during delayed match-to-sample. The Journal of Neuroscience. 2009;29:11880–11890. doi: 10.1523/JNEUROSCI.2245-09.2009. doi:10.1523/JNEUROSCI.2245-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Palmer SL, Goloubeva O, Reddick WE, Glass JO, Gajjar A, Kun L, Mulhern RK. Patterns of intellectual development among survivors of pediatric medulloblastoma: a longitudinal analysis. Journal of Clinical Oncology. 2001;19:2302–2308. doi: 10.1200/JCO.2001.19.8.2302. Retrieved from http://jco.ascopubs.org/content/19/8/2302.full. [DOI] [PubMed] [Google Scholar]
- Petrides M. Functional organization of the human frontal cortex for mnemonic processing. Evidence from neuroimaging studies. Annals of the New York Academy of Sciences. 1995;769:85–96. doi: 10.1111/j.1749-6632.1995.tb38133.x. doi:10.1111/j.1749-6632.1995.tb38133.x. [DOI] [PubMed] [Google Scholar]
- Petrides M, Milner B. Deficits on subject-ordered tasks after frontal- and temporal-lobe lesions in man. Neuropsychologia. 1982;20:249–262. doi: 10.1016/0028-3932(82)90100-2. doi:10.1016/0028-3932(82)90100-2. [DOI] [PubMed] [Google Scholar]
- Qiu D, Kwong DL, Chan GC, Leung LH, Khong PL. Diffusion tensor magnetic resonance imaging finding of discrepant fractional anisotropy between the frontal and parietal lobes after whole-brain irradiation in childhood medulloblastoma survivors: reflection of regional white matter radiosensitivity? International Journal of Radiation Oncology, Biology, Physics. 2007;69:846–851. doi: 10.1016/j.ijrobp.2007.04.041. doi:10.1016/j.ijrobp.2007.04.041. [DOI] [PubMed] [Google Scholar]
- Ranganath C, Johnson MK, D'Esposito M. Prefrontal activity associated with working memory and episodic long-term memory. Neuropsychologia. 2003;41:378–389. doi: 10.1016/s0028-3932(02)00169-0. doi:10.1016/S0028-3932(02)00169-0. [DOI] [PubMed] [Google Scholar]
- Reddick WE, Russell JM, Glass JO, Xiong X, Mulhern RK, Langston JW, Gajjar A. Subtle white matter volume differences in children treated for medulloblastoma with conventional or reduced dose craniospinal irradiation. Magnetic Resonance Imaging. 2000;18:787–793. doi: 10.1016/s0730-725x(00)00182-x. doi: 10.1016/S0730-725X(00)00182-X. [DOI] [PubMed] [Google Scholar]
- Reeves CB, Palmer SL, Reddick WE, Merchant TE, Buchanan GM, Gajjar A, Mulhern RK. Attention and memory function among pediatric patients with medulloblastoma. Journal of Pediatric Psychology. 2006;31:272–280. doi: 10.1093/jpepsy/jsj019. doi:10.1093/jpepsy/jsj019. [DOI] [PubMed] [Google Scholar]
- Rueckriegel SM, Driever PH, Blankenburg F, Ludemann L, Henze G, Bruhn H. Differences in supratentorial damage of white matter in pediatric survivors of posterior fossa tumors with and without adjuvant treatment as detected by magnetic resonance diffusion tensor imaging. International Journal of Radiation Oncology, Biology, Physics. 2010;76:859–866. doi: 10.1016/j.ijrobp.2009.02.054. doi:10.1016/j.ijrobp.2009.02.054. [DOI] [PubMed] [Google Scholar]
- Schatz J, Kramer JH, Ablin A, Matthay KK. Processing speed, working memory, and IQ: a developmental model of cognitive deficits following cranial radiation therapy. Neuropsychology. 2000;14:189–200. doi: 10.1037//0894-4105.14.2.189. doi:10.1037//0894-4105.14.2.189. [DOI] [PubMed] [Google Scholar]
- Schon K, Hasselmo ME, Lopresti ML, Tricarico MD, Stern CE. Persistence of parahippocampal representation in the absence of stimulus input enhances long-term encoding: a functional magnetic resonance imaging study of subsequent memory after a delayed match-to-sample task. Journal of Neuroscience. 2004;24:11088–11097. doi: 10.1523/JNEUROSCI.3807-04.2004. doi:10.1523/JNEUROSCI.3807-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith EE, Jonides J. Neuroimaging analyses of human working memory. Proceedings of the National Academy of Science of the USA. 1998;95:12061–12068. doi: 10.1073/pnas.95.20.12061. doi:10.1073/pnas.95.20.12061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sowell ER, Thompson PM, Tessner KD, Toga AW. Mapping continued brain growth and gray matter density reduction in dorsal frontal cortex: Inverse relationships during postadolescent brain maturation. The Journal of Neuroscience. 2001;21:8819–8829. doi: 10.1523/JNEUROSCI.21-22-08819.2001. Retrieved from http://www.jneurosci.org/content/21/22/8819.full. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Swanson HL. What develops in working memory? A life span perspective. Developmental Psychology. 1999;35:986–1000. doi: 10.1037//0012-1649.35.4.986. doi:10.1037//0012-1649.35.4.986. [DOI] [PubMed] [Google Scholar]
- Vardy J, Tannock I. Cognitive function after chemotherapy in adults with solid tumours. Critical Review Oncology Hematology. 2007;63:183–202. doi: 10.1016/j.critrevonc.2007.06.001. doi:10.1016/j.critrevonc.2007.06.001. [DOI] [PubMed] [Google Scholar]
- Waber DP, Pomeroy SL, Chiverton AM, Kieran MW, Scott RM, Goumnerova LC, Rivkin MJ. Everyday cognitive function after craniopharyngioma in childhood. Pediatric Neurology. 2006;34:13–19. doi: 10.1016/j.pediatrneurol.2005.06.002. doi:10.1016/j.pediatrneurol.2005.06.002. [DOI] [PubMed] [Google Scholar]
- Wechsler D. Wechsler Abbreviated Scale of Intelligence. Harcourt Assessment; San Antonio, Texas: 1999. [Google Scholar]
- Wechsler D. Wechsler Adult Intelligence Scale - Third Edition. Psychological Corporation; San Antonio, TX: 1999. [Google Scholar]
- Wechsler D. Wechsler Intelligence Scale for Children- Fourth Edition, Integrated. Psychological Corporation; San Antonio, TX: 2003. [Google Scholar]
- Wilkinson GS. Wide Range Achievement Test- Third Edition. Wide Range Inc; Wilmington, Delaware: 1994. [Google Scholar]