Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Apr 1.
Published in final edited form as: J Int Neuropsychol Soc. 2013 Jan 28;19(4):380–389. doi: 10.1017/S1355617712001567

The Utility of Parent Report in the Assessment of Working Memory among Childhood Brain Tumor Survivors

Robyn A Howarth 1, Jason M Ashford 2, Thomas E Merchant 3, Robert J Ogg 4, Victor Santana 5, Shengjie Wu 6, Xiaoping Xiong 6, Heather M Conklin 2,*
PMCID: PMC3630283  NIHMSID: NIHMS426379  PMID: 23351399

Abstract

Childhood brain tumor survivors are at increased risk for neurocognitive impairments, including working memory (WM) problems. WM is typically assessed using performance measures. Little is known about the value of parent ratings for identifying WM difficulties, the relationship between rater and performance measures, or predictors of parent-reported WM problems in this population. Accordingly, the current study examined the utility of parent report in detecting WM difficulties among childhood brain tumor survivors treated with conformal radiation therapy (n=50) relative to siblings (n=40) and solid tumor survivors not receiving CNS-directed therapy (n=40). Parents completed the Behavior Rating Inventory of Executive Function (BRIEF). Participants were administered WM measures (digit span, self-ordered search tasks). Findings revealed parents rated brain tumor survivors as having significantly more WM problems (p<.01) compared to controls. However, the BRIEF-WM scale demonstrated poor sensitivity and specificity for detecting performance-based problems. Significant, albeit modest, correlations were found between the BRIEF-WM scale and performance measures (r=−.24 −.22; p<.05) for the combined group. Age at testing, socioeconomic status, and IQ were significant predictors of parent reported WM problems. Rater and performance measures offer complimentary yet different information in assessing WM, which reiterates the importance of utilizing both within the context of clinical assessment.

Keywords: cancer, cognitive late effects, pediatrics, BRIEF, rater-based measures, executive functions

Survivors of childhood cancer who received treatment directed at the central nervous system (CNS) are at an increased risk for experiencing cognitive deficits secondary to their disease and treatment (e.g., Mulhern & Butler, 2004). The emergence of cognitive impairments is associated with a slower rate of acquiring new skills, academic performance problems, and reduced quality of life (e.g., Mitby et al., 2003; Mostow, Byrne, Connelly, & Mulivhill, 1991). With increased survival rates among children diagnosed with cancer, efforts are now focused on delineating the nature of cognitive deficits, developing interventions to mitigate their impact, and, ultimately, optimizing quality of survival.

Historically, much of the survivorship literature focused on global cognitive outcomes, with a continued need to better understand specific cognitive changes associated with the diagnosis and treatment of childhood cancer. Greater specification allows researchers to identify neural systems most susceptible to disruption following treatment, clarify mechanisms of cognitive impairment, and develop appropriate interventions. More recently, findings suggest core deficits in sustained attention, processing efficiency, and working memory (WM) may be associated with CNS-directed therapies, thus contributing to global declines (Dennis et al., 1992; Reeves et al., 2006; Waber et al., 2006). CNS-directed therapies, such as radiation therapy and high-dose chemotherapy, are also well-established causes of changes to cerebral white matter and researchers have demonstrated poor cognitive outcomes are associated with reductions in white matter volume (e.g., Reddick et al., 2003).

WM represents a core executive function responsible for facilitating “online” storage and manipulation of information used to guide thinking and behavior (Baddeley, 1986). WM subserves higher-order cognitive processes including language comprehension, problem-solving, and academic performance (Hanten, Levin & Song, 1999; Moran & Gillon, 2005; Smith, Marshuetz, & Geva, 2002). WM is likely supported by a distributed neural network, although previous neuroimaging studies have demonstrated frontal-parietal brain areas, including the prefrontal cortex, are primary regions of activation (e.g., D’Esposito, 2007; Smith & Jonides, 1999). The prefrontal cortex has been shown to be protracted in neurodevelopment (Giedd, 2004; Sowell, Thompson, Tessner, & Toga, 2001), with evidence that it is one of the last brain areas to myelinate. Given the role of the prefrontal cortex in WM performance, coupled with white matter susceptibility to neurotoxicity associated with CNS-directed therapy, WM abilities may be at particular risk for disruption. Prior studies have demonstrated impairments on performance measures of WM among childhood cancer survivors (e.g., Conklin et al., 2012; Dennis et al., 1992; Reimers et al., 2007; Robinson et al., 2010) and some researchers have investigated attention ratings (e.g., Kahalley et al., 2011; Mabbott et al., 2005; Mulhern et al., 2004) such as the Conners’ Rating Scales (CRS; Conners, 1989) and Child Behavior Checklist (CBCL; Achenbach, 1991), which are highly correlated with the Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000). However, there are no known prior investigations using the BRIEF with this population.

WM is typically assessed using performance-based, clinical measures administered in a one-on-one setting. Such tasks require children to actively hold and manipulate information in mind (e.g., span, serial addition, self-ordered search tasks). Digit span backward is a frequently used clinical measure for assessing WM and a common threshold for identifying relative deficits is performance at least one standard deviation (1SD) below the mean. Childhood cancer survivors often participate in serial neurocognitive assessments, allowing clinicians to monitor the emergence of cognitive late effects and make appropriate recommendations for intervention. Nevertheless, there are obstacles associated with conducting successive evaluations including time, cost, and accessibility of professional resources. In addition, some children may display better performance during evaluations relative to what is observed in real world settings (e.g., school) given the structured nature of the testing environment. As a result, rater-based questionnaires may be a useful tool to measure behavioral manifestation of WM difficulties in the child’s daily environment.

Rater measures may offer several advantages over performance-based instruments with pediatric oncology patients. Rating scales are readily available, can be mailed to families for convenience, and do not require a lot of time or training to complete or score. Questionnaires also lend themselves well to assessing efficacy associated with interventions, particularly when interested in effects across settings. The BRIEF (Gioia et al., 2000) is a commonly used rating scale designed to examine the real-world manifestation of executive functions in children, including WM. Executive functions represent a multi-dimensional construct that includes interrelated processes necessary for purposeful and goal-directed behaviors such as planning and organization, behavioral regulation, and self-monitoring. Given the overlap among executive processes secondary to involvement of the prefrontal cortex, terms are often used interchangeably. For the purpose of this paper, WM is specifically defined as an executive function responsible for the “online” storage and manipulation of information, a construct that has been well-defined and mapped out neuroanatomically using functional brain imaging studies (e.g., Petrides, 1995; Smith & Jonides, 1998).

The utility of rater-based measures in the assessment of WM among childhood cancer survivors is unclear. A retrospective cohort study of long-term adult survivors of childhood cancer was recently completed with participants from the Childhood Cancer Survivor Study (CCSS; Krull et al., 2008). A brief, self-report neurocognitive questionnaire (CCSS-NCQ) consisting of 25-items selected from an early experimental adult version of the BRIEF was developed to assess functions commonly impacted by cancer treatment. Authors concluded the questionnaire demonstrated good reliability, construct and discriminant validity (Krull et al., 2008). Of note, the CCSS-NCQ is a self-report measure completed by adults. However, self-report measures could be problematic for use with survivors of childhood cancer who are often young in age at the time of treatment, and present with a higher incidence of neurocognitive impairment compared to same-age healthy peers, which may impact insight into the extent of cognitive difficulties. Adult survivors of childhood cancer have also been shown to under-report difficulties (O’Leary, Diller, & Recklitis, 2007). Thus, it is important to obtain ratings from an adult informant who is familiar with the child (e.g., parent, teacher) in order to accurately assess behavioral manifestations of neurocognitive impairments in the daily environment.

Prior research has not specifically investigated the relationship between rater- and performance-based measures of WM in childhood cancer survivors. A majority of previous studies examining this association have focused on executive functions more broadly. The BRIEF has generally been used within the context of traumatic brain injury (TBI) in children, with largely inconsistent findings. Some studies found parent ratings are more sensitive than performance measures for identifying executive deficits (Vriezen & Pigott, 2002), while others revealed evidence of executive dysfunction on both types of measures among children with TBI (Conklin et al., 2008), frontal lobe pathology (e.g., early treated phenylketonuria, hydrocephalus, and frontal lesions [Anderson et al., 2002]) and neurofibromatosis type 1 (Payne et al., 2011). Mangeot and colleagues (2002) found children who sustained a TBI were rated as displaying significantly more executive dysfunction compared to controls. In addition, index and composite ratings on the BRIEF were consistently related to performance on a measure of WM (Consonant Trigrams; Paniak, Miller, Murphy, Andrews, & Flynn, 1997). Prior research has generally examined the Behavioral Regulation Index, Metacognition Index, and Global Executive Composite scores in relation to performance measures (Mangeot et al., 2002; Payne et al., 2011; Vriezen & Pigott, 2002), with fewer studies specifically investigating clinical scales, namely WM (Anderson et al., 2002; Conklin et al., 2008).

Since previous studies have not focused on survivors of pediatric cancer, little is known about the value of parent ratings for specifically identifying WM problems, the relationship between rater- and performance-based measures of WM, or the factors most predictive of difficulties among childhood cancer survivors. The primary aim of the current study was to examine the utility of the BRIEF in detecting WM problems among childhood cancer survivors. We hypothesized parents of brain tumor survivors would rate them as displaying higher levels of WM problems compared to parents of healthy sibling and solid tumor participants who did not receive CNS-directed therapy. The second aim was to investigate the inter-relationships among the BRIEF-WM scale and performance measures of WM with this population. Based on the childhood TBI literature, we predicted parent reports of WM functioning would be modestly but significantly associated with performance-based measures of WM (digit span, computerized self-ordered search tasks). Finally, predictors of parent-reported WM problems were explored.

Method

Participants

Participants were between 8 and 18 years of age at the time of evaluation and included: childhood brain tumor survivors treated with conformal radiation therapy (n=50), healthy siblings of brain tumor survivors (n=40), and solid tumor survivors who had not received CNS-directed therapy (n=40). All three groups were stratified by gender and age (8–12; 13–18), with the brain tumor group additionally stratified by tumor location (infratentorial; supratentorial). Healthy siblings were included since norms were not available for experimental measures used in this study and to increase the likelihood that groups would be balanced with regard to potentially contributory variables (e.g., SES). Solid tumor patients were added to control for aspects of the cancer experience (e.g., extended absence from school).

Eligible participants were approached and enrolled in the order of visits for routine medical appointments with an overall participation rate of 82%. Participants and non-participants did not differ with regard to gender, age at diagnosis, or age at assessment (for brain tumor patients and siblings). Solid tumor non-participants were significantly older than participants, with all those who declined in the older age range (13–18).

Brain tumor survivors were treated for a primary CNS tumor (ependymoma, low-grade glioma, or craniopharyngioma) on an institutional phase II trial of conformal radiation therapy (RT-1; NCT00187226). Treatment was initiated at least two years prior to study enrollment and patients were required to have no evidence of recurrent disease. Patients received conformal radiation therapy given in daily fractions of 1.8 Gy over the course of six to seven weeks to a totaling dose of 54–59.4 Gy. The target volume included the tumor or tumor bed surrounding a clinical target volume margin of 1 cm that did not extend beyond anatomic limits of tumor extension, and an additional geometric planning target volume margin of 0.3–0.5 cm to account for set-up uncertainty.

Sibling participants were healthy siblings of brain tumor patients treated at this institution. Of note, only 15 (of 40) were siblings of brain tumor patients included in the current study. The remaining 25 were siblings of brain tumor patients treated at this institution who did not participate.

Solid tumor patients were diagnosed at least two years prior to study enrollment (Ewing sarcoma, osteosarcoma, soft tissue/rhabdomyosarcoma, neuroblastoma, or Wilms’ tumor) and treatment did not include CNS-directed therapy (i.e., cranial radiation therapy, intrathecal chemotherapy, or high dose methotrexate). The most common tumor locations included the abdomen (e.g., kidney) or extremity (e.g., femur, forearm). All solid tumor participants underwent surgery and a majority received adjuvant therapy. Approximately half received radiation and the most common chemotherapy agents were cyclophosphamide, doxorubicin, vincristine, etoposide, and ifosfamide.

Individuals with global intellectual impairment were excluded from participation to increase the likelihood of completing cognitive tasks. Intellectual impairment was operationalized a priori as an intellectual quotient (IQ) less than 70 for brain tumor patients, per most recent RT-1 protocol testing. Comparison participants were initially excluded for a history of pull-out special education services since IQ scores were not available at the time of enrollment. Following enrollment, data was to be removed from analyses for comparison participants with an IQ less than 70; however, no participants were excluded for this reason. Additional exclusion criteria included a history of CNS injury or disease (predating cancer diagnosis in brain tumor patients), documented Attention Deficit/Hyperactivity Disorder (AD/HD; predating cancer diagnosis for brain tumor patients), treatment with psychotropic or stimulant medication within two weeks of study participation, or a major sensory and/or motor impairment that would compromise the validity of testing (e.g., blindness, hemiparesis, poorly controlled seizures). This study was approved by the Institutional Review Board and written informed consent was required prior to participation.

Procedure

Rater-Based Measure(s) of Working Memory

Parents completed the BRIEF (Gioia et al., 2000). The BRIEF consists of 86-items, from which eight clinical scales (including WM), two indices (i.e., Behavioral Regulation and Metacognitive), and an overall Global Executive Composite (GEC) are derived. BRIEF T-scores have a mean of 50 and SD of 10. Scores are standardized by age and gender with T-scores greater than 65 (1.5 SD above mean) reflective of clinically significant concerns. In the current study, scores for all clinical scales were examined, although WM was the primary scale of interest. Internal consistency for the BRIEF is high across clinical scales (.80–.98), including WM (.89–.92). Test-retest reliability is also high for the clinical scales combined (.81) and the WM scale individually (.82–.85) over a two week interval. Convergent validity has been demonstrated with measures of behavioral functioning (e.g., inattention, impulsivity) whereas BRIEF scales correlate less with measures of emotional functioning (Gioia et al., 2000). Further, clinical studies have differentiated children with various clinical diagnoses (e.g., AD/HD, TBI, documented brain lesions) from normative groups regarding characteristics of executive functioning (Gioia et al., 2000). Data from participants with elevated validity indices (e.g., Inconsistency or Negativity scales) were excluded due to potential parent bias.

Performance-Based Measures of Working Memory

Participants were administered Digit Span from the age appropriate Wechsler Scale (Wechsler Intelligence Scale for Children-Fourth Edition [WISC-IV; Wechsler, 2003] or Wechsler Adult Intelligence Scale-Third Edition [WAIS-III; Wechsler, 1999]). The Digit Span Task is comprised of Digit Span Forward, a measure of auditory attention and immediate recall, and Digit Span Backward, a measure of WM based on the need to mentally manipulate information before responding (Lezak, 1995). Z-scores were computed separately for longest forward and backward spans to combine scores across the WISC-IV and WAIS-III.

Two experimental, computerized WM measures were also administered (Self-Ordered Search-Verbal [SOS-V] and Self-Ordered Search-Object [SOS-O]), which have been described elsewhere by our group (Conklin et al., 2012). Briefly, participants were seated in front of a computer screen on which stimuli were presented, over four trials, in arrays of varying length. For the SOS-V, array lengths included 2 × 2 (four words), 3 × 2 (six words), 3 × 3 (nine words), and 4 × 3 (12 words). The SOS-O task is parallel in format to the verbal task except geometric objects were presented instead of words and the most recently shown object location was covered with a black square. Therefore, SOS-O array lengths included 2 × 2 (three objects), 3 × 2 (five objects), 3 × 3 (eight objects), and 4 × 3 (eleven objects). The participant was instructed to select each word or object only once, with the goal of completing each trial with as few responses as possible. Each trial ended when the participant had selected all the target stimuli or after 3N responses (N is the number of stimuli in a trial), whichever happened first. The error score (E) was the dependent variable of interest and reflected the number of erroneous attempts in the process of identifying unselected stimuli. Previous behavioral studies of typically developing children have incorporated these tasks and demonstrated improvements with age (Conklin et al., 2007; Luciana et al., 2005). Further, findings from a positron emission tomography (PET) study revealed a relationship between task performance and regional cerebral blood flow in the dorsolateral prefrontal cortex, with greater blood flow to that region associated with better performance (Curtis et al., 2000).

Measure of General Cognitive Ability

To obtain an estimate of general cognitive functioning, all participants were administered the two subtest version of the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999). This estimated IQ is highly correlated with the full scale (FSIQ) obtained from administration of the Wechsler scale (WISC-IV=.82, Wechsler, 2003; WAIS-III=.87, Wechsler, 1999).

Demographic Characteristics

Caregivers completed a structured questionnaire that included questions about general child characteristics and development (e.g., age, gender, ethnicity, past emotional or behavioral problems). Additional questions about family demographic characteristics (e.g., parental education, occupation, income) were also included to derive an index of socioeconomic status (SES). Specifically, the Barratt Simplified Measure of Social Status was used (Barratt, 2006), which takes into account parental education and occupation.

Identification of Clinical Variables

Based on the existing literature, clinical variables of interest were obtained from the medical record for patient groups. Based on neuroimaging scans conducted at the time of diagnosis, tumor location was categorized as infratentorial or supratentorial. A majority of brain tumor survivors underwent surgical resection prior to radiation and extent of resection was categorized as biopsy, subtotal resection, near total resection, or gross total resection. In addition, treatment with chemotherapy and the presence of hydrocephalus were categorized as yes or no. Finally, medical records were reviewed to identify those brain tumor patients who received placement of a ventriculo-peritoneal (VP) shunt for the management of hydrocephalus.

Data Analyses

Descriptive analyses of demographic and clinical variables were conducted to characterize the three participant groups. In order to establish group similarity, groups were statistically compared using analyses of variance (ANOVA) or chi-square. To address the primary aim of the study, group differences on the BRIEF were examined using ANCOVA, controlling for IQ, and appropriate post-hoc comparisons for significant findings. Next, participants were categorized as elevated or not elevated on the BRIEF-WM scale using a 1SD cut-off to capture those “at risk” for difficulty, and the rate of elevation was examined relative to comparison groups and normative data. Then, the sensitivity and specificity of the BRIEF-WM scale for detecting performance-based WM problems was investigated for the brain tumor group, again using a 1SD cut-off to determine usefulness as a screening tool.

For the second aim of the study, the relationship between the BRIEF-WM scale and performance measures of WM was examined using Spearman rank partial correlations, controlling for group membership. Finally, univariate linear models were used to explore demographic (e.g., gender, age, SES) and clinical (e.g., diagnosis, tumor location, hydrocephalus) predictors of parent-reported WM problems.

Results

Demographic and Clinical Characteristics

Descriptive analyses of demographic and clinical variables as well as group comparisons (Table 1) revealed the three groups were balanced with regard to demographic factors (e.g., gender, race, SES, age at participation). Participants were, on average, 13.11 years of age (SD=2.98) at the time of study participation. The solid tumor group was significantly younger at the time of diagnosis (p=.02), further from diagnosis (p<.01), and further from treatment completion (p<.01) compared to the brain tumor group. Significant group differences were also revealed on an abbreviated measure of IQ, with significantly lower performance for the brain tumor survivors than solid tumor (p<.01) and sibling groups (p<.01), who did not differ from one another.

Table 1.

Demographic Characteristics by Group

Brain Tumor n = 50 Sibling n = 40 Solid Tumor n = 40 pa
Gender (% male) 50 50 50 1.00
Race (% Caucasian) 92 95 95 0.81
SESb, c 37.61 ±12.20 42.63 ± 12.04 42.09 ± 13.32 0.11
Age at assessment (years) 13.18 ± 2.88 12.91 ± 2.62 13.21 ± 3.46 0.88
Age at diagnosis (years) 6.38 ± 3.43 NA 4.50 ± 4.19 0.02
Time since diagnosis (years) 6.80 ± 2.60 NA 8.71 ± 3.94 <.01
Time since treatment (years) 5.62 ± 2.24 NA 7.66 ±4.08 <.01
Treatment duration (years) 1.18 ±1.79 NA 1.05 ± 0.77 0.65
WASI IQ (standard score)d 98.20 ± 13.91 109.00 ± 12.78 107.88 ± 12.44 <.01
Open in a new tab
a

P-values are from ANOVAs for analyses with three groups and independent t-tests for analyses with two groups.

b

SES based on the Barratt Simplified Measure of Social Status. Scores range from 8 to 66 with higher scores indicative of higher SES.

c

All values are presented as mean ± standard deviation.

d

WASI=Wechsler Abbreviated Scale of Intelligence; standard scores have a mean of 100 and standard deviation of 15.

The brain tumor group was balanced with respect to diagnosis and tumor location (Table 2). On average, brain tumor patients were 6.38 years of age (SD=3.43) at the time of diagnosis and 6.80 (SD=2.60) years from diagnosis. Only six patients received chemotherapy prior to radiation and half the sample underwent at least a near total resection. For the solid tumor group, participants were not balanced with regard to diagnosis (p=.02) and a majority of patients were diagnosed with Ewing sarcoma (30%), neuroblastoma (30%), or Wilms’ tumor (25%). Solid tumor participants were approximately 4.50 years of age (SD=4.19) at the time of diagnosis and 8.71 years (SD=3.94) from diagnosis.

Table 2.

Clinical Characteristics of Brain Tumor survivors

N % p
Tumor Diagnosis
 Ependymoma 22 44 0.22
 Low Grade Glioma 12 24
 Craniopharyngioma 16 32
Tumor Location
 Infratentorial 22 44 0.40
 Supratentorial 28 56
Pre-CRT Chemotherapy
 No 44 88 <0.01
 Yes 6 12
Extent of Surgical Resectionb
 Biopsy/STR 25 50 1.00
 NTR/GTR 25 50
Hydrocephalus
 No 21 42 0.26
 Yes 29 58
CSF Shunting
 No 29 58 0.26
 Yes 21 42
Open in a new tab
a

P-value indicates whether group is equally distributed across sub-categories using Chi-square.

b

Biopsy (tumor sampling to determine pathology), STR=subtotal resection (incomplete tumor resection with gross residual disease present on post-operative neuroimaging), NTR=near total resection (incomplete tumor resection with minimal residual disease present on post-operative neuroimaging), GTR=gross total resection (resection of tumor without residual disease observed by the operating neurosurgeon and confirmed on post-operative neuroimaging)

Parent Report of Working Memory

To investigate whether parents of brain tumor survivors would endorse higher levels of WM difficulties compared to healthy sibling and solid tumor participants, group differences on the BRIEF clinical scales were examined using ANCOVA, accounting for IQ (Figure 1). Data from one brain tumor participant was excluded due to an elevated validity scale (Negativity). Results revealed significant group differences on the WM (p=.03) and planning/organization (p=.04) scales in addition to a trend for significance on the initiation (p=.07) and monitoring (p=.08) scales. Post-hoc pairwise comparisons indicated parents rated brain tumor survivors as significantly more elevated in these areas compared to solid tumor and healthy sibling groups, who did not differ from one another. There was a significant interaction for the organization of materials scale; there was a stronger relationship with IQ for the brain tumor group but no significant group differences.

Figure 1.

Figure 1

Open in a new tab

BRIEF Clinical Scales by Group. T-scores have a mean of 50 and standard deviation of 10. Higher scores are indicative of more difficulty. The BRIEF clinical scales include: Inhibit=Inhibition, Shift=shifting, Emot Cont=emotional control, Initiate, WM=working memory, Plan Org=planning/organization, Org Mat=organization of materials, and Monitor=monitoring.

Participants were then categorized as elevated or not elevated on the BRIEF-WM scale based on a 1SD cut-off above the mean. Brain tumor participants displayed a significantly higher percentage of elevation compared to control groups (see Table 3) as well as the normative rate of 16%. In contrast, the rate of elevation among the sibling and solid tumor groups did not significantly differ from the normative rate. Finally, sensitivity and specificity of the BRIEF-WM scale to detect WM problems identified by digit span backward was investigated for the brain tumor group. Using the 1SD cut-off, 26 of 49 brain tumor participants (53%) were correctly classified, resulting in a sensitivity of 0.40 and specificity of 0.56.

Table 3.

Rate of Elevation on BRIEF-WM Scale

1SD Cut-Off
n % pb
Brain Tumora 21 42.9 <.01
Sibling 4 10.0
Solid Tumor 6 15.0
Open in a new tab
a

One brain tumor participant was excluded from analyses due to an elevated Negativity Scale.

b

P-value from Fisher exact test.

Rater-versus Performance-Based Working Memory Measures

As previously reported by Conklin and colleagues (2012), brain tumor survivors performed significantly worse across performance-based WM measures compared to siblings and solid tumor survivors. In short, brain tumor survivors performed significantly worse on digit span backward than both solid tumor and healthy sibling groups (p<.01), who did not differ from one another (p=.61). For both the SOS-Verbal and SOS-Object tasks, linear mixed models revealed main effects for array size (with worsening performance as size increased) and group. For both tasks, post-hoc comparisons indicated brain tumor survivors performed significantly worse than both solid tumor (p<.01) and healthy sibling groups (p<.05).

To examine the relationship between the BRIEF-WM scale and each of the performance-based measures, Spearman rank partial correlation coefficients were calculated, controlling for group. Results revealed significant, yet modest, correlations between the BRIEF-WM scale and digit span forward (r=−.24, p<.01), digit span backward (r=−.21, p=.02), SOS-V (r=0.22, p=.02), and SOS-O (r=.22, p=.01), all in the expected direction.

Clinical Predictors

Finally, univariate linear models were used to explore demographic and clinical predictors of parent-reported WM concerns. For the combined group, age at time of testing (p=.02), SES (p<.01), and IQ (p<.01) were significant predictors of elevations on the WM scale whereas other demographic (e.g., gender, parent education) and clinical factors (e.g., diagnosis, treatment history) were not statistically significant. More specifically, older age at testing, lower SES, and lower IQ were predictive of more parent-reported concerns with WM. When looking individually at the brain tumor group, gender (p=.04), age at testing (p=.03), and extent of surgical resection (p=.02) were significant predictors of elevations on the WM scale. Specifically, greater WM concerns were reported for male brain tumor survivors who were older at the time of testing and had a less extensive resection (e.g., biopsy, subtotal resection). While specific brain tumor diagnosis was not a significant predictor, chi-square analyses revealed an association between diagnosis and extent of resection, with a majority of those receiving a less extensive resection diagnosed with either craniopharyngioma (48%) or low-grade glioma (44%).

Discussion

The current study sought to determine the utility of the BRIEF for detecting WM problems among survivors of pediatric brain tumors. Consistent with a priori hypotheses, results revealed that parents rated brain tumor survivors as displaying significantly greater difficulty with everyday WM abilities compared to solid tumor and healthy sibling participants. However, the BRIEF-WM scale demonstrated poor sensitivity and specificity for identifying WM problems among brain tumor survivors as identified by performance-based measures. Other areas of executive functioning were also rated as more problematic for children treated for a brain tumor relative to comparison groups, such as planning/organization. This pattern reiterates the multi-dimensional nature of executive functions and suggests the BRIEF scales measure overlapping constructs. While WM may be specifically vulnerable among executive functions, processes loading onto the metacognition index may also be more generally at risk among childhood brain tumor survivors. Significant, albeit modest, correlations were found between the BRIEF-WM scale and performance measures of WM. Parental endorsement of WM concerns significantly correlated with both traditional and experimental, computerized performance-based measures of WM. Further, exploratory analyses revealed age at testing, SES, and IQ were significant predictors of parent ratings of WM among participants, whereas other demographic and clinical factors were not. The notion that higher SES was predictive of fewer parent reported problems with WM could reflect greater cognitive reserve secondary to better access to resources and/or correlation with parental IQ. No significant relationships have been found when these same factors were examined as predictors of performance on WM measures (Conklin et al., 2012).

Among brain tumor survivors, more WM concerns were reported for males who were older at the time of testing and underwent a less extensive resection. These findings are not entirely consistent with the previous literature, which has identified risk factors for cognitive deficits to most reliably include female gender (Ris et al., 2001), younger age at treatment (Jannoun & Bloom, 1990), longer time since treatment (Ellenberg et al., 1987), higher treatment intensity (Hoppe-Hirsch et al., 1995), and complicating medical factors (Merchant et al., 2006). Although there is not a clear explanation for why males were reported as having more WM problems compared to females, recent studies of survivors of pediatric acute lymphoblastic leukemia (ALL; Jain et al., 2009) and brain tumors (Conklin et al., 2008) have demonstrated greater male vulnerability when examining specific cognitive skills, such as attention and language-based processes.

Current findings of greater WM concerns reported for participants who were older at the time of testing may be reflective of the “growing into deficits” phenomenon, which has been described in the TBI literature (Dennis, 1989; Giza & Prins, 2006). Specific difficulties may become more apparent as a child enters adolescence secondary to brain maturational changes, decreased environmental supports, and increased demands for independence associated with this stage of development.

With regard to resection, there could also be a relationship between tumor location and extent of resection performed. The possibility of achieving a gross or near total resection may be limited when a tumor is located near or around critical brain structures. For example, cerebellar tumors are more often associated with gross total resection compared to tumors located within deep, subcortical structures. It is plausible that children receiving a less invasive or extensive resection may have had residual and/or diffuse disease, which subsequently impacted neural pathways in the brain. Consistent with this hypothesis, a majority of those participants with a less extensive resection were diagnosed with either craniopharyngioma or low-grade glioma.

WM problems were not found in the solid tumor group, despite being younger at the time of diagnosis, and further out from diagnosis and treatment completion. Despite evidence of CNS disruption associated with some solid tumor diagnoses (e.g., Wilms’ tumor and chronic kidney disease), the higher rate of elevation reported among brain tumor survivors is more likely related to CNS disease and/or CNS-directed treatment than the general cancer experience. Consistent with current findings, recent studies examining the cognitive late effects associated with systemic chemotherapy have revealed little impact (Minisini et al., 2004; Vardy & Tannock, 2007). While the notion of “chemo-brain” has received a great deal of attention in the recent literature, particularly among adult cancer patients (e.g., Boykoff et al., 2009; Hede, 2008; Wefel et al., 2010), the current data are not suggestive of risk.

Consistent with the TBI literature, current findings revealed significant, yet modest, correlations between the BRIEF-WM scale and performance-based WM measures. The modest correlations, coupled with poor sensitivity and specificity of the BRIEF-WM scale relative to digit span backward, suggest the BRIEF is likely not a sufficient tool for fully capturing the nature of WM difficulties experienced by childhood cancer survivors. The limited correlation could also indicate rater measures are more valid for use with unselected, healthy populations. Rater and performance measures may offer complimentary yet distinctly different information in assessing cognitive late effects. As such, despite the number of advantages associated with using questionnaires, it does not seem appropriate to use them in isolation or instead of performance measures in the assessment of WM and other executive functions among childhood cancer survivors. Because rater measures offer information about a child’s performance in the natural setting, the BRIEF may be a simple and quick compliment to clinical assessment. Rather than relying on rater or performance measures of WM alone, it seems optimal to utilize both types of instruments.

Within the context of these findings, study limitations are acknowledged, which provide directions for future research. First, the current study employed a cross-sectional design. A longitudinal study would afford a more in-depth examination of parent reported WM problems over time. Second, examining a larger cohort may help to more comprehensively explore demographic and clinical predictors of WM difficulties. Next, while less than half of siblings were siblings of brain tumor patients included in this study, the potential for non-independence of observations is acknowledged. Finally, rating scales relied on one informant and a measure of negative affect was not used. Previous studies have shown that symptoms of negative affect may contribute to self-report ratings (e.g., Hermelink et al., 2010). As such, biases may exist when using rater measures with characteristics of the informant reflected in responses (e.g., depressed parents may over-report their child’s symptoms), regardless of whether ratings are obtained from a parent, teacher, or the individual. Thus, future research may cross-validate parent report by also administering teacher and/or self-report versions of the BRIEF, including measures of negative affect.

In conclusion, both rater and performance measures identified WM concerns among childhood cancer survivors, although the correlation between them was modest. Performance measures appear to be proximal measures that better tap into the classically defined core WM construct whereas the BRIEF may more accurately capture one’s ability to independently execute and apply these skills in the everyday environment; hence, it identifies more distal outcomes that, despite ecological validity, are subject to potential confounds and intervening influences. Accordingly, the BRIEF does not appear to be a good screening tool for assessing WM among brain tumor survivors in isolation or in lieu of performance measures. This pattern of findings highlights the multi-faceted nature of executive dysfunction experienced by childhood cancer survivors, while also offering some neurodevelopmental clues about patterns of vulnerability, specific areas to monitor, and appropriate targets for developing interventions.

Acknowledgments

The authors would like to acknowledge Clay Curtis, Catalina Hooper and Matt Scoggins for their contributions in developing the experimental computerized self-ordered search tasks. The authors thank the patients and their families who volunteered their time to participate in this study. This work was supported in part by the National Cancer Institute (St. Jude Cancer Center Support [CORE] Grant [P30 CA21765]; H.C., R21 CA131616); the International Neuropsychological Society (H.C., Rita G. Rudel Award); and the American Lebanese Syrian Associated Charities (ALSAC). Portions of this paper were presented at the annual meeting of the American Academy of Clinical Neuropsychology in Washington, D.C., June 8–11, 2011. The authors have no conflicts of interest to disclose.

References

  1. Achenbach TM. Manual for the Child Behavior Checklist/4–18 and 1991 Profile. Burlington, VT: University of Vermont, Department of Psychiatry; 1991. [Google Scholar]
  2. Anderson VA, Anderson P, Northam E, Jacobs R, Mikiewicz O. Relationships between cognitive and behavioral measures of executive function in children with brain disease. Child Neuropsychology. 2002;8(4):231–240. doi: 10.1076/chin.8.4.231.13509. [DOI] [PubMed] [Google Scholar]
  3. Baddeley A. Working memory. Oxford, England: Oxford University Press; 1986. [DOI] [Google Scholar]
  4. Barratt WR. The Barratt Simplified Measure of Social Status. 2006 Retrieved from the Indiana State University website: http://wbarratt.indstate.edu/socialclass/Barratt_Simplified_Measure_of_Social_Status.pdf.
  5. Boykoff N, Moieni M, Subramanian SK. Confronting chemobrain: An in-depth look at survivors’ reports of impact on work, social networks, and health care response. Journal of Cancer Survivors. 2009;3(4):223–232. doi: 10.1007/s11764-009-0098-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Conklin HM, Luciana M, Hooper CJ, Yarger RS. Working memory performance in typically developing children and adolescents: behavioral evidence of protracted frontal lobe development. Developmental Neuropsychology. 2007;31:103–128. doi: 10.1207/s15326942dn3101_6. [DOI] [PubMed] [Google Scholar]
  7. Conklin HM, Li C, Xiong X, Ogg RJ, Merchant TE. Predicting change in academic abilities after conformal radiation therapy for localized ependymoma. Journal of Clinical Oncology. 2008;26(24):3965–3970. doi: 10.1200/JCO.2007.15.9970. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Conklin HM, Salorio CF, Slomine BS. Working memory performance following pediatric traumatic brain injury. Brain Injury. 2008;22(11):847–857. doi: 10.1080/02699050802403565. [DOI] [PubMed] [Google Scholar]
  9. Conklin HC, Ashford JA, Howarth RA, Merchant TE, Ogg RJ, Santana V, Xiong X. Working memory performance among childhood brain tumor survivors. Journal of the International Neuropsychological Society. 2012 doi: 10.1017/S1355617712000793. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Conners CK. Conners’ Rating Scales Manual. North Tonawanda, NY: Multi-Health Systems; 1989. [Google Scholar]
  11. Curtis CE, Zald DH, Pardo JV. Organization of working memory within the human prefrontal cortex: a PET study of self-ordered object working memory. Neuropsychologia. 2000;38:1503–1510. doi: 10.1016/S0028-3932(00)00062-2. [DOI] [PubMed] [Google Scholar]
  12. D’Esposito M. From cognitive to neural models of working memory. Philosophic Transactions of Royal Society B. 2007;362:761–772. doi: 10.1098/rstb.2007.2086. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dennis M. Language and the young damaged brain. In: Boll TB, editor. Clinical neuropsychology and brain function: Research, measurement and practice. Washington, DC: American Psychological Association; 1989. [Google Scholar]
  14. Dennis M, Spiegler BJ, Obonsawin MC, Maria BL, Cowell C, Hoffman HJ. Brain tumors in children and adolescents-III. Effects of radiation and hormone status on intelligence and on working, associative and serial order memory. Neuropsychology. 1992;30:257–275. doi: 10.1016/0028-3932(92)90004-6. [DOI] [PubMed] [Google Scholar]
  15. Ellenberg L, McComb JG, Seigel SE, Stowe S. Factors affecting intellectual outcome in pediatric brain tumor patients. Neurosurgery. 1987;21:638–644. doi: 10.1097/00006123-198711000-00006. [DOI] [PubMed] [Google Scholar]
  16. Giedd JN. Structural magnetic resonance imaging of the adolescent brain. Annals of the New York Academy of Sciences. 2004;1021:105–109. doi: 10.1196/annals.1308.009. [DOI] [PubMed] [Google Scholar]
  17. Gioia GA, Isquith PK, Guy SC, Kenworthy L. Behavior Rating Inventory of Executive Function. Psychological Assessment Resources, Inc; Odessa, Florida: 2000. [Google Scholar]
  18. Giza CC, Prins ML. Is being plastic fantastic? Mechanisms of altered plasticity after developmental traumatic brain injury. Developmental Neuroscience. 2006;28:364–379. doi: 10.1159/000094163. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hanten G, Levin HS, Song JX. Working memory and metacognition in sentence comprehension by severely head injured children: A preliminary study with implications for rehabilitation. Developmental Neuropsychology. 1999;16:393–414. doi: 10.1207/S15326942DN1603_23. [DOI] [Google Scholar]
  20. Hede K. Chemobrain is real but may need new name. Journal National Cancer Institute. 2008;100(3):162–163. 169. doi: 10.1093/jnci/djn007. [DOI] [PubMed] [Google Scholar]
  21. Hermelink K, Kuchenhoff H, Untch M, Bauerfeind M, Lux MP, Buhner M, Munzel K. Two different sides of “chemobrain:” Determinants and non-determinants of self-perceived cognitive dysfunction in a prospective, randomized, multi-center study. Psycho-Oncology. 2010;19(12):1321–1328. doi: 10.1002/pon.1695. [DOI] [PubMed] [Google Scholar]
  22. Hoppe-Hirsch E, Brunet L, Laroussinie F, Cinalli G, Pierre-Kahn A, Renier D, Hirsch JF. Intellectual outcome in children with malignant tumors of the posterior fossa: Influence of the field of irradiation and quality of surgery. Child’s Nervous System. 1995;11:340–346. doi: 10.1007/BF00301666. [DOI] [PubMed] [Google Scholar]
  23. Jain N, Brouwers P, Okcu MF, Cirino PT, Krull KR. Sex-specific attention problems in long-term survivors of pediatric acute lymphoblastic leukemia. Cancer. 2009;115(18):4238–4245. doi: 10.1002/cncr.24464. [DOI] [PubMed] [Google Scholar]
  24. Jannoun L, Bloom HJ. Long-term psychological effects in children treated for intracranial tumors. International Journal of Radiation Oncology, Biology, and Physics. 1990;18:747–753. doi: 10.1016/0360-3016(90)90393-X. [DOI] [PubMed] [Google Scholar]
  25. Kahalley LS, Tyc VL, Wilson SJ, Nelms J, Hudson MM, Wu S, Hinds PS. Adolescent cancer survivors’ smoking intentions are associated with aggression, attention, and smoking history. Journal of Cancer Survivorship. 2011;5:123–131. doi: 10.1007/s11764-010-0149-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Krull KR, Gioia G, Ness KK, Ellenberg L, Recklitis C, Leisenring W, Zeltzer L. Reliability and validity of the Childhood Cancer Survivor Study neurocognitive questionnaire. Cancer. 2008;113(8):2188–2197. doi: 10.1002/cncr.23809. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Lezak M. Neuropsychological Assessment. 3. New York, New York: Oxford University Press; 1995. [Google Scholar]
  28. Luciana M, Conklin HM, Hooper CJ, Yarger RS. The development of nonverbal working memory and executive control processes in adolescents. Child Development. 2005;76:697–712. doi: 10.1111/j.1467-8624.2005.00872.x. [DOI] [PubMed] [Google Scholar]
  29. Mabbott DJ, Spiegler BJ, Greenberg ML, Rutka JT, Hyder DJ, Bouffet E. Serial evaluation of academic and behavioral outcome after treatment with cranial radiation in childhood. Journal of Clinical Oncology. 2005;23(1):2256–2263. doi: 10.1200/JCO.2005.01.158. [DOI] [PubMed] [Google Scholar]
  30. Mangeot S, Armstrong K, Colvin AN, Yeates KO, Taylor HG. Long-term executive function deficits in children with traumatic brain injuries: Assessment using the Brief Rating Inventory of Executive Function (BRIEF) Child Neuropsychology. 2002;8(4):271–284. doi: 10.1076/chin.8.4.271.13503. [DOI] [PubMed] [Google Scholar]
  31. Merchant TE, Kiehna EN, Kun LE, Mulhern RK, Li C, Xiong X, Sanford RA. Phase II trial of conformal radiation therapy for pediatric patients with craniopharyngioma and correlation of surgical factors and radiation dosimetry with change in cognitive function. Journal of Neurosurgery. 2006;104:94–102. doi: 10.3171/ped.2006.104.2.5. [DOI] [PubMed] [Google Scholar]
  32. Minisini A, Atalay G, Bottomley A, Puglisi F, Piccart M, Biganzoli L. What is the effect of systemic anticancer treatment on cognitive function? Lancet Oncology. 2004;5:774–779. doi: 10.1016/S1470-2045(04)01465-2. [DOI] [PubMed] [Google Scholar]
  33. Mitby PA, Robison LL, Whitton JA, Zevon MA, Gibbs IC, Tersak JM, Mertens AC. Utilization of special education services and educational attainment among long-term survivors of childhood cancer: a report from the Childhood Cancer Survivor Study. Cancer. 2003;97:1115–1126. doi: 10.1002/cncr.11117. [DOI] [PubMed] [Google Scholar]
  34. Moran C, Gillon G. Inference comprehension of adolescents with traumatic brain injury: A working memory hypothesis. Brain Injury. 2005;19(10):743–751. doi: 10.1080/02699050500110199. [DOI] [PubMed] [Google Scholar]
  35. Mostow EN, Byrne J, Connelly RR, Mulivhill JJ. Quality of life in long-term survivors of CNS tumours of childhood and adolescence. Journal of Clinical Oncology. 1991;9:592–599. doi: 10.1200/JCO.1991.9.4.592. [DOI] [PubMed] [Google Scholar]
  36. Mulhern RK, Butler RW. Neurocognitive sequelae of childhood cancers and their treatment. Pediatric Rehabilitation. 2004;7:1–14. doi: 10.1080/13638490310001655528. [DOI] [PubMed] [Google Scholar]
  37. Mulhern RK, Khan RB, Kaplan S, Helton S, Christensen R, Bonner M, Reddick WE. Short-term efficacy of Methylphenidate: A randomized, double-blind, placebo-controlled trial among survivors of childhood cancer. Journal of Clinical Oncology. 2004;22:4795–4803. doi: 10.1200/JCO.2004.04.128. [DOI] [PubMed] [Google Scholar]
  38. O’Leary TE, Diller L, Recklitis CJ. The effects of response bias on self-reported quality of life among childhood cancer survivors. Quality of Life Research. 2007;16:1211–1220. doi: 10.1007/s11136-007-9231-3. [DOI] [PubMed] [Google Scholar]
  39. Paniak C, Miller HB, Murphy D, Andrews A, Flynn J. Consonant Trigrams Test for children: Development and norms. The Clinical Neuropsychologist. 1997;11:198–200. doi: 10.1080/13854049708407051. [DOI] [Google Scholar]
  40. Payne JM, Hyman SL, Shores EA, North KN. Assessment of executive function and attention in children with neurofibromatosis type 1: Relationships between cognitive measures and real-world behavior. Child Neuropsychology. 2011;17(4):313–329. doi: 10.1080/09297049.2010.542746. [DOI] [PubMed] [Google Scholar]
  41. Reddick WE, White HA, Glass JO, Wheeler GC, Thompson SJ, Gajjar A, Mulhern RK. Developmental model relating white matter volume to neurocognitive deficits in pediatric brain tumor survivors. Cancer. 2003;97(10):2512–2519. doi: 10.1002/cncr.11355. [DOI] [PubMed] [Google Scholar]
  42. Reeves CB, Palmer SL, Reddick WE, Merchant TE, Buchanan GM, Gajjar A, Mulhern RK. Attention and memory function among pediatric patients with medulloblastoma. Journal of Pediatric Psychology. 2006;31:272–280. doi: 10.1093/jpepsy/jsj019. [DOI] [PubMed] [Google Scholar]
  43. Reimers TS, Mortensen EL, Schmiegelow K. Memory deficits in long-term survivors of childhood brain tumors may primarily reflect general cognitive dysfunctions. Pediatric Blood Cancer. 2007;48:205–212. doi: 10.1002/pbc.20818. [DOI] [PubMed] [Google Scholar]
  44. Ris MD, Packer R, Goldwein J, Jones-Wallace D, Boyett JM. Intellectual outcome after reduced-dose radiation therapy plus adjuvant chemotherapy after medulloblastoma: A Children’s Cancer Group study. Journal of Clinical Oncology. 2001;19:3470–3476. doi: 10.1200/JCO.2001.19.15.3470. [DOI] [PubMed] [Google Scholar]
  45. Robinson KE, Livesay K, Campbell LK, Scaduto M, Cannistraci CJ, Anderson AW, Compas BE. Working memory in survivors of childhood acute lymphocytic leukemia: Functional neuroimaging analyses. Pediatric Blood Cancer. 2010;54(4):585–590. doi: 10.1002/pbc.22362. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Smith EE, Jonides J. Storage and executive processes in the frontal lobes. Science. 1999;283:1657–1661. doi: 10.1126/science.283.5408.1657. [DOI] [PubMed] [Google Scholar]
  47. Smith EE, Marshuetz C, Geva A. Working memory: Findings from neuroimaging and patient studies. In: Boller F, Grafman J, editors. Handbook of Neuropsychology. 2. 2002. pp. 55–72. [Google Scholar]
  48. Sowell ER, Thompson PM, Tessner KD, Toga AW. Mapping continued brain growth and gray matter density reduction in dorsal frontal cortex: Inverse relationships during postadolescent brain maturation. The Journal of Neuroscience. 2001;21:8819–8829. doi: 10.1523/JNEUROSCI.21-22-08819.2001. Retrieved from http://www.jneurosci.org/content/21/22/8819.full. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Vardy J, Tannock I. Cognitive function after chemotherapy in adults with solid tumours. Critical Review Oncology Hematology. 2007;63:183–202. doi: 10.1016/j.critrevonc.2007.06.001. [DOI] [PubMed] [Google Scholar]
  50. Vriezen ER, Pigott SE. The relationship between parental report on the BRIEF and performance-based measures of executive function in children with moderate to severe traumatic brain injury. Child Neuropsychology. 2002;8:296–303. doi: 10.1076/chin.8.4.296.13505. [DOI] [PubMed] [Google Scholar]
  51. Waber DP, Pomeroy SL, Chiverton AM, Kieran MW, Scott RM, Goumnerova LC, Rivkin MJ. Everyday cognitive function after craniopharyngioma in childhood. Pediatric Neurology. 2006;34:13–19. doi: 10.1016/j.pediatrneurol.2005.06.002. [DOI] [PubMed] [Google Scholar]
  52. Wefel JS, Saleeba AK, Buzdar AU, Meyers CA. Acute and late onset cognitive dysfunction associated with chemotherapy in women with breast cancer. Cancer. 2010;116(14):3348–3356. doi: 10.1002/cncr.25098. [DOI] [PubMed] [Google Scholar]
  53. Wechsler D. Wechsler Abbreviated Scale of Intelligence. San Antonio, Texas: Harcourt Assessment; 1999. [Google Scholar]
  54. Wechsler D. Wechsler Adult Intelligence Scale. 3. San Antonio, TX: Psychological Corporation; 1999. [Google Scholar]
  55. Wechsler D. Wechsler Intelligence Scale for Children. 4. San Antonio, TX: Psychological Corporation; 2003. Integrated. [Google Scholar]

RESOURCES