Skip to main content
PMC home page
Journal of Pediatric Psychology logoLink to Journal of Pediatric Psychology
. 2016 Jun 24;42(2):220–231. doi: 10.1093/jpepsy/jsw057

Long-Term Efficacy of Computerized Cognitive Training Among Survivors of Childhood Cancer: A Single-Blind Randomized Controlled Trial

Heather M Conklin 1,✉, Jason M Ashford 1, Kellie N Clark 1, Karen Martin-Elbahesh 1, Kristina K Hardy 2,3, Thomas E Merchant 4, Robert J Ogg 5, Sima Jeha 6, Lu Huang 7, Hui Zhang 7
PMCID: PMC5896595  PMID: 27342301

Abstract

Objective To investigate the long-term efficacy of computerized cognitive training in improving cognitive outcomes among childhood cancer survivors. Methods Sixty-eight survivors of childhood acute lymphoblastic leukemia (ALL) or brain tumor (BT) were randomly assigned to computerized cognitive intervention (23 ALL/11 BT, age = 12.21 ± 2.47) or a waitlist control group (24 ALL/10 BT, age = 11.82 ± 2.42). Cognitive assessments were completed pre-, immediately post-, and 6 months postintervention. Results A prior report showed training led to immediate improvement in working memory, attention and processing speed. In the current study, piecewise linear mixed effects modeling revealed that working memory and processing speed were unchanged from immediate to 6 months postintervention (intervention β = −.04 to .01, p = .26 to .95; control β = −.06 to .01, p = .23–.97), but group differences on an attention measure did not persist. Conclusion Cognitive benefits are maintained 6 months following computerized cognitive training, adding to potential clinical utility of this intervention approach.

Keywords: brain tumor, computerized cognitive training, leukemia, pediatric, working memory.

Children receiving central nervous system (CNS) directed therapy for the treatment of cancer are at significant risk for cognitive problems. Survivors of malignant brain tumor (BT) or acute lymphoblastic leukemia (ALL) are at the greatest risk, with well-documented declines in intellectual functioning (Mulhern & Butler, 2004). Recent research suggests impairments in attention, working memory (WM), and processing speed are core, proximal, contributors to global intellectual declines (Conklin et al., 2012; Mabbott, Penkman, Witol, Strother, & Bouffet, 2008; Reeves et al., 2006; Schatz, Kramer, Ablin, & Matthay, 2000). Cognitive late effects of cancer treatment are associated with reduced academic, social, and vocational attainment (Crom et al., 2007; Mitby et al., 2003). With a growing cancer survivor population (DeSantis et al., 2014), efforts to improve long-term cognitive outcomes take on added importance. Yet, there are few empirically supported interventions that mitigate cognitive problems arising secondary to childhood cancer.

Stimulant medications, particularly methylphenidate, have been shown to be efficacious in addressing attention and social problems experienced by some childhood cancer survivors (Conklin et al., 2010; Mulhern et al., 2004). However, there are a number of children for whom stimulant medications are not a viable treatment option owing to medical contraindication (e.g., poorly controlled seizures or uncorrected endocrinopathies), side effects, parental preference, or poor response (Conklin et al., 2007, 2009, 2010). Therapist-delivered cognitive remediation, which often includes massed practice and teaching of compensatory strategies, has been associated with improved metacognitive skills and academic performance among childhood cancer survivors (Butler et al., 2008; Moore, Hockenberry, Anhalt, McCarthy, & Krull, 2012; Patel, Katz, Richardson, Rimmer, & Kilian, 2009). Yet, participation rates and adherence tend to be low, while time and financial costs are high for modest benefits (Butler et al., 2008; Moore et al., 2012; Patel et al., 2009). Further, children must live in close proximity to geographically limited providers. All things considered, there is a great need for safe, time-efficient, portable cognitive interventions with demonstrated efficacy.

Computerized cognitive training refers to a group of software programs that target specific cognitive skills using repetitive exercises of graded difficulty, typically with some level of expert coaching. Advantages of computerized cognitive training include remote administration affording greater geographical reach, reduced time burden with scheduling flexibility, engaging interfaces for children, tailored difficulty level with immediate feedback, easy progress monitoring, and few (if any) medical contraindications. Cogmed®, a computerized WM intervention, is the best-researched of available programs with over 60 published studies including multiple randomized control trials. Research has demonstrated efficacy for populations with developmental (e.g., Attention Deficit-Hyperactivity Disorder [ADHD], low birth weight, Down syndrome) and acquired (e.g., stroke, traumatic brain injury) attention disorders, with improvements achieved on measures of attention, WM, and executive functions (e.g., Bennett, Holmes, & Buckley, 2013; Klingberg et al., 2005; Lohaugen et al., 2011; Lundqvist, Gundstrom, & Romberg, 2010; Weicker, Vilringer, & Thone-Otto, 2016; Westerberg et al., 2007). One of the primary criticisms of Cogmed® is a failure to demonstrate sustained benefits with some (e.g., Holmes, Gathercole, & Dunning, 2009—sustained WM and math reasoning benefits in healthy children with WM deficits; Hovik, Saune, Aarlien, & Egeland, 2013—sustained WM benefits in children with ADHD; Lohaugen et al., 2011—sustained WM benefits in adolescents born at extremely low birth weight) but not all (Kronenberger et al., 2011—lack of sustained WM benefits in children with cochlear implants) studies demonstrating benefits 6–8 months following training.

With respect to childhood cancer survivors, a pilot study by Hardy and colleagues (2013) demonstrated feasibility and acceptability of Cogmed® with cancer survivors but was not powered to evaluate efficacy. Feasibility and acceptability were replicated in a larger geographically dispersed and socioeconomically varied cancer survivor group (Cox, 2015). The same study demonstrated good computer access and literacy, as well as high satisfaction with computerized WM training. More recently it has been shown that childhood cancer survivors completing Cogmed® demonstrate greater immediate improvement than waitlisted controls on measures of WM, attention, and processing speed (Conklin, 2015). Specifically, there was a significant group by time interaction whereby the intervention group demonstrated greater short-term improvement on the primary WM outcome measure (Wechsler Intelligence Scale for Children, Fourth Edition [WISC-IV; Wechsler, 2004] Spatial Span Backward, p = .002), as well as secondary measures of attention (WISC-IV Spatial Span Forward, p = .012; Conners’ Continuous Performance Test-II [CPT-II; Conners, 2004] omissions, p = .036), WM (WISC-IV Digit Span Backward, p = .017; WISC-IV WM Index, p = .022), and processing speed (CPT-II reaction time, p = .020). Parents of intervention participants reported greater reduction in inattention and executive dysfunction (Conners’ Parent Rating Scale-3 [CPRS-3; Conners, 2008] inattention, p = .009, and executive function, p = .002) Further, corresponding changes in neural activation, as demonstrated by functional magnetic resonance imaging (i.e., reduction from pre- to posttraining in left lateral and bilateral medial frontal areas known to support WM), suggest increased cognitive efficiency indicative of training-related neuroplasticity (Conklin, 2015). While these findings are very encouraging, if benefits do not persist following completion of training, the intervention lacks clinical utility. This maintenance of benefits has yet to be demonstrated for childhood cancer survivors who are susceptible to the gradual emergence of late effects and increasing discrepancy in cognitive performance relative to peers over time (Mulhern & Butler, 2004).

In the current study, we used a randomized, waitlist-controlled design to investigate the long-term efficacy of computerized cognitive training in children who received CNS-directed therapy for a BT or ALL. The primary objective was to investigate whether benefits found and previously published (Conklin, 2015) immediately after training would persist, on average, 6 months following completion of training. Based on studies of Cogmed® with other pediatric patient populations, we hypothesized benefits would be maintained at 6 months following completion of training on our primary WM outcome measure, Spatial Span Backward, as well as several of the other secondary outcome measures showing short-term improvement.

Method

Participants

This investigation represents the 6-month follow-up time point for a randomized, single-blind (psychological examiner), waitlist-controlled, parallel-group trial of computerized cognitive training with childhood cancer survivors. Methods, summarized here, have been previously described (Cox, 2015; Conklin, 2015). Participants were childhood BT or ALL survivors who received cranial radiation therapy and/or intrathecal chemotherapy and were off treatment for at least 1 year without disease recurrence. English speakers between 8 and 16 years of age were recruited. Children with IQ < 70, as indicated by primary treatment protocol, were not eligible. Children were excluded for a history of CNS injury/disease, preexisting ADHD, motor or sensory deficit precluding valid testing or completion of the intervention, psychotropic medications within 2 weeks of enrollment, or a psychological condition that would preclude or take precedence over cognitive intervention. The study was conducted at St. Jude Children’s Research Hospital, a hospital dedicated to the treatment and investigation of life-threatening pediatric diseases that accepts patients from around the world without respect to pay. Study recruitment occurred between October 2010 and November 2012 with study completion in December 2013. This trial was approved by the institutional review board and registered with ClinicalTrials.gov (NCT01217996). Written informed consent was obtained before participation.

Procedures

Participants were recruited in order of upcoming medical appointments. At the first visit, patients completed screening/preintervention cognitive assessment to determine eligibility based on the presence of cognitive difficulties. WM problems were defined by Digit Span, Letter-Number Sequencing, or Spatial Span performance (WISC-IV; Wechsler, 2004) greater than 1 SD below the normative mean or the individual’s IQ (Wechsler Abbreviated Scale of Intelligence [WASI; Wechsler, 1999b]) to allow patients to qualify either based on absolute difficulties (norm referenced) or relative difficulties (referenced to individual IQ). These eligibility criteria allow for investigation of intervention response of children with global cognitive deficits as well as children with specific WM problems. Qualifying participants were randomized to computerized training (Cogmed®; Pearson Education, Inc.) or waitlist control groups. Group randomization was 1:1 and stratified by diagnosis (ALL/BT), age (8–11, 12–16), and gender. Block-randomization as proposed by Zelen (1974) was performed by computer using a system housed within the Biostatistics Department. The person completing randomization (H.C.) did not have advanced knowledge of the next computer algorithm determined group allocation. Only the assigned coach was notified of results of randomized group assignment, with individuals completing enrollment and assessment of cognitive outcomes blind to group assignment. Any documents that revealed randomization outcome or computerized training status were maintained separate from the research chart until study completion to assist in maintaining the blind. A sample size of 30 was targeted for each group to afford 80% power to detect a medium size effect (.65) between groups on WM measures at a significance level of .05.

Computers and/or Internet access were provided to participants randomized to the intervention as needed. The Cogmed® (www.Cogmed.com) intervention group was asked to complete 25 training sessions at home over 5–9 weeks. Training sessions consisted of visual-spatial and verbal WM exercises presented as games, with each session lasting approximately 30–45 min. Exercises increased or decreased in difficulty based on performance. Training progress was monitored over the Internet. Weekly coaching phone calls were used to provide feedback and help maintain motivation. Participants demonstrating slower-than-desired progress (i.e., score gain of <20 after 20 sessions) were offered five additional sessions.

Approximately 10 weeks after baseline assessment, all study participants completed postintervention/waitlist cognitive assessments. Six months later, all participants had a final cognitive assessment and control group members were offered the intervention off-study. Incentives were used to encourage continued participation in training sessions and cognitive assessments. Both groups were provided equal incentives to minimize motivational differences. Participants received $10 gift cards after completing 9, 17, and 25 sessions (or 2, 4, and 6 weeks for controls), as well as after completing pre-, post-, and 6-month follow-up appointments.

Cognitive Measures

Participants were assessed with the same battery of cognitive measures at study outset, 10 weeks and 6 months postintervention/waitlist. All measures have age-specific norms from representative standardization samples, and demonstrated reliability and validity. Psychological examiners who performed testing were blind to participant’s group status.

An age-standardized abbreviated IQ was derived from the WASI (1999b) Vocabulary (participant verbally defines provided words) and Matrix Reasoning (participant selects the best picture to finish a visual pattern) subtests. This abbreviated IQ has a normative mean of 100, standard deviation of 15, and is highly correlated with full IQs (Wechsler, 1991, 1999a). Internal consistency reliability for the WASI abbreviated IQ is high (r = .93). The WASI was chosen over other abbreviated IQ measures to reduce practice effects from Wechsler scales included in primary treatment protocols.

The WISC-IV-Integrated Spatial Span, Digit Span, and Letter-Number Sequencing tasks (Wechsler, 2004) were the performance-based WM measures. Spatial Span Backward was the primary outcome measure because it is a nontrained WM task in the spatial modality, and was used to assess Cogmed® training effects in children with ADHD (Klingberg et al., 2005). Other performance-based and parent measures were secondary outcomes. For Spatial Span, the examiner taps sequences of blocks using 10 blue blocks fastened to a white board. The participant repeats block taps in the same order (Spatial Span Forward—a measure of attention) or in reverse order (Spatial Span Backward—a measure of WM). Digit Span includes Digit Span Forward (participant repeats digits verbatim) and Digit Span Backward (participant repeats digits in reverse order). For Letter-Number Sequencing, the examiner presents sequences of numbers and letters after which the participant repeats the numbers in ascending order followed by the letters in alphabetical order These tasks each provide an age-standardized score with mean of 10 and standard deviation of 3. Internal consistency reliability for these subtests is high (r = .87, .90, and .80, respectively; Wechsler, 2004).

The CPT-II (Conners, 2004) is a computerized measure of sustained attention. Letters are presented on a computer screen, and children press the space bar as quickly and accurately as possible for any letter except the letter “X.” The CPT-II program computes performance indices including an omission score, as an index of inattention, and a reaction time score. Scores are age-standardized T-scores with mean of 50 and standard deviation of 10. Construct validity is indicated by performance differences between children with and without ADHD (Seidel & Joschko, 1990). The CPT is used regularly to monitor medication response in children with ADHD and has negligible practice effect for repeat administration (Conners, 1995).

Reading Fluency and Math Fluency subtests of the Woodcock Johnson III Tests of Achievement (WJ-III; Woodcock, McGrew, & Mather, 2001) were administered to evaluate potential generalization of WM training benefits to academic skills. Reading Fluency requires the participant to read simple sentences and decide if they are true or false. Math Fluency requires the participant to solve simple mathematical calculations. Both subtests measure the number of items correctly completed within a 3-min time limit. Scores are age-standardized with mean of 100 and standard deviation of 15. Test–retest reliabilities are high for the study age range (Reading Fluency—r= .90; Math Fluency—r= .89).

The CPRS-3 (Conners, 2008) was used as a parent-report measure of attention and executive functioning. This form consists of 110 items rated on a scale from 0 (not true at all) to 3 (very much true). Primary scales of interest were Inattention and Executive Functioning. Scaled scores are age- and gender-standardized with mean of 50 and standard deviation of 10. Internal consistency reliabilities range from .85 to .94 for the parent form (Conners, 2008). Evidence for criterion-oriented validity includes significant correlations with Behavior Assessment System for Children, Second Edition (BASC-2; Reynolds & Kamphaus, 2004; r= .72 between the BASC-2 Attention Problems and CPRS-3 Inattention scales).

The Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000) is a parent questionnaire designed to assess behavioral manifestations of executive functioning. It consists of 86 items rated as occurring “never, sometimes or often.” Primary scales of interest were WM and Metacognition Indices. All scaled scores are age- and gender-standardized with mean of 50 and standard deviation of 10. Internal consistency reliabilities for all scales are high (r = .82–.98). The WM Scale correlates moderately with the BASC Attention Problems Scale (r = .69; Reynolds & Kamphaus, 1992).

Statistical Analyses

Demographic and clinical variables were characterized using descriptive statistics and compared between groups to demonstrate group similarity. Linear mixed-effects models were used to examine the contribution of clinical and demographic factors to change from baseline to immediate postintervention. Piecewise linear mixed-effect models were used to evaluate change over time in each group for the time between baseline and immediate postintervention/waitlist cognitive assessment (Slope 1) and between immediate postintervention and six months postintervention/waitlist cognitive assessment (Slope 2) for the primary and secondary cognitive outcome measures. Modeling allowed for comparisons within groups (by evaluating the direction and significance of change in slope) and between groups (by comparing the difference between group slopes). All available data at each time point were included in the models.

Results

Participants

Participant enrollment and adherence with study procedures are briefly summarized here, as well as presented in Figure 1. Of 128 patients screened, 80 qualified for the intervention study based on demonstrated WM problems. Among qualifiers, five were excluded, seven declined participation, and 68 were randomized (34 in each group). Of those randomized to the intervention, 30 (88%) completed at least 20 of 25 sessions (a priori criterion for compliance based on Klingberg et al., 2005 and Hardy, Willard, Allen, & Bonner, 2013) and all returned for postintervention assessments. The average time between end of training and immediate postintervention assessment was 24.87  ±  1.65 days, with no difference in primary WM outcome based on time since intervention completion (<25 or >25 days, t = 0.67, p = .51). Of those randomized to the control group, 32 returned for postwaitlist assessment and 30 for six-month postwaitlist assessment. Several control group participants initiated the intervention off-study (n = 23) with 14 completing training.

Figure 1.

Figure 1.

Open in a new tab

Consort diagram.

Participants in this study were equal in gender distribution (53% male), predominantly Caucasian (78%), and generally of middle-class socioeconomic status (Table I). About two-thirds (69%) of the participants were treated for ALL, typically with chemotherapy alone (87%). The majority of BT participants had received radiation therapy (73%). On average, participants were 12 years of age and 5 years from completion of treatment at study enrollment. In keeping with group stratification procedures, intervention and control groups were well-balanced with respect to gender, age, and diagnosis; there were no group differences in socioeconomic status, age at diagnosis, times since treatment, or treatment intensity. Baseline IQ was trending toward higher among the intervention group (106.9 vs. 99.9, p = .06).

Table I.

Participant Characteristics

Demographic/Clinical Variable Intervention n = 34 Control n = 34 p
Demographic
Gender
 Female 16 (47%) 16 (47%) 1.00
 Male 18 (53%) 18 (53%)
Race/Ethnicity
 African American 1 (3%) 5 (15%) .39
 Asian/Pacific Islander 1 (3%) 1 (3%)
 Caucasian 27 (79%) 26 (76%)
 Hispanic 2 (6%) 1 (3%)
 Other/Multiple races 3 (9%) 1 (3%)
Mean SES (BSMSS)a 39.68 ± 15.37 40.46 ± 12.20 .82
Clinical
Acute lymphoblastic leukemia 23 (68%) 24 (71%) 1.00
Brain tumor 11 (32%) 10 (29%)
 Ependymoma 1 (9%) 3 (30%) .33
 Glioma 2 (18%) 0 (0%)
 Medulloblastoma/PNET 8 (73%) 7 (70%)
Mean age at diagnosis 5.15 ± 2.92 4.62 ± 2.68 .43
Mean age at enrollment 12.21 ± 2.47 11.82 ± 2.42 .51
Mean time since treatment 4.97 ± 3.02 5.04 ± 2.41 .91
Treatment intensity
 Chemo only 20 (59%) 22 (65%) .95
 CSI w/or w/o Chemo 8 (24%) 7 (21%)
 CRT w/or w/o Chemo 3 (9%) 3 (9%)
 Chemo + BMT w/or w/o TBI 3 (9%) 2 (6%)
Mean baseline IQ 106.90 ± 15.74 99.85 ± 14.01 .06
Open in a new tab

Note. Chemo =  chemotherapy; CSI =  craniospinal irradiation; CRT =  conformal radiation therapy; BMT =  bone marrow transplant; TBI =  total body irradiation.

p-values indicate whether group is equally distributed across subcategories using independent t-test, Chi-square, or Fisher’s Exact Test, as appropriate.

aBarrett Simplified Measure of Social Status (BSMSS). Derived from maternal and paternal education and occupation; scores range from 8 to 66 with higher scores indicative of higher SES.

Intervention

Immediate Change

Group means and standard deviations for each cognitive outcome measure, at all three time points, are included in Table II. Piecewise linear mixed-effect models revealed both groups (control and intervention) demonstrated significantly greater cognitive problems than normative expectations at baseline for parent report of attention and executive function difficulties, as well as academic fluency (CPRS-3 Inattention and Executive Function Scales; BRIEF WM and Metacognition Indices; WJ-III Reading and Math Fluency; Intercept Estimates, ps < .05; Table III). The control group demonstrated significantly greater cognitive problems than normative expectations on attention and WM measures (WISC-IV Digit Span Forward, Digit Span Backward, WM Index, and Spatial Span Forward; Intercept Estimates, ps < .05; Table III), with the intervention group showing trends for problems on most of these measures (WISC-IV Digit Span Forward, Digit Span Backward, and WM Index; Intercept Estimates, ps < .10; Table III). In the period between baseline and immediate postcognitive assessment, the intervention group showed greater improvement than the control group on the primary outcome measure, WISC-IV Spatial Span Backward, as indicated by a significant difference in their change scores (Slope 1 Difference Estimate, p < .001; Table III). The intervention group also demonstrated greater improvement than the control group on secondary measures of attention (WISC-IV Spatial Span Forward and CPT-Omissions; Slope 1 Difference, ps < .05; Table III), WM (WISC-IV Digit Span Backward and WM Index; Slope 1 Difference, ps < .05; Table III) and processing speed (CPT-II Reaction Time; Slope 1 Difference, p <  .05; Table III). Parents of participants in the intervention group reported greater reduction in inattention and executive dysfunction than parents of participants in the control group (CPRS-3 Inattention and Executive Function; Slope 1 Difference, ps < .05; Table III). There were no other significant differences in change in performance between groups from baseline to immediate postcognitive assessment (Table III). Linear mixed-effect models revealed that higher baseline IQ and more Cogmed training sessions were predictive of greater change in our primary WM outcome, WISC-IV Spatial Span Backward (Supplementary Table I). There were no adverse events reported for either group.

Table II.

Pretraining, Posttraining, and 6-Month Follow-Up Cognitive Scores

Intervention n = 30
 Control n = 32
Cognitive Outcome Pretraining Posttraining Follow-up Pretraining Posttraining Follow-Up
WISC-IV–Digit Span Forwarda 9.00 ± 2.52 9.93 ± 2.90 10.23 ± 2.64 8.16 ± 3.47 9.00 ± 3.60 9.03 ± 3.21
WISC-IV–Digit Span Backwarda 8.97 ± 2.80 11.17 ± 3.05 10.53 ± 3.58 8.59 ± 3.00 9.22 ± 2.35 9.40 ± 3.01
WISC-IV–Letter Number Sequencinga 9.87 ± 2.91 11.33 ± 1.99 11.13 ± 2.05 9.47 ± 2.70 10.03 ± 2.98 9.80 ± 2.54
WISC-IV–Working Memory Indexb 95.33 ± 12.73 104.50 ± 12.33 103.37 ± 12.47 92.50 ± 14.26 96.47 ± 16.11 95.97 ± 13.74
WISC-IV–Spatial Span Forwarda 9.83 ± 3.36 13.13 ± 3.53 11.63 ± 3.68 8.66 ± 2.38 9.91 ± 2.66 10.07 ± 2.89
WISC-IV–Spatial Span Backwarda 9.50 ± 3.35 12.63 ± 2.99 12.60 ± 2.39 10.03 ± 2.85 10.78 ± 2.73 10.90 ± 2.40
Conners 3 Parent–Inattentionc 63.73 ± 13.86 56.47 ± 7.63 55.67 ± 9.82 61.59 ± 15.32 60.88 ± 15.19 59.63 ± 15.37
Conners 3 Parent–Executive Functionc 62.47 ± 13.30 55.73 ± 8.58 55.50 ± 11.66 58.97 ± 15.20 59.38 ± 14.02 58.13 ± 12.83
BRIEF–Working Memoryc 60.63 ± 11.34 57.23 ± 8.64 56.97 ± 10.57 60.25 ± 14.33 59.53 ± 14.67 56.47 ± 13.62
BRIEF–Metacognitive Indexc 59.53 ± 10.98 55.53 ± 8.08 54.60 ± 9.15 57.75 ± 12.89 56.66 ± 12.56 55.17 ± 12.07
CPT-II–Omissionsc 51.24 ± 11.80 50.46 ± 8.83 51.05 ± 10.16 50.09 ± 9.29 55.76 ± 14.78 52.50 ± 11.56
CPT-II–Hit RTc 50.26 ± 11.22 48.51 ± 9.28 48.41 ± 10.94 49.52 ± 8.52 52.05 ± 10.12 50.07 ± 9.77
WJ-III–Reading Fluencyb 97.57 ± 18.79 99.33 ± 18.79 100.50 ± 20.45 90.29 ± 15.93 94.32 ± 16.45 94.69 ± 17.01
WJ-III–Math Fluencyb 89.53 ± 14.57 90.43 ± 15.68 87.40 ± 16.40 87.59 ± 13.85 89.25 ± 14.74 88.70 ± 12.54
Open in a new tab

Note. WISC-IV = Wechsler Intelligence Scale for Children–Fourth Edition. BRIEF = Behavior Rating Inventory of Executive Function; CPT-II = Conners’ Continuous Performance Test–Second Edition; Hit RT = Hit Reaction Time; WJ-III = Woodcock Johnson Tests of Achievement–Third Edition. All values are presented as mean ± standard deviation.

aScaled Score: mean = 10, standard deviation = 3; higher score is better.

bStandard Score: mean = 100, standard deviation = 15; higher score is better.

cT Score: mean = 50, standard deviation = 10; higher score is worse.

Table III.

Piecewise Linear Mixed Effects Modeling With Random Intercept

  Randomization Intercept
 Slope 1
 Slope 2
Cognitive Outcome Estimate SE p-value* Estimate SE p-value* Estimate SE p-value*
WISC-IV–Digit Span Forwarda Control 8.10 0.54 <.001 0.08 0.05 .080 −0.00 0.02 .828
Intervention 9.00 0.56 .081 0.09 0.05 .061 0.01 0.02 .545
Difference 0.90 0.78 .253 0.01 0.07 .897 0.02 0.03 .561
WISC-IV–Digit Span Backwarda Control 8.58 0.52 .009 0.06 0.05 .247 0.01 0.02 .777
Intervention 8.97 0.54 .062 0.22 0.06 <.001 −0.02 0.02 .256
Difference 0.38 0.75 .611 0.16 0.08 .043 −0.03 0.03 .315
WISC-IV–Letter-Number Sequencinga Control 9.31 0.45 .133 0.06 0.04 .126 −0.01 0.01 .538
Intervention 9.87 0.47 .779 0.15 0.04 <.001 −0.01 0.01 .597
Difference 0.55 0.65 .399 0.09 0.05 .088 0.00 0.02 .948
WISC-IV–Working Memory Indexb Control 91.89 2.38 .001 0.40 0.15 .009 −0.04 0.06 .554
Intervention 95.33 2.51 .067 0.92 0.15 <.001 −0.04 0.06 .463
Difference 3.44 3.46 .322 0.52 0.21 .017 −0.01 0.08 .923
WISC-IV–Spatial Span Forwarda Control 8.56 0.55 .010 0.12 0.06 .030 0.01 0.02 .786
Intervention 9.83 0.57 .770 0.33 0.06 <.001 −0.06 0.02 .012
Difference 1.28 0.79 .107 0.21 0.08 .014 −0.06 0.03 .049
WISC-IV–Spatial Span Backwarda,c Control 10.04 0.49 .940 0.08 0.05 .133 0.00 0.02 .967
Intervention 9.50 0.51 .330 0.31 0.05 <.001 −0.00 0.02 .948
Difference −0.54 0.71 .449 0.24 0.07 .001 −0.00 0.03 .940
Conners 3 Parent–Inattentiond Control 61.77 2.32 <.001 −0.07 0.19 .708 −0.05 0.08 .507
Intervention 63.73 2.42 <.001 −0.73 0.20 <.001 −0.03 0.08 .687
Difference 1.96 3.35 .560 −0.65 0.28 .019 0.02 0.11 .851
Conners 3 Parent–Exec Functiond Control 59.33 2.26 <.001 0.04 0.19 .830 −0.06 0.08 .412
Intervention 62.47 2.36 <.001 −0.67 0.20 <.001 −0.01 0.08 .905
Difference 3.13 3.27 .340 −0.71 0.27 .010 0.05 0.11 .618
BRIEF–Working Memoryd Control 60.50 2.19 <.001 −0.07 0.15 .637 −0.12 0.06 .057
Intervention 60.63 2.30 <.001 −0.34 0.16 .033 −0.01 0.06 .866
Difference 0.13 3.17 .967 −0.27 0.22 .223 0.11 0.09 .214
BRIEF–Metacognition Indexd Control 58.04 1.95 <.001 −0.11 0.13 .387 −0.07 0.05 .183
Intervention 59.53 2.05 <.001 −0.40 0.13 .003 −0.04 0.05 .474
Difference 1.49 2.83 .599 −0.29 0.18 .111 0.03 0.07 .654
CPT-II–Omissionsd Control 50.93 2.00 .644 0.57 0.18 .002 −0.18 0.07 .015
Intervention 51.24 2.08 .553 −0.07 0.19 .713 0.02 0.07 .793
Difference 0.32 2.89 .912 −0.64 0.26 .015 0.19 0.10 .055
CPT-II–Hit RTd Control 49.61 1.75 .824 0.25 0.13 .058 −0.06 0.05 .231
Intervention 50.26 1.83 .886 −0.22 0.14 .116 0.01 0.05 .803
Difference 0.65 2.53 .797 −0.47 0.19 .015 0.08 0.08 .306
WJ-III–Reading Fluencyb Control 90.04 3.10 .002 0.40 0.12 .001 0.00 0.05 .985
Intervention 97.57 3.24 .456 0.18 0.12 .155 0.04 0.05 .346
Difference 7.52 4.49 .097 −0.23 0.17 .193 0.04 0.07 .521
WJ-III–Math Fluencyb Control 87.95 2.53 <.001 0.17 0.14 .243 −0.05 0.06 .396
Intervention 89.53 2.67 <.001 0.09 0.15 .538 −0.12 0.06 .040
Difference 1.59 3.68 .667 −0.08 0.20 .710 −0.07 0.08 .395
Open in a new tab

Note. WISC-IV = Wechsler Intelligence Scale for Children–Fourth Edition; BRIEF = Behavior Rating Inventory of Executive Function; CPT-II = Conners’ Continuous Performance Test–Second Edition; Hit RT = Hit Reaction Time; WJ-III = Woodcock Johnson Tests of Achievement–Third Edition.

*p-values are from piecewise linear mixed-effect models. p-values for intercepts are for baseline estimates relative to published normative mean scores. p-values for Slope 1 are for slope estimates in comparison to zero indicating whether there was a significant change from baseline to immediate postassessment for the control group, intervention group, or a comparison between the control group and intervention group slopes (difference), respectively. p-values for Slope 2 are for slope estimates in comparison to zero indicating whether there was a significant change from immediate postassessment to 6-month postassessment for the control group, intervention group, or a comparison between the control group and intervention group slopes (difference), respectively.

aScaled Score: mean = 10, standard deviation = 3; higher score is better.

bStandard Score: mean = 100, standard deviation = 15; higher score is better.

cPrimary intervention outcome.

dT Score: mean = 50, standard deviation = 10; higher score is worse.

Six-Month Maintenance

In the period between immediate postcognitive and six-month postcognitive assessments (mean = 6.30 months), neither group (control or intervention) showed any change on the primary outcome measure, WISC-IV Spatial Span Backward, indicative of stable performance, or maintenance of benefits over time (Slope 2 Estimates, ps > .90; Table III). For most other measures on which the intervention group demonstrated greater improvement than the control group between baseline and immediate postcognitive assessments, performance remained stable between immediate postassessment and six-month postassessment (WISC-IV Digit Span Backward and WM Index; CPT-II Reaction Time; CPRS-3 Inattention and Executive Function; Slope 2 Estimates, ps = .23–.95; Table III, Supplementary Figure 1). Exceptions were WISC-IV Spatial Span Forward, on which the intervention group declined (Slope 2 Estimate, p < .05; Table III), and CPT-II Omissions, on which the control group improved (Slope 2 Estimate, p < .05; Table III). Linear mixed-effects models that include only baseline and 6 months postintervention time points confirm significantly greater improvement for the intervention group relative to the control group on the primary outcome, Spatial Span Backward, as well as other attention and WM measures (e.g., Spatial Span Forward and Letter-Number Sequencing; Supplementary Table II).

Discussion

Results of this randomized controlled trial have revealed computerized cognitive training is feasible, acceptable, efficacious, and associated with training-related neuroplasticity among childhood cancer survivors. These findings are consistent with the greater Cogmed® literature that indicates short-term cognitive benefits for individuals with developmental and acquired WM problems (e.g., Bennett et al., 2013; Westerberg et al., 2007), as well as brain-based changes on neuroimaging (Olesen, Westerbeg, & Klingberg, 2004). The current study added a critical piece to this line of investigation by demonstrating maintenance of benefits 6 months following completion of training, with stable improvements across both performance measures in the clinical setting and rater measures of behavior in the real-world setting. Further, findings revealed performance on measures was within or better than normative expectations 6 months after training. Sustainable improvements in core skills such as attention, WM, and processing speed among cancer survivors are notable, as these skills are building blocks for higher-level skills, and research has shown early emerging deficits in these areas can translate into downstream problems with academic performance (e.g., Jacola et al., 2016).

Findings revealed better maintenance of WM and processing speed improvements than of attention benefits. Performance on attention measures has been shown to vary more with situational factors such as environment, adequate sleep, arousal level, and caffeine intake (e.g., Helps, Bamford, Sonuga-Barke, & Soderlund, 2014; Louca & Short, 2014). Persistent benefit on parent ratings of attention, if not performance-based attention measures, may be indicative of more meaningful benefits over time. These findings may also suggest the need for combining attention-specific interventions (e.g., behavioral modifications such as reducing environmental distractions, gaining participant attention before presenting information, or teaching of self-monitoring strategies, or pharmacologic intervention such as stimulant medications) with computerized WM training for best maintenance of benefits over time.

This study is not without limitations. While 6 months’ maintenance is the current standard for demonstrating sustained cognitive benefit (e.g., Holmes et al., 2009; Lohaugen et al., 2011), longer-term monitoring would be beneficial. There was mixed evidence for generalizability of cognitive benefits, with improvement in processing speed and parent report of executive functions but not academic fluency. This finding is consistent with the literature that has shown less evidence for generalization of benefits to more distal cognitive outcomes (Robinson et al., 2014). The current study only screened academic fluency with the need for future studies to demonstrate academic benefits of computerized cognitive training. Further, to establish clinically significant benefits of computerized cognitive training, studies should include collateral functional and performance measures including real-world indicators of improved academic, social, and vocational attainment. The current study design included a waitlist control group rather than an active control condition; however, research has clearly shown that intensity and adaptivity are the active ingredients, with Cogmed® consistently outperforming nonadaptive versions and video games (e.g., Klingberg et al., 2005; Thorell et al., 2009). Investigations into prophylactic training and head-to-head comparisons with other empirically supported interventions are also warranted.

Conclusions

Despite well-established findings of cognitive late effects among childhood cancer survivors, and their negative impact on quality of life, there are few empirically supported interventions for this growing group. While methylphenidate and some therapist-delivered interventions have empirical support, these approaches are limited by medical contraindications, parental preference, and availability of providers. Findings from this study offer a new option that may be particularly appealing for those looking for a nonpharmacologic approach that can be completed within the home with scheduling flexibility. Study participation rates, training adherence, and satisfaction ratings all indicate these are desired intervention features (Cox, 2015). Further, studies show the vast majority of families have the requisite technology and computer literacy to be successful (Cox, 2015). The current study provides initial evidence for sustained benefits of computerized cognitive training that further supports the merits of this intervention and suggests training-related improvements might serve as a buffer for delayed emergence of late effects. Additional studies that also demonstrate long-term maintenance of benefits, as well as generalization of benefits to functional outcomes, are needed to establish the clinical utility of computerized cognitive training.

Supplementary Data

Supplementary data can be found at http://www.jpepsy.oxfordjournals.org/.

Supplementary Material

Supplementary Data
Click here for additional data file. (405.1KB, pdf)

Acknowledgement

The authors thank the patients and their families who volunteered their time to participate in this study. Cogmed® software was provided by Pearson Education, Inc. for research purposes. Cogmed and Cogmed Working Memory Training are trademarks of Pearson Education, Inc. or its affiliate(s). Pearson did not play a role in design or conduct of the study; analysis or interpretation of the data; preparation or approval of the manuscript.

Funding

This work was supported in part by the National Cancer Institute at the National Institutes of Health (St. Jude Cancer Center Support [CORE] Grant [P30 CA21765]); the American Cancer Society (RSGPB CPPB-119423 to HC); and the American Lebanese Syrian Associated Charities (ALSAC).

Conflicts of interest: None declared.

References

  1. Bennett S., Holmes J., Buckley S. (2013). Computerized memory training leads to sustained improvement in visuo-spatial short-term memory skills in children with Down syndrome. American Journal on Intellectual and Developmental Disabilities, 118, 179–192. doi: 10.1352/1944-7558-118.3.179 [DOI] [PubMed] [Google Scholar]
  2. Butler R. W., Copeland D. R., Fairclough D. L., Mulhern R. K., Katz E. R., Kazak A. E.,, Noll R. B., Patel S. K., Sahler O. J. (2008). A multicenter, randomized clinical trial of a cognitive remediation program for childhood survivors of pediatric malignancy. Journal of Consulting and Clinical Psychology, 76, 367–378. doi: 10.1037/0022-006X.76.3.367 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Conklin H. M., Ashford J. M., Howarth R. A., Merchant T. E., Ogg R. J., Santana V. M.,, Reddick W. E., Xiong X. (2012). Working memory performance among childhood brain tumor survivors. Journal of the International Neuropsychological Society, 18, 996–1005. doi: 10.1017/S1355617712000793 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Conklin H. M., Helton S., Ashford J., Mulhern R. K., Reddick W. E., Brown R.,, Bonner M., Jasper B. W., Wu S., Xiong X., Khan R. B. (2010). Predicting methylphenidate response in long-term survivors of childhood cancer: A randomized, double-blind, placebo-controlled, crossover trial. Journal of Pediatric Psychology, 35, 144–155. doi: 10.1093/jpepsy/jsp044 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Conklin H. M., Khan R. B., Reddick W. E., Helton S., Brown R., Howard S. C., Bonner M., Christensen R., Wu S., Xiong X., Mulhern R. K. (2007). Acute neurocognitive response to methylphenidate among survivors of childhood cancer: A randomized, double-blind, cross-over trial. Journal of Pediatric Psychology, 32, 1127–1139. doi: 10.1093/jpepsy/jsm045 [DOI] [PubMed] [Google Scholar]
  6. Conklin H. M., Lawford J., Jasper B. W., Morris E. B., Howard S. C., Ogg S. W. (2009). Side effects of methylphenidate in childhood cancer survivors: A randomized placebo-controlled trial. Pediatrics, 124, 226–233. doi: 10.1542/peds.2008-1855 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Conklin H. M., Ogg R. J., Ashford J. M., Scoggins M. A., Zou K. N., Clark K. N., Zhang H. (2015). Computerized cognitive training for amelioration of cognitive late effects among childhood cancer survivors: a randomized controlled trial. Journal of Clinical Oncology, 33, 3894–3902. doi:10.1200/JCO.2015.61.6672 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Conklin H. M., Reddick W. E., Ashford J., Ogg S., Howard S. C., Morris E. B.,, Brown R., Bonner M., Christensen R., Wu S., Xiong X., Khan R. B. (2010). Long-term efficacy of methylphenidate in enhancing attention regulation, social skills, and academic abilities of childhood cancer survivors. Journal of Clinical Oncology, 28, 4465–4472. doi: 10.1200/JCO.2010.28.4026 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Conners C. K. (1995). Conners’ continuous performance test. Toronto, ON: Multi-Health Systems. [Google Scholar]
  10. Conners C. K. (2004). Conners’ continuous performance test II. San Antonio, TX: Pearson Corporation. [Google Scholar]
  11. Conners C. K. (2008). Conners’ rating scales (3rd ed). Toronto, ON: Multi-Health Systems, Incorporated. [Google Scholar]
  12. Cox L. E., Ashford J. M., Clark K. N., Martin-Elbahesh K., Merchant T. E., Ogg R. J., Zhang H. (2015). Feasibility and acceptability of a remotely-administered computerized intervention to address cognitive late effects among childhood cancer survivors. Neuro-oncology Practice, 2, 78–87. doi: 10.1093/nop/npu036 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Crom D. B., Lensing S. Y., Rai S., Snider M. A., Cash D. K., Hudson M. M. (2007). Marriage, employment, and health insurance in adult survivors of childhood cancer. Journal of Cancer Survivorship, 1, 237–245. doi: 10.1007/s11764-007-0026-x [DOI] [PubMed] [Google Scholar]
  14. DeSantis C. E., Lin C. C., Mariotto A. B., Siegel R. L., Stein K. D., Kramer J. L., Alteri R., Robbins A. S., Jemal A. (2014). Cancer treatment and survivorship statistics, 2014. CA: A Cancer Journal for Clinicians, 64, 252–271. doi: 10.3322/caac.21235 [DOI] [PubMed] [Google Scholar]
  15. Gioia G. A., Isquith P. K., Guy S. C., Kenworthy L. (2000). Behavior rating inventory of executive function. Odessa, FL: Psychological Assessment Resources, Incorporated. [Google Scholar]
  16. Hardy K. K., Willard V. W., Allen T. M., Bonner M. J. (2013). Working memory training in survivors of pediatric cancer: A randomized pilot study. Psychooncology, 22, 1856–1865. doi: 10.1002/pon.3222 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Helps S. K., Bamford S., Sonuga-Barke E. J., Soderlund G. B. (2014). Different effects of adding white noise on cognitive performance of sub-, normal and super-attentive school children. PLoS One, 9, e112768. doi: 10.1371/journal.pone.0112768 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Holmes J., Gathercole S. E., Dunning D. L. (2009). Adaptive training leads to sustained enhancement of poor working memory in children. Developmental Science, 12, F9–F15. doi: 10.1111/j.1467-7687.2009.00848.x [DOI] [PubMed] [Google Scholar]
  19. Hovik K., Saune B., Aarlien A. K., Egeland J. (2013). RCT of working memory training in ADHD: Long-term near-transfer effects. PloS ONE, 8, e80561. doi: 10.1371/journal.pone.0080561 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Jacola L. M., Krull K., Pui C.-H., Pei D., Cheng C., Reddick W. E., Conklin H. C. (2016). Longitudinal assessment of neurocognitive outcomes in survivors of childhood acute lymphoblastic leukemia treated on a contemporary chemotherapy protocol. Journal of Clinical Oncology, 34, 1239–1247. doi: 10.1200/JCO.2015.64.3205 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Klingberg T., Fernell E., Olesen P. J., Johnson M., Gustafsson P., Dahlstrom K., Gillberg C. G., Forssberg H., Westerberg H. (2005). Computerized training of working memory in children with ADHD- a randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 177–186. doi: 10.1097/00004583-200502000-00010 [DOI] [PubMed] [Google Scholar]
  22. Kronenberger W. G., Pisoni D. B., Henning S. C., Colson B. G., Hazzard L. M. (2011). Working memory training for children with cochlear implants: A pilot study. Journal of Speech, Language, and Hearing Research, 54, 1182–1196. [Database] doi: 10.1044/1092-4388(2010/10-0119) [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Lohaugen G. C., Antonsen I., Haberg A., Gramstad A., Vik T., Brubakk A. M., Skranes J. (2011). Computerized working memory training improves function in adolescents born at extremely low birth weight. Journal of Pediatrics, 158, 555–561. doi: 10.1016/j.jpeds.2010.09.060 [DOI] [PubMed] [Google Scholar]
  24. Louca M., Short M. A. (2014). The effect of one night’s sleep deprivation on adolescent neurobehavioral performance. Sleep, 37, 1799–1807. doi: 10.5665/sleep.4174 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lundqvist A., Gundstrom K., Ronnberg J. (2010). Computerized working memory training in a group of patients suffering from acquired brain injury. Brain Injury, 24, 1173–1183. doi: 10.3109/02699052.2010.498007 [DOI] [PubMed] [Google Scholar]
  26. Mabbott D. J., Penkman L., Witol A., Strother D., Bouffet E. (2008). Core neurocognitive functions in children treated for posterior fossa tumors. Neuropsychology, 22, 159–168. doi: 10.3109/02699052.2010.498007 [DOI] [PubMed] [Google Scholar]
  27. Mitby P. A., Robison L. L., Whitton J. A., Zevon M. A., Gibbs I. C., Tersak J. M.; Childhood Cancer Survivor Study Steering Committee. (2003). Utilization of special education services and educational attainment among long-term survivors of childhood cancer: A report from the Childhood Cancer Survivor Study. Cancer, 97, 1115–1126. (doi: 10.3109/02699052.2010.498007 [DOI] [PubMed] [Google Scholar]
  28. Moore I. M., Hockenberry M. J., Anhalt C., McCarthy K., Krull K. R. (2012). Mathematics intervention for prevention of neurocognitive deficits in childhood leukemia. Pediatric Blood Cancer, 59, 278–284. doi: 10.1002/pbc.23354 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Mulhern R. K., Butler R. W. (2004). Neurocognitive sequelae of childhood cancers and their treatment. Pediatric Rehabilitation, 7, 1–14. doi: 10.1080/13638490310001655528 [DOI] [PubMed] [Google Scholar]
  30. Mulhern R. K., Khan R. B., Kaplan S., Helton S., Christensen R., Bonner M., Brown R., Xiong X., Wu S., Gururangan S., Reddick W. E. (2004). Short-term efficacy of methylphenidate: A randomized, double-blind, placebo-controlled trial among survivors of childhood cancer. Journal of Clinical Oncology, 22, 4795–4803. doi: 10.1200/JCO.2004.04.128 [DOI] [PubMed] [Google Scholar]
  31. Olesen P. J., Westerberg H., Klingberg T. (2004). Increased prefrontal and parietal activity after training of working memory. Nature Neuroscience, 7, 75–79. doi:10.1038/nn1165 [DOI] [PubMed] [Google Scholar]
  32. Patel S. K., Katz E. R., Richardson R., Rimmer M., Kilian S. (2009). Cognitive and problem solving training in children with cancer: A pilot project. Journal of Pediatric Hematology Oncology, 31, 670–677. doi: 10.1097/MPH.0b013e3181b25a1d [DOI] [PubMed] [Google Scholar]
  33. Reeves C. B., Palmer S. L., Reddick W. E., Merchant T. E., Buchanan G. M., Gajjar A., Mulhern R. K. (2006). Attention and memory function among pediatric patients with medulloblastoma. Journal of Pediatric Psychology, 31, 272–280. doi: 10.1093/jpepsy/jsj019 [DOI] [PubMed] [Google Scholar]
  34. Reynolds C. R., Kamphaus R. W. (1992). Behavior assessment system for children .Circle Pines, MN: American Guidance Service. [Google Scholar]
  35. Reynolds C. R., Kamphaus R. W. (2004). Behavior assessment system for children (2nd ed). Bloomington, MN: Pearson Assessments. [Google Scholar]
  36. Robinson K. E., Kaizar E., Catroppa C., Godfrey C., Yeates K. O. (2014). Systematic review and meta-analysis of cognitive interventions for children with central nervous system disorders and neurodevelopmental disorders. Journal of Pediatric Psychology, 39, 846–865. doi: 10.1093/jpepsy/jsu031 [DOI] [PubMed] [Google Scholar]
  37. Schatz J., Kramer J. H., Ablin A., Matthay K. K. (2000). Processing speed, working memory and IQ: A developmental model of cognitive deficits following cranial radiation therapy. Neuropsychology, 14, 189–200. doi: 10.1037/0894-4105.14.2.189 [DOI] [PubMed] [Google Scholar]
  38. Seidel W. T., Joschko M. (1990). Evidence of difficulties in sustained attention in children with ADDH. Journal of Abnormal Child Psychology, 18, 217–229. doi: 10.1007/BF00910732 [DOI] [PubMed] [Google Scholar]
  39. Thorell L. B., Lindqvist S., Bergman Nutley S., Bohlin G., Klingberg T. (2009). Training and transfer effects of executive functions in preschool children. Developmental Science, 12, 106–113. doi: 10.1111/j.1467-7687.2008.00745.x [DOI] [PubMed] [Google Scholar]
  40. Weicker J., Vilringer A., Thone-Otto A. (2016). Can impaired working memory functioning be improved by training? A meta-analysis with a special focus on brain injured patients. Neuropsychology, 30, 190–212. doi: 10.1037/neu0000227 [DOI] [PubMed] [Google Scholar]
  41. Wechsler D. (1991). Wechsler intelligence scale for children (3rd ed). San Antonio, TX: Psychological Corporation. [Google Scholar]
  42. Wechsler D. (1999a). Wechsler adult intelligence scale (3rd ed). San Antonio, TX: Harcourt Assessment. [Google Scholar]
  43. Wechsler D. (1999b). Wechsler abbreviated scale of intelligence. San Antonio, TX: Harcourt Assessment. [Google Scholar]
  44. Wechsler D. (2004). Wechsler intelligence scale for children (4th ed). San Antonio, TX: Pearson Corporation. [Google Scholar]
  45. Westerberg H., Jacobaeus H., Hirvikoski T., Clevberge P., Ostensson M. L., Bartfai A., Klingberg T. (2007). Computerized working memory training after stroke- a pilot study. Brain Injury, 21, 21–29. doi: 10.1080/02699050601148726 [DOI] [PubMed] [Google Scholar]
  46. Woodcock R. W., McGrew K. S., Mather N. (2001). Woodcock-Johnson III tests of achievement. Itasca, IL: Riverside Publishing. [Google Scholar]
  47. Zelen M. (1974). The randomization and stratification of patients to clinical trials. Journal of Chronic Disease, 27, 365–375. doi: 10.1016/0021-9681(74)90015-0 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data
Click here for additional data file. (405.1KB, pdf)

Articles from Journal of Pediatric Psychology are provided here courtesy of Oxford University Press

RESOURCES