Working Memory Training in Survivors of Pediatric Cancer: A Randomized Pilot Study

Abstract

Objectives

Survivors of pediatric brain tumors and acute lymphoblastic leukemia (ALL) are at increased risk for neurocognitive deficits, but few empirically-supported treatment options exist. We examined the feasibility and preliminary efficacy of a home-based, computerized working memory training program, CogmedRM, with survivors of childhood cancer.

Methods

Survivors of brain tumors or ALL (n = 20) with identified deficits in attention and/or working memory were randomized to either the success-adapted computer intervention or a non-adaptive, active control condition. Specifically, children in the adaptive condition completed exercises that became more challenging with each correct trial, whereas those in the non-adaptive version trained with exercises that never increased in difficulty. All participants were asked to complete 25 training sessions at home, with weekly, phone-based coaching support. Brief assessments were completed pre- and post-intervention; outcome measures included both performance-based and parent-report measures of working memory and attention.

Results

Eighty-five percent of survivors were compliant with the intervention, with no adverse events reported. After controlling for baseline intellectual functioning, survivors who completed the intervention program evidenced significant post-training improvements in their visual working memory and in parent-rated learning problems compared with those in the active control group. No differences in verbal working memory functioning were evident between groups, however.

Conclusions

Home-based, computerized cognitive training demonstrates good feasibility and acceptability in our sample. Children with higher intellectual functioning at baseline appeared to benefit more from the training, though further study is needed to clarify the strength, scope, and particularly the generalizability of potential treatment effects.

It is well known that survivors of brain tumors and acute lymphoblastic leukemia (ALL) are at increased risk for neurocognitive late effects [1–3]. More specifically, many survivors develop primary impairments in attention, working memory, and processing speed that are then associated with declines in cognitive and academic functioning [4,5], as well as poor long-term social and vocational outcomes [6–8]. As such, recent efforts have focused on developing and testing interventions designed to ameliorate these deficits [9].

A number of treatment approaches have been evaluated in recent years, including medication, clinic-based cognitive remediation, and home-based computerized cognitive training [9]. Psychostimulant medication, long established as an effective treatment for symptoms of attention-deficit hyperactivity disorder (ADHD), has also been found to reduce attention deficits in survivors [10–16]. Despite these positive results, there remain several concerns about the use of psychostimulants, including a greater susceptibility to side effects in survivors than in children with ADHD [11, 17], unknown long-term effects [18], and a potential reluctance to using psychoactive medications [19].

Cognitive remediation has also been evaluated as a treatment approach for survivors with neurocognitive deficits. Based on an approach developed for use with individuals with traumatic-brain injury (TBI), these programs typically focus on the repetition of tasks thought to develop core cognitive skills (e.g., immediate and working memory), as well as the acquisition and practice of strategies to compensate for deficits in these areas (e.g., organizational skills, self-monitoring) [20]. To date, clinic-based programs involving primarily face-to-face work with one or more providers have been the focus of most empirical study. A multi-center randomized trial [21] of a hospital-based cognitive remediation program was completed with 161 survivors of pediatric cancer. While treatment efficacy was demonstrated through increases in attention and academic achievement, effect sizes were small for a number of functional outcomes. Conclusions about the effectiveness of the program also were mitigated by a sizable percentage of participants who failed to complete the 6-month program [21]. A similar intervention that focused on problem-solving training was piloted with a small group of survivors [22]. Again, preliminary efficacy was demonstrated through improvements on each outcome measure; however, this study was also characterized by a low participation rate and suboptimal adherence. The authors noted that the most frequent reason given for lower adherence was perceived inconvenience of coming to the clinic. Collectively, these results suggest that intensive, therapist-directed, in-person interventions may not be practical or desirable for some subgroups of the survivor population.

Given the limitations associated with these treatment approaches, there remains a critical need for efficacious interventions targeting symptoms associated with neurocognitive late effects in pediatric cancer survivors. Home-based, computerized cognitive training has recently emerged as a cognitive remediation paradigm with the potential to address core neuropsychological deficits in survivors [23, 24]. This approach is associated with a low risk of side effects (compared to pharmacological treatment) and may carry a reduced treatment burden (compared to clinic-based interventions) because it can be completed at home, any time of day. Computerized cognitive training employs game-like exercises to target core cognitive skills such as working memory and attention. Such programs have demonstrated efficacy across a wide variety of individuals with cognitive difficulties: children with ADHD [25–31], TBI and stroke [32, 33], schizophrenia [34, 35], extremely low birth weight [36], cochlear implants [37], and borderline intellectual disabilities [38].

A recent study piloted the use of one such intervention with a small group (n = 9) of survivors [23]. Results were promising, with survivors demonstrating improvements in both performance-based and questionnaire measures of attention. However, several limitations were noted, including a small sample size, lack of a control group, and improvement in measures of attention, but not working memory [23]. Kesler and colleagues [24] subsequently investigated the preliminary efficacy of an internet-based cognitive rehabilitation program with a small sample of survivors (n = 23). Survivors demonstrated improvements in processing speed, cognitive flexibility, and visual and verbal declarative memory. However, there were no significant changes observed in either working memory or attention. Of interest, fMRI data documented significant increases in dorsolateral prefrontal cortex activation in participants who completed training [24]. Results from both intervention studies [23, 24] suggest that computerized cognitive training programs are feasible for use and potentially efficacious with survivors of pediatric cancer, though neither used a randomized, controlled design.

In order to extend the literature in this area, we evaluated an existing computerized cognitive training program, CogmedRM, with survivors of pediatric cancer. CogmedRM has significant advantages over other computerized cognitive training programs for the survivor population. First, unlike the previous programs evaluated with cancer survivors, its use has been associated with significant efficacy in a number of well-designed and controlled trials with children and adolescents [27, 29–31]. Moreover, CogmedRM specifically targets working memory skills, which have been proposed to underlie the changes in intelligence and academic performance frequently seen in survivors with cognitive late effects [4, 5]. Finally, CogmedRM consists of a fixed “dose” of training (i.e., 25 sessions) and also has an active control version of the program, making it ideally suited for empirical study in a randomized, controlled design. The objectives of this study were to describe the feasibility and preliminary efficacy of this program in a small, randomized clinical trial with survivors of pediatric brain tumors and ALL. Outcomes included an assessment of treatment adherence and acceptability, as well as improvement on parent-rated and performance-based measures of working memory. Specifically, in keeping with prior pilot work using this program, we evaluated a near-transfer task as our primary endpoint (WRAML-2 Symbolic Working Memory) [27,28]. Secondary endpoints included performance-based measures of verbal working memory (WRAML-2 Verbal Working Memory), auditory and visual short-term memory (WRAML-2 Letter Number and Finger Windows), and caregiver ratings of attention and learning problems (Conners’ 3 Rating Scale – Inattention and Learning Problems).

Methods

Participants

Our sample included survivors aged 8 to 16 (85% Caucasian) with a history of brain tumor or ALL. The decision to include both groups was informed by the different profile and severity of clinical impairments in these children. Specifically, brain tumor survivors typically have a wider range of cognitive and physical impairments than ALL survivors. As such, including both groups allowed us to determine feasibility and preliminary efficacy of the intervention on the widest possible range of functioning. This is also consistent with most intervention studies conducted with the pediatric cancer survivor population [11, 15, 21, 22]. Potential participants who had been off-treatment and medically stable for at least one year were recruited from the Division of Pediatric Hematology-Oncology at a large academic medical center in the Southeast over a 20-month period. The majority of participants were screened following presentation for routine evaluation of their neuropsychological functioning, which is part of the standard-of-care services offered to survivors of ALL and BT followed through our clinic. Families of survivors meeting age and diagnostic inclusion criteria were sent a letter describing the study prior to their scheduled appointment for neuropsychological evaluation. At their appointment, we presented all interested families with the option of being screened for participation in the study. In order to reduce burden on participants, we informed families who had reported no difficulties with attention and working memory during their clinical evaluation that their children would likely not qualify for participation in the intervention phase of the study, and most of those families elected not to proceed with screening for the study (see Figure 1). Finally, two participants were screened after their parents learned of the study through the clinical trials website for the National Institutes of Health. See Figure 1 for a flow diagram detailing patient recruitment, enrollment, and screening, and Table I for demographic and medical information of the 20 participants who qualified for participation in the treatment phase of the study.

Figure 1.

Table I.

Demographic and treatment information and baseline functioning for the 20 participants enrolled and randomized to the intervention.

	Adaptive (n = 13)		Nonadaptive (n = 7)
	M ± SD	N (%)	M ± SD	N (%)
Demographic Variables
Age (years)	12.7 ± 2.77		10.7 ± 1.89
Gender
Male		8 (61.5)		4 (57.1)
Female		5 (38.5)		3 (42.9)
Treatment Variables
Age at Diagnosis (years)	4.9 ± 3.54		5.7 ± 2.88
Years Off-Treatment	6.0 ± 2.98		3.0 ± 1.77
Diagnosis
ALL		7 (53.8)		4 (57.1)
Medulloblastoma/PNET		2 (15.4)		2 (28.6)
Ependymoma		3 (23.1)
Other Tumor Type		1 (7.7)		1 (14.3)
Treatment Types
Neurosurgery		5 (38.5)		3 (42.9)
Chemotherapy		10 (76.9)		6 (85.7)
Cranial Radiation		5 (38.5)		4 (57.1)
Baseline Functioning
Estimated IQ	101.8 ± 18.75		97.6 ± 13.21

Note. Two children in the adaptive group were on a stimulant medication during the trial. Children in the adaptive group were significantly further off-therapy at baseline: t = 2.4, p < .05.

Procedures

Following IRB-approved consent (parent or guardian) and assent (children over 12) procedures, survivors underwent a screening battery that included a brief IQ test, attention/working memory testing, and parent-report questionnaires. To be eligible for the intervention phase, participants needed to demonstrate intellectual functioning sufficient to understand the training tasks (estimated IQ ≥ 70). Additionally, participants were required to show evidence of difficulties with attention and working memory on either performance-based measures or parent-ratings of functional impairment. Specifically, survivors had to meet at least one of the following criteria: a T-score greater than the 75^th percentile on the Inattention subscale of the Conners’ Parent Rating Scale (Conners-3)[39]; and/or one or more standard deviations below the mean on the Attention/Concentration or Working Memory Indices of the WRAML2 [40] or Attention/Concentration or Working Memory Indices that were one or more standard deviations below the participant’s estimated IQ. Of note, the majority of survivors (n = 12; 60%) qualified for the trial based on at least two indices, with a quarter (n = 5) qualifying on all three indices. Seven survivors (35%) qualified based on parent-ratings only.

As may be seen in Figure 1, an initial pool of 40 patients was identified as meeting initial diagnosis, age, and time off treatment criteria from among those participants followed in our survivor neuropsychology clinic over the 20-month study recruitment period, out of 95 evaluations conducted during that time. Eighty-seven percent (n = 35) of those families expressed initial interest in the study during their routine neuropsychology visits. To reduce participant burden, only those whose caregivers expressed concerns about attention-related impairment during the assessment were consented for a full screening evaluation. Twenty-three participants (11 brain tumor, 12 ALL) underwent full screening procedures; 20 patients (87%) met inclusion criteria. The three who did not meet inclusion criteria (2 brain tumor, 1 ALL) did not evidence performance-based deficits in either attention or working memory, and questionnaire measures failed to reach our threshold of eligibility despite parent-reported concerns with everyday attentional functioning. See Tables I and II for baseline functioning of the 20 randomized participants.

Table II.

Pre- and post-intervention scores for neuropsychological assessment and rating scales

	Baseline	Post-Intervention		3-month Follow-up
	M ± SD	M ± SD	Cohen’s d	M ± SD	Cohen’s d
WRAML2 Subtests
Finger Windows			.67		.30
Adaptive	9.6 ± 2.40	12.4 ± 3.91		11.1 ± 3.48
Non-Adaptive	10.3 ± 3.82	9.7 ± 2.42		10.0 ± 2.10
Number Letter			.28		−.05
Adaptive	9.5 ± 3.84	10.4 ± 3.86		9.7 ± 2.12
Non-Adaptive	10.0 ± 3.51	9.3 ± 2.07		9.5 ± 3.83
Verbal Working Memory			−.12		−.21
Adaptive	8.5 ± 1.98	9.9 ± 3.75		9.6 ± 2.55
Non-Adaptive	8.3 ± 2.14	9.8 ± 2.56		9.5 ± 2.88
Symbolic Working Memory			.94		1.22
Adaptive	9.8 ± 3.78	12.4 ± 4.57		11.4 ± 3.40
Non-Adaptive	9.3 ± 1.98	10.4 ± 2.30		8.2 ± 4.17
Conners-3 Parent Ratings
Inattention			.40		.21
Adaptive	70.2 ± 14.11	60.8 ± 15.42		57.8 ± 13.59
Non-Adaptive	74.6 ± 16.08	74.0 ± 8.08		68.3 ± 8.38
Learning Problems			.80		.38
Adaptive	74.4 ± 12.55	61.6 ± 8.74		63.6 ± 11.27
Non-Adaptive	69.1 ± 17.90	70.0 ± 13.62		66.2 ± 13.92

Note. WRAML2 scores are scaled scores (mean = 10, SD = 3) and Conners-3 scores are T-scores (mean = 50, SD = 10). Scores presented are uncorrected for IQ.

Following screening procedures, eligible participants were randomized in an allocation ratio of 2:1 to either the intervention (65%) or control (35%) arms, using a blocked randomization procedure, stratified by diagnosis. The CogmedRM program consists of twelve visually-engaging exercises that target visuo-spatial and auditory working memory skills. Difficulty of the tasks is automatically adjusted on a trial-by-trial basis throughout each of 25 training sessions, such that as the child becomes more proficient, the exercises become more difficult. Children in the intervention arm were asked to complete 25 training sessions of CogmedRM, whereas children in the active control condition were asked to complete 25 sessions of MegaMemo, a computer program that consists of the same exercises as CogmedRM, but the level of difficulty never increases. Thus, the nonadaptive MegaMemo condition was designed to deliver a non-therapeutic “dose” of the program. Our decision to include a nonadaptive training condition, rather than a passive control arm (e.g., wait-list control) was informed by our desire to have a comparable comparison group against which to measure any test-retest effects, to reduce the likelihood that any post-treatment changes in parent ratings or cognitive performance could be attributable to expectancy effects [41]. While the administration of the program precluded a double-blind scenario, examiners who performed the evaluations were blind to participants’ randomization status. Moreover, neither participants nor caregivers were informed about which training program their child had been assigned to prior to the intervention; rather, participants and their families were informed that there were two versions of the program reflecting different levels of difficulty.

Participants in both treatment arms were asked to complete three to five training sessions a week for a total of 25 sessions over 5–8 weeks. To assist participants as they completed the intervention, a treatment “coach” was available to survivors in both training conditions. The coach made brief, weekly contact with all families in order to facilitate the child’s motivation for training, as well as to ensure that any problems (e.g., training issues, adverse events, technical problems) were addressed efficiently. To further maintain survivors’ interest and progress in the program, participants earned gift cards following sessions 9, 18 and 25. Additionally, subjective feasibility data was collected from parents and survivors by phone following completion of the intervention.

Immediately following the intervention and three months later, survivors returned to clinic to complete follow-up testing. Those participants who were randomized to the control condition were given the option to complete the adaptive intervention program following completion of the study.

Measures

Feasibility Survey

There is no standardized measure of feasibility and acceptability for use with this type of intervention. Therefore, we developed a 13-item survey for parents and children assessing technical feasibility (2 questions; e.g., “Did the computer program work each time you/your child tried it?”), adherence (3 questions; e.g., “How often did you/your child complain about having to complete training?”), satisfaction (e.g., “How satisfied were you/your child with using the program?”), and ease-of-use (8 questions total; e.g., “How often did you/your child experience frustration/feel bored/enjoy completing the program?”). This survey was administered at the end of the intervention to parents and participants; data were described and interpreted on an item-level basis.

Wechsler Abbreviated Scale of Intelligence (WASI)[42]

The WASI is a brief 4-subtest measure of estimated IQ, fully comparable to IQ scores obtained from longer batteries. The measure was used at baseline to ascertain that survivors had the necessary cognitive functioning to complete the intervention (IQ ≥ 70).

Wide Range Assessment of Memory and Learning, 2^nd edition (WRAML2)[40]

The WRAML2 is a widely-used measure of visual and verbal memory in children and adolescents aged 6–17. Two Attention/Concentration subtests (Number Letter and Finger Windows) and two Working Memory subtests (Symbolic and Verbal Working Memory) were used as outcome measures, with the Symbolic Working Memory task as the identified primary endpoint.

Conners’ Parent Rating Scales, 3^rd edition (Conners-3)[39]

The Conners’ Rating Scales are questionnaire measures of children and adolescents’ attention and behavioral functioning. Factor structure is similar for ADHD and cancer survivor samples [43]. The Inattention and Learning Problems scales were used as outcome measures.

Side Effects Rating Scale (SERS)[44]

Data from the SERS was collected in order to systematically record any adverse events. This measure consists of 17 items (e.g., headaches, nausea), rated on a scale ranging from 0 (absent) to 9 (serious). The SERS has been used in previous randomized clinical trials with cancer survivors in order to track treatment-related adverse events [11–13]. Given the low frequency of reported side effects, these data were analyzed descriptively according to each symptom.

Data Analyses

Given the exploratory nature of this study, we used descriptive and summary statistics to report parent and child ratings of the intervention’s technical feasibility, ease-of-use, and satisfaction. Our primary goal in this pilot trial was to examine method effectiveness, and thus, we employed a Per Protocol analytic strategy in which all participants who completed the treatment arms according to our protocol were included in subsequent analyses [45]. In our analysis of preliminary efficacy, we compared participants’ outcome scores (i.e., WRAML2 and Conners-3) between groups, both post-intervention, and at the three-month follow-up. Specifically, we used a general linear model controlling for both baseline scores and estimated IQ to examine group differences and to estimate effect sizes for each outcome. Finally, given our small sample size, we also employed Reliable Change Index (RCI) analyses in order to determine whether any participants had exhibited clinically meaningful increases in our performance-based measures or ratings of functional impairment [46,47].

Results

Feasibility

As noted above, twenty participants were randomized to the intervention phase. The definition of compliance used was based on previous reports describing the feasibility and efficacy of CogmedRM [27–31]. Of the 20 participants randomized to the two treatment arms, 17 (85%) were compliant with the training, as defined by completion of at least 80% (20 sessions) of the required sessions, and 15 (75%) completed all training sessions. All noncompliant participants were male: two were randomized to the adaptive version of the program, and one to the nonadaptive (control) program. One of these participants did not complete any training sessions (adaptive arm), having changed his mind about participating after returning home on the day of the baseline evaluation. In addition, one child completed 9 sessions of non-adaptive training, and the third completed 14 sessions of the adaptive training. Importantly, none of the three noncompliant participants returned to clinic to complete either of follow-up evaluations, despite our attempts to obtain these data. In addition, we are lacking data for the primary endpoint from the post-treatment evaluation for one participant in the non-adaptive arm due to examiner error. Therefore, our final analyses included complete data from 16 participants, with near-complete data from one additional child, and all of the children included were treatment-compliant. Of the participants meeting the compliance criterion, the mean number of training sessions completed was comparable across the adaptive (M = 24.5, SD = 0.54) and nonadaptive (M = 24.7, SD = 0.33) groups.

No adverse events were reported by any participant in either treatment arm on the SERS. In terms of feasibility, parents reported few problems with the technical use of the computer program: four parents reported having occasional trouble connecting to their home wireless systems to upload data (this did not impact their children’s training schedule) and one parent reported that the program temporarily “froze” during training on two occasions. Similarly, 100% of families reported that it was either “very easy” or “somewhat easy” to log into and start the training program, and to use the mouse to navigate the activities. In addition, all but one parent (94%) reported that they “rarely” or “never” needed to closely supervise their child during training. As seen in Table III, a large percentage of parents of children in both the adaptive (54.5%) and nonadaptive (50.0%) groups reported that their children either “often” or “always” enjoyed completing their training sessions. Participants in the adaptive training group were more likely to report feeling “often” frustrated while completing the exercises (27.3%), whereas those in the nonadaptive group were more likely to report feeling frequent boredom (33.3%). Finally, satisfaction with training programs in both arms was high, with 88.2% of parents rating themselves as either “very” (64.7%) or “somewhat” (23.5%) satisfied with their child’s training experience.

Table III.

Responses to feasibility questionnaire completed by parents at the post-intervention assessment.

Questions: “How often…”	Parent Responses
	Adaptive (n = 11)			Non-Adaptive (n = 6)
	Never/Rarely	Sometimes	Often/Always	Never/Rarely	Sometimes	Often/Always
Did you help your child with the exercises?	10 (90.9%)	1 (9.1%)	-	6 (100%)	-	-
Did your child complain about completing the exercises?	6 (54.5%)	3 (27.3%)	2 (18.2%)	1 (16.7%)	4 (66.6%)	1 (16.7%)
Did your child experience physical pain or discomfort?	10 (90.9%)	1 (9.1%)	-	5 (83.3%)	1 (16.7%)	-
Did your child experience frustration?	2 (18.2%)	6 (54.5%)	3 (27.3%)	5 (83.3%)	1 (16.7%)	-
Did your child feel bored?	7 (63.6%)	3 (27.3%)	1 (9.1%)	2 (33.3%)	2 (33.3%)	2 (33.3%)
Did your child enjoy completing the exercises?	-	5 (45.5%)	6 (54.5%)	2 (33.3%)	1 (16.7%)	3 (50%)
	Somewhat Dissatisfied	Somewhat Satisfied	Very Satisfied	Somewhat Dissatisfied	Somewhat Satisfied	Very Satisfied
How satisfied were you with your child’s participation?	-	2 (18.2%)	8 (72.7%)	1 (16.7%)	2 (33.3%)	3 (50%)

Note. 17 of 20 (85%) of participants enrolled in the trial completed the intervention and the post-intervention assessment.

Efficacy

Unadjusted scores for the performance-based and parent rating data at each timepoint are presented in Tables II and IV, along with between-group effect sizes, reported as Cohen’s d [48]. Our primary outcome measure was the Symbolic Working Memory Task from the WRAML2 (Figure 1a depicts group means adjusted for covariates). On this task, controlling for estimated IQ and baseline performance, scores for the adaptive group increased significantly compared with the nonadaptive (control) group (F = 4.57, p = .05), though this effect was no longer significant at the 3-month follow up (F = 3.65, p = .08). The effect size was large both immediately following treatment (Cohen’s d = 0.94) and at follow-up (d = 1.22). Moreover, over a third of participants in this group (36.3%) exceeded the RCI threshold for the measure versus none of the children in the nonadaptive arm. In contrast, none of the group differences for the secondary performance-based outcomes reached statistical significance. Of note, a medium effect size was observed for increases in scores on the visual attention task (WRAML2 Finger Windows; d = 0.67), but effect sizes for both verbal-based working memory/attention tasks were very small. With regard to ratings of functional impairment, there was a significant group difference in parent-reported Learning Problems on the Conners-3 (F = 4.65, p = .05), with participants in the adaptive group experiencing a greater improvement over those in the nonadaptive (control) arm at the post-intervention timepoint (d = 0.80; Figure 1b depicts adjusted group means). Moreover, 45% of children in the adaptive arm exhibited improvement consistent with the RCI versus one of the children in the nonadaptive arm. This effect was not maintained at the 3-month follow-up, however (d = 0.38). In addition, parent ratings on the Inattention scale did not differ significantly between groups, either at the post-intervention assessment (d = 0.40) or at the 3-month follow-up (d = 0.21). Broadly speaking, baseline estimated IQ scores were moderately correlated with the magnitude of change in performance-based outcomes following training. For example, children who had higher baseline IQ scores tended to show greater improvements in visual working memory following training (r = 0.45, p = 0.08), but there was no association between IQ and change in parent-rated learning problems (r = 0.00, NS).

Table IV.

General linear model analyses

	Post-Intervention				3-Month Follow-Up
	n	F	R2	p	n	F	R2	p
WRAML2
Finger Windows	17	2.06	.31	.18	15	0.31	.18	.59
Number Letter	17	0.32	.54	.58	15	0.02	.64	.90
Verbal Working Memory	17	0.14	.58	.71	15	0.21	.71	.65
Symbolic Working Memory	16	4.57	.85	.05	15	3.65	.65	.08
Conners-3 Parent Ratings
Inattention	17	1.83	.58	.20	15	1.40	.33	.26
Learning Problems	17	4.65	.72	.05	15	0.13	.50	.73

ANCOVAs, controlling for IQ and baseline scores

Discussion

Using a randomized, controlled design,we examined the feasibility and preliminary efficacy of a home-based, computerized cognitive training program with a small group of school-aged survivors of brain tumors and ALL. The majority of participants were fully compliant with the intervention. Participants and their families reported high levels of satisfaction with training, and no adverse events. Results also indicated that participants who were compliant with the adaptive training program exhibited significant improvement in visual working memory skills compared to those who completed the nonadaptive version of the program. Moreover, group differences in visual working memory skills persisted after 3-months without further training. There were also group differences in parent ratings of learning difficulties, such that participants who completed the adaptive training were rated as having greater improvements in functioning immediately post-treatment. Collectively, these results, and corresponding effect sizes, are consistent with previous reports of CogmedRM training as evaluated with children with ADHD and low working memory skills [27–31] suggesting that some neurocognitive deficits acquired by pediatric cancer survivors may be amenable to improvement to a similar degree as those associated with neurodevelopmental disorders. Of importance, however, transfer of training effects to other tasks and maintenance of gains over time were less robust in our sample than in prior ADHD studies, and this leaves open the question of the generalizability of this approach to real-world functioning, as discussed below.

Our findings provide preliminary support for the feasibility of using this home-based, computerized working memory training program. Compliance among participants in our study was quite consistent with the two previously published studies of computerized cognitive training with pediatric cancer survivors, and in general, exceeds rates of completion obtained by other intervention studies with this population involving clinic-based treatment [21, 22]. Specifically, clinic-based cognitive remediation approaches have been associated with completion rates of approximately 29% [22] to 60% [22]; in our study, 75% of participants completed all training sessions. Even when broader adherence criteria are considered, home-based training compares favorably with clinic-based approaches: 85% of participants in our sample completed at least 80% of the intervention, compared with 64% who completed at least 70% of the sessions in the clinic-based study conducted by Patel and colleagues [22] and 80% who attended three-quarters of the sessions in Butler and colleagues’ program [21]. Because survivors and their families often live at a substantial distance from regional medical centers where specialized neurorehabilitation services are offered, there may be significant financial and logistical barriers to attending weekly therapeutic sessions over an extended time period. As such, it is likely that the ability to complete cognitive training on a flexible, but frequent, schedule at home contributed to the high rate of compliance among our sample. In this way, home-based cognitive training programs like CogmedRM may be particularly advantageous for survivors who are geographically limited in their access to rehabilitative care. Even for participants who can readily obtain services locally, including home-based training as part of a comprehensive neurorehabilitation program may help to increase compliance and thereby improve treatment outcomes. However, it is noteworthy that our three noncompliant participants were all adolescent males. It is possible that these participants were less interested in completing the intervention because the computerized activities were not as visually stimulating as other video games that are played for recreational purposes, particularly among this demographic. Although further data are needed to explore this hypothesis, it clearly cannot be concluded from our data that all children and adolescents will participate in computerized training with the same level of compliance.

The largest and most stable differences between treatment groups in our study occurred for our primary outcome, visual working memory. Although the majority of exercises in CogmedRM train visual/visuo-spatial working memory, the Symbolic Working Memory task is very different than the training exercises in terms of the stimuli used and manner of responding. In this way, our data provide preliminary evidence of the near transfer of training to non-trained working memory tasks. It is important to note, however, that significant gains were not evident across most measures of neurocognitive functioning. In particular, there were no discernible effects of training on verbal working memory or attention tasks. Although the majority of tasks in the CogmedRM program are visual-spatial, some activities train auditory working memory. Thus, we would have expected some gains on these tasks, particularly for children who evidenced large increase in their performance on visual working memory tasks. Indeed, in previous trials of this program, gains have been made by participants on tasks of verbal working memory [27–31]. One possible reason we failed to see a training effect on verbal aspects of working memory could be our choice of task. In previous trials, significant changes in tasks of reverse digit span have been seen, and this task is more analogous to one of the CogmedRM activities than the Verbal Working Memory task from the WRAML2. It may also be that this population requires more intensive training in the auditory component of working memory than is provided in the standard CogmedRM training schedule, or that the verbal training exercises are targeting a different neuropsychological mechanism than that which underlies verbal working memory difficulties in pediatric cancer survivors [49].

It is important to note that we also utilized a limited battery of working memory and attention tasks in this pilot study. The effect sizes we observed were nevertheless fairly consistent (i.e., medium to large for near-transfer tasks) with what other researchers have obtained for similar measures using this treatment with other child clinical samples. It is therefore reasonable to suppose these effects may be present on a wider range of neurocognitive outcomes in a larger trial. In addition, although the relation between laboratory/performance-based measures of working memory and real-world functioning is not always robust [50], it is important that participants in the adaptive training group also experienced significant reductions in parent-rated learning problems immediately following treatment. These results should be interpreted cautiously in light of the possibility that parents in our study may have guessed their child’s treatment assignment and then consciously or unconsciously based their ratings on expectations for improvement (or lack thereof). Differences in parent ratings between the adaptive and non-adaptive groups were no longer significant at the 3-month follow up evaluation, lending support to this theory. However, the vast majority of parents in our study reported that they provided very little oversight of their child’s training, so it is reasonable to hypothesize that many parents may not have discerned the assignment. Moreover, our findings are consistent with the parent-rated improvements in attention that have been reported in trials of CogmedRM with other populations [27, 31]. To clarify these results, it will be critical for future work to obtain measures of everyday functioning from raters truly blind to the training condition. Teachers may be particularly salient informants in this respect, and their ratings may also be relevant to understanding any possible transfer effects to classroom performance in this population.

Importantly, participants’ estimated IQ significantly predicted improvement following training, such that children with higher intellectual functioning at baseline had larger training-related improvement. In studies of CogmedRM with other patient populations, intellectual functioning has also been a predictor of change, though children with intellectual disabilities still manifested working memory increases in at least one trial [38]. Because survivors of pediatric brain tumors are often at increased risk for declines in IQ scores [51], it may be that these children would benefit from a higher “dose” of training (i.e., more than 25 sessions) than is required for children with intellectual functioning that is comparable to typically-developing peers. Additionally, it will be important for future investigators to evaluate factors, potentially related to intellectual functioning, such as motivation and effort, which may impact adherence with and effectiveness of cognitive training. Finally, although improvements on tasks of visual working memory largely persisted over time for children who completed adaptive training, the magnitude of gains was attenuated to some extent, and (as noted above), parent ratings of improvement were no longer significantly different between treatment groups. This raises questions about the longevity of neurocognitive gains that can be expected from training with this population. In healthy children with developmental ADHD or working memory deficits, training gains have been largely maintained in previous trials [27, 30] over periods of up to 6 months. That we did not see conservation of gains to this extent suggests that our “dose” of training may have been insufficient, that we cannot expect the same level of benefit that this training has afforded to children with neurodevelopmental conditions, or that the neurocognitive/neurobiological mechanisms which underlie training-related improvement in other children are different for cancer survivors. Further trials are warranted to examine factors related to development and maintenance of training effects, and to seek to optimize the program for this unique group.

Limitations

While this study provided preliminary support for the feasibility of the CogmedRM intervention program with survivors of ALL and brain tumors, results must be interpreted in light of limitations. First, our sample size was small, with only 20 survivors randomized on the trial, and only 17 who completed training. However, as this study was designed primarily to evaluate the feasibility of this program, it was felt that this outcome could be maximized with a smaller number of participants while still allowing for the initial evaluation of efficacy as well. We also relied on traditional paper-and-pencil measures of working memory and attention to evaluate efficacy in our sample. While our primary outcome measure provided a psychometrically-solid assessment of the two main domains of interest (working memory and attention) it does not provide alternate forms. As such, it is possible that the improvements in scores were due to practice effects from repeated administrations of the measure. As the same measure and administration schedule was also used with the control group, it was felt that any practice effects would be accounted for by this comparison. Indeed, the magnitude of the difference in pre- and post-assessment was much greater in the intervention group as compared to the control group, thus suggesting that practice effects could not account for all the variance in scores. It will be important for additional evaluation of this intervention to include measures that are robust to practice effects, such as computerized assessment batteries.

Future Directions

Future trials of CogmedRM also should include a more comprehensive assessment of functional improvements associated with the intervention. While there were improvements in parent-reported attention-related learning problems over the course of the intervention, it will be important to determine how training affects everyday aspects of survivors’ life, such as academic performance and social functioning. Anecdotally, several parents reported improvements in their children’s school performance over the course of the trial; however, we did not include a measure to quantitatively capture these changes. Recent reports by Holmes and colleagues [29] have documented improvement in academics six months following completing of CogmedRM. Future studies should make a concerted effort to include assessment of academic achievement, as well as teacher report of attention and executive functioning, and perhaps social functioning as well.

It may also be of benefit to evaluate the feasibility and effectiveness of CogmedRM and other interventions as preventative tools. Specifically, it is possible that if children and adolescents who are at risk for neurocognitive deficits were to engage in these programs during treatment, that these programs may be able to delay or mitigate the development of neurocognitive late effects [52]. Moore and colleagues [53] tested the preliminary efficacy of one such program that targeted math skills in children newly diagnosed with ALL. The intervention included 40–50 hours of individualized instruction completed over a one-year period. Results revealed that children assigned to the intervention achieved gains in both mathematics ability and visual working memory, with gains generally maintained over a one-year follow-up, as compared to children randomized to standard care. While the results of this intervention are promising, the authors noted that the intensive, in-person nature of the intervention led to more attrition than was expected [53]. This limitation is similar to those described by other in-person interventions [21, 22] and again highlights the need for interventions such as computerized programs that allow more flexibility in delivery.

Conclusion

This report reflects the results of the first randomized trial of CogmedRM in survivors of ALL and brain tumors. Future work with this program should include a larger, randomized sample and focus on potential moderators of treatment effects, including diagnosis, baseline intellectual functioning, and optimal training “dose.” In keeping with recent findings suggesting changes in activity in the pre-frontal cortex following computerized cognitive training [24], future studies should also incorporate functional imaging outcomes when possible.

Figure 2.