Feasibility and acceptability of a remotely administered computerized intervention to address cognitive late effects among childhood cancer survivors

Abstract

Background

Childhood cancer survivors frequently develop working memory (WM) deficits as a result of disease and treatment. Medication-based and therapist-delivered interventions are promising but have limitations. Computerized interventions completed at home may be more appealing for survivors. We evaluated the feasibility and acceptability of a remotely administered, computerized WM intervention (Cogmed) for pediatric cancer survivors using a single-blind, randomized, wait-list control design.

Methods

Of 80 qualifying patients, 12 were excluded or declined to participate. Participants randomized to intervention (n = 34/68) included survivors of childhood brain tumors (32%) or acute lymphoblastic leukemia (ALL; 68%) between the ages of 8 and 16 years ($\overline{x}$ = 12.2) who were at least 1 year post therapy ($\overline{x}$ = 5.0). The majority of brain tumor participants were treated with cranial radiation therapy (72.7%), whereas most of the ALL participants were treated with chemotherapy only (87%). Participants completed 25 WM training sessions over 5–9 weeks at home with weekly phone-based coaching.

Results

Participants lived in 16 states. Compliance was strong, with 30 of the 34 participants (88%) completing intervention. Almost all participants completed pre- and postintervention neuroimaging exams (91% and 93%, respectively). Families had the necessary skills to utilize the computer program successfully. Caregivers reported they were generally able to find time to complete training (63%), viewed training as beneficial (70%), and would recommend this intervention to others (93%).

Conclusions

Cogmed is a feasible and acceptable intervention for childhood cancer survivors. It is a viable option for survivors who do not live in close proximity to cancer care centers. Efficacy and neural correlates of change are currently being evaluated.

Survivors of pediatric brain tumors and acute lymphoblastic leukemia (ALL) are at significant risk for neurocognitive late effects secondary to disease and treatment, with specific risk to attention and working memory (WM).^1–3 Attention and WM are thought to be foundational cognitive skills, such that deficits in these domains can have deleterious effects on social, vocational, and academic attainment.^4,5 Recent advances in treatment have resulted in improved survival with 5-year event-free survival rates of ∼90% among ALL populations.⁶ While some pediatric brain tumor diagnoses continue to be associated with poor prognosis, survival is >70% among a number of frequently diagnosed brain tumor populations.⁷ Improved survival rates^1,8 have led to an increased focus on survivorship and quality of life, including investigation of targeted methods for mitigating neurocognitive late effects.

Current options for remediating late effects include pharmacological interventions, therapist-delivered cognitive remediation, and computerized interventions. Psychostimulants have been used extensively and successfully to treat attention deficit/hyperactivity disorder (ADHD).⁹ Due to similarity of cognitive late effects with symptoms of ADHD, clinical trials have investigated the utility of psychostimulant medications for cancer populations.^10–12 Methylphenidate (MPH) trials have suggested that MPH is efficacious for improving attention and social skills in childhood cancer survivors with learning impairments, demonstrating both immediate^10,11 and long-term benefits.¹² While efficacious, drug trials have typically excluded individuals with medical contraindications (eg, history of uncontrolled seizures), rendering these patients inappropriate for pharmacological intervention.¹¹ Side effects in children treated with MPH are generally comparable with those in ADHD patients; however, there appears to be a subset of childhood cancer survivors who experience increased rates of adverse side effects.¹³ Additionally, childhood cancer survivors respond to MPH at a lower rate than the ADHD population (eg, 45% childhood cancer vs 75% ADHD).¹³ Finally, many parents are hesitant to put their child on a psychostimulant.¹¹ While pharmacological interventions show promise for many childhood cancer survivors, these findings suggest that nonpharmacological interventions are needed to address cognitive deficits in those who are not viable candidates for medication.

Butler and Copeland developed an intensive cognitive remediation program based on techniques from traumatic brain injury rehabilitation, special education, and clinical psychology,¹⁴ which involved 20 individual therapy sessions over 4–5 months. Survivors demonstrated improvement in academic and metacognitive skills, as well as parent ratings of attention, but no improvement on attention or memory performance measures.¹⁴ Other programs utilize similar strategies and/or focus on particular domains known to be at risk for survivors.^15,16 Support for therapist-directed cognitive remediation interventions includes modest academic benefit, involvement of parents and educators who help maintain gains, and lack of medical contraindications. However, these programs require intensive one-on-one intervention with a trained therapist, necessitating close proximity to specialized cancer care centers, while providing only modest training benefit. Ultimately, existing pharmacological and therapist-directed interventions are not tenable for all survivors and thus highlight the need for safe, portable, time-efficient, and efficacious interventions for this population.

Computerized interventions are software programs that involve massed practice, graded difficulty, and expert coaching. Home-based computerized training allows immediate feedback, customization for individual client needs/ability levels, ease of data collection for monitoring progress, and an engaging interface.¹⁷ Additionally, computerized interventions are not constrained by proximity to a specific facility, which may reduce treatment burden because they can be completed in the home, and have considerably fewer (if any) side effects compared with pharmacological intervention. Computerized training programs have demonstrated efficacy with a wide variety of populations including ADHD¹⁸ and traumatic brain injury.¹⁹

Currently, there are a few different computerized cognitive training programs on the market that have been tested for use by cancer survivors. Captain's Log, developed by Brain Train (www.braintrain.com), involves computer game-like activities that are designed to strengthen memory, attention, concentration, listening skills, self-control, patience, and processing speed. While initial results indicated efficacy in a small sample,²⁰ several aspects of the program, including an outdated presentation of graphics and lack of ease of administration (eg, dose standardization, progress monitoring), have limited its use. Lumos Laboratories created a Cognitive Rehabilitation Curriculum that specifically targets cognitive flexibility, WM, and attention.²¹ An initial single-arm trial with cancer survivors demonstrated moderate success with improvements in processing speed, cognitive flexibility, and visual and verbal memory but no improvement in attention or WM.²²

Cogmed, a software program created by neuroscientists and game developers at the Karolinska Institute, is designed to exercise different cognitive processes through a series of brain-training sessions. The program has been empirically tested in a wide variety of populations, including children with ADHD,^21,22 with notable efficacy. Cogmed specifically targets WM, a domain frequently affected in survivors of pediatric cancer, which makes it particularly well suited for this population.^23,24 Hardy et al²⁵ conducted a small pilot study of Cogmed with a sample of 20 childhood cancer survivors who exhibited deficits in attention and WM. Participants were randomized to the adaptive computerized intervention or to a nonadaptive, active control group with brief cognitive assessments completed before and after intervention. Results showed that 85% of participants were compliant with the intervention with no adverse events reported. Preliminary findings suggest postintervention gains in visual WM but call for more rigorous examination in a larger sample.²⁵

In sum, limited research has demonstrated that computerized interventions can reduce treatment burden and the need for proximity to a facility when compared with therapist-delivered cognitive remediation interventions, and result in a much more desirable side-effect profile than pharmacological interventions. Computerized interventions such as Cogmed appear to be particularly well suited for clinical practice. However, past studies were conducted with small sample sizes, using mostly local patient populations with a limited range of socioeconomic statuses, and have not included neuroimaging to investigate early neural correlates of change. With these omissions in mind, the current study aimed to (i) replicate the preliminary feasibility and acceptability demonstrated by Hardy et al²⁵ with a nonlocal sample that was also powered for efficacy and (ii) investigate the feasibility of pre- and postintervention neuroimaging (eg, functional MRI) with a pediatric population. Separate reports with efficacy and neuroimaging findings are forthcoming.

Materials and Methods

Participants

Survivors of brain tumors or ALL treated at St. Jude Children's Research Hospital (SJCRH) with CNS-directed therapy (ie, intrathecal chemotherapy and/or cranial radiation therapy) were recruited to study the efficacy of a remotely administered, computerized intervention targeting late neurocognitive effects. All participants were between 8 and 16 years of age, spoke English as a primary language, and had been off treatment for at least one year with no evidence of recurrent disease. Brain tumor survivors were required to have an infratentorial tumor location for neuroimaging purposes and included individuals diagnosed with medulloblastoma/primitive neuroectodermal tumor, ependymoma, or glioma treated with cranial radiation therapy with or without chemotherapy. ALL patients received systemic and intrathecal chemotherapy. Individuals were excluded from participation if they had significant impairment in global intellectual functioning (operationalized as IQ <70) demonstrated via standardized testing routinely conducted on primary treatment protocols. Additionally, individuals were excluded if they had a history of CNS injury/disease or ADHD predating cancer diagnosis, treatment with psychostimulant or psychotropic medication within 2 weeks of study participation, or major sensory or motor impairment that would preclude valid testing or intervention completion (eg, significant bilateral paresis or ataxia, blindness, or photosensitive seizures). Recruited participants were contacted in the order of their upcoming medical appointments.

Qualification for the intervention phase was based on WM impairment, operationalized as an age-scaled score on the Digit Span, Letter-Number Sequencing, or Spatial Span subtests (Wechsler Intelligence Scale for Children-IV²⁶) at least one standard deviation below the normative mean (SS = 10) or their IQ (Wechsler Abbreviated Scale of Intelligence²⁷). Participants also had to complete neuroimaging procedures without sedation (eg, no orthodontic appliances, tattoos, or known claustrophobia). Those with a major psychological condition that would preclude or take precedence over study participation (eg, significant oppositionality, autism spectrum disorder, severe mood disorder) were excluded from intervention. Qualifying participants (n = 68) were randomly assigned to the Cogmed intervention group or a wait-list control group. Randomization was stratified based on diagnosis (ALL/brain tumor), age (8–11 y, 12–16 y), and sex. Figure 1 presents a consort diagram detailing participant flow. This study was approved by the Institutional Review Board, written informed consent was obtained prior to participation, and the trial was registered on ClinicalTrials.gov (NCT01217996).

Figure 1.

Consort diagram. WM, working memory. ^†Completed visit 1 fMRI, n = 31; One participant supplied partial preintervention fMRI data due to fatigue. ^¥Completed visit 2 fMRI, n = 28.

Procedures

At their first visit, participants took part in the screening/preintervention assessment to determine qualification and, as appropriate, conduct randomization procedures. The intervention group completed a preintervention neuroimaging exam on the same day as the screening/preintervention cognitive assessment. Intervention participants were trained on intervention procedures and provided with instructions on how to use Cogmed. Computers and/or Internet access were provided as needed.

Approximately 10 weeks later, the intervention participants returned to SJCRH for their second visit, where they completed a postintervention cognitive assessment and neuroimaging exam. Control participants also returned at this time to complete the cognitive assessment. Participants returned again to SJCRH 6 months following their intervention training period to take part in a final cognitive assessment. While not a crossover design, controls were offered participation in the intervention off-study at this time point.

Incentives were offered to encourage motivation and continued participation with training sessions. With commercial use, Cogmed recommends the use of a rewards program to families. Both intervention and control groups received equal incentives so as not to introduce motivational differences and still adhere to clinical practice. Survivors who completed the initial screening assessment but did not qualify for the study were compensated $10 for their time. Participants received $10 gift cards after completing 9, 17, and 25 training sessions and were awarded $10 gift cards after completing pre-, post-, and 6-month follow-up appointments.

Computerized Training

Cogmed (www.Cogmed.com) was the primary cognitive intervention in the current study. Intervention participation involved 25 sessions completed over the course of 5–9 weeks. Each session required ∼30–45 minutes. Training consisted of rotating exercises, presented as games, designed to train visual-spatial and verbal WM. Cogmed tasks are adaptive, such that the difficulty level increases or decreases based on performance. The intervention was facilitated online to allow progress monitoring over the Internet. Weekly coaching calls provided participants with information about training progress and allowed study staff to monitor motivation and respond to any concerns. Intervention participants who made less progress than desired (operationalized as training index score <20 after 20 training sessions) were offered 5 additional training sessions.

Neuroimaging

Neuroimaging examinations were conducted before and immediately following Cogmed intervention. Exams included both functional magnetic resonance imaging (fMRI) as well as diffusion tensor imaging. A short PowerPoint presentation provided information about scanning procedures and pictures of the MRI machine. Participants were given the opportunity to become familiar with tasks used during data collection and the pneumatic squeeze ball response mechanism. All participants underwent conventional imaging to identify morphological abnormalities, to facilitate spatial normalization of brain images, and to visualize functional imaging results. A grid-based task assessing spatial WM by Olesen et al,²⁸ as well as the classic n-back task,²⁹ were utilized during fMRI. In sum, the neuroimaging exam was completed within ∼1 hour.

Measures

Intelligence Testing

Each participant was administered the Vocabulary and Matrix Reasoning subtests of the Wechsler Abbreviated Scales of Intelligence (WASI).²⁷ These subtests allow the computation of an age-standardized abbreviated IQ with a mean of 100 and a standard deviation of 15. Abbreviated IQ, as measured by the WASI, is highly correlated with the full Wechsler intelligence scales.^26,30

Computer Literacy Questionnaire

Parents of each participant completed a computer use questionnaire developed by the research team to assess familiarity with computer operating systems. This measure required ∼5 minutes to complete. Participants indicated computer ownership, Internet access, hours per week spent on a computer, and typical activities performed on a computer (eg, e-mail, social networking, work, homework, games, music, and videos). Additionally, 12 items assessed comfort level with different computer activities. Items were rated on a 5-point Likert-type scale with anchors ranging from 1 (not at all comfortable) to 5 (very comfortable) and referenced activities including use of the mouse/keyboard, use of Windows operating system, installing software, saving data, and e-mail usage. For this measure, questions were completed based on the “best user” in the family. For some, this was a parent or caregiver; for others, this was the patient.

Satisfaction Questionnaire

Caregivers of each participant completed the parent satisfaction questionnaire, created by the researchers, to assess their overall satisfaction with the intervention. A 5-point Likert-type scale, with anchors ranging from 1 (strongly disagree) to 5 (strongly agree) was used to evaluate agreement with statements regarding different aspects of the intervention, including how well the program worked, utility of weekly coaching phone calls, child agreeableness to training, and other relevant domains. Short answer items inquiring about ways caregivers mitigated resistance from their child (if encountered) were included. Child participants completed a similar but shorter questionnaire (participant satisfaction questionnaire). Items addressed enjoyment of the intervention, attention to the program, and ease of session completion. Both versions assessed treatment satisfaction. Several items on the participant report mirrored items on the parent report to allow direct comparison. Questionnaires were completed by parents and participants midway through and upon completion of the intervention.

Statistical Considerations

Descriptive statistics for demographic and clinical variables were calculated to characterize patient groups. Percentages were calculated to determine overall level of agreement with items on the computer use questionnaire and on the end-point parent and participant satisfaction questionnaires. T-tests were used to evaluate differences in responding on the satisfaction questionnaires from mid- to endpoint. Similarity in responding between parents and participants (ie, interrater agreement) was examined using the weighted kappa statistic.³¹

Results

Table 1 presents clinical and demographic information for the intervention and control groups. The majority of brain tumor participants were treated with cranial radiation therapy (72.7%), whereas most of the ALL participants were treated with chemotherapy only (87%). None of the ALL participants received cranial radiation therapy. The majority (n = 42) of ALL participants received combined intrathecal and systemic chemotherapy only, and the remainder (n = 5) received a combination of chemotherapy and bone marrow transplant (pretransplant conditioning included total body irradiation [12 Gy] for 3 of these participants). Consistent with stratification procedures, intervention and control groups were matched by sex, age, and diagnosis. There were no significant differences between groups in socioeconomic status, mean age at diagnosis, time since treatment, or treatment intensity. However, the intervention group trended towards a higher baseline IQ ($\overline{x}$ = 106.9) than the control group ($\overline{x}$ = 99.8; P = .06). Participants completed the intervention while inhabiting 16 states across the United States (including states on both coasts and several in the southeastern United States; Fig. 2).

Table 1.

Participant characteristics

			Intervention	Control	P value
Demographic	Sex	Female	16 (47%)	16 (47%)	1.00
		Male	18 (53%)	18 (53%)
	Race	African American	1 (3%)	5 (15%)	.39
		Asian/Pacific Islander	1 (3%)	1 (3%)
		Caucasian	27 (79%)	26 (76%)
		Hispanic	2 (6%)	1 (3%)
		Other/multiple races	3 (9%)	1 (3%)
	SES (BSMSS)*		39.68 ± 15.37	40.46 ± 12.20	.82
Clinical	Acute lymphoblastic leukemia		23 (68%)	24 (71%)	1.00
	Brain tumor		11 (32%)	10 (29%)
	Mean age at diagnosis, y		5.15 ± 2.92	4.62 ± 2.68	.43
	Mean age at enrollment. y		12.21 ± 2.47	11.82 ± 2.42	.51
	Mean time since treatment		4.97 ± 3.02	5.04 ± 2.41	.91
	Brain tumor group	Ependymoma	1 (9%)	3 (30%)	.33
		Glioma	2 (18%)	0 (0%)
		Medulloblastoma/PNET	8 (73%)	7 (70%)
	Treatment intensity#	Chemo only	20 (59%)	22 (65%)	.95
		CSI w/ or w/o chemo	8 (24%)	7 (21%)
		CRT w/ or w/o chemo	3 (9%)	3 (9%)
		Chemo + BMT w/ or w/o TBI	3 (9%)	2 (6%)
	Mean Baseline IQ	106.90 ± 15.74	99.85 ± 14.01	0.06

Abbreviations: BMT, bone marrow transplant; chemo, chemotherapy; CSI, craniospinal irradiation; CRT, conformal radiation therapy; PNET, primitive neuroectodermal tumor; SES, socioeconomic status; TBI, total body irradiation; w/ or w/o, with or without.

*Barrett Simplified Measure of Social Status (BSMSS). Derived from maternal and paternal education and occupation; scores range from 8 to 66 with high being indicative of higher SES.

^#No child with ALL received CSI or CRT; ALL patients received systemic and intrathecal chemotherapy.

Figure 2.

Geographical distribution of Cogmed training. SJCRH, St. Jude Children's Research Hospital.

Compliance

Of those who qualified for inclusion in the study, an equal number (n = 34) were randomized to intervention and control groups. Intervention training session compliance was strong, with 88% of participants completing the required training, operationalized as completion of at least 20 sessions (based on previous reports of Cogmed^21,22,25). Of those who did not complete training (12%), retrospective analysis revealed that exactly one-half participated for altruistic reasons (rather than perceived personal benefit) and the remaining half experienced significant family situations that interfered with participation (eg, estrangement between divorced parents with shared custody). Although the small number of noncompliers precluded parametric analyses, there were no apparent demographic or clinical differences between compliant and noncompliant participants. Of those who complied, participants completed an average of 26 sessions (± 2.39; range = 21–30 sessions) and spent 47 days training (± 12.48; range = 31–71 days). Active training time averaged 38 minutes (± 5.71, range = 29.16–48.48 min), and time spent paused (ie, on a break) during training sessions averaged 16 minutes (± 19.88, range = 1.00–91.42 min). Participants with active or paused training times >2 standard deviations above the mean were considered outliers and were removed from analysis (active n = 2; pause n = 1).

At the 10-week follow-up, all participants who completed the intervention provided postintervention assessment data. Most control participants (94%) also took part in the 10-week assessment. All 30 intervention compliers completed the 6-month follow-up assessment, as did 30 of the 34 control participants. Several participants randomized to the control group initiated the intervention off-study (n = 23); 14 completed training, with most returning to complete an additional cognitive assessment (n = 11).

Computer Literacy

The majority of intervention participants had their own computer (97%) and Internet connection (90%). However, laptops and/or Internet cards were provided to those who requested them (laptops n = 13; Internet n = 7). Thirteen families (43%) reported spending 11 or more hours on computers per week. Families reported that their daily computer use consisted of activities such as computer games (79%), e-mail/social networking (66%), and completion of work/homework (66%). Table 2 provides information about comfort with using a computer. A high degree of familiarity and comfort with use of computers was reported by the overwhelming majority of participants and their families. Responses indicated that families had the skills necessary for general computer use (97%), including turning it on and off (93%), using a mouse and keyboard (100%), using software applications (93%), and connecting to the Internet (90%). Families who did not report comfort and familiarity with computers were provided instruction as needed to ensure they could access the intervention program appropriately.

Table 2.

Computer literacy questionnaire

	Uncomfortable* (1 and 2) n (%)	Neutral (3) n (%)	Comfortable (4 and 5) n (%)	Mean ± SD
General use of computer		1 (3%)	29 (97%)	4.73 ± 0.52
Safely turn computer on and off		2 (7%)	28 (93%)	4.87 ± 0.51
Use mouse			30 (100%)	4.90 ± 0.31
Use keyboard			30 (100%)	4.83 ± 0.38
Use Windows operating system	6 (20%)	1 (3%)	23 (77%)	4.17 ± 1.37
Save data to hard drive/portable media	5 (17%)	2 (7%)	23 (77%)	4.03 ± 1.43
Install software	5 (17%)	2 (7%)	23 (77%)	4.10 ± 1.37
Start and close software application		2 (7%)	28 (93%)	4.73 ± 0.58
Connect to Internet	1 (3%)	2 (7%)	27 (90%)	4.70 ± 0.88
Use e-mail to open and read messages	4 (13%)	1 (3%)	25 (83%)	4.30 ± 1.39
Use e-mail to create and send message	4 (13%)	2 (7%)	24 (80%)	4.27 ± 1.41
Upload files to Internet or software	4 (13%)	5 (17%)	21 (70%)	4.00 ± 1.34

*1 = Not at all comfortable; 2 = Not very comfortable; 3 = Neutral; 4 = Somewhat comfortable; 5 = Very comfortable.

Neuroimaging

Participants randomized to intervention received fMRI assessments during the screening and at immediate postintervention appointments. Efforts were made to ensure that fMRI appointments were conducted during routinely scheduled medical visits (as were all study-related appointments). Of those randomized to intervention, 91% completed preintervention fMRI examinations. Preintervention fMRI data were also provided by 3 of the 4 participants who were taken off the study. At postintervention assessment, 93% of participants provided fMRI data. One participant supplied partial preintervention fMRI data only due to fatigue, although these data were usable and were included in analyses. Two participants completed the intervention but did not provide pre- or post-fMRI data due to claustrophobia. No data were excluded for motion artifact or poor performance.

Acceptability

Table 3 presents parent satisfaction questionnaire items and responses. Results generally show that parents were satisfied with the intervention. Parents agreed that the computer program worked and that instructions were helpful and easy to understand. More than half of parents agreed that finding time to complete training sessions was easy (63%). Parents reported that their child was agreeable to completing sessions (63%), although it appears that enthusiasm declined over the course of training (t(19) = 2.99, P = .008). About half of parents agreed that their child enjoyed the training program (47%), although considerably fewer felt their child enjoyed it as much as other video games (28%). Approximately half of parents agreed that their child was not bored during training sessions (50%), although boredom increased as the intervention went on (t(19) = 3.20, P = .005). All parents reported that their child was able to complete training sessions independently (100%). Finally, most parents agreed that their child benefitted from the intervention (70%) and that they would recommend this study to other parents (93%).

Table 3.

Parent satisfaction questionnaire

	Disagree* (1 and 2) n (%)	Neutral (3) n (%)	Agree (4 and 5) n (%)	Mean ± SD
The Cogmed computer program worked each time it was used	2 (7%)	1 (3%)	27 (90%)	4.30 ± 0.84
The instructions provided to me were helpful and easy to understand			30 (100%)	4.73 ± 0.54
The weekly telephone calls accurately addressed any difficulty my child was having			30 (100%)	4.77 ± 0.43
The weekly telephone calls provided useful tips to improve my child's performance			30 (100%)	4.80 ± 0.41
I think sending my child's progress weekly was a good idea		4 (13%)	26 (87%)	4.50 ± 0.73
Scheduling time for my child to complete the daily sessions was easy	6 (20%)	5 (17%)	19 (63%)	3.67 ± 1.16
My child was agreeable to completing the sessions	4 (13%)	7 (23%)	19 (63%)	3.70 ± 0.95
My child enjoyed this training program	4 (13%)	12 (40%)	14 (47%)	3.47 ± 0.90
My child enjoyed this as much as other video games he/she usually plays	12 (41%)	9 (31%)	8 (28%)	2.79 ± 0.94
My child was not easily bored during the sessions	4 (13%)	11 (37%)	15 (50%)	3.40 ± 0.89
My child did not get frustrated during the sessions	11 (37%)	8 (27%)	11 (37%)	3.03 ± 1.03
My child was able to complete the sessions independently			30 (100%)	4.53 ± 0.51
My child looked forward to playing the racing game at the end of the session	3 (10%)	7 (24%)	19 (66%)	3.93 ± 1.03
The gift card provided motivation for my child to complete the activities	2 (7%)	4 (13%)	24 (80%)	4.27 ± 0.94
I was able to upload the information to the Internet each week	1 (4%)	2 (7%)	25 (89%)	4.39 ± 0.79
I noticed a change in my child during this study	1 (3%)	13 (43%)	16 (53%)	3.60 ± 0.86
Other people (eg, teachers) noticed a change in my child during this study		22 (73%)	8 (27%)	3.33 ± 0.61
My child's grades improved during this study	2 (7%)	23 (79%)	4 (14%)	3.07 ± 0.46
My child benefitted directly from this study		9 (30%)	21 (70%)	3.83 ± 0.66
I would recommend this study to other parents		2 (7%)	28 (93%)	4.33 ± 0.61

*1 = Strongly disagree; 2 = Disagree; 3 = Neutral; 4 = Agree; 5 = Strongly agree.

Table 4 presents participant satisfaction questionnaire items and responses. Participants were able to understand the rules of the games during training sessions (93%). Half of participants agreed that it was easy to find time to complete training sessions (50%). Approximately one-third reported that they enjoyed the training sessions (37%); however, a sizable minority (17%) did not enjoy training. Regardless, the vast majority of participants did not enjoy training games as much as other video games (70%), and it appears that enjoyment diminished as the intervention went on (t(25) = 2.27, P = .03). Slightly fewer than half of participants agreed that training sessions kept their attention (43%). However, almost all participants reported that they were able to complete training sessions without help from their parents (93%).

Table 4.

Child satisfaction questionnaire

	Disagree* (1 and 2) n (%)	Neutral (3) n (%)	Agree (4 and 5) n (%)	Mean ± SD
I understood the rules of the games		2 (7%)	28 (93%)	4.53 ± 0.63
It was easy to find time to complete my daily sessions	6 (20%)	9 (30%)	15 (50%)	3.37 ± 0.89
I rarely complained when completing the sessions	6 (20%)	13 (43%)	11 (37%)	3.23 ± 0.97
I enjoyed these games	5 (17%)	14 (47%)	11 (37%)	3.17 ± 1.12
I enjoyed these games as much as other video games I usually play	21 (70%)	4 (13%)	5 (17%)	2.17 ± 1.05
The sessions kept my attention	3 (10%)	14 (47%)	13 (43%)	3.43 ± 0.82
I was able to complete the sessions without help from my parent		2 (7%)	28 (93%)	4.70 ± 0.60
I looked forward to playing the racing game at the end of the session	6 (20%)	10 (33%)	14 (47%)	3.47 ± 1.31
The gift card motivated me to complete the activities	3 (10%)	10 (35%)	16 (55%)	3.69 ± 1.07
These games helped me to do better work at school	2 (7%)	13 (45%)	14 (48%)	3.45 ± 0.83
I think other children my age would like being in this study	9 (31%)	12 (41%)	8 (28%)	2.93 ± 1.16

*1 = Strongly disagree; 2 = Disagree; 3 = Neutral; 4 = Agree; 5 = Strongly agree.

Interrater Reliability

The parent and participant versions of the satisfaction questionnaire were specifically created to have overlapping items so that reliability between parent and participant reporting could be investigated. Cohen's weighted kappa statistic was computed for each of the 10 overlapping items.³¹ Kappas ranged from 0.07 to 0.40 ($\overline{x}$ = 0.23), indicating slight to fair agreement³² between parent/participant dyads. Statistically significant differences between parent and participant reporting were found on 4 survey items including enthusiasm for training, enjoyment of training games as much as other video games, and recommendation of training program to others (P < .05), with parents generally more favorable than participants. However, the participants endorsed school-related improvements more favorably than did their parents.

Discussion

Our study demonstrated the feasibility of a remotely administered computerized intervention to mitigate neurocognitive late effects in survivors of childhood cancer using a single-blind, randomized control design. Compliance with study-related procedures (assessments, intervention training sessions, and neuroimaging) was strong. The vast majority of intervention participants completed training sessions as prescribed (88%), and 14 of the control participants completed the intervention off-study. All intervention participants and almost all control participants returned for their postintervention cognitive assessments. Similarly, almost all intervention participants provided pre- and postintervention neuroimaging data (93%).

The computerized, remotely administered nature of Cogmed requires that participants and caregivers have basic computer proficiency. We generally found this to be the case in our sample. There were no significant technological problems that interfered with use of Cogmed. The online format of Cogmed enabled centralized coaching and monitoring, streamlined intervention administration, and allowed multiple families to participate simultaneously. Further, weekly coaching calls helped maintain motivation and compliance. Finally, Cogmed allowed families substantial independence in deciding when to complete training sessions. Families were able to choose on a day-to-day basis how to best fit training into their already busy schedules, although within set parameters (ie, complete individual session within one day and complete training within 5–9 weeks). This reduced family burden considerably and likely makes Cogmed a more feasible option for survivors than in-person, therapist-administered cognitive remediation programs. Moreover, these features make Cogmed particularly amenable for dissemination via clinical practice.

Overall compliance results were commensurate with those of Hardy et al²⁵ and reports from the ADHD literature,^21,22 lending further support to the feasibility of Cogmed for childhood cancer survivors. Compared with therapist-directed cognitive interventions,^14–16 compliance and feasibility appeaed to be much improved. Four participants were removed for noncompliance; 2 had adverse family situations that precluded meaningful participation in training sessions. Further analysis revealed that the remaining 2 participated for altruistic reasons rather than perceived benefit, suggesting that perhaps altruism alone is not a strong enough motivator for compliance. Subsequent participants were informed that “wanting to give back” to the institution was appreciated but was not sufficient, and only those seeking potential individual benefit from training for identified attention and/or WM difficulties should take part. It is possible that screening out those with purely altruistic motivations helped bolster compliance. Several participants (n = 14) originally randomized to the control group elected to complete Cogmed training off-study. Most of these participants (n = 11) returned to SJCRH for post-wait list cognitive testing to determine benefit. While participants remained enthusiastic about the possible benefits of Cogmed training, participation rates might be lower than in a crossover design, given that all patients already participated in monitoring and assessments as controls.

Participants took part in the intervention while inhabiting 16 states across America. Given that many survivors do not live in close proximity to specialized cancer centers, it is quite promising that Cogmed would be a feasible option for survivors who do not have the option of frequent face-to-face meetings with specialized therapists. While remotely administered computerized interventions provide ease of administration, the removal of face-to-face weekly therapy sessions may make it more difficult for providers to keep in touch with their patients. However, Cogmed allows providers to monitor progress online by giving detailed information about training progress. Weekly phone calls for progress monitoring required ∼15–20 minutes (well within the parameters of a routine therapeutic relationship), which further supports the capability of disseminating Cogmed to this population. In this way, Cogmed appears to be uniquely positioned for clinical practice and represents a portable and convenient option for remediating neurocognitive late effects, which practitioners can provide online and thus extend the reach of their clinical practice. Geography is particularly relevant when we take into account that many prior interventions have had limited success due to family reluctance to return to their home hospital at frequent intervals. Moreover, the socioeconomic range of our sample is broader than previous reports,²⁰ providing support for use of this intervention with families of diverse socioeconomic statuses.

Limitations of the current study include a mixed diagnostic group comprising both ALL and brain tumor survivors, with multiple tumor types represented. Although this group is representative of the majority of survivors, it will be essential to explore each diagnostic category separately and evaluate the possibility of different outcomes. The current study did not include any formal measure of adverse events or side effects. None were reported, although this was not explicitly measured. Our evaluation of computer skills was a brief summary measure and not an objective behavioral assessment. It would also be interesting to evaluate alternative ways to motivate compliance. The use of gift cards seemed appropriate for this study, but it may be helpful to consider other less costly means for future use by clinicians. Finally, our study did not include a comparison group of individuals treated with stimulant medication. This may be an interesting point of comparison in future studies.

Overall, Cogmed appears to be a feasible and acceptable intervention for the remediation of neurocognitive late effects in survivors. Future endeavors should evaluate efficacy, maintenance and generalization of effects, and neuroimaging changes that might be indicative of training-related neuroplasticity. Furthermore, subsequent research should consider the timing of intervention, including prophylactic administration during active treatment, and possible combination of interventions that might improve patient outcomes. Multiple prophylactic trials are currently underway for childhood cancer survivors.

Funding

This work was supported in part by the National Cancer Institute (St. Jude Cancer Center Support [CORE] Grant [P30 CA21765]); the American Cancer Society (H.C., RSGPB-11-009-01-CPPB); and the American Lebanese Syrian Associated Charities (ALSAC). Portions of this paper were presented at the annual meeting of the International Neuropsychological Society in Seattle, Washington, 2014.