Exploring N-Back Cognitive Training for Children With ADHD

Abstract

The efficacy of n-back training for children with Attention Deficit Hyperactivity Disorder (ADHD) was tested in a randomized controlled trial. Forty-one children aged 7–14 with ADHD trained on an n-back task, and their performance was compared with that of an active control group (n = 39) who trained on a general knowledge and vocabulary task. The experimental group demonstrated transfer of training to a non-trained n-back task as well as to a measure of inhibitory control. These effects were correlated with the magnitude of training gains. Our results suggest that n-back training may be useful in addressing some of the cognitive and behavioral issues associated with ADHD.

Attention Deficit Hyperactivity Disorder (ADHD) is a developmental disorder characterized by inattention, hyperactivity, and impulsive behavior. It is common; the National Survey of Children’s Health found that in 2011, 11% of US children ages 4–17 (i.e., 6.4 million) had ever been diagnosed with ADHD and 9% of children had a current diagnosis of ADHD (Visser et al., 2014) though prevalence rates vary across studies and hover around 6% in international samples (Moffitt et al., 2015; Polanczyk, Willcutt, Salum, Kieling, & Rohde, 2014). ADHD symptoms typically arise early in life (median age of diagnosis is 6 years; Visser et al., 2014), generally continue through adolescence, and persist into adulthood for 30–60% of individuals diagnosed (Barkley, 1997; Barkley, Fischer, Smallish, & Fletcher, 2002; Faraone, Biederman, & Mick, 2005; Matte et al., 2015; Moffit et al., 2015). Approximately 80% of children with ADHD have academic performance problems and have a higher incidence of grade retention and dropping out of school compared to children without ADHD (DuPaul & Stoner, 2014). As a result, ADHD often contributes to academic, social, and employment difficulties throughout the lifespan (Barkley et al., 2002). Given the prevalence of ADHD, the chances are high that classrooms with 20 or more students include at least one individual with ADHD (Abikoff et al., 2002). Because children with ADHD can be disruptive, learning of all students in the classroom can be affected (August et al., 1996). Thus there is a strong, practical need for the development of effective interventions to mitigate the symptoms and consequences of ADHD.

Though there are many facets to ADHD, the present study focuses primarily on working memory, inhibition, and the ADHD behaviors associated with those executive functions. Children with ADHD show impairments on a variety of working memory-related and executive control tasks. On average, children with ADHD perform worse than typically developing children on standard working memory tasks (Karatekin & Asarnow, 1998; McInnes, Humphries, Hogg-Johnson, & Tannock, 2003; Sonuga-Barke, Dalen, Daley, & Remington, 2002; Westerberg, Hirvikoski, Forssberg, & Klingberg, 2004), and a number of studies have documented impairments among children with ADHD in executive function tasks, including the Wisconsin Card Sorting Test (Pineda et al., 1998; Reeve & Schandler, 2001), verbal fluency tasks (Pineda et al., 1998), the Stroop task (Reeve & Schandler, 2001), task-switching (Cepeda, Cepeda, & Kramer, 2000; White & Shah, 2006), and the Tower of London task (Cornoldi, Barbieri, Gaiani, & Zocchi, 1999). This is likely due to lower working memory capacity and reduced cognitive control in individuals with ADHD. As inhibitory and working memory skills are implicated in scholastic and non-scholastic achievement (Friso-Van Den Bos, Van Der Ven, Kroesbergen, & Van Luit, 2013; Mcvay & Kane, 2012; St Clair-Thompson & Gathercole, 2006), interventions specifically targeting these mechanisms could be of great benefit for children with ADHD.

Interventions for ADHD

There are various approaches for ameliorating the challenges associated with ADHD; the most common method is stimulant medication. There are also non-pharmacological strategies such as behavioral interventions and cognitive training; some of these methods are often used in combination with stimulant medication.

Medication.

Although medication can be of great help, they are ineffective for up to 30% of individuals with ADHD (Banaschewski, Roessner, Dittmann, Santosh, & Rothenberger, 2004). Furthermore, though ADHD medications are often seemingly effective in laboratory contexts, they seem to have less impact on everyday functional outcomes (Pelham et al., 2017). In addition, they are often associated with side effects such as reduced appetite and sleep disturbances (Banaschewski et al., 2004). For these and other reasons, approximately 20% of individuals with ADHD cease stimulant use within the first year of taking them (Toomey, Sox, Rusinak, & Finkelstein, 2012). It is not our aim to report a thorough overview on this topic, but interested readers may refer to Rubia et al. (2014) for a meta-analysis.

Behavioral Therapy.

The most widely used non-pharmacological interventions for children with ADHD is behavioral management therapy (Evans, Owens, & Bunford, 2013; Wolraich et al., 2011). In these interventions, teachers (behavioral classroom management), or parents (behavioral parent therapy) are taught to reinforce children when they perform a desired behavior and to ignore undesired behaviors. This approach helps children to learn self-regulation skills, with typical outcomes including improved compliance to parent and teacher directions, as well as decreased disruptive behavior in the classroom. While behavioral therapies are generally considered the most effective non-pharmacological approaches for addressing ADHD symptomatology (Evans et al., 2013; Fabiano et al., 2009), many issues prevent these interventions from being available and effective for all children. Behavioral therapies can incur significant financial and time investments. Furthermore, new meta-analytic work suggests that although behavioral therapies may be successful in individual studies, a variety of factors related to both the design of the intervention and individual difference factors across participants may limit their effectiveness (Sonuga-Barke et al., 2013).

Cognitive Training.

Another approach to treat ADHD is the use of “cognitive training” programs aimed at the core executive function deficits associated with ADHD. This approach is attractive in that, if effective, it may be lower in cost in terms of time, money, and potential side effects. Indeed, a recent meta-analysis of cognitive training (Cortese et al., 2015) and non-pharmacological interventions for ADHD more broadly (Sonuga-Barke et al., 2013), cognitive training was identified as an area that requires further exploration. As such, our study contributes to the growing literature on cognitive training interventions for individuals with ADHD. Currently, the most widespread and highly researched cognitive intervention for ADHD is Cogmed working memory training (CWMT), an online working memory training program that targets primarily the storage aspects of both verbal and visuospatial working memory. It is marketed to schools and clinicians as a tool for improving cognitive abilities, such as attention and reasoning (Roberts et al., 2016). Some studies of CWMT in children with ADHD have shown improvements in ADHD-related symptoms as a function of training (Beck, Hanson, Puffenberger, Benninger, & Benninger, 2010; Klingberg et al., 2005; Klingberg, Forssberg, & Westerberg, 2002; Mezzacappa & Buckner, 2010), and a recent study by Bigorra, Garolera, Guijarro, and Hervas (2016) demonstrated reductions in inhibitory control, as measured by CPT errors of commission. However, others have failed to replicate those findings, leading to conflicting reviews and meta-analyses (Chacko et al., 2013; van der Donk, Hiemstra-Beernink, Tjeenk-Kalff, van der Leij, & Lindauer, 2015; Shipstead, Hicks, & Engle, 2012). Additional challenges regarding CWMT include lengthy sessions (approximately 40 minutes per day over the course of 5 weeks), the necessity of a coach, and the significant cost (Chacko et al., 2014).

Other cognitive training work has attempted to target inhibition directly. Shavlev, Tsal, & Mevorach (2002), found improvements in attention and academic outcomes following an attentional control training program. Conversely, a recent large, randomized controlled trial of inhibitory control training found no evidence for training and transfer effects (Enge et al., 2014). Although some of the evidence is mixed, it may remain possible to train inhibition, especially if it occurs in combination with training working memory, as there seems to be considerable overlap between the domains of working memory and inhibitory control (Hsu, Jaeggi, & Novick, 2017). Specifically, we have demonstrated transfer to inhibition via an inhibition and working memory-demanding executive functioning task, the n-back task, in young adults (Hsu, Buschkuehl, Jonides, & Jaeggi, 2013) and typically developing children (Jaeggi, Buschkuehl, Jonides, & Shah, 2011). The n-back task requires participants to respond to a series of stimuli by judging whether each stimulus is the same as the one presented n-items previously. The n-back task requires both active maintenance and updating of items in working memory as well as deleting items that are further back in a sequence. It also requires inhibiting responses to near target items, or lures. In Jaeggi and colleagues (2011), inhibition was measured by the Conners’ Continuous Performance Task (CPT), which requires children to respond to all alphabetic letters appearing intermittently on a but withhold responses to the letter “X.” On average, children in the n-back training group exhibited fewer errors of commission compared to peers in a control condition. These results suggest that n-back training may be especially beneficial for children with ADHD, given that they are thought to suffer from underlying issues of inhibitory control. Furthermore, the CPT is often used as a diagnostic measure for ADHD in neuropsychology testing (e.g., Corkum & Siegel, 1993).

Based on the previous work in this domain, the goal of the present study was to examine whether training on the n-back task would improve the response inhibition skills of children with ADHD. Children with ADHD trained on a visuospatial version of the n-back task that was previously used with typically developing children (Jaeggi, Buschkuehl, Jonides, Shah, et al., 2011). The participants’ performance was compared with that of an active control group that trained on vocabulary and general knowledge. Based on our prior work finding improvements on measures of response inhibition in typically-developing children and young adults, our primary hypothesis was that children with ADHD would also show improvements in response inhibition as measured by the CPT. Furthermore, because inhibitory control is purported to be a mechanism underlying self-regulation and cognitive/achievement deficits in children with ADHD, we expected training on the n-back task may also lead to decreased ADHD symptoms (as measured by parent report) and improved academic ability (as measured by performance on math and reading measures from the Woodcock Johnson III; Woodcock, McGrew, & Mather, 2003). Such a result would inform developing theories of cognitive training, offer support for theories implicating low inhibitory control as a contributor to ADHD, and provide preliminary evidence of the efficacy of n-back training as a useful intervention for children with ADHD. We also assessed general knowledge at posttest and delayed posttest as a control measure. For this task, which was largely identical to the training performed by the control group, we predicted that the control group would outperform the training group at delayed posttest.

Previous work has demonstrated improvements in working memory and fluid reasoning in typically-developing young adults following n-back training (Au, Sheehan, et al., 2015; Au, Buschkuehl, Duncan, & Jaeggi, 2016). And furthermore, typically-developing children who showed improvement on the n-back task also demonstrated improvements on measures of fluid reasoning (Jaeggi, Buschkuehl, Jonides, Shah, et al., 2011). We predicted that the training group in our study might outperform the control group on measures of working memory and fluid reasoning, however, fluid reasoning was considered exploratory, as there has been only limited evidence of fluid reasoning improvements in samples of children with learning or attention disorders. Based on our previous work, we also tested whether performance may be related to quality of training on the n-back task.

Children were assessed before training, immediately after training, and again after a three-month delay. The purpose of the immediate posttest was to test for immediate changes as a result of training, and the purpose of the delayed posttest was twofold: to see whether any immediate changes could be sustained over time and to see whether any changes would manifest after a delay even if they were not measurable at immediate posttest. If improved inhibition leads to improved self-regulation, such changes may not be detectable in parent reports of ADHD symptomology without some period of time post-training. Likewise, if improved inhibition is beneficial for learning, and such changes may not have time to be reflected in achievement till students can use their enhanced inhibitory control skills in learning situations. Indeed, other cognitive interventions that impact academic achievement often do so only after delay (e.g., Klauer & Phye, 2008).

Finally, there is evidence that in cognitive training not all participants benefit equally from training (Hussey & Novick, 2012; Katz, Jones, Shah, Buschkuehl, & Jaeggi, 2016; Jaeggi et al., 2011). Those who perform well on the training task tend to benefit more from the training, demonstrating greater improvements on untrained tasks such as reasoning and reading comprehension (Hussey & Novick, 2012; Jaeggi, et al., 2011). It is possible that others lack interest during training, or that they experience difficulty coping with the frustrations of the task as it became more challenging. The latter explanation is particularly likely in the case of children with ADHD, who tend to struggle with overcoming frustration (Seymour, Macatee, & Chronis-Tuscano, 2016). Accordingly, we predicted that the children with ADHD in our sample would vary in terms of training task performance, and that the training task performance would be related to performance on transfer tasks and parent report measures.

Method

Participants

This study was approved by the appropriate ethics committee and performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments. One hundred and eight children with ADHD from the communities around the University of Michigan were recruited to participate in the study via fliers and advertisements posted in primary care offices, schools and local newspapers. Parents were told that children would participate in a training protocol that might improve attention and problem solving skills. The only compensation for participants were small prizes (e.g., tickers, pencils, or small stuffed animals.). Informed consent was obtained from all individual participants included in the study. Most children were from upper middle-class backgrounds (see Table 1). Seven were excluded due to comorbid diagnoses in addition to ADHD (e.g., autism). For data analyses, we included only participants who completed at least 15 out of 20 training sessions and who trained for at least 4 weeks, but not longer than 6 weeks, and who had no major training or posttest scheduling irregularities (such as training three times per day instead of once a day). Based on those criteria, we excluded twenty-one children for failure to comply with the training schedule. Our final sample consisted of 80 children (mean age: 10.14 years; SD: 2.02; range: 7–14; 25 girls). All participants met diagnostic criteria on the The Conners’ Parent Questionnaire – Revised: Long Form (CPRS–R:L; Conners, Sitarenios, Parker, & Epstein, 1998; Conners, 2001) or the The Child Behavior Checklist (CBCL; Achenbach, 1991; Chang, Wang, & Tsai, 2016). Additionally, all participants had been diagnosed with ADHD by a clinician (i.e., pediatrician, psychologist or psychiatrist), according to Diagnostic and Statistical Manual of Mental Disorders (DSM-IV Diagnostic and Statistical Manual of Mental Disorder, 1994) criteria. Although the ADHD diagnoses made by clinicians were not standardized across participants, we chose this present approach as it is representative for how ADHD is typically diagnosed in the community, and furthermore, our main goal was to test the efficacy of our intervention in an ecologically valid context. Participants could be no younger than seven, as the control task required participants to read at a somewhat fluent level. Participants could also be no older than 14, as the task design and background stories of the intervention were created with younger children in mind, and might not have been appropriately engaging for older adolescents.

Table 1.

Participant Demographic Information

	Training Group Mean (SD)	Control Group Mean (SD)	p-value	BF
N	41	39
Age	10.22 (2.04)	10.05 (2.03)	0.71	0.25
Grade	4.78 (2.13)	4.72 (2.08)	0.90	0.23
ADHD Severity	23.15 (7.34)	23.68 (7.03)	0.74	0.25
Num. Girls	13	12	0.93	0.25
Num. Medicated	20	18	0.93	0.29
Mother’s Education	16.57 (2.5)	16.44 (2.30)	0.39	0.25
Family income	$25,000–99,000	$25,00–99,000	0.69	0.003

Note: ADHD Severity as measured by parent report on the CPRS-R:L ADHD index at pretest, mother’s education is measured in years, and income mode is presented, as income was measured in bins. The p-values and Bayes factors are from t-test from continuous variables and from chi-squared tests for non-continuous variables. Bayes factors over 1 are considered evidence in favor of the alternative hypothesis.

As they were recruited, participants in the two groups were continuously matched on measures of fluid reasoning (TONI and SPM), age, gender and ADHD symptom severity (as measured by parent rated CPRS–R:L Comprehensive Behavior Rating Scales) by a researcher who was neither involved in testing nor in training of the participants. Based on this matched pairing, participants were randomly assigned to one of two conditions: n-back training (n=41) or an active knowledge training control condition (n=39). Of these participants, 19 in the training group and 15 in the control group were actively taking medication for ADHD. See table 1 for full demographic information.

Procedure

Participants came to the lab for two 60-minute sessions to complete pretest measures. Pretest measures included tasks that were structurally similar to the training tasks (termed “near transfer”), tasks that were dissimilar to the training tasks (“far transfer”), and parent report measures. Participants were required to train at home once per day for 20 sessions with parent supervision, and in addition, children came to train in the lab, a library, or another public place once a week for a researcher-supervised training session (16 sessions at home, 4 sessions supervised, over the course of about 5 weeks). The children returned for a posttest session immediately following training completion, and they returned again for a second, delayed posttest to complete the same measures three months later to discern whether any differences between groups persisted over time.

All pretest, posttest, and delayed posttest descriptive statistics along with re-test reliability estimates and effect sizes are provided in the supplemental online material (SOM) tables 1–3.

Table 3.

ANCOVA Posttest Results

Outcome Measure	F (1, 77)	Non-adjusted p-value	Adjusted p-value	η2p	BF
Object n-back	11.41	.001***	0.01**	0.13	29.50
Trivia	.10	.76	0.76	0.00	0.25
CPT	6.48	.01**	0.03*	0.08	3.73
Working Memory	1.99	.16	0.26	0.03	0.61
School-related tasks	1.54	.22	0.26	0.02	0.32
Matrix Reasoning	1.79	.19	0.26	0.02	0.53

Note: Adjusted and unadjusted p-values are reported. Adjusted p-values were calculated using the Hochberg-Benjamini method. Significance levels are indicated as:

^***= 0.001

^** = 0.01

^* = 0.05.

Bayes factors over 1 are considered evidence in favor of the alternative hypothesis.

School-related tasks did not control for pretest performance, as assumptions for ANCOVA were not met in this case.

Measures

N-Back Training.

Participants in the experimental condition trained on a game-like computerized cognitive training task similar to that used in previous studies with typically developing children (Jaeggi, Buschkuehl, Jonides, Shah, et al., 2011). This spatial n-back task presented participants with images at one of six locations on the screen at a rate of three seconds each, each image presented for 500 ms followed by a 2,500 ms interstimulus interval. The participants’ task was to decide whether a stimulus appeared at the same location as the one presented n items back in the sequence. Participants pressed the ‘A’ key each time the current image was in the same location as the one presented n items previously (targets) and the ‘L’ key if the image did not match (non-targets). There were five randomly positioned targets per block of trials, and each block included 15 + n trials. Each training session consisted of 10 rounds lasting approximately one minute each. Participants were provided with performance feedback at the end of each round as well as at the end of the training session. A complete training session lasted approximately 15 minutes.

The levels in the game corresponded to the n back that the participant had to remember. The n was adaptive, such that successfully completing a round at a particular level (three or fewer errors) resulted in the next round being more difficult by increasing the level of n, whereas poor performance (four or more errors) resulted in “losing a life.” At the end of each round, points were calculated and awarded in the form of virtual gold coins. At higher levels that were more challenging, more points were awarded per correct response and more points could be earned per round. If three lives were lost on a same level, the player would be moved down one level and receive three more lives. Moving down a level was not intended as a penalty, but rather as a means to keep the difficulty of the game within the participants’ level of maximum ability.

Control Training.

Participants in the active control training group completed a general knowledge and trivia program as used before (Jaeggi, Buschkuehl, Jonides, Shah, et al., 2011), in which they answered questions about vocabulary, history, and general-knowledge facts. Time on task and reinforcers were similar as time for the n-back training group. Questions such as “What is Orion’s Belt?” were presented and participants selected from four answer alternatives shown below the question. Participants were provided with feedback on their performance, and questions which were answered incorrectly were repeated in the next session. This game also had levels and was adaptive, but only minimally implicated working memory or controlled attention.

There were four different themes in both the n-back and the control training. The themes were implemented in order to make the training task more interesting for children. After five training sessions, the theme would automatically switch to the next one. The general training features (e.g., levels, bonuses, scores) would stay the same regardless of the presented theme. While playing the games at home, the child would be earning points and virtual gold coins, which he or she would later be able to trade in for trivial prizes (stickers, pencils, stuffed animals, etc.) after the supervised training sessions. Participants received no other compensation.

Near Transfer Tasks.

The participants completed an untrained (non-spatial/identity) version of the n-back task as previously used (Katz, Jaeggi, Buschkuehl, Stegman, & Shah, 2014), in which they were presented with a series of colored objects (e.g. a penguin, a flower, a lemon) in the center of the screen, and they had to indicate whether or not the current stimulus was the same as the one n positions back in the sequence. Unlike the training task, which was adaptive, the n-back level was fixed at 2-back, and children completed only three rounds (20+n trials each). The fixed level of n-back was chosen based on our previous work with typically developing children (Jaeggi et al., 2011), showing that this task at that level and using those stimuli is adequate in terms of difficulty, in that we were expecting enough variability allowing us to assess transfer. The fixed level also allowed us to assess participants’attention and impulsivity (noticing and rejecting lure trials) in addition to working memory. The dependent variable was the proportion of hits minus the proportion of false alarms. This task was administered at pretest, posttest, and at delayed posttest. Participants also completed a multiple-choice quiz similar to the control training task. This was included as a near transfer task for the control group and a control task for the n-back training group as we did not predict improvements in general knowledge as a function of cognitive training. This quiz was administered at posttest and at delayed posttest only.

Cognitive Far Transfer Tasks.

These measures were administered at pretest, posttest, and delayed posttest. We used the CPT to assess sustained attention and inhibitory control by having participants respond as quickly as possible to stimuli (letters; go trials), but to withhold responses for a small percentage of trials (i.e., upon presentation of the letter “X”; no-go trials). The letters were presented in the center of the screen (white on black background) for a duration of 250 ms. Our task was modeled after Conners et al., (2003), consisting of 18 blocks with 20 trials each (360 total trials). Each block differed in the duration of the interstimulus interval, which was 1,000, 2,000, or 4,000 ms long. The order of the blocks was randomized. The ratio of go- vs. no-go trials was 9:1. Our dependent measure was false alarm rates. Split-half reliability for the CPT has been reported as 0.83 (range 0.66 to 0.95; Conners et al., 2000; 1994).

All participants completed two verbal working memory measures that were not n-back tasks. For analyses, a composite score of working memory was created using standardized z-scores averaged from both tasks, i.e. Digit Span (forward and backward) and Following Directions. The digit span required participants to read lists of digits and repeat them either in order (forward digit span) or in the reverse order of presentation (backward digit span; Wechsler, 2003). List lengths varied from two to nine (two trials per list length), and the task was ended if participants failed at both trials of a particular list length. The dependent variable was the number of correctly recalled sequences. Parallel-test versions (counterbalanced across participants) were used for pre- and posttest, however, for the delayed posttest, the version used at pretest was administered. In the following directions task (Gathercole, Durling, Evans, Jeffcook, & Stone, 2008), participants were seated in front of an array of familiar objects (boxes, folders, pencils, rulers, erasers) in three different colors (blue, yellow, and red). Participants were asked to execute a spoken instruction such as “Pick up the blue pencil and put it in the yellow box.” The task instructions became increasingly complex (i.e., containing more actions) throughout the measure, up to a level when the participant made three consecutive errors within one level of complexity. The number of actions ranged from one to seven (levels 1–7), and each level contained six trials. The dependent variable was calculated as the sum of the number of actions in sequence that the child could accurately complete all trials of, plus 0.25 for each correct trial at the next span level. For example, if a child successfully completed six trials with three actions (this counts three towards the dependent variable), completes one correct trial with four actions (this counts 0.25 towards the dependent variable), then fails the subsequent trials with four actions, their score would be a span of 3.25 (Holmes, Gathercole, & Dunning, 2009; see also Ramani, Jaeggi, Daubert, & Buschkuehl, 2017). Parallel-test versions (counterbalanced across participants) were used for pre- and posttest, and for the delayed posttest, the version used at pretest was administered.

Far Transfer to ADHD Symptoms.

Three standardized parent questionnaires were administered at pretest and at the delayed posttest three months after the training had ended, providing insight into whether there were any observable behavioral differences resulting from cognitive training. Here we focus on specific sub-scales measuring ADHD symptoms related to executive functioning, but for completeness, the descriptive data for all scales are provided in SOM Table 3.

The CBCL assesses behavioral problems in children. Cronbach’s alpha for the narrow scales (syndrome and DSM-oriented) is reported as 0.83, and for the broad scales (internalizing, externalizing, and total problems), it is 0.94 (Bullard, Griss, Greene, & Gekker, 2013). The Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000) measures executive functioning in children. Cronbach’s alpha for the scales range from 0.80–0.98 (Gioia et al., 2000). The CPRS–R:L provides seven scales: oppositional, cognitive problems/inattention, hyperactivity-impulsivity, anxious-shy, perfectionism, social, and psychosomatic problems. Cronbach’s alpha for the scales range from 0.75–0.95 (Conners et al., 1998).

For the purpose of the present paper, specific scales were selected for analyses based on our hypotheses. Parent report of executive functioning was measured using the BRIEF Executive Functioning sub-scale. As we were interested in inattention and hyperactivity, we used composite z-scores from the following scales: attention problems and ADHD problems from the CBCL and the inattention and hyperactivity scales from the CPRS–R:L. This approach was taken to minimize familywise error inflation, reducing the overall number of comparisons. However, for completeness, the descriptive results of all scales are reported in SOM Table 3.

School-Related Far Transfer Tasks.

Three school-related tasks were administered. For analyses, a composite score for academic ability was created using z-scores from the Woodcock Johnson III Passage Comprehension, Math Applied Problem Solving, and Reading Fluency (Woodcock et al., 2003). Individual test reliabities using a split half method are reported as 0.80 or higher (Grenwelge, 2009). For all school-related tasks, parallel-test versions (counterbalanced across participants) were administered for pre- and post-test, and for the delayed posttest, the version used at pretest was administered. The Passage Comprehension subtest of the Woodcock Johnson III was used to measure participants’ reading comprehension. Participants read progressively more challenging passages and answered questions about them. The cut-off for this task was six sequential incorrect responses within a set, and the dependent variable was the total number of correct responses. The Math Applied Problem Solving subset of the Woodcock Johnson III was used to measure participants’ broad math skills. Participants solved math word problems. The cut-off for this task was six sequential incorrect responses in a set, and the dependent variable was the total number of correct responses. The Reading Fluency subtest of the Woodcock Johnson III was used to measure participants’ automaticity in reading simple sentences. Participants were given three minutes to read simple sentences and judge if those sentences were true or false. The dependent variable was the number of correctly answered questions.

Far Transfer to Matrix Reasoning.

Participants were presented with incomplete, abstract patterns to be completed by selecting the missing part that logically completes the pattern or series. We used two standardized versions, the Ravens Standard Progressive Matrices (SPM; Raven, 2000) and the Test of Nonverbal Intelligence (TONI; Brown, 2003) The odd-even split-half corrected coefficient of reliability for SPM is 0.96 (Raven, 2000), and Cronbach’s alpha for TONI is reported in the 0.80 and 0.90 range (Brown, 2003). The cut off for the Ravens was 10 minutes, and the cut off for the TONI was three incorrect answers out of a sequence of five items. The dependent measure was the amount of correctly solved problems. We created a composite score for matrix reasoning using standardized z-scores averaging the scores of both measures. These tasks were administered at pretest, posttest, and at delayed posttest. Parallel-test versions (counterbalanced across participants) were administered for pre- and posttest, and for the delayed posttest, the version used at pretest was administered.

Results

Primary Analyses

Data were analyzed using IBM SPSS Statistics Version 22 and JASP Version 0.8.5.1. Descriptive data for the near and far transfer tasks are provided in SOM tables 1, 2, and 3. SOM tables 4 and 5 provide correlations between outcome measures at pretest, training gain, and gains on outcome measures. Note that there were no significant differences between groups on any of the variables of interest at pretest (all ps > .20). We interpret significance at the p < 0.05 level, but as this is an exploratory study, we provide the reader with rich information with which to assess our findings. This includes uncorrected p-values and p-values corrected for multiple comparisons using the Hochberg-Benjamini method (Hochberg & Benjamini, 1990), and in addition, we conducted Baysean analyses using default priors in JASP, which allow us to report Bayes Factors that quantify evidence for and against the null hypothesis (Wagenmakers et al., 2017).

Table 2.

Demographic Information for included and excluded participants

	Included Mean (SD)	Excluded Mean (SD)	p-value	BF
N	80	21
Age	9.9 (0.43)	10.13 (0.22)	0.63	0.24
Grade	4.62 (0.45)	4.75 (0.23)	0.79	0.23
ADHD Severity	25.95 (1.61)	23.41 (0.79)	0.16	0.02
Num. Girls	25	4	0.14	0.29
Num. Medicated	38	14	0.26	1.19
Mother’s Ed	16.63 (0.54)	16.51 (0.28)	0.84	0.01
Family income	$25,000–99,000	$25,000–99,000	0.89	0.26

Note: ADHD Severity as measured by parent report on the CPRS-R:L ADHD index at pretest, mother’s education is measured in years, and income mode is presented, as income was measured in bins. The p-values and Bayes factors are from t-test from continuous variables and from chi-squared tests for non-continuous variables. Bayes factors over 1 are considered evidence in favor of the alternative hypothesis.

Table 4.

ANCOVA Delayed Posttest Results

Outcome Measure	F (1, 54)	Non-adjusted p-value	Adjusted p-value	η2p	BF
Object n-back	4.56	.04*	.11	0.08	1.74
Trivia	0.54	.47	.54	0.01	0.33
CPT	4.34	.04*	.11	0.07	1.55
Working Memory	5.48	.02*	.11	0.09	2.32
Inattention and Hyperactivity	3.38	.07	.14	0.07	1.11
Executive Functioning	1.99	.17	.27	0.05	0.68
School-related tasks	.93	.34	.45	0.02	0.44
Matrix reasoning	0.02	.87	.87	0.00	0.26

Note: Adjusted and unadjusted p-values are reported. Adjusted p-values were calculated using the Hochberg-Benjamini method. Significance levels are indicated as:

^*** = 0.001

^** = 0.01

^* =0.05.

Bayes factors over 1 are considered evidence in favor of the alternative hypothesis.

Table 5.

Descriptive statistics by group for the outcome variables at pretest, posttest, and delayed posttest

	Active Control Group			N-Back Group
Outcome Measure	Pretest Mean (SD)	Posttest Mean (SD)	Delayed Posttest Mean (SD)	Pretest Mean (SD)	Posttest Mean (SD)	Delayed Posttest Mean (SD)
Object n-back	0.34 (0.22)	0.39 (0.23)	0.42 (0.28)	0.40 (0.24)	0.59 (0.27)	0.54 (0.21)
Trivia	n/a	9.12 (3.72)	9.53 (3.49)	n/a	8.86 (3.57)	8.83 (3.71)
CPT	62.06 (17.05)	52.20 (21.15)	52.27 (22.15)	58.60 (22.77)	39.77 (27.36)	41.44 (23.36)
Working Memory	−0.03 (0.91)	−0.12 (0.89)	−0.17 (0.92)	0.08 (0.82)	0.14 (0.84)	0.20 (0.66)
Inattention and Hyperactivity	0.03 (0.71)	n/a	0.17 (0.89)	−0.05 (0.79)	n/a	−0.29 (0.68)
Executive Functioning	67.50 (9.05)	n/a	66.14 (9.84)	65.48 (9.66)	n/a	59.73 (10.93)
School-related tasks	−0.07 (1.04)	−0.16 (1.05)	−0.02 (1.06)	0.03 (0.83)	0.10 (0.71)	0.07 (0.66)
Matrix reasoning	−0.02 (0.95)	−0.12 (0.96)	0.07 (0.94)	0.01 (0.90)	0.09 (0.92)	−0.02 (0.94)

Note: n/a indicates that values are not applicable for tests that were not administered at those time points.

First, data were analyzed to test whether any significant group differences existed at pretest. Both groups completed an equivalent number of training sessions (experimental group: mean: 19.15; SD: 2.69; control group: mean: 19.62, SD: 1.71; t(78) = −0.93, p = 0.36, d = 0.21). On average, children with ADHD improved by approximately half an n-back level, which is slightly below the improvement that has been observed in typically developing children (Jaeggi et al., 2011). Then, our general approach to testing for differences between the training and the control group resulting from cognitive training was to conduct two-group analyses of covariance. We analyzed the posttest performance using pretest as a covariate to test for immediate training efficacy. All analyses met the assumptions for ANCOVA with the exception of school-related skills at posttest, which violated the assumption of homogeneity of slopes. The analysis for this particular measure was therefore performed as an Analysis of Variance without a covariate. These posttest results are presented in table 3. Then we analyzed the delayed posttest with the pretest as a covariate to test for sustained changes. All these analyses met the assumptions for ANCOVA, and the results are presented in table 4.

In short, at post-test, we observed strong evidence of group differences in the non-trained n-back task (F(1,77) = 11.41, p = 0.001, _η²_p = 0.13, BF = 29.50), as well as well as substantial evidence for group differences in the measure of inhibitory control (CPT; F(1, 77) = 6.48, p = 0.01, _η²_p = 0.08, BF = 3.73). At delayed post-test, the group differences remained, however, the effects were less pronounced than at post-test, and furthermore, they did not survive corrections for multiple comparisons (non-trained n-back: F(1, 54) = 4.56, p = 0.04, _η²_p = 0.08, BF = 1.74; CPT: (F(1, 54) = 4.34, p = 0.04, _η²_p = 0.07, BF = 1.55), see figure 1. We also observed anecdotal evidence of reduced ADHD symptoms as reported by the parents (F(1, 43) = 3.38, p = 0.07, _η²_p = 0.07, BF = 1.11; see figure 2). Descriptive statistics by group for pretest, posttest, and delayed posttest are displayed in table 5.

Figure 1:

Performance on impulsivity task (CPT errors of commission) by training group. Lower scores indicate better performance. Error bars represent standard errors.

Figure 2:

Parent rating of inattention and hyperactivity by training group. Note that better performance is represented by negative numbers (i.e. fewer symptoms). Error bars represent standard errors.

Finally, multiple regression analyses were conducted to test whether the amount of training improvement in the training group predicted performance on untrained measures (see table 6). Training improvement was measured as the average score on the last two training sessions minus the average score on the first two training sessions. This approach was guided by our hypothesis that cognitive training is most effective when participants are actively engaged and put effort into doing their best. Assumptions for multiple regressions were tested and met. The analyses suggested that training performance was predictive for transfer in several measures, specifically, in CPT at posttest (b = −6.93, p = 0.03, BF = 1.34), parent-reported ADHD symptoms of inattention and hyperactivity (b = −0.26, p = 0.02, BF = 2.58), posttest school-related tasks (b = 0.14, p = 0.04, BF = 0.26), matrix reasoning at posttest (b = 0.21, p = 0.05, BF = 0.93), and matrix reasoning on delayed posttest (b = 0.36, p = 0.01, BF = 3.90).

Table 6.

Regression analyses for the n-back group

	beta	SD	t	p-value	BF
CPT Errors of Commission Posttest
Pretest	.96	.12	8.17	.00***	5.05×106
Training gain	−6.93	3.15	−2.20	.03*	1.34
CPT Errors of Commission Delayed Posttest
Pretest	.80	.16	4.87	.00***	352.39
Training Gain	−7.17	3.84	−1.87	.07	1.09
Working Memory Posttest
Pretest	.77	.13	5.88	.00***	27,915.89
Training gain	.00	.13	.02	.99	0.20
Working Memory Delayed Posttest
Pretest	.40	.16	2.48	.02*	5.22
Training Gain	.06	.14	.43	.67	0.44
Inattention & Hyperactivity Delayed Posttest
Pretest	.72	.12	6.00	.00***	2,134.52
Training gain	−.26	.11	−2.49	.02*	2.58
Executive Functioning Delayed Posttest
Pretest	.92	.18	5.24	.00***	1,173.57
Training gain	.43	2.01	0.21	.83	0.23
School Tasks Posttest
Pretest	0.72	.07	10.86	.00***	2.60×109
Training gain	.14	.06	2.16	.04*	0.95
School Tasks Delayed Posttest
Pretest	.71	.10	7.34	.00***	152,512.54
Training gain	.08	.08	0.93	.36	0.26
Matrix Reasoning Post
Pretest	.74	.09	7.8	.00***	2.46×106
Training gain	.21	.10	1.98	.05*	0.93
Matrix Reasoning Delayed Posttest
Pretest	.07	0.13	5.14	.00***	625.09
Training gain	.36	0.14	2.64	.01**	3.90

Note: The effect of training gains on outcome measures controlling for pretest scores. Betas are unstandardized. Significance levels are indicated as:

^***=0.001

^**=0.01

^*=0.05.

Bayes factors over 1 are considered evidence in favor of the alternative hypothesis.

Exploratory Analyses

Motivation.

To investigate the possibility that any effects were due to motivational differences between groups, all children were asked at the end of each training session to rate how much they enjoyed the training on a scale from 1–5 where 1 was “I really did not enjoy the game” and 5 was “I really enjoyed the game.” The average rating of the control group was 3.84 (SD = 0.89), and the average rating of the training group was 3.27 (SD = 0.84). The difference between the groups was significant (p = 0.004; d = 0.66), indicating that the control group enjoyed their task more than the training group. This suggests that any differences between groups cannot be attributed to the experimental group enjoying their task more than the control group.

Additionally, participants were asked to rate how much effort they put into the game in order to test for differences in engagement between group. Effort was rated on a 5-point scale where 1 was “very little” and 5 was “too much.” The average rating of the control group was 3.25 (SD = 1.13), and the average rating of the experimental group was also 3.23 (SD = 1.17). The difference between the groups was not significant (p = 0.92; d = 0.02), suggesting that neither group was more engaged than the other.

Medication.

Although our limited sample size makes it difficult to compare children in the training group who were medicated (n=19) to children in the training group who were not medicated (n=15), the question of whether medication influences training efficacy is important enough to warrant an exploratory analysis. The training gain (the average score on the last two training sessions minus the average score on the first two sessions) was compared between medicated and un-medicated children. There was no significant training gain difference between groups (medicated: mean gain = .55, SD = .85; not medicated: mean gain = .84, SD = .77; p = .31, d = .36), suggesting that the training program was equally beneficial for children who were and were not taking medication for their ADHD symptoms.

ADHD Severity.

To explore the relationship between ADHD severity and training gain, a linear regression of ADHD symptoms (as measured by the Connors at pretest) on training gain was performed. ADHD severity did not significantly predict training gain (b = −.04, p = .08).

Discussion

Children who struggle with ADHD are hypothesized to have underlying executive functioning deficits, particularly in inhibitory control and working memory. In the present study, we tested the hypothesis that training children with ADHD in those two domains would lead to improvements on executive functioning skills that rely upon inhibition and working memory, and also to reductions in ADHD symptoms related to those executive functioning skills. Our results indicate that training inhibition, embedded in what is classically considered a working memory task n-back training that targets both working memory and inhibition resulted in substantial improvements in a non-spatial variant of the n-back task, and in addition, we observed improved inhibitory control (as measured by the CPT), albeit with small effect sizes, and marginally reduced parent-reported symptoms of inattention and hyperactivity three months after training completion. However, this latter finding, which approached significance when uncorrected, failed to reach significance when adjusted for multiple comparisons. Nonetheless, we feel that this outcome is worth exploring in future studies with larger sample sizes. It remains possible that these symptoms may be reduced through cognitive training, which is relatively easy to implement compared to some other previously discussed interventions. Though these data are not conclusive, we believe that these findings are worth reporting to the clinical and research communities as they may help guide future research in this domain.

Our regression analyses revealed that the amount of training gain predicted the amount of transfer for CPT, inattention and hyeractivity, school tasks, and matrix reasoning (see Table 5). This finding supports our hypothesis and corroborates previous work that the extent to which training is effective is relevant for transfer (Jaeggi, Buschkuehl, Jonides, Shah, et al., 2011). If students do not actually improve throughout training, one would not expect that practice per se would yield any transfer (Solomon & Perkins, 1989). This finding highlights the importance of examining training performance in cognitive training studies. It also suggests that one way of boosting effectiveness of cognitive training interventions is to ensure that participants maintain engagement with training tasks and are able to learn from their practice. Future studies could address the extent to which motivational factors embedded within the training task or different adaptivity parameters or scaffolding are helpful. This issue may be especially important for studies of children with ADHD, as they are more likely to be discouraged when facing challenging cognitive tasks (Seymour et al., 2016).

The effect of training gain on transfer is helpful for interpreting our positive transfer effects on parent ratings of behavior. Specifically, one might question the objectivity of a parent reported measures, arguing that a parent could figure out whether his or her child was in the training or the control group, and that this knowledge would bias questionnaire responses (Cortese et al., 2015). However, though parents may discover which condition their child was in, they are likely to be unaware of whether they actually improved on the training task, as the training was completed in the home and as there was no interaction between the participating families. The fact that training gain was associated with parent reports of behavior suggests that the findings were not likely due to parent expectations.

On the near transfer measure, the object n-back task, there was a significant difference between groups, as predicted. However, the regression analysis revealed no significant relationship between the amount of training gain on the object n-back performance. Although we did predict that greater training gains would be associated with greater transfer performance, the lack of such a finding on this particular measure may not be, in retrospect altogether surprising, because the object n-back task was fixed at a 2-back level. For all participants in the training group, the average n-back level on the very first day of training was 2.4. Thus, even low-performing participants who did not achieve high levels of n-back training still had ample practice at 2-back, providing no advantage to the higher-performing participants on this particular measure.

Although we predicted that the control group would improve relative to the training group on the trivia quiz (the near transfer measure of general knowledge), no significant differences were found between the two groups. One explanation might be that both groups began with high enough general knowledge that this control intervention had no significant effect, or alternatively, that both groups improved equally. As we have no pretest measure of general knowledge, we cannot evaluate these potential explanations.

Our composite measure of working memory yielded no significant differences between groups. The working memory tasks used to form the composite score were digit span and following directions, both of which seem to require active recall rather than updating and recognition, which are primarily involved in n-back, and thus, transfer from n-back to those tasks might be limited (Jaeggi, Studer-Luethi, et al., 2010; Jaeggi, Buschkuehl, Perrig, & Meier, 2010). Furthermore, both working memory outcome measures were verbal tasks while the training task was spatial. One might reasonably expect little transfer across these distinct working memory subsystems, especially also given previous literature that has shown larger correlations between n-back and visual WM tasks as compared to verbal WM tasks (Waris et al., 2017), as well as limited transfer to verbal domains in general (Au et al., 2015; Buschkuehl et al., 2008; Colom et al., 2013). In a recent meta-analysis of n-back cognitive training by Soveri, Antfolk, Karlsson, Salo, and Laine (2017), transfer to working memory tasks yielded smaller effect sizes as compared to the effect sizes found following cognitive training using more traditional working memory training paradigms (Melby-Lervåg & Hulme, 2013; Weicker, Villringer, & Thöne-Otto, 2015; Schwaighofer, Fischer, & Bühner, 2015). However, the fact that an effect seemed to emerge at delayed posttest with the uncorrected p-value leaves open the possibility (BF = 2.32) that transfer to other working memory domains may be possible, but that a considerable amount of time might be required for such effects to occur or that robust effects might require more training.

We tentatively predicted transfer of n-back training to our composite measure of matrix reasoning especially for children who improved in the task based on previous work with typically developing young adults and children (Jaeggi et al., 2011; Jaeggi, Buschkuehl, Shah, & Jonides, 2014). We also expected to find transfer to school-related tasks, as compared to typically developing children, children with ADHD might suffer from restriction in school-related performance due to their ADHD symptoms. Unfortunately, we found transfer to neither matrix reasoning nor school-related tasks. The failure to observe group differences could reflect the fact that children with ADHD struggle with attention, which is strongly required in tests of matrix reasoning, math, and reading ability. Although we were able to increase attention as a result of our cognitive training, it’s possible that we did not increase it enough. The levels achieved by these children with ADHD were lower than those typically achieved by children without ADHD and even certainly lower than typical adult performance. Perhaps with a longer training regimen, transfer to reasoning and school-related skills could be achieved in children with ADHD. Another possible explanation is that performance on the achievement measures require more crystalized knowledge compared to the measures for which we did find effects. As our intervention did not target crystalized knowledge, it is perhaps unsurprising that we did not find improvements in those areas.

Although we found transfer to objective measures of executive functioning, namely inhibition and, to some extent, working memory, we found no transfer to parent-reported executive functioning symptoms of ADHD. One explanation for this lack of transfer may be that, though laboratory and behavioral mesures are related in untrained populations, these relationships are no longer present following training (for example, if participants are learning strategies effective for laboratory tasks that may not apply to everyday behavior). Alternatively, it is possible that there was not enough improvement in inhibition to yield noticeable impact on parent reports of behavior. This second interpretation is consistent with our finding that training gain is correlated with the parent ratings; if training gains were larger, it is possible we would have seen significant improvements in parent ratings. If so, a more effective or longer-term n-back intervention may be necessary to see significant improvements in behavior.

Although the small effect sizes demonstrated here must necessarily temper the interpretation of these findings, the highlight of this study is the finding that sustained attention and inhibitory control can be improved via n-back training. This has the potential to be beneficial for children with ADHD who struggle with inhibition, attention, and hyperactivity. An added benefit of this intervention, compared to CWMT and behavioral therapies, is that it can be completed at home in the absence of a trained professional and can be easily distributed at minimal cost to families of children who struggle with ADHD. Additionally, this study has much to offer theoretically: It provides support for the theory that inhibitory control underlie issues associated with ADHD, namely inattention and hyperactivity. It also offers theoretical support for cognitive training more broadly, and provides insight into the relationship between training gains and transfer.

Limitations

The present study contributes to understanding of cognitive training and to our efforts of ameliorating the difficulties associated with ADHD. However, we acknowledge that our study has limitations. Specifically, outcome measures were all either lab based or parent report. Lab-based measures often lack in ecological validity and thus have limited bearing on everyday life, although the CPT used in our study has been shown to be highly predictive to ADHD symptomatology (Perugini, Harvey, Lovejoy, Sandstrom, & Webb, 2000). Parent report, on the other hand, is rooted in everyday experiences with the child. Thus, parent report measures offer a great deal of ecological validity and clinical relevance, but may not be fully objective. Future work should include more objective assessments of whether improvements resulting from n-back training impact the child’s behavior at school or in the home. Another issue was attrition and dropout. Twenty-one participants (20.79%) did not complete the training or failed to adhere to the training schedule, a rate that seems high on first sight, however, it is lower than what has been reported in studies using similar interventions with non-ADHD samples (e.g., Jaeggi, Buschkuehl, Shah, & Jonides, 2014; Redick et al., 2013) Importantly however, the excluded participants did not vary systematically from the included participants; alleviating concerns regarding selection bias (see Table 2).

Conclusion

The present study introduced n-back training as a potential cognitive intervention for children with ADHD. Children with ADHD who completed the training demonstrated improved inhibitory control relative to their peers in the control condition as assessed by lab-based measures. Our findings have significant implications given that models of ADHD consider inhibitory control to be an underlying core deficit in ADHD (Barkley, 1997). Additionally, the training group’s marginally significant reduction in parent-reported symptoms of inattention and hyperactivity has practical relevance: Our data suggest that improving inhibitory control via n-back training may result in decreased ADHD symptoms. Finally, unlike other cognitive interventions that require a trained clinician or researcher to administrator the training, ours was done at home by participants and supervised by parents in an ecologically valid setting. We chose at-home training rather than a laboratory-based training because we wanted to capitalize on one of the primary advantages of computerized cognitive training over or in conjunction with pharmacological or behavioral interventions, namely, that it can be easily distributed and inexpensively deployed. Thus, our findings offer significant practical relevance regarding the development of interventions for children with ADHD.