Cogmed Working Memory Training for Youth with ADHD: A closer examination of efficacy utilizing evidence-based criteria

Abstract

Objective

The current review applied the evidence-based treatment (EBT) criteria espoused by the Society for Clinical Child and Adolescent Psychology (Silverman & Hinshaw, 2008) to specifically evaluate the short- and longer-term efficacy of Cogmed Working Memory Training (CWMT) as a treatment for youth with Attention-Deficit/Hyperactivity Disorder (ADHD).

Method

Utilizing a systematic literature search, seven studies that employed the school-age version of CWMT were identified for this review.

Results

The data reviewed herein suggest mixed findings regarding the benefit of CWMT for youth with ADHD. Two randomized controlled studies have demonstrated that CWMT led to improvements in neuropsychological outcomes and parent-rated ADHD symptoms relative to wait-list control and placebo treatment conditions. Another study demonstrated effects of CWMT relative to a placebo condition on an analog observation of behavior during an academic task, although this study did not find an effect of CWMT on parent-rated ADHD. Finally, an additional study utilizing an active comparison control condition did not find incremental benefits of CWMT on parent- or teacher-rated ADHD. Critical issues in interpreting existing studies include lack of alignment between demonstrated outcomes and the hypothesized model of therapeutic benefit of CWMT, issues with equivalence of control conditions, and individual differences that may moderate treatment response.

Conclusions

Collectively, the strengths and limitations of the studies reviewed suggest that CWMT is best defined as a Possibly Efficacious Treatment for youth with ADHD. We suggest future directions for research and conclude with clinical implications of our findings for the treatment of youth with ADHD.

Behavioral interventions (i.e., contingency management in the classroom and behavioral parent training) as well as pharmacological interventions, most prominently stimulant medication, are considered the best-supported treatments for Attention-Deficit/Hyperactivity Disorder (ADHD) (American Academy of Pediatrics [AAP], 2011; Pelham & Fabiano, 2008). These interventions result in significant benefits for youth with ADHD across multiple domains of functioning (Pelham & Fabiano, 2008). Importantly, these treatment modalities are not without limitations. While stimulant medication is relatively easy to implement, and for many, provides rapid therapeutic benefit, a significant minority of youth with ADHD do not respond to stimulant medication (10–30%; Goldman, Genel, Bezman, & Slanetz, 1998) or experience significant side-effects that prohibit continued use (< 10%; Graham & Coghill, 2008). Parental perceptions of the impact of stimulant medication on overall health, as well as parental preference for alternative treatments, influence the acceptance of and adherence to stimulant medication (see Chacko, Newcorn, Feirsen, & Uderman, 2010 for a review).

Behavioral interventions, in contrast, are often more difficult to sustain over long periods of time, are generally more costly, and, arguably, may be less effective than stimulant medications—particularly for the core symptoms of ADHD (MTA Cooperative Group, 1999). The availability of providers who utilize behavioral interventions has also been noted as a limiting factor in providing effective treatment for ADHD (AAP, 2011). In addition, there are several limitations that both stimulant medication and behavioral interventions share. First, although efficacious, these interventions do not normalize the behavior of a significant number of youth with ADHD (Swanson et al., 2001) nor do treatment effects persist past the point of active dosing/implementation (Chronis, et al 2001; Chronis, Pelham, Gnagy, Roberts, & Aronoff, 2003). Lastly, there appears to be very little data supporting the effects of either intervention on key areas of functioning, such as academic achievement (Raggi & Chronis, 2006).

Given these limitations there has been a continued interest in developing alternative interventions for ADHD. In particular, development of interventions that more directly address the pathophysiology of the disorder, underlying compensatory mechanisms or identified deficiencies may yield benefits that persist after formal treatment discontinuation. This treatment approach may more likely impact the negative trajectories experienced by youth with ADHD (Halperin & Healey, 2011). There has been a growing recognition of the neuropsychological deficits underlying ADHD (Makris, Biederman, Monuteaux, & Seidman, 2009), which over the past decade have focused substantially on the role of executive function deficiencies. As such, there has been increasing efforts to target these deficits in youth with ADHD (e.g., Sonuga-Barke, Thompson, Abikoff, Klein, & Brotman, 2006).

Working memory has received particular attention as an important factor in understanding ADHD in youth. First, although youth with ADHD have an array of executive functioning deficits, working memory has been observed as one of the most significantly impaired (Willcutt, Doyle, Nigg, Faraone, & Pennington, 2005). Working memory has also been posited to serve a basic fundamental function that underlies more complex executive functions and core behavioral symptomatology characteristic of ADHD (Rapport, Chung, Shore, & Isaacs, 2001). There is also increasing recognition that youth diagnosed with ADHD and executive functioning deficits (including working memory deficits) may represent a distinct subgroup of ADHD (Lambek et al., 2010). Working memory is also an essential cognitive ability that is related to reading and math achievement (e.g., Swanson & Jerman, 2007; Swanson, Jerman & Zheng, 2008).

Given the role of working memory deficits in individuals with ADHD, targeted interventions focused on improving working memory appear to be an important line of empirical inquiry. Notably, if working memory can be improved in youth with ADHD, not only should ADHD symptoms and other key impairments associated with ADHD and working memory (e.g., academic impairments) improve but the underlying change in working memory should result in more enduring treatment-related changes that generalize across settings and persist following treatment discontinuation.

One intervention focused on improving working memory in youth with ADHD which has both undergone empirical investigation and is now commercially available for use is Cogmed Working Memory Training (CWMT; www.cogmed.com). CWMT is a computerized training program designed to improve working memory by effectively increasing working memory capacity over a five-week training period. The program is built around three age-specific software applications for preschool children, school-age children and adults and was developed to target both storage and manipulation of verbal and nonverbal working memory components. CWMT can only be delivered in settings where there is internet access (e.g., home, school, etc.) making wide-spread dissemination more feasible relative to other interventions.

Although the school-age version includes 12 different working memory exercises, each individual completes only eight exercises during a given session. For the school-age version, training takes place in approximately 30–45 minute increments over five days per week (25 training-days total), with 20 or more training sessions completed within five weeks considered compliance to CWMT. In the school-age version, exercises consist of tasks that require the user to store and/or manipulate visual and/or auditory information. As an example, the user may be presented digits verbally and asked to recall these digits but in reverse order using a visual number pad. In a given training session, the order of the eight work memory exercises are decided by the user but all eight exercises must be completed for the training session to be considered complete. The amount of time to complete an exercise is often approximately 3–4 minutes but will largely depend on how quickly responses are provided by the user for each trial within an exercise.

For all versions of CWMT, the trials are individualized/titrated to the capacity of the individual using an adaptive, staircase design that adjusts the difficulty of the program on a trial-by-trial basis. That is, correct trials are followed by successive trials with heightened working memory demands, whereas incorrect trials result in subsequent trials with diminished working memory load. Contingent reinforcement is integrated within the program such as earning small rewards (e.g., toys, stickers, etc.) for successful completion of a training-day or training-week. Finally, each individual’s training is supervised by a training aide (typically a parent or guardian) and a certified CWMT coach, who is able to closely track (via online access) each individual’s performance. The roles of the training aide and CWMT coach are essential, particularly for youth with ADHD where motivational issues and/or oppositional behavior may detract from compliance to CWMT. During CWMT, the training aide is responsible for supporting the user through reinforcing on-task behavior, effort, and completion of CWMT by providing praise, encouragement and contingent rewards. On a weekly basis by phone, the CWMT coach, training aide, and the user review training and problem-solve struggles with adherence, ranging from motivational to logistic (e.g., scheduling) challenges.

In light of the growing importance of working memory in ADHD, and the current active dissemination of CWMT as an intervention for routine clinical practice (approximately 450 certified CWMT coaches internationally; Pearson, personal communication, April 27, 2012) the current review was designed to apply the evidence-based treatment (EBT) criteria espoused by the Society for Clinical Child and Adolescent Psychology (Silverman & Hinshaw, 2008; See Table 1) to evaluate CWMT for youth with ADHD. The most recent EBT review for youth with ADHD (Pelham & Fabiano, 2008) did not explicitly review the CWMT literature. The ultimate aim of this review is to determine, using the EBT criteria, the short- and longer-term efficacy of CWMT for the treatment of ADHD in youth and to inform clinical practice for the treatment of ADHD.

Table 1.

Evidence-Based Criteria (Silverman & Hinshaw, 2008)

Criteria 1: Well-Established Treatment	Criteria 2: Probably Efficacious Treatments	Criterion 3: Possibly Efficacious Treatments	Criterion 4: Experimental Treatments
1.1 There must be at least two good group-design experiments, conducted in at least two independent research settings and by independent investigatory teams, demonstrating efficacy by showing the treatment to be: a) statistically significantly superior to pill or psychological placebo or to another treatment OR b) equivalent (or not significantly different) to an already established treatment in experiments with statistical power being sufficient to detect moderate differences	2.1 There must be at least two good experiments showing the treatment is superior (statistically significantly so) to a wait-list control group OR Criteria 2.2 2.2 One or more good experiments meeting the Well-Established Treatment Criteria with the one exception of having been conducted in at least two independent research settings and by independent investigatory teams	At least one “good” study showing the treatment to be efficacious in the absence of conflicting evidence	Treatment not yet tested in trials meeting task force criteria for methodology
1.2 Treatment manuals or logical equivalent were used for the treatment
1.3 Conducted with a population, treated for specified problems, for whom inclusion criteria have been delineated in a reliable, valid manner
1.4 Reliable and valid outcome assessment measures, at minimum tapping the problems targeted for change were used
1.5 Appropriate data analyses

Methods

To identify studies which specifically utilized CWMT for the treatment of youth with ADHD, we utilized the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (www.prisma-statement.org). We conducted a systematic literature search in electronic databases (i.e., PsychInfo, PubMed) as well as identified studies through CWMT’s website (www.cogmed.com). Key words that were utilized during the electronic search included “working memory training, computerized intervention, Cogmed Working Memory Training, treatment, ADHD, attention deficit disorder, attention deficit hyperactivity disorder, inattention, hyperactivity, impulsivity, ADD, children, youth, and/or adolescents”. A study was included if it 1) utilized CWMT, 2) included youth (ages 4–17) diagnosed with ADHD or identified as displaying ADHD-type behaviors (i.e., parent- and/or teacher-rated inattention, hyperactivity, and/or impulsivity), 3) determined the short and/or longer-term treatment effects of CWMT, and; 4) was accepted/published in a peer-reviewed English language journal. Identified studies were then coded on the following variables: authors and year of publication, total sample size, age range of study participants, inclusion criteria, comparison group, and the outcome measures used.

Similar to Pelham and Fabiano (2008), the quality of the study design was coded utilizing criteria developed by Nathan and Gorman (2002), which include six types of treatment studies. As detailed in Silverman and Hinshaw (2008), Type 1 studies are the most rigorous and involve randomized, prospective clinical trial methodology. They involve comparison groups with random assignment, blinded assessments, clear presentation of the study’s inclusion and exclusion criteria, state-of-the-art diagnostic methods (i.e., obtaining direct reports from parents and teachers regarding ADHD symptoms and associated impairments to obtain an ADHD diagnosis), adequate sample size to offer statistical power, and clearly described statistical methods. Type 2 studies are clinical trials in which an intervention is tested, but at least one aspect of the Type 1 study requirement is missing. Type 3 studies are open trials aimed at obtaining pilot data. Type 4 studies are reviews with secondary data analyses such as meta-analyses. Type 5 studies are reviews that do not include secondary data analyses. Type 6 studies are case studies, essays, and opinion papers. Finally, we reported effect-size outcomes across various domains (i.e., parent and/or teacher-reports [e.g., ADHD symptoms]; observed behavior; trained cognitive tasks [tasks that closely resemble CWMT training exercises]; and non-trained cognitive tasks). Cohen’s d effect sizes were computed by calculating the change in improvement in the CWMT condition minus the change in improvement in the control condition divided by the pooled standard deviation. Effect sizes were interpreted as no (d= −.20−.19), small (d= .20−.49), moderate (d= .50−.79), and large (d > .80) post-treatment effects of CWMT. For studies that reported multiple outcome measures with a domain, an average effect size across measures was calculated.

Results

The systematic literature search identified 1,091 publications and 1 publication was identified by contacting the primary author listed on the CWMT website. Eleven of these studies were screened eligible for study inclusion. Of these 11, four did not meet inclusion criteria (i.e., did not utilize CWMT as an intervention for the treatment of ADHD in youth), leaving seven studies meeting inclusion criteria (i.e., Beck, Hanson, Puffenberger, Benninger, & Benninger, 2010; Gray et al., 2012; Green et al., 2012; Holmes et al., 2010; Klingberg, Forssberg, & Westerberg, 2002; Klingberg, et al., 2005; Mezacappa & Buckner, 2010). Three individuals coded each categorical criterion for each study. Cohen’s kappa for categorical variables was K= 0.97 (p < .0001), and the agreement rate was 97%. Any disagreements between raters were resolved by consulting the respective paper and by discussion. Of the seven studies, five included youth diagnosed with ADHD within the context of the study (Beck et al., 2010; Holmes et al., 2010; Green et al., 2012; Klingberg et al., 2002; 2005) or from previous diagnosis (Gray et al., 2012), while the last (Mezacappa & Buckner, 2010) included youth whose teachers rated them as having significant issues with attention, hyperactivity and/or impulsivity. All of these studies involved the school-age version of CWMT (Cogmed RM) and met Nathan and Gorman (2002) criteria for a Type 1, 2, or 3 study. Below, we discuss the application of the EBT criteria to these studies (please see Table 2 for description of these studies including Nathan and Gorman classification for each study).

Table 2.

Summary of CWMT studies reviewed

Study	ADHD Formally Diagnosed?	Sample Demographics	N (age range, years)	Gender (% male)	Nathan and Gorman (2002) Criteria for Quality of Study Design and Explanation for Classification
Klingberg et al. 2002	Yes	% medicated: 36 Subtype: N/A Additional Diagnoses: N/A Family/Youth Characteristics: N/A	14 (7–5)	79	Type 2 (did not assess ADHD outcomes; did not utilize state-of-the-art assessment procedures for ADHD diagnosis- diagnoses made by pediatricians using DSM criteria but unknown if parent and teacher reports were collected directly)
Klingberg et al. 2005	Yes	% medicated: 0 Subtype: 38 CT; 15 IT Additional Diagnoses: N/A Family/Youth Characteristics: N/A	53 (7–2)	83	Type 1 (utilized state-of-the-art assessment procedures for ADHD diagnosis; assessed ADHD outcomes; between group design with treatment comparison group)
Beck et al. 2010	Yes	% medicated: 61 Subtype: 29% CT, 71% IT Additional Diagnoses: 46% ODD/CD; 39% ANX; 8% mood disorder Family/Youth Characteristics: 96% Caucasian	52 (7–7)	69	Type 2 (waitlist control design)
Holmes et al. 2010	Yes	% medicated: 100 Subtype: 100% CT Additional Diagnoses: N/A Family/Youth Characteristics: N/A	25 (8–1)	84	Type 3 (did not include state-of-the-art assessment for ADHD diagnosis --diagnoses made by pediatricians using DSM criteria but unknown if parent and teacher reports were collected directly; did not assess ADHD outcomes)
Mezzacappa & Buckner 2010	No official diagnosis	% medicated: 0 Subtype: no official diagnosis Additional Diagnoses: N/A Family/Youth Characteristics: 100% ethnic minority; 100% low SES	9 (8–1)	67	Type 3 (open trial; did not use ADHD diagnosis for inclusion)
Gray et al. 2012	Yes (+LD)	% medicated: 98 Subtype: N/A Additional Diagnoses: N/A Family/Youth Characteristics: N/A	60 (12–7)	87	Type 2 (did not utilize state-of-the-art assessment procedures for ADHD diagnosis - diagnosis made in the community before entering study)
Green et al. 2012	Yes	% medicated: 38 Subtype:: 42% CT, 50% IT, 2% H/I Additional Diagnoses: 0% ODD; Family/Youth Characteristics: 65% Caucasian	26 (7–4)	65	Type 2 (did not utilize state-of-the-art assessment procedures for ADHD diagnosis - no direct teacher ratings for diagnosis)

ANX – Anxiety Disorder; CD – Conduct Disorder; CT – Combined Type; H/I – Hyperactive/Impulsive Type; IT – Inattentive Type; LD – Learning Disorder; N/A – not available; ODD – Oppositional Defiant Disorder; SES – Socioeconomic Status

Studies Contributing to EBT Status

Four of these studies (i.e., Beck et al., 2010; Gray et al., 2012; Green et al., 2012; Klingberg et al., 2005) contribute to the evaluation of CWMT EBT criteria as delineated in Silverman and Hinshaw (2008). These four studies were 1) between-group experiments; 2) conducted in independent research settings by independent investigatory teams; 3) utilized an operationally-defined treatment methodology; 4) were conducted with an ADHD population; 5) utilized reliable and valid ADHD outcomes (i.e., ADHD symptoms), and; 6) applied appropriate data analytic procedures. Next, we discuss these four studies that contribute to the EBT criteria and comment on the extent to which the additional identified studies provide support for the efficacy of CWMT for the treatment of youth with ADHD.

In the only Type 1 study, which was conducted by the CWMT developer, Klingberg et al (2005) randomized 53 youth with ADHD to either CWMT or to a non-titrating, low-level working memory version (i.e., placebo) of CWMT. Of the 27 participants assigned to CWMT, three withdrew and an additional four did not complete CWMT. Recruitment procedures were referrals from “pediatricians, child psychiatrists, and special teachers at school” (p. 178). Inclusion criteria were (1) diagnosis of ADHD Combined Type or Predominantly Inattentive Type established at study intake, (2) being between 7 to 12 years of age, and (3) having access to a personal computer with an Internet connection. Exclusion criteria were (1) being treated with stimulants, atomoxetine, a neuroleptic, or any other psychoactive drugs; (2) meeting criteria for diagnoses of Oppositional Defiant Disorder, Autism Spectrum Disorder or Major Depressive Disorder; (3) having a history of seizures during the past two years; (4) having an IQ less than 80 (based on IQ test or the physician’s clinical impression and school history); (5) having a motor or perceptual handicap that would prevent using the computer program; (6) an educational level and socioeconomic situation that made it unlikely that the family would be able to follow the treatment procedure and study requirements; and (7) having a medical illness requiring immediate treatment. Eighty-three percent of participants were compliant with CWMT.

Intent-to-treat analyses demonstrated the effects of CWMT on various aspects of trained and non-trained cognitive tasks which were maintained at 3-month follow-up (see Table 3). Of most interest, this study demonstrated improvements in most measures of parent-rated symptoms of inattention and all measures of hyperactivity/impulsivity symptoms at post-treatment, which were maintained at 3-month follow-up. No statistically significant effects of CWMT were observed on teacher-rated symptoms of inattention or hyperactivity/impulsivity, or on an objective measure of motor activity. Effect size data suggest a moderate effect of CWMT on parent-rated ADHD (d = .54) and a no effect on teacher-rated ADHD (d= .19).

Table 3.

Cogmed Working Memory Training outcomes by domain

Study	Comparison Condition (effect sizea)	Trained Cognitive Tasks (effect sizea)	Non-Trained Cognitive Tasks (effect sizea)	Behavioral Observations (effect sizea)	Parent-Report (effect sizea)	Teacher-Report (effect sizea)
Beck et al. 2010	Waitlist control	N/A	N/A	N/A	DSM ADHD Symptoms (inattention*); CRS*; BRIEF* (.85)	DSM ADHD Symptoms; CRS; BRIEF (.22)
Gray et al. 2012	Academy of Math	WISC-IV Digit Span Forward and Backward*; CANTAB Spatial Span *; CANTAB Spatial WM ; (.06)	WMR; D2; WRAT-PM (.03)	N/A	SWAN; IOWA Conners Scale (−.08)	SWAN; IOWA Conners Scale (.04)
Green et al. 2012	Placebo computerized working memory training	WISC-IV WMI* (.39)	N/A	RAST*b (.84)	CRS (−.21)	N/A
Holmes et al. 2010	Within subject- compared on and off meds pre/post	AWMA* (.57) c	WASI (.12) c	N/A	N/A	N/A
Klingberg et al. 2002	Placebo computerized working memory training	Visuospatial WM Task*; Span-Board Task* (5.60)	Visuospatial WM Task*; Stroop Task*; Raven's*; CRTT (2.38)	# of Head Movements (infrared motion analysis system)* (2.83)	N/A	N/A
Klingberg et al. 2005	Placebo computerized working memory training	WAIS-RNI Span-Board Task*; WISC-III Digit Span* (.74)	Stroop Task*; Raven's* (.61)	# of Head Movements (infrared motion analysis system) (.07)	DSM ADHD Symptoms*; CRS* (.54)	DSM ADHD Symptoms; CRS (.19)
Mezzacappa & Buckner 2010 d	Pre/post- no control group	WISC IV Digit Span*; WRAML Finger Windows*d	N/A	N/A	N/A	DSM ADHD Symptoms; ADHD-RS IV* d

^aCohen’s d reported for each domain. Average effect size calculated for domains where there are multiple measures

^bEffect Size calculated only for continuous data as other categories were coded as binary outcomes.

^cEffect Size determined by difference between no medication condition and CWMT

^dStandard deviations were not reported in study in order to calculate effect sizes.

^*Significant difference between groups in favor of Cogmed Working Memory Training

Note: CRS = Conners Rating Scale; CANTAB = Cambridge Neuropsychological Testing Automated Battery; DSM= Diagnostic and Statistical Manual of Mental Disorders; ADHD = Attention Deficit/Hyperactivity Disorder; WAIS-RNI = Wechsler Adult Intelligence Scale – Revised as a Neuropsychological Instrument; WISC = Wechsler Intelligence Test for Children; BRIEF = Behavior Rating Inventory of Executive Function; AWMA = Automated Working Memory Assessment; WRAML = Wide Range Assessment of Memory and Learning ; WMI = Working Memory Index; WASI = Wechsler Abbreviated Scale of Intelligence; ADHD-RS IV = Attention Deficit/Hyperactivity Disorder Rating Scale, Fourth Edition; WM = Working Memory; WMR = Working Memory Rating Scale; WRAT-PM = Wide-Range Achievement Test-4-Progress Monitoring Version; SWAN = Strengths and Weakness of ADHD-symptoms and Normal-behavior Scale; RAST = Restricted Academic Situations Task; Raven’s = Raven's Colored Progressive Matrices; D2 = The D2 Test of Attention; CRTT = Choice Reaction Time Task

In a randomized clinical trial of CWMT with 52 youth with ADHD, Beck and colleagues (2010) replicated many of the findings of Klingberg et al (2005). Children and adolescents were recruited from a private school for youth with ADHD and/or learning difficulties through flyers sent to parents. All participants met DSM-IV criteria for ADHD using a structured clinical interview at study intake. No additional information was provided regarding inclusion/exclusion criteria. Information gleaned from Beck and colleagues suggest that all participants completed CWMT but no clear data were reported regarding youth compliance to CWMT.

Beck et al. (2010) determined the efficacy of CWMT compared to a wait-list group on parent- and teacher-rated reports of working memory and ADHD symptoms (see Table 3 for details). Youth randomly assigned to the wait-list control condition received CWMT after the post-treatment assessment. All study participants completed a 4-month follow-up assessment. Results from this study indicated that participants in the CWMT condition had improved parent-rated working memory, parent-rated inattention symptoms/problems at post-treatment, which were maintained at follow-up assessment. No statistically significant effect of CWMT was observed on parent-reported hyperactivity/impulsivity symptoms, oppositional-defiant behaviors or on any teacher-rated outcomes. The post-treatment results were mostly replicated when the wait-list control youth participated in the CWMT intervention but significant attrition in the wait-list group prohibited meaningful comparisons of follow-up outcomes. Effect size data suggest a large effect of CWMT on parent-rated ADHD symptoms (d= .85) and a small effect on teacher-rated ADHD symptoms (d= .22).

Green and colleagues (2012) conducted a randomized clinical trial comparing CWMT to a placebo-version of CWMT (i.e., identical to that used in Klingberg et al., 2005) in a sample of 30 youth with ADHD. Youth were recruited from the participating medical center, as well as from flyers and advertisements posted around the community and to a support group. Eligibility for the study involved a multi-step process, including 1) positive phone screening for ADHD; 2) meeting DSM criteria for ADHD based on structured clinical interview at study intake, and; 3) a t-score of at least 65 on the Conners Parent Rating Scale. Families also had to have reliable internet access and speak English fluently. Youth were excluded if they met criteria for mental retardation (i.e., IQ < 70) or severe mental illness (i.e., psychosis, bipolar, major depressive disorder) or autism spectrum disorder. In addition, families were asked not to make changes in treatment during the course of the study.

Green and colleagues (2012) found that, relative to the placebo condition, CWMT resulted in significant benefits on objective working memory tasks (See Table 3). Interestingly, this was the first study to determine the effects of CWMT on observed behaviors. The Restricted Academic Setting Task (RAST) is a 15-minute task where the child, in isolation, engages in academic worksheets (e.g., math) that are one grade level below their current ability. Behaviors observed and quantified during the task include off-task behavior, out of seat behavior, fidgeting, vocalizing, and playing with objects. CWMT led to significant reductions in off-task behavior and in playing with objects. Surprisingly, CWMT did not improve parent-rated ADHD symptoms. Importantly, parents in both treatment conditions reported significant improvements in ADHD symptoms as a function of time. This study only analyzed data from participants who completed the placebo (14 of the 15 participants) and the CWMT (12 of the 15 participants) interventions. As such, results reflect those who are compliant to treatment. Effect size data suggest a small negative effect of CWMT on parent-rated ADHD symptoms (d= −.21); teacher-rated ADHD symptoms were not collected.

Most recently, in a randomized clinical trial of 60 adolescents with severe Learning Disorders (LD) and ADHD, Gray and colleagues (2012) evaluated CWMT compared to an intensive computerized academic instruction program (Academy of Math; www.autoskill.com). Gray and colleagues used an unbalanced design, randomly assigning 36 of the 60 participants to the CWMT condition. Of these 36, four did not complete CWMT (two participants moved from the school and two participants withdrew from CWMT resulting in a compliance rate of 88%). Both interventions occurred during the school day. Interestingly, this was the only study of CWMT delivered within the context of a school-setting by school staff. Youth were recruited from a semi-residential school for adolescents with severely impairing LD and ADHD. Eligibility criteria for the school include co-occurring LD/ADHD previously diagnosed in the community plus severe problems in learning and behavior with poor response to the available standards of care and intervention in their home communities, which included special education and pharmacological treatment. Students with comorbid diagnoses of conduct disorder, severe aggression, depression or anxiety requiring specific and immediate treatments were considered ineligible for the school. Inclusion criteria for this study were: (1) full time attendance at the participating school; (2) age between 12 and 17 years; (3) IQ > 80; and (4) English as the primary spoken language. ADHD diagnosis was based on available reports obtained from previous assessment in the community. Exclusion criteria were uncorrected perceptual, motor or language impairments that would impede usage of the computer program or intelligibility of spoken responses.

Results demonstrated effects of CWMT on two of the three trained working memory tasks. No differences were found on non-trained cognitive tasks. CWMT had no differential effect on parent- or teacher-rated ADHD behavior or oppositional behavior. There were no effects found on academic measures (see Table 3). Supplemental analyses revealed that there was an effect of time on cognitive attention, reading and math as well as parent-reported ADHD behavior. This suggests that the effect of the remedial educational setting combined with stimulant medication improved some key areas of functioning. Effect size data suggest no effect of CWMT on parent-rated ADHD symptoms (d= −.08) and no effect on teacher-rated ADHD symptoms (d= .04).

Additional Studies Focusing on CWMT

Three additional studies support the efficacy of CWMT but were not considered to contribute to the determination of EBT status because these studies did not utilize ADHD-symptom outcomes (Holmes et al., 2010; Klingberg et al., 2002) or did not include youth with a formal diagnosis of ADHD (Mezzacappa & Buckner, 2010). We briefly review these below.

Klingberg and colleagues (2002) randomly assigned 14 youth diagnosed with ADHD at study intake to either CWMT or a placebo version of CWMT. No data regarding recruitment strategies or compliance to CWMT were reported. Several outcomes were assessed (see Table 3). This study found that CWMT resulted in significantly greater improvements in trained and non-trained cognitive tasks as well as observed behaviors (i.e., head movements).

Holmes and colleagues (2010) recruited 25 children through pediatric psychiatrists and community pediatricians to participate in a within-subject study evaluating the effects of medication and CWMT on working memory and IQ. Children were included in the study if they met criteria for ADHD Combined type at study intake and were prescribed stimulant medication. No data on compliance to CWMT was reported. Across a four-time-point assessment schedule, the effects of no medication (time 1), medication (time 2), CWMT (time 3 post CWMT) and 6-month follow-up (time 4) revealed differences between treatments. More specifically, there was no effect of medication on IQ scores but improved several aspects of working memory relative to no medication. Relative to medication, CWMT demonstrated improvements across working memory outcomes immediately after CWMT (time 3) and were maintained at 6-month follow-up (time 4). No effect of CWMT was found on IQ (see Table 3). Importantly, a limitation of this study was that treatments were confounded with time, making interpretation of these results challenging.

Finally, Mezzacappa and Buckner (2010) evaluated CWMT in nine children who were attending an inner-city elementary school. All children were identified for this study based on teacher-reports of ADHD symptoms on rating scales. No child received previous assessment or treatment for any psychiatric difficulties in the past or during the study. Administered by a research assistant, CWMT was implemented individually during the school-day for five weeks. Eight of the nine participants completed CWMT. Interestingly, results from this study demonstrated improvements in teacher-rated total ADHD symptoms and measures of working memory (see Table 3). However, the role of teacher expectancy bias for improvement could not be ruled-out as a possible explanation for these findings; parents did not complete measures of treatment efficacy in this study.

Discussion

Overview of Results from Key Studies

Although only recently developed, CWMT has undergone empirical evaluation over the past decade as a treatment for ADHD. The utilization of randomized clinical trials by multiple, independent investigators, assessing multiple outcomes by various reporters, are notable strengths of these evaluations. Moreover, studies have been conducted with school-age youth and/or adolescents in both home and in school settings. The results of the four randomized clinical trials and the additional studies reviewed suggest more consistent effects of CWMT on working memory outcomes (see Table 3). The results of the four studies that contribute to the EBT status of CWMT suggest mixed findings for ADHD symptom outcomes. Two studies generally support CWMT for school-age children and adolescents with ADHD (Beck et al., 2010; Klingberg et al., 2005). These two studies utilized a wait-list control condition (Beck et al., 2010) or a placebo condition (Klingberg et al., 2005) with a well-diagnosed ADHD population and generally found significant short- and longer-term effects of CWMT on ADHD symptoms as reported by parents. Green and colleagues (2012) found effects of CWMT relative to a placebo condition on two of five observed behavioral categories during an analog academic task but found no effect of CWMT on parent-rated ADHD. Data from Gray and colleagues (2012) do not support CWMT as an intervention for ADHD. In a randomized clinical trial of adolescents with ADHD and severe LD, they reported no effects of CWMT on parent- and teacher-rated ADHD symptoms. The unique strength of the Gray and colleagues study was the rigor of the comparison condition—this was the only randomized clinical trial of CWMT to date with an active treatment comparison condition. Moreover, data document a wide range of effect sizes on parent and teacher reports of ADHD symptoms, which is often related to the type of control condition utilized (Fabiano et al., 2009). For instance, the largest effect size was found in Beck and colleagues study (d=.85) which utilized a wait-list control condition while considerably smaller effects were found in the other three studies, all of which included a treatment control condition. Critical issues in these studies, as discussed below, which mitigate the clarity of the findings, include lack of alignment between outcomes of CWMT studies and the hypothesized model of therapeutic benefit of CWMT, the equivalence of the placebo condition used in several studies, and individual response to CWMT.

Critical Issues for Interpreting CWMT Studies

Alignment between CWMT outcomes and hypothesized model of therapeutic benefit of CWMT

As we have reviewed herein and detailed in Table 3, there are significant inconsistencies in outcomes within as well as across CWMT studies. For example, Klingberg and colleagues (2005) found parent-rated improvements in hyperactivity but no effect of CWMT on head movements (an objective indicator of hyperactivity). Head movements were, however, improved following CWMT in an earlier study (Klingberg et al., 2002). Moreover, although Klingberg and colleagues (2005) found significant effects of CWMT on some parent-rated inattention outcomes and all hyperactivity/impulsivity outcomes, Beck and colleagues (2010) found benefits only on parent-rated inattention symptoms and Gray et al reported no effects of CWMT on either ADHD symptom domain. Although Green and colleagues (2012) found a benefit of CWMT on ADHD-type behaviors observed during an analog academic task, no effect of CWMT was reported on parent-rated ADHD symptoms. No study has found an effect of CWMT on teacher-rated ADHD symptoms. Multiple reasons are likely related to these various discrepancies, including differences in study designs, measurement issues, and individual differences (a point we discuss below). However, a more fundamental issue that should guide interpretation of this data is the extent to which findings from CWMT studies align with the hypothesized model of therapeutic benefit of CWMT.

For CWMT, it is hypothesized that the intervention has a proximal effect on working memory capacity, which then impacts ADHD symptoms, notably inattentive symptoms. This model supposes that proximal effects of CWMT on working memory should have distal effects on inattention symptoms in various settings. This should likely be most evident in settings where working memory is taxed (e.g., classroom settings where youth must follow complex multi-step directions, read and comprehend text, plan and organize class materials; etc.). This model of therapeutic benefit is akin to stimulant medication—underlying neuropsychological-biological processes are proximally improved and improvements in these factors have distal effects on ADHD symptoms. Importantly, improvement in ADHD symptoms should be apparent regardless of setting, given the hypothesized model. The finding that no randomized clinical trial of CWMT, including those with a wait-list comparison condition (i.e., Beck et al., 2010), has found an effect of CWMT on teacher-reports of ADHD is not in-line with the hypothesized model of therapeutic benefit of CWMT. Moreover, findings that both parent-reported and objectively-rated hyperactivity are improved with CWMT are at-odds with the purported therapeutic model. These are fundamental issues that go well beyond the discrepancies and inconsistencies found in the CWMT studies done to date and call into question the true impact of this intervention.

Placebo Condition Equivalence

The study by Klingberg and colleagues (2005) provides the most supportive evidence in favor of CWMT as a treatment for ADHD in youth. The placebo condition used in this study was designed to control for parent involvement as the training aide and child exposure to computer training. Importantly although the placebo condition consisted of the same number of working memory training trials as the active condition (i.e., 90 trials), overall training time was not matched between the two conditions (Pearson, personal communication, November 22, 2011). Since the placebo condition was easier (non-adaptive), it required considerably less time to complete than the active CWMT condition. These two factors may have significantly impacted the experience that parents had during the study, which has direct implications for interpreting parent-rated improvements in ADHD. First, an easier and quicker intervention reduces the amount and quality of interactions the training aide (i.e., parent) has with their child during training. Given that part of the training experience is to support the child through positive reinforcement and encouragement, reduced training time likely also means reduced opportunities for supportive interactions between the parent and child, a noted important aspect of CWMT (Holmes et al., 2010). This difference should not be minimized, as small, daily positive interactions in which the parent provides praise and encouragement can be very helpful in improving parent-child relationships and child behaviors (Harwood & Eyberg, 2006).

In addition, participating in a briefer and easier intervention may also influence the type of support that the CWMT coach provides to a family in the control condition. Much of the responsibilities of the CWMT coach are to work closely with the family to encourage adherence, enhance motivation, and to problem solve issues that may arise in implementing CWMT. The CWMT coach’s ability to work in a close, supportive partnership with the training aide is likely essential for maximizing adherence and engagement to CWMT. The benefits of providing support and collaborative problem-solving should not be underestimated—treatment outcomes studies in youth with ADHD have demonstrated that such support can improve parent reports of ADHD symptoms (Sonuga-Barke, Thompson, Daley, Laver-Bradbury, & Weeks, 2001). It is unlikely that families in the placebo condition received the same level of support and problem-solving opportunities given the non-adaptive and brief nature of the placebo intervention.

Collectively, these differences in interactions as a function of treatment condition may partially explain some of the benefit of participation in CWMT for participants in the Klingberg et al. (2005) study. This may be even more of a significant and confounding issue if the skills learned by parents (i.e. problem solving issues with motivation, using praise and encouragement) extended well beyond CWMT to other contexts (e.g., homework time) where these skills could affect ADHD-related symptoms and difficulties. It is unclear the extent to which specific components of CWMT (i.e., adaptive working memory training) versus other factors (e.g., parents learning methods to support their child’s behavior) contribute to the results found in studies which have compared CWMT to the CWMT placebo condition.

Individuals differences and response to CWMT

The four between group studies of CWMT have recruited strikingly different participants, which poses a significant issue in interpreting the results of these studies. For example, Klingberg and colleagues (2005) excluded youth with oppositional defiant disorder and those who were taking psychoactive medication. This study found improvements in parent-reports of ADHD symptoms as well as other trained and non-trained cognitive tasks. Gray and colleagues (2012) included youth with severe learning problems and ADHD such that available standards of care and intervention in their home communities were not beneficial. Additionally, almost the entire sample was receiving medication. As reviewed above, this study largely did not find differential treatment effects, particularly on parent and teacher-rated ADHD symptoms. One interpretation of these discrepant findings is that individual differences, such as severity of psychopathology, may moderate treatment response. This is not surprising given that these types of individual differences can be related to treatment outcomes (Owens et al., 2003). This issue is particularly challenging in the context of interpreting the CWMT literature given that there are few studies focused on youth with ADHD. As we discuss below in future directions, more representative samples of youth are needed to better gauge the effects of CWMT.

EBT Status of CWMT

Collectively, when accounting for the strengths and limitations of the four CWMT studies that contribute to the classification of EBT status (i.e., Beck et al., 2010; Gray et al., 2012; Green et al., 2012; Klingberg et al., 2005) and noting the fundamental issues with alignment between outcomes of CWMT and hypothesized model of therapeutic benefit purported by CWMT, CWMT best meets criteria as a Possibly Efficacious treatment for youth with ADHD (see Table 1 for EBT criteria). Although Beck et al. (2010) and Klingberg et al. (2005) offer supportive results for CWMT, findings are mitigated by critical issues in these studies. Green and colleagues (2012) provide evidence for the benefits of CWMT on observed behavior; however, the extent to which the analog context of the observation is representative of academic tasks in typical classroom settings is questionable. More specifically, assessing behavior during an analogue situation where a child completes academic work that is at least one grade level below their ability, in isolation, for 15-minutes is not representative of typical academic contexts. Moreover, no normative data were provided on the RAST making it difficult to interpret the clinical utility of the observed improvements in behavior following CWMT. In addition, the lack of effects of CWMT on parent-rated ADHD found in the study by Green and colleagues is not clearly interpretable. Collectively, the data for the most supportive studies of CWMT (Beck; Green; Klingberg et al., 2005) are inconsistent with the therapeutic model of CWMT, and warrant caution in placing high levels of confidence in CWMT. Gray and colleagues (2012) recent study suggests further caution when considering the EBT status for CWMT, although it is likely that the severity of the sample may have moderated response to treatment. Below we discuss future directions for CWMT for the treatment of youth with ADHD and consider clinical implications of the findings of this review.

Future Directions

Increasing the heterogeneity of participants to obtain more clinically-useful data

Each of the four studies that contributed to the classification of the EBT status of CWMT had significant restrictions regarding the inclusion criteria and/or the population recruited which likely impacted study results and implications. For instance, Klingberg and colleagues (2005) emphasized internal validity of CWMT by maximizing the likelihood that youth and families would adhere to CWMT by excluding youth with ODD and families from lower socioeconomic status. Although doing so is important and logical in the early stage development of an intervention, it did limit understanding of the extent to which these findings generalize to more comorbid and diverse samples.

Green and colleagues (2012) conducted the smallest randomized clinical trial reviewed herein and limited information regarding the sample was presented. Moreover, the sample represented a pure ADHD group in that no child met criteria for ODD (J. Schweitzer, personal communication, October 12, 2012). Beck and colleagues (2010) included youth without utilizing significant exclusionary criteria. Despite these improvements, participants were exclusively recruited from one private school setting suggesting that these youth and their families were more homogenous in terms of socioeconomic status. Given that both clearly supportive studies of CWMT (Beck et al., 2010; Klingberg et al., 2005) focused on youth from higher socioeconomic status, and that socioeconomic status and its correlates are often associated with adherence to psychosocial and pharmacological intervention (Chacko et al., 2010), it will be important to determine if the high rates of adherence, feasibility, and palatability seen thus far with CWMT generalize to families with greater psychosocial challenges. These types of data have direct implications for the potential broader dissemination of CWMT.

Gray and colleagues recruited participants from one school that served adolescents who were most impaired in their community school settings. Importantly, it may have been difficult to observe incremental benefits of CWMT (or any other additional intervention) in this study given that all adolescents were involved in intensive academic interventions and stimulant medication as part of their specialized school placement. These limitations and qualifications of the Gray study do not negate the findings of the lack of effects of CWMT. However, it may qualify these findings; that is, CWMT may be unable to provide incremental benefits beyond intensive academic and pharmacological interventions for youth with more severe psychopathology but may be beneficial for more representative youth with ADHD.

Collectively, it appears that studies of CWMT that represent the diversity of socioeconomic backgrounds, psychiatric comorbidities, and functioning of youth with ADHD and their families are needed. Given that CWMT is no longer in a treatment development phase, but being actively disseminated for clinical use, data from more representative studies of youth with ADHD are essential to providers in order to make informed decisions about utilizing CWMT.

Utilizing equivalent placebo control conditions or active treatment comparison conditions

There is clearly a need for studies to be conducted that utilize well-designed control conditions that balance time on task and interactions between the training aide (parent) and the child as well as the CWMT coach and the family. The only study meeting Nathan and Gorman criteria for a Type 1 study (Klingberg et al., 2005) utilized a placebo condition that arguably did not control for these critical factors and therefore may not have provided an equivalent experience for youth and their families in the placebo control condition. Even more important are studies that utilize active treatment comparison conditions, such as what was employed in Gray and colleagues (2012). Such interventions have to be chosen thoughtfully as to be viable and matched on multiple parameters (scaffolding frequency, difficulty level, etc).

Potential benefits of CWMT for subgroups of youth with ADHD

There is the potential for CWMT to be a key intervention for distinct subgroups rather than for the general population of youth with ADHD. For example, theoretically, CWMT may be more effective for youth with ADHD Inattentive subtype, although Klingberg and colleagues (2005) found no differential effects of CWMT by ADHD subtype. Further empirical investigation with larger samples will be required to more fully evaluate this issue.

CWMT may be differentially effective as a function of developmental age. The four randomized clinical trials of CWMT do not provide direct information regarding the potential impact of CWMT as a function of age. Klingberg and colleagues (2005) found benefit of CWMT for school-age youth while Gray and colleagues (2012) reported no benefit in a group of adolescents. Green et al., (2012) found benefit of CWMT on some observed behaviors for school-age youth with ADHD but no effect on parent-rated behavior. Finally, Beck and colleagues demonstrated a positive impact of CWMT in a wide range of youth with ADHD (7–17 year-olds), with no significant differences as a function of age. However, since these studies varied greatly in the type and quality of comparison conditions and participant populations, definitive conclusions about age effects of CWMT are premature.

Theoretically, CWMT may be most effective as an early intervention for ADHD given the increased plasticity and sensitivity of the brain of young children (Halperin & Healey, 2011). Interestingly, CWMT has been evaluated in typically developing preschool children with promising results (Thorell et al., 2009) but data do not yet exist on its efficacy in preschool children at risk for or with ADHD. This appears to be a promising line of empirical investigation.

In addition, CWMT may be effective for youth with LD. Although Gray and colleagues (2012) found no benefit of CWMT in a group of adolescents with ADHD and LD, the severity of LD in this population along with the receipt of intensive academic interventions during the course of the year may have been limiting factors in realizing the benefits of CWMT on academic achievement. Given the role of working memory on reading and math achievement (Swanson & Jerman, 2007; Swanson, Jerman & Zheng, 2008), there is reason to predict that CWMT, which is designed to improve working memory, may beneficially impact academic outcomes. For instance, Dahlin (2011) found that CWMT had a significant impact of academic outcomes in a sample of children, many of whom had significant learning difficulties.

Given that CWMT is purported to have effects on ADHD symptoms via improvement in working memory, an interesting hypothesis is that youth with ADHD and working memory deficits should benefit the most from this intervention. Importantly, although working memory deficits are associated with ADHD, not all youth with ADHD have working memory deficits (Willicutt et al., 2005). As such, this distinct subgroup of youth with ADHD may be most responsive to CWMT. In fact, Gray and colleagues (2012) found a significant positive correlation between improvements in working memory, measured by the CWMT Index score, and parent-rated improvements in ADHD symptoms. This remains a beneficial area of investigation.

Broadening focus of outcomes to include functional impairment

Outcomes of interest in studies of CWMT have focused primarily on neuropsychological outcomes and collateral reports of ADHD symptoms. Additionally, Gray and colleagues (2012) reported on the impact of CWMT on functional impairment in daily activities (academic achievement). Green and colleagues (2012) also assessed the effects of CWMT on behaviors during academic tasks. Importantly, there are clear limitations to the representativeness and potential generalizability of the analog academic task utilized in the Green study. This is not to say that analog/laboratory tasks are not useful as outcomes in treatment studies. Relative to parent/teacher reports, laboratory measures/tasks may be more objective and may be more sensitive to treatment effects (Sonuga-Barke, Coghill, DeBacker, & Swanson, 2009). Parent/teacher reports may be influenced by expectations, biases, and are likely more resistant to change—greater effects of treatment may be necessary for parents/teachers to report change in behavior. We argue that the more closely laboratory/analog tasks resemble/align with the day-to-day contexts that youth with ADHD experience, the more relevant positive outcomes on these laboratory/analog tasks will be for understanding treatment effects.

Functional impairments experienced by youth with ADHD (e.g., parent-child relationships, academic functioning) should be evaluated in addition to neuropsychological deficits and ADHD symptoms for several reasons. First, functional impairments experienced by youth with ADHD are socially valid outcomes in that these are the reasons why parents seek out treatment for their children, and are key predictors of long term outcomes in youth (See Pelham & Fabiano, 2008). Moreover, ADHD symptoms and impairment are related but distinct constructs, suggesting that improving ADHD symptoms may not necessarily lead to substantial reductions in impairment. For instance, Gordon and colleagues (2006) found that the intensity and severity of ADHD symptoms only accounted for approximately 10% of the variance in impairment, suggesting that ADHD symptoms alone do not provide a full conceptualization of an individual’s functioning. Given these findings, studies should focus not only on the extent to which an intervention improves ADHD symptoms alone but, perhaps more importantly, the extent to which an intervention improves a child’s impairment across multiple domains.

Clinical Implications

Behavioral and pharmacological treatments are considered to be the gold-standard of interventions for ADHD. These interventions have been tested in numerous short- and long-term randomized clinical trials, utilizing a wide-range of youth with ADHD and their families, while assessing multiple and various outcomes and have been shown to be acutely effective in improving not only ADHD symptoms but also functional impairment (Pelham & Fabiano, 2008). Effects sizes reported of BPT in between-group studies of ADHD have demonstrated an average small-to-moderate effect on parent-rated ADHD symptoms (d= .39) and an average moderate-to-large effect on teacher-rated ADHD symptoms (d=.79) (Fabiano et al., 2009). Importantly, behavioral interventions are theorized to affect impairment associated with ADHD rather than core symptoms—data support this view with effects sizes for between-group studies of ADHD ranging from moderate-to-large effects (d=.55−.84; Fabiano et al., 2009). Comparatively, effects sizes reported for stimulant medication treatment for ADHD symptoms are large (i.e., d > 1.00; Faraone, 2009).

In comparison, the evidence-base for CWMT consists of a handful of studies conducted with more restricted populations, evaluating specific key behavioral and neurocognitive outcomes. In general effect sizes for ADHD symptoms suggest a small negative effect to a large effect (d= −.21 to .85) of CWMT and appear to vary significantly as a function of quality of control condition, respondent (parent or teacher), and severity of the study participants. As such, at the present time, confidence in the efficacy of behavioral interventions and specific medications for the treatment of ADHD in youth is greater than that for CWMT. Clearly, behavioral interventions and specific pharmacological interventions should be considered first-line interventions for the treatment of ADHD. Given the state of the literature on CWMT, further investigation is needed before CWMT can be considered a clearly viable treatment for youth with ADHD. In particular, data from CWMT studies are not well-aligned with the hypothesized model of therapeutic benefits of CWMT, which suggests further caution in providing clinical recommendations for CWMT as an intervention for youth with ADHD. Utilizing the EBT criteria delineated by Silverman and Hinshaw (2008), our interpretation of the strengths and limitations of the data across several studies suggest that CWMT is currently a Possibly Efficacious treatment for youth with ADHD. Future studies should represent the diversity of youth with ADHD and their families and include diverse outcomes, including observed objective outcomes (cf. Green et al., 2012) and functional impairment outcomes. Closer attention must be paid to conducting studies to specifically evaluate the hypothesized model of therapeutic benefit of CWMT for the treatment of ADHD. Given that multiple clinical trials using CWMT for ADHD in youth are now ongoing, the literature in this area is rapidly progressing and will likely answer many of the questions posed herein and provide greater clarification on several issues regarding CWMT and further contribute to determining the EBT status of CWMT for youth with ADHD.