Can Working Memory Training Work for ADHD? Development of Central Executive Training and Comparison with Behavioral Parent Training

Abstract

Objective

Working memory deficits have been linked experimentally and developmentally with ADHD-related symptoms/impairments. Unfortunately, substantial evidence indicates that extant working memory training programs fail to improve these symptoms/impairments. We hypothesized that this discrepancy may reflect insufficient targeting, such that extant protocols do not adequately engage the specific working memory components linked with the disorder’s behavioral/functional impairments.

Method

The current study describes the development, empirical basis, and initial testing of central executive training (CET) relative to gold-standard behavioral parent training (BPT). Children with ADHD ages 8–13 (M=10.43, SD=1.59; 21 girls; 76% Caucasian/Non-Hispanic) were treated using BPT (n=27) or CET (n=27). Detailed data analytic plans for the pre/post design were preregistered. Primary outcomes included phonological and visuospatial working memory, and secondary outcomes included actigraphy during working memory testing and two distal far-transfer tasks. Multiple feasibility/acceptability measures were included.

Results

The BPT and CET samples did not differ on any pre-treatment characteristics. CET was rated as highly acceptable by children, and was equivalent to BPT in terms of feasibility/acceptability as evidenced by parent-reported high satisfaction, low barriers to participation, and large ADHD symptom reductions. CET was superior to BPT for improving working memory (group x time d=1.06) as hypothesized. CET was also superior to BPT for reducing actigraph-measured hyperactivity during visuospatial working memory testing and both distal far-transfer tasks (group x time d=0.74).

Conclusions

Results provide strong support for continued testing of CET and, if replicated, would support recent hypotheses that next-generation ADHD cognitive training protocols may overcome current limitations via improved targeting.

Working memory deficits are present in a substantial portion of children with attention-deficit/hyperactivity disorder (ADHD; Kasper et al., 2012), and have been linked with core behavioral symptoms of inattention (Kofler et al., 2010; Gathercole et al., 2008), hyperactivity (Hudec et al., 2015; Rapport et al., 2009), and impulsivity (Raiker et al., 2012). Working memory deficits covary longitudinally with ADHD symptom severity (Halperin et al., 2008; Salari et al., 2017; van Lieshout et al., 2016), and age-related reductions in ADHD symptoms appear limited to a subset of children who show age-related improvements in working memory (Karalunas et al., in press). Further, working memory deficits in ADHD have been linked with ADHD-related impairments in academic (Friedman et al., 2017), social (Bunford et al., 2014), organizational (Kofler et al., 2017a) and family functioning (Kofler et al., 2017b).

Given this preponderance of evidence, the recent upsurge in attempts to improve working memory in pediatric ADHD is unsurprising. More surprising, however, is the inefficacy of these protocols, as documented in comprehensive meta-analyses of working memory training studies for children with ADHD (Cortese et al., 2015; Rapport et al., 2013), neurotypical children (Sala & Gobet, 2017), and diverse samples of children and adults (Melby-Lervåg et al., 2016; Shipstead et al., 2012). These independent reports uniformly conclude that the evidence supporting working memory training is limited to uncontrolled or under-controlled studies, is found almost exclusively for unblinded parent ratings (Cortese et al., 2015; Rapport et al., 2013) and tests of short-term memory rather than working memory (Chacko et al., 2014a; Roberts et al., 2016), and/or reflects misinterpretation of interaction effects that show control group decrements rather than treatment group improvements (Redick, 2015). Taken together, the evidence strongly indicates that “claims regarding the academic, behavioral, and cognitive benefits associated with extant cognitive training programs are unsupported in ADHD” (Rapport et al., 2013, p. 1237), and as such working memory training “cannot be recommended as an educational tool” (Sala & Gobet, 2017, p. 682) and “should not be considered a viable treatment for children with ADHD” (Chacko et al., 2014a, p. 247).

Taken together, the literature indicates that working memory shows strong covariation with ADHD’s core behavioral and functional impairments but that working memory training fails to improve these outcomes. The reason for this incongruence is not readily clear; however, two hypotheses show promise. First, working memory’s association with ADHD symptoms may be correlational, and better attributed to a common underlying mechanism (Barkley, 1997a). In this case, the lack of transfer from working memory training to other skills/abilities would be unsurprising because treating a secondary/peripheral symptom is unlikely to travel upstream to affect core, underlying deficits in the absence of transactional or reciprocal influences (Rapport et al., 2001). Similarly, their developmental covariation may reflect an epiphenomenal association in that neurocognitive functioning predicts ADHD symptoms but is not causal (van Lieshout et al., 2013). These possibilities are undermined at least partially, however, based on experimental evidence that manipulating working memory demands can provocate and rarefy objectively-assessed ADHD inattentive (Kofler et al., 2010) and hyperactive behavior (Rapport et al., 2009).

An alternate possibility is that extant training protocols target aspects of the working memory system that are either intact in ADHD or unrelated to the disorder’s phenotypic expression. The possibility of target misspecification was explored in a recent meta-analysis (Rapport et al., 2013), which concluded that current working memory trainings for ADHD focus almost exclusively on short-term memory (simple storage/rehearsal) rather than the dual-processing, continuous updating, and serial-reordering processes that specifically characterize the working memory construct (Wager & Smith, 2003). Further, both Chacko et al. (2014a) and Gibson et al. (2011) differentiated short-term memory (maintenance) from working memory (processing + maintenance), and found that training effects are limited to short-term memory, with no significant improvement in working memory capacity for children with ADHD. A recent, population-based RCT of working memory training found the same: moderate gains on one of two measures of short-term memory, but no significant gains on any test of working memory (Roberts et al., 2016).

Thus, a compelling possibility is that working memory training is ineffective because it does not train working memory. More specifically, current training tasks used in the ADHD literature may place insufficient demands on the specific processes that are most impaired in ADHD and drive working memory’s association with important behavioral and educational outcomes (for review, see Rapport et al., 2013). This hypothesis was the driving force behind the development of Central Executive Training (CET), a computerized training protocol created to provide broad training of the three primary processes that comprise the working memory construct (dual-processing, continuous updating, serial reordering; Wager & Smith, 2003). Dual-processing refers to diverse processes that involve operating on information while concurrently storing the same/different information (often measured by ‘complex span’ tasks; Conway et al., 2005), continuous updating refers to the active addition and deletion of items from working memory (Miyake & Friedman, 2012), and serial reordering refers to the mental manipulation of temporal/sequential order (Rapport et al., 2009). We collectively refer to these three interrelated but separable functions of the mid-lateral prefrontal cortex and interconnected networks (Nee et al., 2013; Wager & Smith, 2003) as the central executive (Baddeley, 2007), or the working components of working memory (Kasper et al., 2012; Rapport et al., 2013; Wells et al., 2018).

Central Executive Training (CET): Core Assumptions

A central tenet of computerized, cognitive training protocols is that meaningful and sustained improvement at the neural substrate level can be accomplished by extensive training involving repetition, practice, and feedback on activities supported by these neural networks (Rapport et al., 2013; Sala & Gobet, 2017; Shipstead et al., 2012). By extension, this improvement is expected to transfer to other skills and abilities that rely on the same neural networks (Simons et al., 2017).

Applied to CET, the following assumptions guided training task development:

1. Central executive deficits are an underlying neurocognitive mechanism that drive, to a significant extent, ADHD’s phenotypic expression for many children with ADHD (Rapport et al., 2009).

2. Central executive abilities will improve with training for children with ADHD.

3. ADHD symptoms will decrease as central executive abilities improve, to the extent that this empirically-identified target is central to ADHD as hypothesized. We do not expect normalization of ADHD symptoms; 10-weeks of training cannot be realistically expected to overcome 3–4 year delays in CE-related cortical maturation found in pediatric ADHD samples (Shaw et al., 2007). We speculate that continued training will produce incremental gains over time or slight alterations in developmental trajectories that may only be realized over time (Halperin & Healey, 2011).

4. Based on the clinical model of psychopathology (Rapport et al., 2001) and the greater impact of proximal confounding influences (e.g., skill/knowledge gaps), CET’s effects on peer, family, academic, and organizational impairments in ADHD will be more modest. CET does not teach social or academic skills; rather, these skills are expected to become more amenable to targeted intervention following improvements in CE-related neural substrates that support planning and maintenance of goal-directed behavior (Chacko et al., 2014b, 2017).

5. More generally, the magnitude of improvement on any untrained outcome will be capped by (a) the magnitude of improvement in central executive working memory, and (b) the strength of central executive working memory’s association with the untrained outcome (Rapport et al., 2013).

6. ADHD is a heterogeneous disorder (Nigg, 2005), with multiple pathways to its phenotypic expression (Sonuga-Barke et al., 2010). Recent evidence suggests that approximately 80% (Kasper et al., 2012) to 85% (Karalunas et al., in press) of children with ADHD have working memory deficits when assessed using tasks with a prominent executive component. At minimum, there will be a subset of children with ADHD who do not respond to CET because central executive deficits do not underlie their behavioral presentation. Optimal targeting will require neuropsychological assessment and an armamentarium of interventions to address each cognitive pathway to ADHD.

The current study describes the development of CET and provides a preliminary test of assumptions 1–3. All assumptions differ from psychosocial treatments that teach compensatory, regulatory, and problem-solving strategies (Rapport et al., 2013), curriculum restructuring approaches intended to accommodate working memory deficits (Elliott et al., 2010), and behavioral approaches that target overt behaviors (Evans et al., 2014). This latter distinction is consistent with recent hypotheses that behavioral treatment may work, in part, by inadvertently circumventing working memory (e.g., breaking down multi-step instructions, providing explicit reminders; Kofler et al., 2017). CET’s assumptions differ also from approaches that emphasize in vivo cognitive training as a prerequisite for transfer of training gains (Halperin et al., 2012). As described below, CET also differs fundamentally from extant ADHD cognitive training protocols in its training targets, methods of adapting task parameters to hit those targets, and emphasis on requiring recall rather than recognition.

Current Study

The current study describes CET’s development and an initial test of CET relative to behavioral parent training (BPT). BPT was selected as an active comparator because it is a current gold standard psychosocial intervention for ADHD (Evans et al., 2014). We hypothesized that CET would be comparable to BPT in terms of feasibility/acceptability for caregivers and children. We further hypothesized that CET would produce significant improvements on untrained working memory tests that would be superior to any improvements associated with BPT. To our knowledge, only two studies have reported effects of behavioral treatment on cognitive outcomes for children with ADHD, and neither found significant changes (Hannesdottir et al., 2014; Steeger et al., 2016). Finally, we predicted that CET and BPT would produce similar improvements in objectively-assessed ADHD symptoms.

Method

Preregistration and Open Data

Primary and secondary outcomes and detailed data analytic plans were preregistered at https://osf.io/9gcbd/. There were no departures from the preregistered plan with one clearly marked exception. The de-identified raw data (.jasp) and detailed results output (including test statistics) are available for peer review as recommended (Redick, 2015): https://osf.io/9gcbd/.

Study Timeline

Families recruited between June 2013 and December 2014 were offered behavioral parent training (BPT). Families recruited between June 2015 and December 2016 were offered central executive training (CET). Recruitment to BPT was closed when CET was ready for testing. Recruitment to this CET pilot study was closed when software for an active training comparator was completed. Both samples reflect consecutive referrals.

Participants

The modified CONSORT study flow diagram is shown in Figure 1 (modified to emphasize sequential design). As shown in Table 1, the treated sample comprised 54 children aged 8–13 years (M=10.43, SD=1.59; 21 girls) from the Southeastern U.S., consecutively referred to a university-based children’s learning clinic (CLC) through community resources. Psychoeducational evaluations were provided to caregivers. All parents/children gave informed consent/assent; IRB approval was obtained. Child race/ethnicity was 76% Caucasian/Non-Hispanic, 9% Hispanic/English-speaking, 9% African American, and 6% multiracial.

Figure 1.

Modified CONSORT diagram (modified to emphasize sequential design)

Table 1.

Pre-Treatment Sample and Demographic Variables

Variable	BPT (n=27)		CET (n=27)		Cohen’s d	BF01	p
	M	SD	M	SD
Gender (Girls/Boys)	10/17		11/16		--	3.01	.79, ns
Age	10.57	1.51	10.29	1.69	0.17	3.08	.52, ns
SES	48.63	10.98	52.22	9.06	−0.36	1.80	.20, ns
WASI-II/WISC-V VCI	109.37	13.23	102.67	13.08	0.51	1.15	.07, ns
Medication (No/Yes)	12/15		18/9		--	1.19	.10, ns
Medication Changes (Stop/No/Add)	1/18/8		1/21/5		--	8.82	.63, ns
Race/ethnicity (White/Non-white)	21/6		20/7		--	3.38	.75, ns
ADHD Presentation (I/H/C/OS)	7/2/18/0		9/0/16/2		--	7.07	.61, ns
Comorbidity (No/Yes)	9/18		9/18		--	3.23	.99, ns
ODD Comorbidity (No/Yes)	17/10		17/10		--	3.16	.99, ns
BASC-2 Attention Problems (T-score)
Parent	67.04	7.33	65.56	7.35	0.20	1.07	.46, ns
Teacher	62.11	6.99	62.74	8.39	−0.08	3.52	.77, ns
BASC-2 Hyperactivity (T-score)
Parent	71.74	13.45	65.30	13.86	0.47	2.91	.09, ns
Teacher	60.33	12.09	58.81	13.84	0.12	3.38	.67, ns
Child Symptom Inventory-IV (Raw Score)
Attention Problems (Parent)	21.22	4.05	20.04	6.36	0.22	2.77	.42, ns
Hyperactivity/Impulsivity (Parent)	17.22	6.17	15.33	7.96	0.27	2.46	.33, ns
Oppositional-Defiant (Parent)	10.96	5.20	9.70	5.64	0.23	2.70	.40, ns
Working Memory Performance Data (Stimuli Correct/Trial)
Phonological Working Memory	3.11	0.82	3.21	0.58	−0.14	3.27	.61, ns
Visuospatial Working Memory	2.51	0.75	2.21	0.94	0.35	1.86	.21, ns
Actigraph-measured Hyperactivity
Baseline 1	56.34	34.29	38.91	27.33	0.56	0.64	.04
PHWM Task	216.83	83.41	222.21	123.80	−0.05	3.27	.85, ns
VSWM Task	154.89	80.31	176.83	117.11	−0.22	2.79	.43, ns
Baseline 2	75.05	42.48	61.87	50.65	0.28	2.34	.31, ns

Note. Raw p-values are presented (uncorrected for multiple comparisons). BASC-2 = Behavior Assessment System for Children (T-scores); BF = Bayes Factor, BF₀₁ is the odds ratio of the evidence favoring the null to the evidence favoring the alternative hypothesis. A value of 1 indicates that the data are equally likely under the null and alternative hypotheses, values >1 favor the null hypothesis that the groups are equivalent, and values ≥3 are considered statistically significant evidence of equivalence. BF₁₀ can be computed as the inverse of BF₀₁ (1/BF₀₁); BPT = behavioral parent training; CET = Central Executive Training; Medication Changes (Stop = Discontinued Medication During Study, No = No Changes Reported, Add = Started Medication During Study); OS = Other Specified ADHD Presentation; PH = Phonological Working Memory; VCI = Verbal Comprehension Index (IQ; standard scores); VS = Visuospatial Working Memory.

^*p < .05,

^**p ≤ .01,

^***p ≤ .001

Inclusion/Exclusion Criteria

All children and caregivers completed an identical evaluation, regardless of group assignment, that included detailed, semi-structured clinical interviewing (K-SADS; Kaufman et al., 1997). The K-SADS (2013 Update) assesses developmental history as well as onset, course, and impairment of DSM-5 (APA, 2013) disorders in children and adolescents. Parent and teacher ADHD ratings were obtained from the Behavior Assessment System for Children (BASC-2; Reynolds & Kamphaus, 2004) and Child Symptom Inventory (CSI-IV; Gadow & Sprafkin, 2002).

Study eligibility required: (1) DSM-5 diagnosis of ADHD (any presentation) by the directing clinical psychologist based on K-SADS; and (2) Borderline/clinical elevations on at least one parent and one teacher ADHD rating scale, or previous psychoeducational evaluation documenting cross-informant symptoms (e.g., for children prescribed medication that reduces ADHD symptoms at school; ‘otherwise specified’ presentation was specified for n=2 children who met criteria based on only one informant). All children had current impairment based on K-SADS. Comorbidities reflect clinical consensus best estimates, and include oppositional defiant (37.0%), anxiety (16.7%), depressive (9.3%), and autism spectrum disorders (3.7%). The BPT and CET groups were equivalent proportionally in comorbidities overall (χ²[1]=0.00, p=.99, ns, BF₀₁=3.23), or across diagnostic categories (χ²[4]=4.39, p=.68, ns, BF₀₁=142.4). BF₀₁ is an odds ratio indicating support for the null hypothesis that the groups are equivalent (H₀) relative to the alternative hypothesis that the groups differ (H₁) (see Bayesian Analyses section below).

Learning disabilities were suspected in 26% of BPT and 19% of CET cases based on score(s) >1.5 SD below age-based norms on one or more KTEA-2/3 core subtests (Kaufman & Kaufman, 2004, 2014); these proportions did not differ significantly (χ²[1]=0.43, p=.51, ns, BF₀₁=2.98). Fifteen children in the BPT group (55.6%) and 9 children in the CET group (33.3%) were currently prescribed psychostimulants. An additional 3 CET children received previous psychostimulant trials. The BPT/CET difference in current medication rates did not reach significance (p=.10, ns, BF₀₁=1.19). The BPT and CET groups were equivalent with regard to medication changes during the study (p=.63, ns, BF₀₁=8.82; Table 1).

Children were excluded for gross neurological, sensory, or motor impairment; seizure disorder, psychosis, or intellectual disability; or non-stimulant medications that could not be withheld for testing. No inclusion/exclusion based on working memory/executive functioning performance was set.

Procedures

Best practices for cognitive training studies were followed as detailed in Supplementary Table 1 (Redick, 2015; Simons et al., 2017). Pre-treatment testing occurred during a larger battery of two, 3-hour sessions. Post-testing occurred during a single, 3-hour session. A 90-minute mid-treatment testing session was conducted for children in the BPT group. As specified in the NIH R34 proposal (R34 MH102499), no mid-treatment sessions were completed for the first wave of CET testing. This design difference was considered acceptable because it favored the null (i.e., test-retest effects, if present, would favor the BPT group). Statistical tests therefore assess pre-post changes as preregistered.

All tests were counterbalanced within/across sessions to minimize order/fatigue effects. Children received brief breaks after each task, and preset longer breaks every 2–3 tasks to minimize fatigue.

Treatments

BPT and CET were both delivered according to manualized procedures in small group format (2–6 families) or individually as needed to accommodate families’ schedules. Schedule changes were accommodated to the extent possible (e.g., make-up sessions the same week). Identical procedures were used for both samples (e.g., 1-hour in-office sessions). Families were not required to withhold psychostimulants prior to child treatment visits. Psychostimulants were withheld for ≥24-hours prior to all child pre/post testing sessions.

Behavioral Parent Training (BPT)

Evidence-based BPT (Evans et al., 2014) was provided using the manualized Barkley (1997b) Defiant Children protocol. BPT was delivered by behaviorally trained, PhD-level clinicians (MJK, DES, HSS). As reported below, parent-reported ADHD symptom reductions in this group were consistent with (van der Oord et al., 2008) or larger than (Sonuga-Barke et al., 2013) expectations based on meta-analysis.

Central Executive Training (CET)

CET is a translational, evidence-informed, hybrid (in-office and at-home), and software-based treatment protocol that includes gaming elements (Prins et al., 2011) and an automated token economy to reinforce training goals and improve player engagement. The final 10-week protocol includes weekly in-office sessions with the child (1 hour) and a concurrent parent psychoeducational group, combined with parent-supervised, in-home training (goal: 15-min/day, 2–3 days/week).

Hitting the Target

We developed nine training tasks (‘games’) to form a clinical protocol and maximize the likelihood that CET would adequately engage central executive (CE) working memory, and not just short-term memory (Table 2)¹. That is, CET is less concerned with increasing the number of items children can hold in short-term memory (the memory components of working memory) and more focused on improving what children can do with those items (the working components of working memory). This targeting was based on (a) neuroimaging meta-analyses demonstrating overlapping and non-overlapping mid-lateral prefrontal and interconnected regions associated with each of the three central executive processes (Nee et al., 2013; Wager & Smith, 2003), (b) cross-sectional evidence of ADHD-related cortical underdevelopment in each implicated region (for reviews, see Kasper et al., 2012; Rapport et al., 2013), and (c) longitudinal evidence of 3–4 year delays in cortical maturation of these regions in pediatric ADHD samples (Shaw et al., 2007). CET emphasizes recall instead of recognition based on compelling evidence that recognition-based tasks share minimal variance (r=.20) with criterion working memory recall tasks (Redick & Lindsey, 2013).

Table 2.

Central executive training (CET) games as a function of central executive processing requirements and stimulus modality. Sample screenshots of each training game are shown below, in the same order as the table.

As shown in Table 2, each CET game targets a unique combination of CE process (dual-processing, continuous updating, serial reordering) and stimulus modality (phonological, visual, spatial). No additional executive functions were targeted based on meta-analytic evidence that ADHD cognitive training protocols are less effective when potency is decreased by targeting multiple executive functions (e.g., near transfer d=0.63 for short-term memory (STM) training vs. d=0.19 for STM + inhibition training; Rapport et al., 2013). Given the limited role of short-term memory deficits in ADHD symptoms/impairments (e.g., Kasper et al., 2012), we created games involving all three short-term memory modalities to facilitate generalization of CE-related improvements.

Adaptive Training

CET was designed as a competence-oriented training in which the child’s basal level is established and they are trained up from there, thus ensuring that each child is constantly working within her/his zone of proximal development (“flow state” in the serious games literature) (Canon-Bowers & Bowers, 2010). A critical distinction between CET and extant, capacity-based protocols (Rapport et al., 2013) is that CET’s algorithms dynamically adjust multiple parameters to incrementally increase demands on central executive processes. These parameters are dependent on training target (Table 2), and include aspects such as target density, categories:stimuli ratio, target:nontarget stimuli ratio, visual discriminability, and search space size. For example, increasing the search space size produces greater visual saccades, which in turn increase central executive demands during spatial working memory tasks because these saccades interrupt spatial rehearsal (Awh et al., 2001; Postle et al., 2004).

CET’s adaptive algorithms were set to ensure incremental increases in central executive demands based on child performance. This process occurred during the current study and involved iterative changes and extensive testing with research assistants (alpha testing) and children with ADHD (beta testing). To ensure breadth of training, the software’s ‘Mission Mode’ automatically selects games that the child has not completed recently. CET’s automated token economy awards ‘tickets’ for successful recall during each game, for completing each game, and for completing the daily ‘Mission Mode.’ These tickets are exchanged for tangible prizes during the weekly in-office sessions.

Beta testing

The CET sample was recruited in 3 waves to facilitate software refinements and testing of key design features. As specified in the NIH grant proposal (R34 MH102499), the first wave trained on one game per week, the second wave had immediate access to all 9 training games, and the third wave (n=9 per wave) tested the final protocol that implemented all CET features (e.g., tickets, ‘Mission Mode’ detailed above). To reduce child expectancies, children were told that they were ‘beta testers’ for our video game design team.

Focus groups

Parents in the first two CET subgroups participated in focus groups. Key CET design changes based on these focus groups included overhauling the on-screen instructions, modifying the home screen to show child progress (daily games completed), and automation of the token economy via children earning on-screen ‘tickets’ that are exchanged for tangible, in-office rewards. Key logistical changes from these focus groups included improved communication with parents (e.g., access/login instructions, progress monitoring of days/games completed at home), modified expectations for at-home training duration (original goal of 30-minutes/day decreased to 15-minutes/day) and frequency (original daily training goal decreased to 2–3 days/week), and modified treatment duration (original 12-weeks decreased to 10-weeks). Finally, parent feedback resulted in the addition of a parent group that began mid-way through testing with subgroup 2.

Parent groups

The parent psychoeducation group occurred in a separate room from the child in-office training session, led by PhD-level study therapists (MJK, KA). The parent component was added during CET wave 2 (described above) and made optional for parents based on parent focus group feedback. Parent groups were intended to promote treatment adherence and troubleshoot difficulties with the at-home training (e.g., demonstrating login procedures, brainstorming feasible days/times for the child to complete training). A further goal of the parent group was to establish the infrastructure anticipated for future trials that combine CET with extant evidence-based psychosocial treatments, because most of these programs involve working with the parent rather than child (e.g., parent training; Chacko et al., 2017). The psychoeducational component is manualized and provides general information about ADHD-relevant topics (e.g., common cormorbidities, evidence-based treatment options, IEP/504 Plans). Importantly, no active treatment components are included in the parent group.

Measures

Intellectual Functioning (IQ) and Socioeconomic Status (SES) at Pre-Treatment

IQ was estimated using the WASI-II (BPT) or WISC-V (CET) Verbal Comprehension Index (Wechsler, 2011, 2014). The changeover was made due to the WISC-V’s release during the course of the study and to provide caregivers with most up-to-date psychoeducational evaluation possible. Hollingshead (1975) SES was estimated based on caregiver(s)’ education and occupation.

Feasibility, Acceptability, and Usability Outcomes

Client Satisfaction Questionnaire (CSQ-8; Nguyen et al., 1983)

The CSQ-8 is an extensively studied, 8-item generic measure of clients’ perceptions of the value of services received (e.g., “to what extent has our program met your needs?”). Parents completed the CSQ-8 at post-treatment. Higher scores indicate higher satisfaction (range=1–4).

Barriers to Treatment Participation Scale (BTPS; Kazdin et al., 1997)

The BTPS is a 44-item measure of perceived treatment barriers completed by parents at post-treatment (e.g., “my job got in the way of coming to a session”); item scores are summed to provide a Total Barriers score (% possible barriers). Higher scores indicate more barriers.

System Usability Scale (SUS; Canon-Bowers & Bowers, 2010)

The SUS is a 10-item, item response theory-developed scale assessing ease of use on a 5-point Likert scale (e.g., “I thought the game system was easy to use”). Children in the CET group completed the SUS at post-treatment. Higher scores indicate greater usability (range = 0–100).

CET Training Duration

The CET software records training duration for each completed training game (time spent actively engaged); total minutes trained and total completed games are reported.

Subjective ADHD symptom changes

As specified in the NIH R34 proposal and following best practices for cognitive training studies (Simons et al., 2017), unblinded parent reports of ADHD symptoms were interpreted as additional evidence of feasibility/acceptability rather than efficacy. Parent-reported Attention Problems and Hyperactivity/Impulsivity were assessed via T-scores on the BASC-2 (age and gender norms; Reynolds & Kamphaus, 2004) and raw symptom severity scores on the CSI-IV (Gadow & Sprafkin, 2002). Higher scores indicate greater symptom quantity/severity.

Primary Outcomes (Working Memory; WM)

The well-validated Rapport et al. (2009) phonological (PHWM) and visuospatial (VSWM) computerized WM tests correctly classify children with vs. without ADHD at similar rates as parent and teacher ADHD rating scales (Tarle et al., 2017), and predict hyperactivity (Rapport et al., 2009), inattention (Kofler et al., 2010), and impulsivity (Raiker et al., 2012). Reliability and validity evidence includes internal consistency (α=.82–.97), 1–3-week test-retest reliability (.76–.90; Sarver et al., 2015), and expected relations with criterion WM complex span (r=.69) and updating tasks (r=.61) (Wells et al., 2018).

Both tasks involve serial reordering of characters presented (numbers, black dot locations), and reordering of a target stimulus (letter, red dot location) into the final serial position recalled. The phonological task involved mentally reordering and verbally recalling a jumbled series of sequentially presented numbers and letters (e.g., 4H62 is correctly recalled as 246H). The visuospatial task involved mentally reordering a sequentially presented series of spatial locations based on what color dot appeared in each location (black dots in serial order, red dot last) and responding on a modified keyboard. Six trials per set size were administered in randomized/unpredictable order for each task (3–6 stimuli/trial; 1 stimuli/second) as recommended (Kofler et al., 2017a). The 24 total trials per task were randomized, then grouped into 2 blocks of 12 trials each, with short breaks between each block (approximately 1 minute) (Kofler et al., 2016). Five practice trials were administered before each task (80% correct required). Task duration was approximately 5 (VSWM) to 7 (PHWM) minutes.

Dependent variables

Stimuli correct per trial were computed at each set size as recommended (Conway et al., 2005), and averaged to provide separate indicators for PHWM and VSWM. Higher scores reflect better working memory.

Secondary Outcomes (Objectively-Assessed ADHD Hyperactivity)

Objective measurement

Micro Motionlogger actigraphs (Ambulatory Monitoring, 2014) are acceleration-sensitive devices that sample movement intensity 16 times per second (16 Hz), collapsed into 1-second epochs. The estimated reliability for actigraphs placed at the same site on the same person ranges from .90 to .99 (Tryon et al., 1991). Actigraphs show expected levels of correspondence with parent- and teacher-reported hyperactivity (r=.32–.58), have superior predictive validity relative to rating scales for differentiating children with all subtypes/presentations of ADHD (including Predominantly Inattentive) from neurotypical children and from children with other clinical disorders at both the group and individual levels, and outperform other mechanical devices for differentiating ADHD from Non-ADHD groups (for meta-analytic review, see Kofler et al., 2016).

Children were told that the actigraphs were “special watches” that let them play the computer learning games. Observer XT (Noldus, 2012) software was used to code start and stop times for each task, which were matched to the time stamps from the actigraphs. Actigraphs were placed on the child’s non-dominant wrist and both ankles.

Following Rapport et al. (2009), a total hyperactivity score (THS) was computed by summing activity level across the three actigraphs. Hyperactivity scores were computed separately for movement during each of the WM tests described above (proximal outcomes), as well as for movement during the control conditions that occurred as the first and last activities of each testing session (distal outcomes). During the control conditions, children used Microsoft Paint for five consecutive minutes at the beginning and end of all pre- and post-treatment sessions. Children sat in the same chair and interacted with the same computer used for the WM tasks.

Bayesian Analyses

Traditional null hypothesis significance tests (p-values) were supplemented with Bayes Factors as recommended (Redick, 2015). The benefits of Bayesian methods are well documented (e.g., Wagenmakers et al., 2016); for our purposes, Bayesian analyses were added because they allow stronger conclusions by estimating the magnitude of support for both the alternative and null hypotheses (Rouder & Morey, 2012). Bayes factor mixed-model ANOVAs, paired-samples t-tests, and independent sample t-tests (Rouder & Morey, 2012; Wagenmakers et al., 2016) with default prior scales were conducted using JASP 0.8.2 (JASP Team, 2017). Instead of a p-value, these analyses provide BF₁₀, which is the Bayes Factor of the alternative hypothesis (H₁) against the null hypothesis (H₀). BF₁₀ is an odds ratio, where values above 3.0 are considered significant evidence supporting the alternative hypothesis (conceptual equivalent of p < .05). BF₁₀ values above 10.0 are considered strong (>30 = very strong, >100 = decisive/extreme support; Wagenmakers et al., 2016).

Conversely, BF₀₁ is the Bayes Factor of the null hypothesis (H₀) against the alternative hypothesis (H₁). BF₀₁ is the inverse of BF₁₀ (i.e., BF₀₁ = 1/BF₁₀), and is reported when the evidence indicates a lack of an effect (i.e., favors the null hypothesis; Rouder & Morey, 2012). BF₀₁ values are interpreted identically to BF₁₀ (>3.0 = moderate, > 10.0 = strong, > 100 = decisive/extreme support for the null hypothesis that a predictor does not have an effect on an outcome; Rouder & Morey, 2012).

Between-group comparisons are supplemented with Cohen’s d effect sizes (converted from η²_p to aid interpretability). Both p-values and Bayes Factors are reported for transparency. P-values were preregistered as the primary indicator if discrepancies arose.

Data Analysis Overview

Data analyses were conducted using JASP 0.8.2 according to the preregistered plan. We initially compared pre-treatment characteristics of treated vs. assessed but untreated children to probe the representativeness of our treatment samples (CONSORT Figure 1). We then compared the treated BPT and CET samples on pre-treatment characteristics, study retention, and feasibility/acceptability outcomes. Finally, BPT and CET were compared for effects on our primary outcomes (WM) and secondary outcomes (objective ADHD symptom assessments). These analyses involved residual gain scores (i.e., between-group comparison of post-treatment scores, covaried for pre-treatment scores) and group x outcome x time mixed-model ANOVAs. Two measures were used for each primary and secondary outcome to maximize power and strengthen interpretation (Shipstead et al., 2012). Exploratory analyses were added to address the mechanism of change, and involved correlating overall changes in working memory with changes in objectively-assessed ADHD symptoms during the proximal and distal activities.

Results

Treated vs. Untreated Samples: Pre-Treatment Characteristics

As shown in CONSORT Figure 1, we evaluated 79 children for eligibility, of whom 54 received treatment and 25 did not. There were no significant differences between the treated (N=54) and untreated samples (N=25) on any of the pre-treatment variables listed in Table 1. The treated and untreated samples were equivalent in terms of gender, IQ, medication status, race/ethnicity, % comorbidity, all parent reported ADHD subscales, teacher-reported hyperactivity, WM test performance, and hyperactivity during the visuospatial and end of session baseline tasks (all p ≥ .10, all BF₀₁ ≥ 3.06). The treated and untreated samples did not differ significantly on age, SES, ADHD subtype, or hyperactivity during the phonological and beginning of session baseline tasks (all p ≥ .15, all BF₀₁ ≥ 1.41). Untreated children were not followed beyond the pre-treatment evaluation.

BPT vs. CET Samples: Pre-Treatment Characteristics

As shown in Table 1, the BPT and CET treated samples did not differ on any demographic variables. Based on these analyses, no demographic covariates were included when assessing differential pre-post change as a function of BPT vs. CET.

Study Retention

Study retention was high for both BPT and CET. Notably, 100% of children in both BPT and CET completed the post-treatment testing, regardless of completer/non-completer status. Of the non-completers, 4 families discontinued BPT after 4–5 sessions but agreed to return for child testing (none completed parent questionnaires), and 3 families discontinued CET after 2–6 sessions but returned for child testing (2 did not complete parent questionnaires). All other families completed at least 8 BPT or CET sessions and all scheduled assessments.

Outlier and Missing Data Handling

Outliers ≥3.0 SD were winsorized relative to the within-group distribution (BPT: 0.62% of data points, CET: 0.76% of data points). Missing data rates were low and did not differ between BPT (M=2.0%, SD = 6.0) and CET (M=1.0%, SD = 3.0)(p=.38, BF₀₁=2.63). Little’s MCAR test indicated that these data were missing completely at random (χ² [167]=144.49, p = .90). Missing data were therefore imputed using the preregistered plan (expectation-maximization based on all available data).

Feasibility/acceptability/usability

CSQ-8, BTPS, SUS, and CET engagement

BPT and CET were highly feasible and acceptable to parents (Table 3a). BPT and CET did not differ in parent-reported treatment satisfaction (p=.18, BF₀₁=1.69) and were equivalent in terms of parent-reported treatment barriers (p=.59, BF₀₁=3.24). Children in the CET sample rated the software as easy to use on the SUS (Table 3a). Mean number of completed training games was 53.00 per child (SD=40.33).

Table 3a.

Post-treatment feasibility/acceptability data

Variable	BPT (n=27)		CET (n=27)		Effect size η2p	Cohen’s d	BF10	p
	M	SD	M	SD
Medication Changes (Stop/No/Add)	1/18/8		1/21/5		--	--	0.11	.63, ns
Caregiver Satisfaction (CSQ-8)	3.53	0.48	3.69	0.37	.03	0.35	0.59	.18, ns
BTPS Total Barriers (% items)	0.09	0.05	0.08	0.05	.01	0.20	0.31	.59, ns
CET Training Time (minutes)	--	--	379.70	260.20	--	--	--	--
CET Training (games completed)	--	--	53.00	40.33	--	--	--	--
CET System Usability Scale	--	--	80.12	17.44	--	--	--	--
BASC-2 Attention Problems (T-score)	63.78	6.45	60.04	8.76	.06	0.51	1.11	.07, ns
BASC-2 Hyperactivity (T-score)	60.33	11.86	57.48	13.02	.01	0.20	0.34	.45, ns
CSI-IV Attention Problems (Raw scores)	11.41	4.34	10.93	5.99	.00	0.00	0.27	.98, ns
CSI-IV Hyperactive/Impulsive (Raw scores)	8.11	4.37	7.19	5.93	.01	0.20	0.28	.45, ns
CSI-IV Oppositional-Defiant (Raw scores)	7.30	4.66	5.56	4.64	.02	0.29	0.45	.29, ns

Note. Effect sizes and statistical tests reflect control for pre-treatment scores on the same measure for BASC-2 and CSI-IV. BF₀₁ can be computed as the inverse of BF₁₀ (1/BF₁₀). BF = Bayes Factor; BPT = behavioral parent training; CET = Central Executive Training; Medication Changes (Stop = Discontinued Medication During Study, No = No Changes Reported, Add = Started Medication During Study).

^*p < .05,

^**p ≤ .01,

^***p ≤ .001

BASC-2

Controlling for pre-treatment scores, CET and BPT did not differ at post-treatment for Attention Problems (d = 0.51, η²_p=.06; p=.07, BF₁₀=1.11) or Hyperactivity/Impulsivity (d=0.20, η²_p=.01; p=.45, BF₀₁=2.93). The group (BPT, CET) x symptom domain (Attention Problems, Hyperactivity/Impulsivity) x time (Pre, Post) mixed-model ANOVA was significant for the main effect of time (p<.001, BF₁₀=9.14 × 10⁸), the time x symptom interaction (p=.004, BF₁₀=7.26), and the time x group x symptom interaction (p=.03, BF₀₁=2.41). The groups showed statistically equivalent change over time (group x time interaction: d=0.06, η²_p=.001; p=.80, BF₀₁=5.38) (Supplementary Fig. 1, top).

Paired-sample t-tests indicated that the CET group showed large reductions in Attention Problems (d=0.97, p<.001, BF₁₀=811.14) and Hyperactivity/Impulsivity (d=0.96, p<.001, BF₁₀=689.11). Likewise, the BPT group showed medium reductions in Attention Problems (d=0.61, p=.004, BF₁₀=10.40) and large reductions in Hyperactivity/Impulsivity (d=1.47, p<.001, BF₁₀=3.43 × 10⁵).

CSI-IV

Controlling for pre-treatment scores, CET and BPT were equivalent at post-treatment for Inattention (d=0.00, η²_p=.000; p=.98, BF₀₁=3.68) and Hyperactivity/Impulsivity (d=0.20, η²_p=.01; p=.45, BF₀₁=3.62). The group (BPT, CET) x symptom domain (Inattention, Hyperactivity/Impulsivity) x time (Pre, Post) mixed-model ANOVA was significant for time (p<.001, BF₁₀=3.26 × 10²⁶) and symptom domain (p<.001, BF₁₀=852.32). The groups showed equivalent change over time (group x time interaction: d=0.17, η²_p=.007; p=.56, BF₀₁=4.26) (Supplementary Figure 1, bottom).

Paired-sample t-tests indicated that the CET group showed large reductions in Inattention (d=1.60, p<.001, BF₁₀=1.54 × 10⁶) and Hyperactivity/Impulsivity (d=1.35, p<.001, BF₁₀=8.29 × 10⁴). Likewise, the BPT group showed large reductions in Inattention (d=1.64, p<.001, BF₁₀=2.42 × 10⁶) and Hyperactivity/Impulsivity (d=1.79, p<.001, BF₁₀=1.25 × 10⁷).

Primary Outcomes (Working Memory)

Controlling for pre-treatment scores, CET was superior to BPT at post-treatment for improving VSWM (d=0.91, η²_p=.17; p<.001, BF₁₀=13.38) and PHWM (d=0.77, η²_p=.13; p<.001, BF₁₀=5.57) (Table 3b). The group (BPT, CET) x task (VSWM, PHWM) x time (Pre, Post) mixed-model ANOVA was significant for time (p<.001, BF₁₀=276.89), task (p<.001, BF₁₀=8.30 × 10¹⁶), and the group x time interaction (d=1.06, η²_p=.22; p<.001, BF₁₀=53.40) (Figure 2).

Table 3b.

Post-treatment outcome data

Variable	BPT (n=27)		CET (n=27)		Effect size η2p	Cohen’s d	BF10	p
	M	SD	M	SD
Primary Outcomes
Phonological Working Memory	3.24	0.81	3.66	0.55	.13	0.77	5.57	***
Visuospatial Working Memory	2.66	0.87	2.91	0.74	.17	0.91	13.38	***
Secondary Outcomes
Control Activity 1 Hyperactivity (PIM)	64.84	34.89	30.07	29.57	.17	0.91	19.06	**
PHWM Task Hyperactivity (PIM)	200.38	97.56	186.68	117.62	.01	0.20	0.34	.50, ns
VSWM Task Hyperactivity (PIM)	153.65	81.41	119.39	92.29	.12	0.74	3.90	**
Control Activity 2 Hyperactivity (PIM)	75.09	52.58	35.30	33.52	.17	0.91	15.98	**

Note. Effect sizes and statistical tests reflect control for pre-treatment scores on the same measure (residualized gain scores). Partial eta-squared effect sizes indicate the percent of variance in post-treatment scores explained by treatment group after accounting for pre-treatment scores, and are interpreted as small = .01; medium = .06; large = .13; BF₀₁ can be computed as the inverse of BF₁₀ (1/BF₁₀). BASC-2 = Behavior Assessment System for Children (T-scores); BF = Bayes Factor; BPT = behavioral parent training; CET = Central Executive Training; PH = Phonological Working Memory (Stimuli Correct/Trial); PIM = proportional integrating measure (assesses movement intensity) VS = Visuospatial Working Memory (Stimuli Correct/Trial).

^*p < .05,

^**p ≤ .01,

^***p ≤ .001

Figure 2.

Working memory performance as a function of group and time. Error bars reflect Bayesian 95% credibility intervals. BPT = behavioral parent training, CET = central executive training.

Paired-sample t-tests indicated that the CET group showed large magnitude improvements in VSWM (d=1.05, p<.001, BF10=2.13 × 10³) and PHWM (d=0.97, p<.001, BF₁₀=812.26). In contrast, the BPT group did not show significant pre/post change for VSWM (d=0.29, p=.14, BF₀₁=1.75) or PHWM (d= 0.22, p=.26, BF₀₁=2.70).

Secondary Outcomes (Objective Hyperactivity Assessment)

Proximal outcomes: actigraphs during working memory testing

Controlling for pre-treatment scores, CET was superior to BPT for reducing post-treatment hyperactivity during VSWM testing (d=0.74, η²_p=.12; p=.01, BF₁₀=3.90) but not during PHWM testing (d=0.20, η²_p=.01; p=.50, BF₀₁=2.98). Given this discrepancy, we departed from the preregistered plan to assess treatment-related change over time separately by task.² The group (BPT, CET) x time (Pre, Post) mixed-model ANOVA for hyperactivity during VSWM testing was significant for time (p=.007, η²_p=.13, BF₁₀=3.88) and the group x time interaction (d=0.74, η²_p=.12; p=.01, BF₁₀=4.87). Paired-sample t-tests indicated that the CET group showed large reductions in hyperactivity (d=0.78, p<.001, BF₁₀=75.09), whereas the BPT group was equivalent at pre and post treatment (d= −0.02, p=.94, BF₀₁=4.89) for hyperactivity during VSWM testing (Supplementary Figure 2, bottom left).

In contrast, hyperactivity during PHWM testing showed nonsignificant time (d=0.55, η²_p=.07; p=.06, BF₁₀=1.05) and group x time interaction effects (d=0.06, η²_p=.001; p=.87, BF₀₁=2.96). Paired-sample t-tests indicated that CET group reductions in hyperactivity failed to reach significance (d=0.36, p=.07, BF₀₁=1.04); the BPT group was equivalent at pre and post treatment (d=0.16, p=.41, BF₀₁=4.89) (Supplementary Figure 2, bottom right).

Distal outcomes: actigraphs during baseline painting

Controlling for pre-treatment scores, CET was superior to BPT for reducing hyperactivity during the beginning of session (d=0.91, η²_p=.17; p=.002, BF₁₀=19.06) and end of session control activities (d=0.91, η²_p=.17; p=.003, BF₁₀=15.98). The group (BPT, CET) x condition (session beginning, end) x time (Pre, Post) mixed-model ANOVA was significant for condition (p<.001, η²_p=.27, BF₁₀=137.67) and the group x time interaction (d=0.74, η²_p=.12; p=.04, BF₁₀=13.66) (Supplementary Figure 3).

Paired-sample t-tests indicated that the CET group showed significantly reduced hyperactivity during the control activities at the beginning (d=0.43, p=.03, BF₁₀=1.74) and end of each session (d=0.60, p=.004, BF₁₀=9.52); the BPT group’s hyperactivity did not change significantly during the beginning (d=−0.23, p=.25, BF₀₁=2.60) or end of session activities (d=0.01, p=.99, BF₀₁=4.91).

Exploratory Analyses: Mechanisms of Change

Exploratory analyses were conducted to test the hypothesis that WM improvements would predict objectively-assessed ADHD symptom reductions (Figure S4). Change scores were computed for each primary and secondary outcome, and averaged separately to estimate overall changes in WM, objective hyperactivity (proximal assessment), and objective hyperactivity (distal assessment). Results indicated that WM improvements were significantly associated with reductions in objectively-measured hyperactivity during both the proximal (r= −.31, p=.01, BF₁₀= 4.34) and distal (r= −.37, p=.003, BF₁₀=14.43) assessments. Combined with the primary analyses above, these findings provide preliminary evidence that CET engages its intended mechanism, and that improvements in this mechanism are associated, to a significant extent, with reductions in objectively-measured ADHD symptoms as hypothesized.

Discussion

The current study described the development, empirical basis, and initial testing of central executive training (CET) relative to behavioral parent training (BPT), a current gold-standard non-medication treatment for ADHD. Confidence in the findings is supported by methodological strengths including detailed preregistration of the study’s outcomes and analytic plan, open data policy, adherence to best practice guidelines for cognitive training studies as outlined in Supplementary Table 1 (Redick, 2015; Simons et al., 2017), and use of outcome measures with strong construct and predictive validity evidence. The developed CET protocol was rated as highly usable by children with ADHD, and was equivalent to BPT on all parent-reported feasibility/acceptability outcomes, including high satisfaction, low perceived treatment barriers, and large perceived ADHD symptom reductions.

As hypothesized, CET was superior to BPT for improving working memory during both criterion tests (d=1.06), suggesting that CET was successful in engaging its intended training target. More importantly, CET but not BPT produced reductions in objectively-assessed hyperactivity during three of the four conditions, including both of the most distal far-transfer outcomes (d=0.74). Further, improvements in working memory correlated with reductions in objectively-assessed ADHD symptoms, and our objective estimates of CET’s effect on hyperactivity during the most distal outcomes were highly consistent with expectations based on juxtaposing CET’s effects on improving working memory (d=1.06) with prior evidence regarding the relation between working memory and actigraph-measured hyperactivity (Rapport et al., 2009). To our knowledge, this is the first cognitive training study to report objective, clinically significant reductions in ADHD symptoms that exceed those associated with gold-standard behavioral treatment (Rapport et al., 2013). These findings provide strong support for moving ahead with a randomized CET efficacy trial and, if replicated, would support recent hypotheses that next-generation ADHD cognitive training protocols may overcome current limitations via improved targeting of etiologically-relevant constructs and neurocognitive systems (Chacko et al., 2014b; Rapport et al., 2013).

With regard to the treatment feasibility/acceptability indicators, reductions in parent-reported ADHD symptoms (d=0.61–1.79) in the current study were somewhat larger than expectations based on recent meta-analyses for both BPT (d=0.40; Sonuga-Barke et al., 2013) and working memory training (d=0.63; Rapport et al., 2013). Several explanations merit consideration, including delivery of BPT by behaviorally trained PhDs (expertise and/or allegiance effects), and parents’ active participation that precluded blinding to treatment. Meta-analytic evidence suggests that effect sizes for unblinded ADHD symptom ratings are inflated similarly for working memory training (Δd=0.36–0.40) and behavioral intervention studies (Δd=0.38) (Rapport et al., 2013; Sonuga-Barke et al., 2013). Thus, while these findings would suggest that our use, in part, of unblinded parent ratings was unlikely to differentially affect the treatments, they do suggest that conclusions regarding effect magnitude should be tempered. Taken together, these findings support our a priori decision to treat unblinded parent ratings as feasibility/acceptability data rather than evidence of treatment efficacy.

Interestingly, BPT failed to produce improvements in objectively-assessed ADHD symptoms during any of the four proximal or distal activities. This finding is consistent with meta-analytic evidence that BPT improves functional impairments but not ADHD symptoms based on blinded ratings (Sonuga-Barke et al., 2013), but may be unsurprising given the highly situation-specific nature of behavioral treatment effects (Chronis et al., 2004). Meta-analytic estimates of blinded ADHD ratings came primarily from school (teacher reports; Sonuga-Barke et al., 2013), and the current study assessed hyperactivity in a clinic setting, both of which are removed from the home setting where parents implement the behavioral techniques. In other words, the current findings should not be interpreted as evidence against BPT efficacy because by design parents were not present to implement contingencies during testing in the clinic. Further, this initial study did not assess for several key benefits of BPT (e.g., parent-child relational benefits, parenting confidence; Evans et al., 2014).

Behavioral parent training did not produce improvements in working memory for children with ADHD. This finding was highly consistent with previous evidence (Hannesdottir et al., 2014; Steeger et al., 2016), and may suggest a potential mechanism to explain the rapid loss of treatment gains when BPT is discontinued (Chronis et al., 2004). Although using nonsignificant findings to explain other nonsignificant findings is tenuous when based on frequentist statistics, the Bayesian models allowed stronger conclusions because they supported the null rather than just failing to reject it. The clinical model of psychopathology posits that treating a disorder’s peripheral symptoms is unlikely to affect upstream, core deficits in the absence of transactional or reciprocal influences (Rapport et al., 2001). To the extent that inattention and hyperactivity/impulsivity reflect the phenotypic expression of underlying working memory deficits for many children with ADHD as hypothesized (Rapport et al., 2013), the low maintenance of BPT gains may be due to a lack of change in the underdeveloped neurobiological substrate that produces these symptoms (Kofler et al., 2016). That is, this model posits that peripheral behavioral symptoms may be improved by targeting them directly (e.g., behavioral interventions), changing the environment to circumvent the underlying impairments (e.g., accommodations), temporarily actuating the underdeveloped substrate (e.g., psychostimulant medication), and/or affecting developmental growth in that substrate (the hypothesized mechanism for cognitive training; Simons et al., 2017). Only the latter is expected to produce lasting improvements beyond the active treatment phase. This explanation is consistent with the current finding that both groups demonstrated ADHD symptom reductions by unblinded parent report, but only the CET group improved on objective tests of working memory and hyperactivity; studies that include neuroimaging and long-term follow-up are needed to test for substrate-level changes and maintenance of gains beyond the active treatment phase.

Focus Group and Beta Testing Lessons: CET Feasibility vs. Ideal Dosing

There were several instances where our parent focus groups and child beta testing resulted in changes that improved treatment acceptability at the expense of reduced CET dosing (e.g., reduced at-home training goals). A critical lesson from this pilot study was the importance of the in-office sessions to ensure at least minimal exposure to the treatment. Among the final CET group, for example, 1/3 of the children completed 39–43 training games, which corresponds to the approximately 4 games per week completed during the in-office sessions. In contrast, the top 1/3 more regularly completed the at-home goals and completed 127–160 training games during the same time period. Most caregivers described school-assigned homework as the biggest obstacle to at-home CET training; opinions were split approximately evenly regarding priority of CET vs. homework, with differences focused primarily on weighing immediate vs. delayed consequences. Examining dose response will be critical in the planned RCT for refining CET. Combining CET with behavioral parent training is likely to increase at-home training, but for this initial study we felt it important to exclude active treatment components from the parent group to prevent masking of effects and maximize certainty regarding effect specificity.

Limitations

As an initial development and feasibility study, there were ongoing changes to the CET software – including substantive changes such as automating the training-specific token economy – that likely produced somewhat different treatment experiences across children. In addition, the sequential design controls for most but not all threats to validity addressed by randomization (e.g., potential differential history effects between 2013–2015 vs. 2015–2017), and inherent differences between behavioral parent training and central executive training introduce potential confounds that preclude strong efficacy conclusions from this initial trial (e.g., primary treatment target: parent vs. child; required parent-child interaction duration). In the context of significant hyperactivity reductions during the visuospatial and both distal tasks, it is unclear whether the equivocal finding during the phonological task suggests that CET’s effects on hyperactivity will be more pronounced in specific settings, as opposed to an artifact of lower power to detect small effects in our pilot study (e.g., the CET effect size of d=0.36 for hyperactivity reductions during the phonological test failed to reach significance at p=.07, with BF₀₁=1.01 indicating that the data are equally likely under the alternative and null hypotheses). Finally, as noted previously, we contend that it is unreasonable to expect a time-limited treatment to overcome the 3–4 year delays in prefrontal cortical maturation associated with pediatric ADHD (Shaw et al., 2007), and as such our far-transfer outcomes focused on continuous measures rather than diagnostic remission.

Clinical and Research Implications

Taken together, the results provide strong support for continued evaluation of central executive training (CET) for ADHD. Children with ADHD rated CET as highly usable, and CET was equivalent to BPT in terms of parent perceptions of high feasibility, acceptability, and efficacy. CET was superior to behavioral parent training for improving working memory based on all tests and for reducing objectively-measured hyperactivity during three of four activities – including both of the most distal far-transfer outcomes. If replicated, these findings would provide strong support for adding CET to our armamentarium of evidence-based treatments for ADHD. Future work is needed to assess long-term maintenance of treatment gains, effects on ADHD-related functional impairments, and objective ADHD symptom reductions outside the clinic.

Public Health Significance Statement.

This study describes the development of central executive training (CET) and compares it with gold-standard behavioral parent training (BPT) for ADHD. Results suggest that CET is feasible and acceptable to families, and may be superior to BPT for improving working memory and hyperactivity.