Central Executive Training for ADHD: Effects on Academic Achievement, Productivity, and Success in the Classroom

Abstract

Objective:

Central executive training (CET) is a ‘level 2’ evidence-based treatment for improving ADHD-related executive dysfunction and behavioral symptoms, but the extent to which these gains extend to the disorder’s well-documented academic difficulties is unknown.

Method:

Across two clinical trials, 108 children with ADHD 8-13 years old (M=10.29, SD=1.50; 32 girls; 75% White/Non-Hispanic) were treated with CET (n=52), inhibitory control training (ICT; n=29), or gold-standard behavioral parent training (BPT; n=27).

Results:

CET was superior to BPT and ICT (d=0.62-0.88) for improving masked teacher perceptions of academic success, impulse control, and academic productivity at 1-2 months post-treatment. At 2-4 month follow-up, CET (d=0.76) and ICT (d=0.54) were superior to BPT for improving objectively-tested academic achievement overall (reading comprehension, math problem-solving, language comprehension), and CET was superior to ICT (d=0.56) for improving math problem-solving. The significant benefits of CET on academic success, academic productivity, reading comprehension, and math problem-solving replicated across both trials and were clinically significant as evidenced by low number needed to treat estimates (NNT=3-7) and significantly higher proportions of individual cases demonstrating reliable improvements in academic success/productivity (33%-36% vs. 0%-18%) and achievement (38%-72% vs. 18%-54%) across outcomes (all p≤.01).

Conclusions:

Results across the two trials provide strong support for the efficacy of CET for ADHD, and are consistent with model-driven hypotheses that academic difficulties in ADHD are due, in part, to these children’s underdeveloped executive functioning abilities.

Pediatric attention-deficit/hyperactivity disorder (ADHD) is a chronic and clinically heterogenous neurodevelopmental disorder that affects 5% of school-aged children and is associated with adverse functional outcomes across the lifespan (Polanczyk et al., 2014). The educational cost of illness incurred from special education placement, grade retention, and disciplinary incidents yields a large financial impact on the U.S. educational system (Robb et al., 2011). Academic impairment is one of the most common adverse outcomes associated with pediatric ADHD and often is identified as the area of most concern by caregivers and teachers (Langberg et al., 2011). Studies indicate that up to 70% of children with ADHD exhibit academic difficulties, with 33% to 63% of children meeting criteria for a specific learning disorder in one or more academic domains (DuPaul et al., 2013). Moreover, co-occurring ADHD and learning difficulties are associated with increased risk for adverse near-and long-term educational outcomes compared to typically developing children and children with a learning disorder alone (Loe & Feldman, 2007).

ADHD and Executive Function Deficits

According to the influential Miyake et al. (2000) model, there are 3 primary executive functions (working memory, inhibitory control, and set shifting); however, meta-analytic evidence suggests that only 2 of these executive functions (working memory and inhibitory control) are present/separable in school-aged children (Karr et al., 2018). Working memory is a primary executive function and refers to the active, top-down manipulation of information held in short term memory (Baddeley, 2007). This active manipulation is accomplished via three interrelated working components of working memory: dual-processing, continuous updating, and temporal/serial reordering (Wager & Smith, 2003). Inhibitory control is the other primary executive function in school-age children (Karr et al., 2018) and refers to the ability to withhold or stop an on-going response, particularly within the context of goal-directed behavior (Alderson et al., 2007). Studies of neurocognitive heterogeneity in pediatric ADHD indicate that 89% of children with ADHD exhibit deficits in at least one executive function, with approximately 62%–85% of children exhibiting working memory deficits and 21%–46% of children exhibiting inhibitory control deficits (for review see Kofler et al., 2017). Growing cross-sectional, experimental, and intervention-based evidence supports the notion that deficits in these higher-order executive functions may reflect core mechanisms underlying ADHD behavioral symptoms and many if not most associated functional impairments, including academic functioning, for a majority of children with ADHD (for reviews seeRapport et al., 2013; Kofler et al., 2018, 2020).

ADHD and Academic Difficulties

The construct of academic functioning is multifaceted and complex, and studies indicate that both neurocognitive (e.g., executive function) and noncognitive (e.g., academic engagement, study skills, motivation, interpersonal skills) factors contribute to academic success (Alloway et al., 2011; DiPerna & Elliot, 2002). The literature typically categorizes academic functioning into two interrelated outcomes: academic achievement and academic performance. Academic achievement refers to acquired academic knowledge and skills that can be measured using standardized achievement tests. Academic performance refers to success in the school setting and includes indicators such as grades, academic productivity (e.g., teacher-reported assignment completion and accuracy), years of schooling completed, and college enrollment (Langberg et al., 2011).

It is well documented that pediatric ADHD is associated with impairment in both academic achievement and performance (Friedman et al., 2017; Rapport et al., 2009). For example, children with ADHD are more likely to score lower on standardized achievement tests of reading, math, and spelling compared to typically developing peers, which persists longitudinally (Sarver et al., 2012). Additionally, children with ADHD are more likely to obtain lower grades (Titz & Karbach, 2014); exhibit decreased productivity and accuracy during reading- and math-related activities (Mautone et al., 2012; Rapport et al., 2009); and have higher rates of grade retention (Molina et al., 2009) compared to their peers. These academic difficulties appear unique to ADHD symptomology and independent of IQ, special education services, socioeconomic status, and comorbid conduct problems (Fried et al., 2016).

ADHD, Executive Dysfunction, and Academic Difficulties

Compelling evidence indicates that executive function abilities and learning behaviors (e.g., task engagement, task persistence) are associated with academic success for children with and without ADHD (Demaray & Jenkins, 2011). Executive dysfunction has been associated with disruptions in learning behaviors necessary for successful academic performance across neurotypical and ADHD samples (Massetti et al., 2008). Moreover, executive dysfunction has been shown to mediate the link between ADHD symptoms and both academic underachievement and underperformance (Colomer et al., 2017; Volpe et al., 2006), and to predict academic difficulties more strongly than ADHD behavioral symptoms (Diamantopoulou et al., 2007; Soto et al., 2021). In particular, evidence indicates that working memory is a strong predictor of reading and math performance for children with and without ADHD (Sjöwall & Thorell, 2014; St Clair-Thompson & Gathercole, 2006). Academic theories also suggest that working memory is associated with myriad reading skills, including word decoding, fluency, and reading and language comprehension (Friedman et al., 2017; Lonigan et al., 2018), and mathematical processes such as numerical facility and reasoning (Alloway & Alloway, 2010; Geary, 2011). Fewer studies have examined the relation between inhibitory control and reading and math performance; however, the relation appears to exist nonetheless, especially for early academic skills (Spiegel et al., 2021).

Clinical Utility of ADHD Treatments for Academic Functioning

Current evidence-based treatments for ADHD are minimally effective for producing or sustaining clinically meaningful improvements in academic outcomes for children with ADHD (Parker et al., 2013). Extant theoretical models suggest that this inefficacy may be because first-line ADHD treatments (i.e., behavioral management training, psychostimulant medication) target behavioral outcomes (e.g., reducing disruptive behavior, improving on-task behavior) rather than the core neurocognitive mechanisms that give rise to those observable behaviors (Chacko et al., 2014; Rapport et al., 2001). Given the direct and indirect links between executive functions and academic outcomes (Karr et al., 2018; Spiegel et al., 2021), interventions directly targeting these neurocognitive mechanisms may have the potential to yield clinically meaningful change, characterized by long-lasting effects in ADHD-related symptoms and academic outcomes that generalize beyond the active treatment phase (Karr et al., 2018).

Unfortunately, however, comprehensive meta-analyses of executive function training studies for children with ADHD indicate that these training protocols do not improve ADHD-related behavioral and functional outcomes (Rapport et al., 2013; Sonuga-Barke et al., 2013). This inefficacy appears to be occurring because these training protocols insufficiently target and/or mistarget the neurocognitive systems (i.e., working memory dual processing, updating, and reordering; inhibitory control action restraint and cancellation) associated with phenotypic ADHD behaviors/impairments (for review see Rapport et al., 2013). In contrast, recent clinical trials suggest promising and replicated benefits from central executive training, a newly developed, cognitively-informed neurocognitive training protocol for children with ADHD. This computerized protocol has been shown in two clinical trials to produce superior improvements in working memory and objective/masked indices of ADHD-related hyperactivity relative to gold-standard behavioral parent training and an active, credible control training protocol (Kofler et al., 2018, 2020). However, the extent to which these improvements extend to the disorder’s well-documented academic difficulties remains unknown.

Current Study

Taken together, cross-sectional, longitudinal, experimental, and meta-analytic studies provide convincing evidence that deficits in executive functions, particularly working memory and to a lesser extent inhibitory control, likely contribute to ADHD-related reading and math difficulties (Friedman et al., 2017, 2018; Miller et al., 2012, 2013). As such, deficits in working memory and/or inhibitory control may reflect common factor(s) that produce, to a significant extent, ADHD-related behavioral symptoms and academic difficulties (e.g., Rapport et al., 2008). Given the evidence that central executive training specifically and significantly targets neurocognitive subcomponents that are impaired in ADHD and linked with ADHD-related symptoms and/or impairments, we hypothesized that the neurocognitive improvements yielded from this protocol would distally transfer to other skills and abilities that rely on the same neural networks – in particular reading achievement, math achievement, and learning behaviors linked to academic success and productivity (Simons et al., 2016). This hypothesis is based on evidence indicating that (1) the majority of children with ADHD exhibit deficits in one or more of these executive functions (Alderson et al., 2007; Kofler et al., 2017); and, (2) these deficits have been linked to ADHD behavioral symptoms and associated functional impairments, including academic functioning (for review see Rapport et al., 2013) in reading, math, and learning behaviors associated with academic success and productivity (Demaray & Jenkins, 2011). The purpose of this study is to (a) build from the evidence linking ADHD-related executive dysfunction and academic difficulties, and (b) examine the extent to which two neurocognitive training protocols, central executive training and inhibitory control training, previously shown to improve working memory and inhibitory control abilities in children with ADHD, respectively (Kofler et al., 2018, 2020), improve objective indices of reading and math achievement and masked teacher-reported academic performance in children with ADHD – both relative to each other and relative to gold-standard behavioral management training.

Method

Study Design and Timeline

The current study reports results from 2 clinical trials conducted consecutively: A sequential trial of central executive training (CET) vs. behavioral parent training (BPT), followed by a randomized controlled trial of CET vs. inhibitory control training (ICT). Families recruited between June 2013 and December 2014 were offered BPT while the CET protocol was being designed/programmed. Recruitment to BPT was closed when CET was ready for testing. Families recruited between June 2015 and December 2016 were offered CET while the ICT software was in development. This initial trial (Study 1: BPT vs. CET) was closed when ICT was ready for testing. Families recruited between March 2017 and March 2019 were randomized with allocation concealment to CET or ICT (Study 2: CET vs. ICT). The current sample reflects all participants included in the primary outcomes manuscripts (Kofler et al., 2018, 2020). No families participated in both Study 1 and Study 2, and no families participated in more than one of the interventions. Please see Table 1 and Kofler et al. (2020) for details on randomization, allocation concealment, and masking maintenance.

Table 1.

Critical evaluation of the current studies relative to best practice guidelines for cognitive training methodology and reporting standards (adapted from Simons et al., 2016 and Redick, 2015)

	Criterion / Commentary
Best practice recommendations from Simons et al. (2016)
✓	Assess pre-treatment baseline performance for all groups
	Both trials used a pre-post/follow-up test design. Pre-treatment performance was assessed and controlled when probing between-treatment differences at post-treatment/follow-up.
✓	Include an active, credible control group matched for expectancies
	Study 1: Behavioral parent training (BPT) served as the active comparator. BPT is currently the gold standard psychosocial treatment for pediatric ADHD (for review, see Evans et al., 2018). The CET and BPT treatments did not differ in caregiver-reported feasibility or acceptability (Kofler et al.,2018).  Study 2: Working memory and inhibitory control are both putative core mechanisms implicated in ADHD and featured in prominent conceptual models of the disorder’s etiology and psychopathology. The two versions are identical except the target mechanism, and served as active, credible controls for each other. The treatments were identical in terms of expectancies and did not differ in caregiver-reported feasibility or acceptability (Kofler et al., 2020).
✓	Include at least 20 participants in each treatment arm
	Study 1: All analyses included BPT n=27 and CET n=27 participants.  Study 2: All analyses included ICT n=29 and CET n=25 participants.
✗✓	Randomly assign children to condition
	Study 1: Children were assigned sequentially based on date of intake with a priori defined cut-off dates (i.e., BPT recruitment was closed when CET was ready for testing). Random assignment was not feasible for this initial study due to CET’s lengthy development cycle (i.e., children could not be assigned to CET before the software existed). Meta-analytic evidence indicates that randomization does not significantly affect estimates of working memory training’s impact on working memory for children, but non-randomized studies inflate far-transfer estimates by d=0.20 (Sala & Gobet, 2017).  Study 2: Children were randomly assigned using unpredictable allocation concealment.
✓	Pre-register the trial, and explicitly acknowledge departures from pre-registered plan
	Both studies’ outcome measures and detailed data analytic plans were pre-registered. Preregistration occurred during data collection and prior to accessing the data. Data analyses were conducted masked to treatment allocation.
✓	Mask raters for all subjective outcome measures
	Teachers were masked to treatment status. Masking was also included for objective outcomes:Clinicians conducting standardized academic testing were masked to treatment status in Study 2. In Study 1 it is reasonable to conclude that clinicians were not masked to treatment status because only one treatment was offered at a time.
✓	Label any analyses conducted after inspecting the data as ‘exploratory’
	The analyses reported herein did not depart from the preregistered plan, with the exception of clearly marked analyses that were added during the peer review process.
✓	Avoid subgroup analyses unless preregistered
	No subgroup analyses were preregistered; therefore, none were conducted. Within-treatment analyses were limited to planned contrasts to characterize the pattern of change for each treatment.
✓	Identify all outcome data collected, including outcomes not reported herein
	A complete list of data collected for secondary research questions can be found on the studies’ OSF preregistration websites.
Additional recommendations from Redick (2015)
✓	Report full pre-test and post-test means and SDs for all groups
	Pre-treatment and post-treatment means and SDs are shown in Tables 2 and 3, respectively.
✓	Provide full, subject-level data as supplementary material
	JASP (.jasp) data files with subject-level data and results output are posted for peer review on the study’s OSF website.
✓	Use likelihood ratios, in particular Bayes Factors
	Traditional p-values are supplemented with Bayes Factors to allow stronger conclusions regarding both between-treatment equivalence and emerging between-treatment differences.
✓	Examine outcomes graphically to ensure that the pattern of pre- to post-test change is theoretically consistent with the expected pattern of results
	Outcomes were examined graphically to ensure the pattern of change was consistent with our descriptions in the text.

Transparency and Openness Statement

Best practice guidelines for cognitive training studies were closely followed as detailed in Table 1. Primary and secondary outcomes and detailed data analytic plans were preregistered at https://osf.io/9gcbd and https://osf.io/abwms for Study 1 and Study 2, respectively. The current study used the preregistered academic outcome measures that were included in Studies 1 and 2 and followed the preregistered analytic plan for the primary outcomes with one exception: We decided a priori to combine the Study 1 and 2 CET samples for the primary analyses to maximize power; sensitivity analyses are reported to examine the impact of this methodological decision, including direct comparison of the Study 1 vs. Study 2 CET samples. Exploratory analyses separated by Study (Study 1 BPT vs. CET, Study 2 CET vs. ICT) are also reported, and were highly consistent with the post-hoc/planned comparisons reported below with minor, explicitly noted exceptions. The de-identified raw data (.jasp) and results output (including analysis scripts and test statistics) are available for peer review as recommended (Redick, 2015): https://osf.io/tzuyr/. We report how we determined our sample size, all data exclusions, all manipulations, and all measures in the study.

Participants

CONSORT study flow diagrams are shown in the primary outcome studies (Kofler et al., 2018, 2020). As shown in Table 2, the treated sample comprised 108 children aged 8-13 years (M=10.29, SD=1.50; 32 girls) from the Southeastern U.S., consecutively referred to a university-based research clinic through community resources. IRB approval was obtained/maintained; all parents/children gave informed consent/assent. Psychoeducational evaluation results were provided to caregivers. Child race/ethnicity was 75% White/Non-Hispanic, 10% Hispanic, 9% Black, and 6% mixed race/ethnicity. All participants spoke English.

Table 2.

Pre-treatment sample and demographic variables

Variable	BPT (n=27)		CET (n=52)		ICT (n=29)		Contrasts	BF 01	p
	M	SD	M	SD	M	SD
Sex (Girls/Boys)	10/17		14/38		8/21		--	9.80	.62, ns
Age	10.57	1.51	10.26	1.53	10.07	1.46	--	5.96	.46, ns
SES	48.63	10.98	48.79	11.43	49.48	9.46	--	10.44	.93, ns
VCI	109.37	13.23	102.90	12.64	105.93	12.29	--	1.99	.13, ns
Medication (No/Yes)	12/15		33/19		20/9		--	2.04	.14, ns
Med. Changes During Study (Stop/No/Start)	1/18/8		2/40/10		1/20/8		--	>100	.86, ns
Race/ethnicity (W/B/H/M)	21/0/3/3		40/6/5/1		20/4/3/2		--	92.92	.37, ns
Maternal Education Level (HS/A/B/G)	1/4/5/17		1/6/27/18		2/5/13/9		--	2.30	.08, ns
ADHD Presentation (I/H/C/OS)	7/2/18/0		15/1/34/2		9/1/19/0		--	>100	.71, ns
BASC Attention Problems (T-score)
Parent	67.04	7.33	67.58	6.77	67.00	7.34	--	10.27	.92, ns
Teacher	62.11	6.99	63.12	7.35	65.52	7.59	--	3.01	.20, ns
BASC Hyperactivity (T-score)
Parent	71.74	13.45	69.48	12.82	66.52	13.01	--	4.01	.28, ns
Teacher	60.33	12.09	60.75	13.83	62.45	11.09	--	9.18	.79, ns
Kaufman Test of Educational Achievement (Standard Scores)
Reading Comprehension	110.60	13.58	96.75	10.81	99.59	13.99	BPT > CET = ICT		<.001
Math Concepts & Applications	110.10	11.27	97.54	14.65	97.93	16.52	BPT > CET = ICT		<.001
Listening Comprehension	110.60	11.27	102.60	13.48	102.70	14.47	BPT > CET = ICT		.03
Academic Performance Rating Scale (Raw Scores)
Academic Success	25.19	6.03	21.48	5.14	21.41	6.52	BPT > CET = ICT	0.39	.01
Impulse Control	8.30	2.33	7.40	1.97	8.14	2.57	--	2.48	.17, ns
Academic Productivity	39.04	8.03	37.63	7.59	35.86	8.59	--	4.61	.33, ns
Kaufman Test of Educational Achievement (GSV Scores)							--
Reading Comprehension	525.70	14.07	513.1	18.51	511.70	25.41	BPT > CET = ICT	0.31	.01
Math Concepts & Applications	525.40	13.43	511.30	20.88	508.70	27.39	BPT > CET = ICT	0.20	.007
Listening Comprehension	516.6	12.08	511.30	9.51	509.60	13.71	--	1.17	.06, ns

Note. Raw p-values are presented (uncorrected for multiple comparisons).

BASC = Behavior Assessment System for Children (T-scores); BF = Bayes Factor, BF₀₁ is the odds ratio of the evidence favoring the null to the evidence favoring the alternative hypothesis. A value of 1 indicates that the data are equally likely under the null and alternative hypotheses, values >1 favor the null hypothesis that the groups are equivalent, and values ≥3 are considered statistically significant evidence of equivalence. BPT = behavioral parent training; CET = Central Executive Training; ICT = Inhibitory Control Training; Race/ethnicity (W = White, B = Black, H = Hispanic, M = Mixed); Maternal Education Level (HS = High School diploma or equivalent, A = at least 1 year of college, Associate’s degree or specialized training, B = Bachelor’s/4-year college degree, G = Graduate degree); VCI = Verbal Comprehension Index (Wechsler IQ; standard scores).

Inclusion/Exclusion Criteria

All families completed a comprehensive evaluation that included detailed semi-structured clinical interviewing (K-SADS; Kaufman et al., 1997) and age/sex norm-referenced parent and teacher ADHD ratings (ADHD-RS-4/5 and BASC-2/3; DuPaul et al., 2016; Reynolds & Kamphaus, 2015). Additional details regarding the psychoeducational evaluation and differential diagnosis process can be found on our preregistration website. Study eligibility required: (1) DSM-5 diagnosis of ADHD (any presentation) by the directing clinical psychologist and multidisciplinary treatment team based on K-SADS (2013 update for DSM-5) and differential diagnosis considering all available clinical information indicating onset, course, duration, and severity of ADHD symptoms consistent with the ADHD neurodevelopmental syndrome; (2) clinical/borderline elevations on at least one parent and one teacher ADHD rating scale (i.e., >90^th percentile), or previous psychoeducational evaluation documenting cross-informant symptoms (e.g., for children prescribed psychostimulant medication that reduces ADHD symptoms at school); and (3) current impairment based on parent report. Additional inclusion criteria for the Study 2 sample included below average or lower scores on at least one of the pre-treatment working memory tests (n=2 excluded based on this criterion); no inhibitory control thresholds were specified. Children were excluded for gross neurological, sensory, or motor impairment; seizure disorder, psychosis, or intellectual disability; or non-stimulant medications that could not be withheld for testing.

Diagnosed comorbidities included anxiety (21%), autism spectrum (10%), depressive (5%), oppositional defiant (22%)¹, SLD reading (15%), and SLD math (10%). The BPT, CET, and ICT treatment groups did not differ in terms of comorbidities overall or within each diagnostic category (Kofler et al., 2018, 2020). These treatment groups were also equivalent or did not differ significantly on any demographic characteristic, the number of children prescribed psychostimulants, or medication changes during the study (Table 2).

Procedures

Pre-treatment academic testing occurred during a 3-hour psychoeducational battery that also included IQ/neuropsychological testing, norm-referenced child internalizing disorder screeners, and additional standardized measures administered clinically as needed to inform differential diagnosis and accurate assessment of comorbidities (e.g., child clinical interviews, additional testing). Follow-up academic testing was conducted by a masked evaluator who was not involved with the child’s treatment, and was administered during a single, 3-hour session that occurred 2-4 months after the post-treatment evaluation (M=81.40 days, SD=45.33; the treatment groups were equivalent in the time between treatment completion and follow-up testing, p=.90, BF₀₁=7.22). Academic testing was not conducted at immediate post-treatment because conceptual models describe executive functions as foundational abilities that support academic skills acquisition (Chacko et al., 2014). That is, these trainings do not directly teach academic skills, and as such we expected that any downstream improvements in academic functioning would take time to develop. Families were not required to withhold psychostimulants prior to child treatment or psychoeducational testing visits.

Teacher questionnaires were sent during the post-treatment session and were completed by teachers approximately 1-2 months post-treatment (M=49.07 days, SD=43.87; the treatment groups were equivalent in the time between treatment completion and teacher behavioral ratings, p=.55, BF₀₁=4.52). Of note, our preregistration called for a 2-month follow-up testing session; however, in running the trial we elected to allow more flexibility to facilitate retention by maximally accommodating families’ availability to attend the follow-up appointment. As such, the relatively wide follow-up testing and teacher completion SDs were to maximize retention through follow-up (e.g., children unavailable during the summer due to travel or out-of-town custody arrangements); the follow-up testing session is therefore described as a 2-4 month follow-up to better characterize the obtained range. For children completing treatment during their summer holiday, teacher questionnaires were sent 1 month after the start of the school year to allow teachers sufficient time to observe the child.

Treatments

BPT, CET, and ICT were delivered according to manualized procedures in small group format or individually as needed to accommodate families’ schedules. Schedule changes were accommodated to the extent possible (e.g., make-up sessions the same week). Identical procedures were used for all treatments (e.g., 10 in-office sessions at 1 hour each). As reported in Kofler et al. (2018, 2020), the interventions were equivalent or did not differ in terms of parent satisfaction, feasibility, acceptability, and barriers to treatment participation.

Behavioral Parent Training (BPT)

Evidence-based BPT (Evans et al., 2018) was provided using the manualized, 9-session Barkley (1997) Defiant Children protocol. BPT was delivered by behaviorally trained, PhD-level clinicians. Parent-reported ADHD symptom reductions in the BPT treatment group (d_mean=1.38) were reported in Kofler et al. (2018) and were consistent with (van der Oord et al., 2008) or larger than (Sonuga-Barke et al., 2013) expectations based on meta-analysis. Additional evidence supporting treatment fidelity, feasibility, acceptability, accessibility, and perceived utility are reported in Kofler et al. (2018).

Central Executive Training (CET)

CET is a translational, evidence-based, hybrid (in-office and at-home), and software-based treatment protocol that trains the ‘working’ components of working memory (dual-processing, continuous updating, serial/temporal reordering; Kofler et al., 2018, 2020) via repeated practice and feedback using a suite of 9 computer ‘games’ developed based on the Baddeley (2013) and Wager & Smith (2003) models of working memory. The 10-week protocol includes weekly in-office sessions with the child (1 hour) combined with parent-supervised, in-home training (goal: 15-min/day, 2-3 days/week). CET has been shown to be superior to both BPT and ICT for improving working memory (d=0.96-1.25 across studies), with mechanism of change analyses linking CET-related working memory improvements with CET-related ADHD symptom reductions (Kofler et al., 2018, 2020). In terms of primary clinical outcomes (ADHD symptoms), CET demonstrated superiority to both BPT and ICT for reducing objectively-assessed hyperactivity during clinic-based testing (Kofler et al., 2018, 2020). Similarly, CET was superior to ICT for reducing parent- and masked teacher-reported ADHD symptoms (Kofler et al., 2020).

Inhibitory Control Training (ICT)

ICT was identical to CET in all aspects except for the active ingredient (targeting inhibitory control instead of central executive working memory). For example, each of the 9 matched pairs of ICT/CET training games were identical in terms of website address, name, art, animations, storylines, layouts, interfaces, and use of adaptive training algorithms to maximize internal/construct validity. More generally, best practice guidelines for cognitive training studies were closely followed (Table 1), allowing strong conclusions regarding emerging group differences as a function of training target (Redick, 2015). ICT has been shown to be superior to CET for improving stop-signal inhibitory control (d=1.12), despite not improving go/no-go inhibition (Kofler et al., 2020). Notably, targeting multiple neurocognitive systems within a single intervention protocol reduces potency and thus limits efficacy (i.e., dividing training time across more targets = lower dosage per target), as shown in recent ADHD cognitive training meta-analyses (Rapport et al., 2013). Thus, CET and ICT were developed as distinct interventions that serve as active, credible controls for each other (Simons et al., 2016).

Measures

Intellectual Functioning (IQ) and Socioeconomic Status (SES) at Pre-Treatment

IQ was estimated using the WASI-II (BPT) or WISC-V (CET, ICT) Verbal Comprehension Index (Wechsler, 2011, 2014). Hollingshead (1975) SES was estimated based on caregiver(s)’ education and occupation.

Academic Outcomes

Subjective outcomes.

The Academic Performance Rating Scale (APRS; DuPaul et al., 1991) was completed by each child’s teacher to assess academic performance (2-week test-retest=.93-.95; α=.94-.95). This measure provides subjective estimates of learning behaviors that contribute to academic success (e.g., Demaray & Jenkins, 2011) which in turn have been previously shown or hypothesized to depend at least in part on working memory and/or inhibitory control processes (e.g., Kofler et al., 2016). The APRS contains three subscales: The Academic Success scale contains seven items that assess the quality of reading and spoken work, how quickly children learn new material, and how well they retain new information. The Impulse Control scale contains 3 items that assess carelessness in beginning and completing work, and handwriting neatness. The Academic Productivity scale contains 12 items that assess academic efficiency (e.g., percentage of classwork completed correctly) and consistency, following group instructions, and timely completing work. Teachers were masked to treatment allocation; masking maintenance was excellent (Kofler et al., 2020).² Raw scores were selected a priori as recommended for research purposes (Achenbach, 1991). Higher scores reflect better academic functioning and impulse control at school.

Objective outcomes.

The Kaufman Test of Educational Achievement (KTEA-2/3; Kaufman & Kaufman, 2004, 2014) was used to assess academic achievement (1-2 week test-retest=.80-.96; α=.92-.99). Three subtests were administered to provide objective estimates of higher-order academic skills previously shown or hypothesized to depend, at least in part, on working memory and/or inhibitory control (e.g., Spiegel et al., 2021): Reading comprehension was measured by the Reading Comprehension subtest, which involves reading words, sentences, and passages, and responding to comprehension questions. Math problem-solving was measured by the Math Concepts & Applications subtest, which involves applying number concepts, arithmetic, time and money, and measurement skills to solve math problems that relate to real life situations. Language comprehension was measured by the Listening Comprehension subtest, which involves listening to sentences or passages, and responding to comprehension questions. To minimize test-retest effects, parallel forms were used, with Form A administered at pre-treatment and Form B administered at follow-up.

Standardized growth scale values (GSV) were used as recommended by the test publisher for tracking response to intervention (RTI; Kaufman & Kaufman, 2014). GSV scores are based on the same normative sample used to construct standard scores but are constructed using item response theory specifically to assess change over time. They are a linear transformation of the Rasch ability scale for each subtest that indexes the child’s absolute level of performance. Because the Rasch calibration of each subtest was based on Forms A and B combined, GSVs are the same across forms, have true equal intervals, and are more sensitive/better suited than raw or standardized scores for measuring change over time (Farmer et al., 2020; Kaufman & Kaufman, 2014).³ For all subtests, the average GSV for grade 3 is anchored at 500. Higher scores reflect higher achievement.

Bayesian Analyses

Traditional null hypothesis significance tests (p-values) were supplemented with Bayes Factors as recommended (Redick, 2015). BF₁₀ is the Bayes Factor (BF) indicating how much more likely the alternative hypothesis (H₁) is relative to the null hypothesis (H₀). Values ≥3.0 are considered moderate support for the alternative hypothesis (Wagenmakers et al., 2018). BF₀₁ is the inverse of BF₁₀ (i.e., BF₀₁=1/BF₁₀), and is reported when the evidence favors the null hypothesis (Rouder & Morey, 2012). BF₀₁ is interpreted identically to BF₁₀ (≥3=moderate, >10=strong, >100=decisive evidence that the treatment groups are equivalent on an outcome). Both p-values and Bayes Factors are reported. We refer to findings of BF₁₀ ≥ 3 as significant evidence for an effect (i.e., support for the alternative hypothesis of an effect at/above pre-specified evidentiary thresholds), and findings of BF₀₁ ≥ 3 as significant evidence against an effect (i.e., support for the null hypothesis of no effect at/above pre-specified evidentiary thresholds). We refer to effects as ‘marginally significant’ when results indicate p<.05 but BF₁₀ < 3.0 (i.e., when the effect is supported by null hypothesis testing but the Bayes Factor suggests evidentiary value below our prespecified threshold).

Data Analysis Overview

Data analyses were conducted with default prior scales using JASP 0.14 (JASP Team, 2020) according to the preregistered plan. We organized our analyses into three analytic Tiers. In Tier 1 we (a) compared the BPT, CET, and ICT treatment groups on pre-treatment characteristics and (b) compared BPT, CET, and ICT for effects on the subjective and objective academic outcomes (masked teacher-reported academic performance, objectively-assessed academic achievement). These analyses involved treatment x subscale/subtest x time mixed-model ANOVAs, with post-hocs following significant interactions and preregistered planned contrasts to characterize the pattern of change over time separately for each treatment group. In Tier 2, exploratory sensitivity analyses were added to (a) probe the extent to which the pattern of results replicated when analyzed separately for the Study 1 and 2 samples described above, and (b) directly compare CET effects across Study 1 vs. Study 2. Tier 3 included additional exploratory analyses to (a) address the mechanisms of change, which involved correlating changes in working memory and inhibitory control between pre- and post-treatment (as reported in Kofler et al., 2018, 2020) with changes in the subjectively- and objectively-assessed academic outcomes between pre-treatment and follow-up, and (b) investigate clinical significance by estimating the proportion of children in each treatment group who demonstrated reliable change on each outcome measure and using these data to compute the Number Needed to Treat (NNT).

Results

Power Analysis

Our sample size was determined by our preregistered stopping rule (Kofler et al., 2018, 2020), which was in turn determined by best practice recommendations for cognitive training studies (Simons et al., 2016). Power analysis using G*Power 3.1 (Faul et al., 2007) indicated that for α=.05, β=.80, and 3 treatments (BPT, CET, ICT), our N=108 is powered to detect within-subject effects of time at d ≥.28, treatment x time interactions of d ≥.30, and between-treatment effects of d ≥.44. For the mechanism of change analyses, our N is powered to reliably detect bivariate correlations of r >.26. Thus, the study is sufficiently powered to address its primary aims.

Study Retention, Outliers, and Missing Data Handling

Study retention was high for all three treatment groups. Notably, 85% of BPT families completed treatment, with 96% of BPT completers returning for the follow-up child testing session and 70% of teachers returning the post-treatment questionnaires. Completion rates for ICT (83% completers, 75% returning for follow-up testing, 79% of teachers completing post-treatment questionnaires) and CET (94%, 83%, 71%, respectively) were similarly high. The treatments were equivalent on these completion/return rates (all p > .15, all BF₀₁ > 3.06). In addition, the evidence indicated that missing data were missing completely at random (i.e., not systematically related to any child demographic characteristics or outcome measures; Little’s MCAR test p=.99). Thus, there was no evidence for differential attrition by treatment group. We therefore analyzed the data based on the a priori plan. Missing data were imputed using the preregistered plan (expectation-maximization based on all available data). This maximum likelihood-based approach has been shown to produce unbiased results for missingness rates at/above the current levels when data are missing at random (Kristman et al., 2004) as was the case in the current study.

Tier 1. Pre-Treatment Characteristics

As shown in Table 2, the BPT, CET, and ICT treatment groups were equivalent or did not differ demographically at pre-treatment (p>.13, all BF₀₁>1.99) and were equivalent in terms of ADHD symptom severity (all p>.20, all BF₀₁>3.01). In contrast, the BPT treatment group demonstrated a significant pre-treatment advantage on Math Concepts & Applications (p=.007, BF₁₀=5.00, d=0.75-0.77) and Reading Comprehension (p=.01, BF₁₀=3.24, d=0.68-0.74) and a marginally significant advantage on teacher-perceived Academic Success (p=.01, BF₁₀=2.55, d=0.60-0.68). Thus, despite the treatments being equivalent or not differing in terms of age, sex, IQ, SES, maternal education level, ethnicity, medication status, and ADHD symptom severity, the BPT treatment group demonstrated an unexpected pre-treatment advantage on up to 50% of the academic outcomes. Pre-treatment scores were therefore included in all analyses, residual gain score analyses (follow-up scores covaried for pre-treatment scores) were included, and analyses were repeated using just the randomized Study 2 (CET vs. ICT; n=54) sample cases, who were equivalent or did not differ at pre-treatment on all academic measures (all p>.18, all BF₀₁>1.71).

Tier 1. Subjective Academic Outcomes (Teacher-Reported Academic Performance at 1-2 Months Post-Treatment)

The Treatment (BPT, CET, ICT) x APRS Subscale (Academic Success, Impulse Control, Academic Productivity) x Time (Pre, Post) mixed-model ANOVA was significant for main effects of Time (p=.004; BF₁₀=2.89 x 10³), Treatment (p=.09, BF₁₀=18.21), and Subscale (p<.001; BF₁₀=8.84 x 10¹²), and for the Treatment x Time (p=.03; BF₁₀=3.99), Treatment x Subscale (p=.008; BF₁₀=41.40), and Time x Subscale (p<.001; BF₁₀=8.27) interactions.⁴ The 3-way interaction was not significant (p=.42; BF₀₁=2.96). Post-hocs for the significant interactions indicated that CET was superior to both BPT (Treatment x Time interaction: d=0.54, p=.02, BF₁₀=4.51) and ICT (Treatment x Time interaction: d=0.44, p=.05, BF₁₀=3.88) for improving overall teacher-perceived academic performance at school. Corrected for multiple comparisons, CET portended significant pre-post improvements in Academic Success (d=0.55, p=.001, BF₁₀=104.18), Impulse Control (d=0.48, p=.02, BF₁₀=24.99), and Academic Productivity (d=0.53, p=.002, BF₁₀=64.08). In contrast, neither BPT (d= −0.10 to 0.15, all p>.99, all BF₀₁>3.67) nor ICT (d=0.00 to 0.27, all p>.99, all BF₀₁>1.98) produced significant improvements on any APRS subscale (Figure 1; Table 3).

Figure 1.

Treatment-related improvements in subjective and objective academic outcomes. * = significant improvement at follow-up relative to pre-treatment (p<.05 and BF₁₀>3.00).

Table 3.

Academic functioning at 1-2 month follow-up (teacher reports) and 2-4 month follow-up (achievement testing)

Variable	BPT (n=27)		CET (n=52)		ICT (n=29)		Pre-Treatment to Post-Treatment/Follow-up Change (Cohen’s d Effect Sizes)
	M	SD	M	SD	M	SD	BPT	CET	ICT
Teacher-Perceived Academic Functioning (APRS Raw Scores)
Academic Success	24.82	4.72	23.83	4.46	21.41	4.12	−0.09, ns	0.55 *	0.00, ns
Impulse Control	8.15	1.98	8.19	2.28	8.41	1.99	−0.10, ns	0.48 *	0.15, ns
Academic Productivity	40.26	6.07	42.04	6.65	37.76	7.71	0.15, ns	0.53 *	0.27, ns
Objectively-Assessed Academic Achievement (KTEA GSV Scores)
Reading Comprehension	527.44	16.03	522.96	12.93	519.83	21.44	0.25, ns	0.78 *	0.86 *
Math Concepts & Applications	531.82	14.32	523.10	17.61	516.52	23.32	0.96 *	1.30 *	0.87 *
Listening Comprehension	516.67	12.67	512.90	9.98	511.21	14.73	0.01, ns	0.24, ns	0.20, ns

Note. Effect sizes and statistical tests reflect paired-sample T-tests (pre/post or pre/follow-up). Cohen’s d effect sizes are interpreted as small = .20; medium = .50; large = .80.

^*= statistically significant change (p < .05 and BF₁₀ > 3.0).

BPT = behavioral parent training; CET = Central Executive Training; ICT = Inhibitory Control Training; APRS = Academic Performance Rating Scale; KTEA = Kaufman Test of Educational Achievement; GSV = standardized growth scale values (for all subtests, the average GSV for grade 3 is anchored at 500). Higher scores = better functioning on all subscales/subtests.

Tier 1. Objective Academic Outcomes (Academic Achievement at 2-4 Month Follow-Up Testing)

The Treatment (BPT, CET, ICT) x KTEA Subtest (Reading Comprehension, Math Concepts & Applications, Listening Comprehension) x Time (Pre, Follow-Up) mixed-model ANOVA was significant for main effects of Time (p=.001; BF₁₀=1.57 x 10¹⁰), Treatment (p=.02, BF₁₀=14.58), and Subtest (p<.001; BF₁₀=2.01 x 10¹⁴), for the Treatment x Subtest (p=.06; BF₁₀=9.06) and Time x Subtest (p<.001; BF₁₀=1.41 x 10³) interactions, and marginally for the Treatment x Time interaction (p=.005; BF₁₀=2.47). The 3-way interaction was not significant (p=.08; BF₀₁=1.89). Post-hocs for the significant interactions indicated that CET was superior to BPT for improving overall academic achievement (Treatment x Time interaction: d=0.76, p=.001, BF₁₀=16.36). ICT was also marginally superior to BPT for improving overall academic achievement (Treatment x Time interaction: d=0.54, p=.05, BF₁₀=1.21). The pattern for CET vs. ICT was more nuanced: The Treatment x Time interaction for overall academic functioning (d=0.28, p=.22, BF₀₁=5.56) was not significant, whereas there was a significant Time x Subtest interaction (p=.002, BF₁₀=390.56). A priori planned contrasts indicated that CET was superior to ICT for improving Math Concepts & Applications (Treatment x Time interaction: d=0.56, p=.02, BF₁₀=7.93) but that CET and ICT did not differ in terms of improvements in Reading Comprehension (Treatment x Time interaction: d=0.14, p=.52, BF₀₁=2.72) or Listening Comprehension (Treatment x Time interaction: d=0.04, p=.97, BF₀₁=5.80).

Corrected for multiple comparisons, post-hoc tests and a priori planned comparisons indicated that treatment effects differed by treatment group for Reading Comprehension (Treatment x Time interaction: p=.006; BF₁₀=17.22) and Math Concepts & Applications (Treatment x Time interaction: p=.02; BF₁₀=8.33) but not Listening Comprehension (Treatment x Time interaction: p=.67; BF₀₁=11.63). The CET (d=0.78, p<.001, BF₁₀=1.96 x 10⁴) and ICT (d=0.86, p=.001, BF₁₀=348.86) treatments both demonstrated significant pre/follow-up improvements in Reading Comprehension, whereas the BPT treatment did not (d=0.25, p=.99, BF₁₀=2.34). In contrast, all three groups demonstrated large improvements in Math Concepts & Applications; the significant interaction was due to larger improvements for CET treatment (d=1.30, p<.001, BF₁₀=7.10 x 10⁹) relative to the ICT (d=0.87, p<.001, BF₁₀=385.69) and BPT (d=0.96, p=.99, BF₁₀=708.48) groups. Finally, none of the groups showed significant improvements in Listening Comprehension: CET (d=0.24, p=.99, BF₀₁=1.58), ICT (d=0.20, p=.99, BF₁₀=3.04), BPT (d=0.01, p=.99, BF₁₀=4.90) (Figure 1).

Tier 2. Sensitivity Analyses: Comparing CET in Study 1 vs. Study 2

To probe the impact of our a priori decision to combine the CET samples, we conducted two sets of exploratory analyses. First, we directly compared the Study 1 vs. Study 2 CET samples. This involved repeating each of the primary analyses above, this time with just the CET sample classified as Study 1 vs. Study 2. Next, we repeated the primary analyses again, this time separately for Study 1 (BPT vs. 1^st CET sample) and Study 2 (ICT vs. 2^nd CET sample). Reporting is truncated for readability.

CET effects in study 1 vs. study 2.

Consistent with the primary analyses above, there were significant main effects of Time in all models (p<.001, BF₁₀>47.81) except Listening Comprehension (d=0.24, p=.09, BF₀₁=1.32), reflecting the overall pattern of results that CET portended significant improvements in teacher-perceived Academic Success (d=0.55, p<.001, BF₁₀=75.24), Impulse Control (d=0.51, p<.001, BF₁₀=47.81), and Academic Productivity (d=0.52, p<.001, BF₁₀=70.61) at 1-2 months post-treatment; and Reading Comprehension (d=0.78, p<.001, BF₁₀=1.08 x 10⁴) and Math Concepts & Applications (d=1.29, p<.001, BF₁₀=2.71 x 10⁹) at 2-4 month follow-up. The CET phase x time interaction was significant only for APRS Impulse Control (p=.02, BF₁₀=6.58), with post-hocs indicating that CET in Study 2 (d=1.01, p<.001, BF₁₀=624.50) showed larger improvements than CET in Study 1 (d=0.17, p=.40, BF₁₀=3.51). All other CET phase x time interactions were non-significant at p>.47 (BF₀₁>1.94) except Listening Comprehension, which failed to reach significance at p=.06 (BF₀₁=1.36; d improved with marginal significance for CET in Study 1, d=0.46, p=.03, BF₁₀=2.21 but not in Study 2, d= −0.04, p=.85, BF₀₁=4.67) (Figure S1).

Replicability of findings across study 1 and study 2.

Overall, the pattern of results reported above were directly replicated across Study 1 (BPT vs. CET) and Study 2 (CET vs. ICT), with minor exceptions that appear attributable to the reduced power when limiting the analyses to half of the available CET participants. Effect sizes and statistics reflect the Treatment x Time interaction for each comparison. CET was superior to BPT (Study 1) and ICT (Study 2) for improving Academic Success (both d=0.62, BF₁₀=2.16-3.60, both p=.03), superior to ICT (Study 2) for improving Impulse Control (d=0.63, BF₁₀=2.84, p=.03), superior to BPT (Study 1) for improving Reading Comprehension (d=0.64, BF₁₀=4.70, p=.03), and superior to both BPT (Study 1) and ICT (Study 2) for improving Math Concepts & Applications (Study 1 BPT vs. CET: d=0.71, BF₁₀=11.20, p=.01; Study 2 CET vs. ICT: d=0.46, BF₁₀=1.76, p=.046). In the few instances where Treatment x Time interactions failed to reach significance, significant main effects of Time were detected (p<.001, BF₁₀>100), consistent with the finding above that CET demonstrated significant improvements on all assessed outcomes except Listening Comprehension (Figure S1).

Residual gain scores (post-treatment covaried for pre-treatment).

Given the BPT treatment group’s unexpected pre-treatment advantage on a portion of the academic outcomes noted above, we computed residual gain scores to further characterize the pattern of change over time. Results for the Study 1 (BPT vs. CET) sample indicated that the BPT treatment group demonstrated marginally significant to significantly higher pre-treatment academic functioning than the CET treatment group on all academic outcomes (p<.046, BF₁₀=1.50-34.78) except Academic Productivity (p=.31, BF₀₁=2.38), but that the CET treatment group improved disproportionately to the BPT group and became equivalent, or not significantly different, than BPT on all academic outcomes at post-treatment/follow-up (p=.22-.99, BF₀₁=1.94-3.65). In contrast, in Study 2 the CET and ICT treatment groups were equivalent or did not differ at pretreatment (p=.18-.97, BF₀₁=1.71-3.64), whereas the CET treatment group improved disproportionately and was therefore superior to ICT at post-treatment/follow-up for Academic Success (d=0.88, p=.002, BF₁₀=16.46), Impulse Control (d=0.63, p=.03, BF₁₀=2.32), Academic Productivity (d=0.67, p=.02, BF₁₀=3.00), and Math Concepts & Applications (d=0.69, p=.02, BF₁₀=3.51); the CET scores were also descriptively higher relative to ICT at post-treatment for Reading Comprehension and Listening Comprehension, but these contrasts were not significant (d=0.20-0.38, p>.17, BF₀₁>1.62).

Tier 2. Sensitivity Analyses: Medication Status, Medication Changes, and Maturation

Next, we conducted sensitivity analyses to probe for alternate explanations for the pattern of results. Despite the groups not differing in terms of pre-treatment medication status, medication changes during the course of treatment, age, or time between completing treatment and follow-up testing/teacher reports (Table 2), it was possible that the significant main effects of time were attributable to one or more of these potential confounds rather than the tested treatments. This hypothesis was unsupported: The pattern, significance, and interpretation of all results were unchanged when medication status, medication changes, age, time between post-treatment and teachers completing the APRS, or time between post-treatment and the follow-up KTEA testing were added to the models.⁵ In addition, none of these covariates showed significant main effects, with the exception of an expected main effect of age in the KTEA model (older children have learned more academic skills than younger children), suggesting that the findings were not attributable to medication effects, medication changes, or maturation.

Tier 3. Exploratory Analyses: Neurocognitive Mechanisms of Change

Exploratory analyses were conducted to test the mechanisms of change. This involved computing change scores (post or follow-up – pre-treatment) to index overall change for each participant for each outcome. These scores were then correlated with the changes in working memory and inhibitory control reported in Kofler et al. (2018, 2020) to test the hypothesis that children who demonstrated greater pre/post improvements in executive functioning would make greater pre/follow-up gains academically. Inhibitory control was only assessed as an outcome in Study 2; therefore, analyses were run separately for the two studies. In Study 1 (BPT vs. CET), improvements in working memory were associated with improvements in math problem-solving (r=.31, p=.01, BF₁₀=4.49) and language comprehension (r=.24, p=.04, BF₁₀=1.51). These findings replicated in Study 2 (CET vs. ICT), with greater improvements in working memory predicting greater gains in math problem-solving (r=.33, p=.008, BF₁₀=5.49) and language comprehension (r=.27, p=.03, BF₁₀=2.13). In addition, Study 2 showed significant relations between improvements in working memory and improvements in teacher-perceived academic success (r=.31, p=.01, BF₁₀=4.05) and impulse control (r=.26, p=.03, BF₁₀=1.99). Similarly, improvements in inhibitory control were associated with improvements in impulse control (r=.35, p=.005, BF₁₀=8.97), academic success (r=.26, p=.03, BF₁₀=1.92), and academic productivity (r=.29, p=.02, BF₁₀=2.99) in Study 2. Combined with the primary analyses above, these findings provide additional evidence that the improvements in executive functioning produced by these neurocognitive training protocols produce, to a significant extent, improvements in some, if not most, of the academic outcomes assessed herein.

Tier 3. Exploratory Analyses: Reliable Change and Number Needed to Treat

A final set of analyses were added to estimate the proportion of children in each treatment who exhibited reliable change on each outcome measure, and then compute the Number Needed to Treat (NNT). These analyses were not preregistered but were added during the peer review process. Briefly, NNT refers to the number of patients who need to be treated to get one more patient better than would have improved without the treatment; 1/NNT estimates the probability that any given patient will benefit from the treatment (Wang et al., 2000). Smaller values of NNT suggest more effective treatments; NNT values are rounded to the nearest whole number. Biederman et al. (2019) considered NNTs ≤ 10 to be indicative of effective treatment for ADHD given the high cost of illness and associated functional impairments/consequences; a more general rule-of-thumb based on Chong et al. (2006) suggests that NNTs ≤ 5 indicates an effective treatment that should be ‘ruled in’ by clinicians whereas NNTs > 15 suggests small/minimal treatment benefits that should be ‘ruled out’ by clinicians. Reliable change was computed using the Zakzanis (2001) ‘percent non-overlap’ method, which estimates the percentage of participants in each treatment group whose post-treatment/follow-up scores fell outside of the pre-treatment range of scores (based on the pre-post/follow-up Cohen’s d main effects of time for each treatment).⁶ Results are shown in Table 4. Focusing on the outcomes for which CET showed significant improvements that replicated across both trials in the primary analyses, 33%-36% of children treated with CET demonstrated reliable improvements in teacher-reported academic success and productivity relative to 0-18% of BPT/ICT cases (all p≤.01), which corresponded to NNTs of 3 for academic success and 4-7 for academic productivity. Similarly, 38%-72% of children treated with CET showed reliable improvements in reading and math achievement relative to 18%-54% of BPT/ICT cases (all p≤.008), which corresponded to NNTs of 4-5.

Table 4.

Proportion of children exhibiting reliable change and the number needed to treat (NNT) for central executive training vs. behavioral parent training (Study 1) and inhibitory control training (Study 2).

	Study 1				Study 2
	% Reliably Improved		χ2 p-value	Number Needed to Treat	% Reliably Improved		χ2 p-value	Number Needed to Treat
	BPT	CET			CET	ICT
Teacher-reported academic performance
Academic success	0	33	< .001	3	36	0	< .001	3
Impulse control	0	11	< .001	3	55	8	< .001	2
Academic productivity	8	33	< .001	4	33	18	.01	7
Objectively-assessed academic achievement
Reading comprehension	18	38	.002	5	72	50	.001	4
Math concepts & applications	54	72	.008	5	72	50	.001	4
Listening comprehension	0	30	< .001	3	0	15	<.001	−4

Note: Number needed to treat was computed for central executive training, based on the proportion of children in the CET treatment group who demonstrated reliable change from CET relative to the proportion of children who demonstrated reliable change in the active, credible control conditions in Study 1 and Study 2. Number needed to treat (NNT) refers to the number of patients who need to be treated to get one more patient better than would have improved without the treatment; 1/NNT estimates the probability that any given patient will benefit from the treatment (Wang et al., 2000). Smaller positive values suggest more effective treatments; the negative NNT value for listening comprehension in study 1 indicates that fewer children reliably improved in the CET relative to ICT treatment group, consistent with the primary analyses indicating that CET does not produce significant benefits on listening comprehension. NNTs ≤ 5 based on the current clinical trials are bolded; boxes reflect outcomes where CET showed significant benefits in the primary analyses that replicated across Study 1 and Study 2 (these outcomes also showed low NNTs suggestive of clinically significant benefits). P-values computed based on 2 (treatment: CET, BPT/ICT) x 2 (status: improved, not improved) χ² tests with Yates continuity correction.

Discussion

The current study was the first to examine academic far-transfer effects of multiple neurocognitive training protocols relative to an active, credible control condition (behavioral parent training) previously shown to improve child behavior but not academic functioning (van der Oord et al., 2008). Confidence in the findings is supported by study strengths including the preregistration of both trials, open data, use of both subjective evaluation by masked teachers and objective testing by masked evaluators, sequential and randomized designs that minimize most if not all threats to validity, and inclusion of multiple interventions that served as active, credible controls for each other.

Did Behavioral Parent Training Improve Academic Outcomes?

Overall, behavioral parent training (BPT) was not associated with significant improvements on any academic outcomes, as expected, with one exception: BPT was associated with large improvements in math problem-solving. These gains were nonetheless inferior to those detected for central executive training and inhibitory control training and likely reflect developmental growth and the impact of explicit instruction and practice at school rather than effects of BPT per se. This pattern of results was largely expected based on prior meta-analyses indicating small or null effects of BPT on academic achievement and masked teacher ratings (Sonuga-Barke et al., 2013; van der Oord et al., 2008), and likely reflects the highly setting-specific and time-limited effects of behavioral interventions (e.g., Chronis et al., 2004).

Did Central Executive Training Improve Academic Outcomes?

In contrast to BPT, central executive training (CET) and to a lesser extent inhibitory control training (ICT) portended significant improvements in academic functioning, which provides strong support for academic far-transfer effects of CET and potentially ICT but not BPT. Specifically, CET produced significant improvements in teacher-perceived academic performance and objectively-assessed academic achievement that in most cases (a) were superior to BPT and/or ICT, (b) replicated across Study 1 and Study 2 trials, and (c) were significantly correlated with the CET-related improvements in executive functioning reported in the primary outcomes papers (Kofler et al., 2018, 2020). That is, CET showed significant effects on all 6 academic outcomes in at least one study, with 4 of these 6 outcomes replicating across studies. Specifically, CET portended significant, medium to very large magnitude improvements in teacher-reported academic success, impulse control, academic productivity and objectively-assessed reading comprehension and math problem-solving.

Direct comparison of CET samples from Study 1 and Study 2 showed that in most cases these gains were superior to gains associated with ICT and/or BPT. CET was superior to both BPT and ICT for improving all teacher-reported academic outcomes at 1-2 months post-treatment (d=0.48-0.55) and objectively-assessed math problem-solving at 2-4-month follow-up (d=0.56-0.76). This replicated finding provides strong efficacy evidence – particularly in the context of Study 2’s rigorous RCT design, which addressed the few potential validity threats not controlled by Study 1’s sequential design but nonetheless replicated CET’s superiority relative to an active, credible, and adaptive control (i.e., ICT) for improving academic success, impulse control, academic productivity, and math problem-solving (d=0.63-0.88). The clinical significance of CET’s effects on these four academic outcomes were evident in (a) the medium-to-large treatment x time interaction effect sizes favoring CET, (b) the significantly higher proportion of treatment responders in the CET vs. BPT/ICT groups, as well as the low ‘number needed to treat’ values that fell within the range for treatments that should be ‘ruled in’ for children with ADHD (Biederman et al., 2019; Chong et al., 2006).

Did Inhibitory Control Training Improve Academic Outcomes?

Despite the mixed and generally disappointing findings for ICT in the primary outcomes paper (Kofler et al., 2020), ICT produced somewhat more promising – albeit still mixed – results for academic functioning. ICT failed to improve teacher perceived academic functioning, and was inferior to CET for improving math problem-solving, but was equivalent to CET and superior to BPT for improving reading comprehension and overall academic achievement. Although this finding needs replication, it adds to a small but growing literature base linking inhibitory control to reading skills, while seeming to contradict research indicating stronger relations with math than reading skills, especially in the early academic years (Allan et al., 2014; Son et al., 2019). Nonetheless, several studies support the unique contribution of inhibitory control to early word reading skills (Blair & Razza, 2007) and to later comprehension skills such as inhibiting irrelevant word meanings (Barnes et al., 2004), ignoring irrelevant contextual information (Cain, 2006; Kieffer et al., 2013), and inhibiting irrelevant information from retrieval (Borella et al., 2010).

Why Did These Neurocognitive Training Protocols Yield Academic Far-transfer Benefits?

To our knowledge, this study is the first to show academic far-transfer benefits of a neurocognitive training protocol relative to both gold-standard behavioral parent training and an active, credible, and adaptive neurocognitive training intervention (inhibitory control training; Rapport et al., 2013; Simons et al., 2016). However, while these findings are consistent with model-driven predictions regarding the foundational role of working memory for academic success (Kofler et al., 2018; Rapport et al., 2001, 2008; Simone et al., 2018; Spiegel et al., 2021), at first glance they may appear surprising because (a) CET does not explicitly teach academic skills; and (b) first-generation working memory training protocols generally failed to improve academic outcomes for children with ADHD (Rapport et al., 2013). Based on conceptual models linking executive dysfunction with academic underachievement in ADHD (e.g., Rapport et al., 2001) and developmental models linking executive functioning with academic functioning more generally (e.g., Fuhs et al., 2014; Rabiner et al., 2016; Willoughby et al., 2012), we propose two potential mechanisms that may explain why CET produced clinically significant academic benefits that replicated across both clinical trials.

First, as argued by Chacko et al. (2014), improving central executive and related executive functions is expected to result in improved potential due to the improved neurocognitive foundation. That is, with better developed executive functions and resulting improvements in attentive behavior in the classroom (Kofler et al., 2020), children treated with central executive training may be better able to benefit from the direct instruction and practice in these skills that occurs at school. Alternatively, it may be that the treatment-related behavioral gains seen at home, at school, and in the clinic (Kofler et al., 2018, 2020) decrease the impact of interfering behaviors on achievement test performance. This hypothesis is consistent with conceptualizations of ADHD as reflecting performance rather than skills deficits, as well as empirical evidence suggesting that at least some functional impairments in ADHD may be attributable to difficulties selecting and implementing learned skills in the moment rather than a lack of knowledge or skills (Aduen et al., 2018), with these performance difficulties in turn linked with these children’s underlying working memory deficits (Kofler et al., 2018). In this case, the differential improvements in academic testing produced by CET may have occurred, in part, because children with better developed working memory are better able to sustain attention to tasks and maintain task goals in mind during testing (e.g., Kofler et al., 2010), rather than resulting from a differential increase in academic knowledge per se. Future work with larger samples and a broader academic assessment is needed to test these hypotheses.

Limitations

The current study has several strengths, including the relatively large sample, masked assessment of both teacher-perceived and performance-based/objective academic functioning several months after children completed treatment, and rigor of the intervention trials as detailed in Table 1 and Kofler et al. (2018, 2020). Nonetheless, the following limitations should be considered when interpreting results. First, despite the study’s relatively large sample size and that the treatment groups were equivalent with regard to race/ethnicity, SES, and maternal education level, the majority of the participants in the study identified as White and relatively high maternal education levels were reported (i.e., majority of mother’s reported having a Bachelor’s degree or higher). As such, the findings from the study may not generalize to historically-excluded racial minority groups as well as children who have mothers with relatively lower levels of education. Second, although parents were explicitly asked about changes in medication and other treatments or services, none of them reported new or modified academic interventions/ accommodations, and thus we were unable to probe for potential additive effects of treatment + school-based services. Similarly, although the link between ADHD and academic underachievement is well established (e.g., DuPaul et al., 2013), academic difficulties were not part of the study’s inclusion criteria, and children with learning disabilities made up a relatively small proportion of the sample. Thus, we were unable to examine the extent to which CET and/or ICT may provide greater or fewer benefits for children with more severe learning difficulties/underachievement.

Relatedly, despite clear evidence from Study 1 that CET improved – and improved differentially relative to an active, credible control – on most academic outcomes, history and ceiling/regression to the mean effects were plausible alternative explanations for the superiority of CET over BPT at the end of Study 1. However, it stands to reason that the findings from Study 2 rule out both history and ceiling/regression effects as plausible explanations for CET’s effects in Study 1, because Study 2 was an RCT in which the CET and active, credible control treatment (ICT) were equivalent at pretreatment on all assessed academic outcomes. History effects were ruled out because the interventions were implemented at the same time, and ceiling effects (and regression to the mean) were ruled out because the CET treatment again showed significant and medium/large magnitude improvements on most outcomes – replicating Study 1 – and again showed differential improvements on the masked teacher reported academic outcomes as well as 2 of the 3 academic achievement outcomes. Nonetheless, given the BPT treatment group’s pre-treatment advantage on a subset of the current study’s outcomes of interest, some readers may prefer to limit interpretation to the Study 2 (CET vs. ICT) findings, which included randomized groups that did not differ academically at pre-treatment but differed on most outcomes at post-treatment due to larger gains associated with CET relative to ICT. Next, although we included multiple, core components of academic achievement, additional research is needed to examine the extent to which CET and/or ICT produce improvements in other academic domains (e.g., written/oral expression, decoding, academic fluency). Finally, although the study design provided a robust replication of CET’s benefits on academic functioning across two trials, ICT was only tested in Study 2 and thus the extent to which its more limited benefits will replicate in future trials remains unknown.

Clinical and Research Implications

Taken together, the current findings and the results from the primary outcome studies (Kofler et al., 2018, 2020) provide strong and replicated evidence that central executive training (CET) improves executive functioning, reduces ADHD symptoms at home, school, and in the clinic, and exerts medium to large magnitude downstream effects on academic functioning based on both masked teacher perceptions and objective testing that are apparent several months after treatment is terminated. Thus, CET meets all criteria for a ‘well established treatment’ for ADHD-related executive dysfunction, behavioral symptoms, and academic difficulties except for replication by an independent research team (Evans et al., 2018, 2020). In addition to replicated evidence of CET’s direct benefits for children with ADHD, the current findings also have implications for conceptual models of academic functioning by providing causal evidence that improving working memory results in improvements in overall academic achievement, math and reading skills specifically, and academic productivity and success in the classroom. Additional research is needed to examine the extent to which CET-related improvements in working memory also portend improvements in other functional domains (e.g., peer/social and family functioning, organizational skills, activities of daily living; Abikoff et al., 2013; Irwin et al., 2021; Mikami & Normand, 2015) and cognitive functions (e.g., set shifting; Irwin et al., 2019, 2020) that are impaired in children with ADHD and linked with their working memory difficulties.

Key Points.

Question:

Central executive training (CET) improves the ‘working’ components of working memory and has been shown to improve ADHD-related executive dysfunction and behavioral symptoms, but do these gains extend to the disorder’s well-documented academic difficulties?

Findings:

CET was superior to behavioral parent training (BPT) and/or inhibitory control training (ICT) for improving masked teacher perceptions of academic success and academic productivity as well as objectively-assessed academic achievement in reading comprehension and math problem-solving across two clinical trials.

Importance:

CET yields direct benefits for children with ADHD and demonstrates academic far-transfer benefits relative to both gold-standard behavioral parent training and an active, credible, and adaptive neurocognitive training intervention (ICT).

Next Steps:

Future work is needed to evaluate CET’s potential benefits across a broader array of academic skills as well as other domains of functional impairment associated with ADHD (e.g., peer, family, organizational skills).