A preliminary randomized, controlled trial of executive function training for children with autism spectrum disorder

Abstract

This randomized, controlled study examined the initial efficacy of an executive function (EF) training program for children with autism spectrum disorder (ASD). Seventy 7- to 11-year-olds with ASD and IQ ≥ 80 were randomly assigned to receive a web-based set of EF training games combined with in-person metacognition coaching or to a waitlist. Primary outcomes were evaluated for neural responses related to EF, lab-based EF behavior, and generalization of EF skills. Secondary outcomes included measures of social function. Post-testing and analyses were conducted by staff naïve to group assignment. Children exhibited a change in neural response following training relative to the waitlist group (η_p² = .14). Training effects were not detected via lab-based tasks (η_p²s < .02) or generalize to caregiver-reported EF skills outside the lab (η_p² = .001). However, the training group demonstrated reduced symptoms of repetitive behavior (η_p² = .15) following training. There were no adverse events or attrition from the training group. Findings suggest that brief, targeted computer-based training program accompanied by coaching is feasible and may improve neural responses and repetitive behaviors of school-aged children with ASD.

Lay Abstract

Executive function (EF), which is a set of thinking skills that includes stopping unwanted responses, being flexible, and remembering information needed to solve problems, is a challenge for many children on the autism spectrum. This study tested whether EF could be improved with a computerized EF training program under the guidance of a coach who reinforced the use of EF skills. Seventy children with autism spectrum disorder (ASD) from age 7 to 11-years of age participated in the study. They were randomly assigned to receive training or to a waiting group. The tests most likely to determine whether the training may be effective were chosen from a larger battery before the study started and included one task measuring brain responses, two measures of EF in the lab, and a parent questionnaire. Changes in social functioning and repetitive behaviors were also explored. All children assigned to training completed the program and families generally reported the experience was positive. Brain responses of the training group changed following training, but not within the waiting group during a similar time period. Children who received training did not exhibit behavioral changes during the two the lab-based tasks. Parent report on questionnaires indicated that neither group showed a significant change in their broad use of EF in other settings. Yet, children who received training were reported to have fewer restricted and repetitive behaviors following training. These initial findings suggest that short EF training activities are feasible and may improve some functioning of school-aged children on the autism spectrum.

Autism spectrum disorder (ASD) is associated with lifelong impairments in executive function (EF) (Demetriou et al., 2017; Geurts, de Vries, & van den Bergh, 2014; Kenworthy, Yerys, Anthony, & Wallace, 2008). EF is the ability to manage complex or conflicting information in the service of attaining a goal and encompasses inhibition (i.e., the ability to deliberately suppress a dominant response or competing information), set-shifting (i.e., moving flexibly between tasks or mental representations), and working memory (i.e., holding information in mind and updating it while using it to solve a problem) (Lehto et al., 2003; McAuley & White, 2011; Miyake et al., 2000; Miyake & Friedman, 2012). EF ability relates to the severity of social and repetitive symptoms of ASD (Faja & Nelson Darling, 2019; Geurts et al., 2014)

EF training programs that engage typically developing children (TD) in practice just beyond their current level of competency result in behavioral and neural changes (Diamond, 2013). Many EF training programs are delivered electronically (Jaeggi et al., 2011; Karbach & Kray, 2009; Karbach & Unger, 2014; Rueda et al., 2005; Rueda, Checa, & Cómbita, 2012; Thorell et al., 2009) and have small to moderate effect sizes for school-aged children (Takacs & Kassai, 2019), whereas more comprehensive and longer EF training programs typically result in greater transfer of skills and larger improvements (Diamond, 2013). Adding in-person metacognitive scaffolding during computerized EF training also enhances efficacy (Pozuelos et al., 2019). Finally, some EF training effects appear to generalize to theory of mind (Kloo & Perner, 2003) and social competence (Greenberg, 2006; Riggs et al., 2006).

For children with ASD, EF training has received relatively little research attention. A randomized trial of a curriculum-based EF training program, Unstuck and On Target, compared children in classrooms who received either EF or social skills training. Better problem-solving, flexibility, planning, and EF behavior in the classroom were observed in the EF training group compared to the social skills group (Kenworthy et al., 2014). Although Unstuck and On Target has research support, individual intervention may be more feasible to implement outside educational settings and computer-based approaches are engaging and highly motivating (Bolte et al., 2010; Chen & Bernard-Opitz, 1993; Dichter et al., 2012), less socially demanding for children with ASD, and allow for individualized difficulty and pacing. A randomized trial of self-guided computer-based EF training (i.e., Braingame Brian) for 24 x 45min sessions over 6 weeks included active engagement in one of three conditions: working memory, flexibility, or mock training activities. This program found comparable improvements for groups assigned to training and to an active control condition (de Vries et al., 2015). A pilot investigation with semi-random assignment to a self-guided app-based Neuroracer intervention (Project EVO) involved multi-tasking between a perceptual discrimination attention and memory task and a continuous visuomotor driving task for 100 x 5min sessions over 4 weeks. Project EVO reported moderate-to-large within group effect sizes on measures of Attention Deficit/Hyperactivity Disorder (ADHD) symptoms, EF challenges, and social skills among a group of children with ASD+ADHD (Yerys et al., 2019). These studies suggest EF training may be useful in improving EF and social functioning in ASD and raise the possibility that face-to-face EF training combined with computer-based practice may be beneficial, particularly to support strategies that children with ASD may not spontaneously employ.

The combined approach has yet to be examined in ASD. To address this need, we selected computer games that emphasized set-shifting, inhibition, and spatial working memory with levels of difficulty that incrementally increased. Set-shifting and planning are impaired in ASD, with moderate to large effect sizes reported in meta-analyses (Demetriou et al., 2017; Willcutt et al., 2008). Inhibition is also affected in children with ASD, with comparable performance to groups with ADHD (Craig et al., 2016; Geurts, van den Bergh, & Ruzzano, 2014; Schmitt et al., 2018; Sergeant, Geurts, & Oosterlaan, 2002; Tye et al., 2014). Finally, working memory, particularly in the context of visuospatial tasks, is reduced in ASD (Kenworthy et al., 2008; Habib et al., 2019). We also developed a companion coaching manual to teach metacognition skills.

Outcomes in EF training studies include lab-based behavioral measures, parent report measures, and electrophysiological responses. Neural measures such as electroencephalography complement lab-based behavioral tasks by probing aspects of EF not directly captured by overt behavioral responses, such as response preparation and inhibition (Banaschewski & Brandeis, 2007), and may precede behavioral changes. Lab-based tasks allow for sensitive measurement of initial changes and differences in specific aspects of EF whereas broad surveys capture more global changes in behavior.

The N2 event-related potential (ERP) has been used as a measure of improved EF in several prior training studies (Liu at al., 2017; Millner et al., 2012; Rueda et al., 2012) as it is thought to reflect detection and monitoring of conflicting information (Abundis-Gutiérrez et al., 2014; Heil et al., 2000) or inhibition of competing information (Folstein & Van Petten, 2008; Van’t Ent, 2002). These abilities are closely related to EF (Buss et al., 2011; Miyake et al., 2000; Rueda et al., 2004). N2 amplitude is generally larger (i.e., more negative) for trials with greater conflict (Brydges et al., 2014; Espinet, Anderson, & Zelazo, 2012, 2013) and for younger children who also do not exhibit clear condition effects under age six at typical N2 time windows (Buss et al., 2011; Rueda et al., 2004). For children with ASD, N2 amplitudes are more negative than controls overall and N2 amplitude relates to EF behavior (Faja, Clarkson, & Webb, 2016; Jodo & Kayama, 1992; Pfefferbaum et al., 1985). The extent to which the N2 ERP component changes as a result of EF intervention in children with ASD is unknown.

The Current Study

The objective of the current project is a test of the initial efficacy of a brief computer-based EF training combined with in-person coaching for young children with ASD at the behavioral and neural level. Because of the preliminary nature of the study, children were not selected on the basis of an EF impairment and multiple outcomes were examined. We focused on young, verbal school-aged children because the training games were originally developed for preschoolers and young school-aged children without ASD. EF impairments are detected by this age in ASD (Kenworthy et al., 2008), and successful intervention could impact subsequent academic and social function. Neural and behavioral outcome measures were selected because they were used in investigations of similar training with typically developing children (Pozuelos et al., 2019; Rueda et al., 2012) or other EF interventions for children with ASD (Kenworthy et al., 2014; Yerys et al., 2019).

We report the findings for primary outcomes selected to examine three levels of analysis: (1) neural changes associated with EF (N2 ERP during the Flanker Task); (2) lab-based cognitive changes in EF subdomains emphasized in training (inhibition and flexibility); and (3) generalization of EF skills via a broadband measure of real-world EF. Secondary outcomes explored (1) performance on EF domains not addressed by intervention (verbal working memory, decision-making); (2) social functioning; and (3) a neural measure associated with EF that had lower acquisition rates in piloting (N2 ERP during the Go/Nogo Task). Finally, given the relation between EF and repetitive behaviors, we also explored repetitive symptoms.

We hypothesized that children with ASD who received training would exhibit changes in electrophysiological responses relative to children with ASD randomly assigned to a waitlist control group. Although outcome measures differed from training, we predicted that lab-based behavioral measures that were most closely related to training activities would be most sensitive to initial changes. Given the clinical importance of generalizing newly acquired EF, we also measured parent report of improved EF skills at home. Yet, we predicted that neural changes are likely to precede behavioral changes and generalization beyond the lab; thus behavioral changes would only be detected if neural changes were observed. Finally, we explored untrained domains of EF, social function, and repetitive behaviors with the expectation that generalization would only be observed if changes were detected in primary outcomes. In sum, the battery is designed to detect the presence of a signal from a brief intervention via neural and lab-based measurement and, if detected, begin to explore the potential generalization of skills to other tasks and settings.

Method

Participants

Seventy children (7 girls), aged 7 to 11 years old, diagnosed with ASD participated. Inclusion criteria included age (7-11 years at enrollment), full scale IQ ≥ 80, and a diagnosis of ASD (described below). See Figure 1 for the number of eligible participants at each stage of the study. Sample size was determined for this preliminary efficacy study based on generic effect sizes for planned analyses. Participants were recruited from 2015 to 2017 until the planned sample size was enrolled. Exclusionary criteria included colorblindness, inability to complete procedures in English or due to sensory or motor impairments, medical disorders that impact the central nervous system, prolonged prenatal substance exposure, and a history of seizures or use of seizure medication. Other medications were non-exclusionary and use did not differ by group (Table 1). The study was conducted at Boston Children’s Hospital in the United States and approved by its Human Subjects Division; all parents provided consent and all children provided written assent to participate. The study, including selection of outcome measures and analyses, was pre-registered on ClinicalTrials.gov (NCT02361762).

Figure 1.

CONSORT Diagram.

Table 1.

Participant Characteristics (N = 70)

	Training Group	Waitlist Group	Statistical Test
Age (in years)	M=9.15 (1.38)	M=9.10 (1.34)	t(68) = .18, p = .86
Reported Gender	91% boys	89% boys	χ2(1) = −16, p = .69
Race	0% Asian	3% Asian
	3% Black	9% Black
	88% White	77% White
	9% Biracial	11% Biracial
Ethnicity	11% Hispanic	3% Hispanic	χ2(1) = 1.94, p = .16
Household Income	15% <35K	13% <35K	χ2(4) = 2.11, p = .72
	6% 36-65K	16% 36-65K
	27% 66-100K	29% 66-100K
	26% 101-160K	19% 101-160K
	26% >161K	23% >161K
Primary Caregiver Education	9% High School	6% High School	χ2(3) = 1.60, p = .66
	26% Associate	15% Associate
	28% Some College	30% Some College
	37% Bachelor	49% Bachelor
Stimulant Medication Use	23%	21%	χ2(1) = .05, p = .82
Non-stimulant ADHD Medication	14%	26%	χ2(1)= 1.58, p = .21
Other Medication Use	34%	32%	χ2(1) = .02, p = .87
ADOS-2 Comparison Score	8.89 (1.6)	8.89 (1.2)	t(68) = .00, p = 1.00
Social Affect CS	8.20 (1.6)	8.40 (1.2)	t(68) = −.59, p = .56
Restricted Repetitive CS	9.09 (1.0)	9.06 (1.3)	t(68) = 11, p = .92
ADI-R Social Raw Score	17.29 (4.8)	18.26 (5.2)	t(68) = −.82, p = .42
ADI-R Verbal Communication	15.51 (4.0)	16.63 (4.5)	t(68) = −1.09, p = .28
ADI-R Restricted & Repetitive	7.86 (3.1)	8.43 (2.3)	t(68) = −.87, p = .39
CBCL ADHD Scale T-Score	60.83 (6.67)	63.74 (8.38)	t(67) = −1.60, p = .12
WASI-2 Full Scale IQ	108.43 (13.6)	102.83 (12.2)	t(68) = 1.81, p = .07
Verbal Comprehension Index	106.60 (15.2)	102.40 (12.4)	t(68) = 1.27, p = .21
Perceptual Reasoning Index	108.54 (15.0)	102.74 (13.1)	t(68) = 1.72, p = .09
BRIEF Global Executive	66.31 (12.0)	68.18 (10.2)	t(67) = −.69, p = .49
Metacognition Index	65.03 (12.2)	67.59 (9.8)	t(67) = −.96, p = .34
Behavioral Regulation Index	65.46 (13.2)	67.06 (11.4)	t(67) = −.54, p = .59

Note: ADHD = Attention deficit/hyperactivity disorder; ADOS-2 = Autism Diagnostic Observation Schedule, Second Edition, CS = Comparison score; ADI-R = Autism Diagnostic Interview-Revised; CBCL = Child Behavior Checklist; WASI-2 = Wechsler Abbreviated Scale of Intelligence, Second Edition; BRIEF = Behavior Rating Inventory of Executive Function

Procedure

Parents of potential participants completed phone screening to establish initial eligibility. Diagnostic and cognitive eligibility were assessed at the first visit under the supervision of a licensed psychologist. During two additional baseline visits, neural and behavioral responses to a battery of EF and social cognition tasks were collected while parents completed questionnaires. Then, children were randomized equally to either active EF training or waitlist control (i.e., parallel design, 1:1 allocation ratio). Randomization order (simple) was computer generated by a staff member not involved with visits and assignments were concealed in sequential, sealed envelopes. Children returned for two post-testing visits conducted by staff who were unaware of group status. Post-testing followed the same procedures as baseline. Groups did not differ in the duration between randomization and the first post-testing visit, t(66) = 0.54, p = .59; M_Training = 11.37 weeks (SD = 2.50), M_Waitlist = 11.07 weeks (SD = 1.95). All Training participants returned for post-testing. In the Waitlist Group, one family withdrew at randomization and one family was lost to contact. Training was offered to the Waitlist Group at the conclusion of the trial. No adverse events were reported.

Symptom assessment.

Existing diagnosis of ASD was confirmed according to DSM-5 (American Psychiatric Association, 2013) criteria based on expert clinical judgement, the Autism Diagnostic Interview–Revised (Rutter, LeCouteur, & Lord, 2003 scored according to Sung et al., 2005), and the Autism Diagnostic Observation Schedule, Second edition (Lord et al., 2012). Symptoms of ADHD were assessed via the Child Behavior Checklist (Achenbach & Rescorla, 2001).

Cognitive assessment.

Overall cognitive ability was determined via the Wechsler Abbreviated Scale of Intelligence-2 (Wechsler, 2013).

Intervention

Training involved up to ten hour-long visits with children approximately once a week led by a research assistant or graduate student under the supervision of a clinical psychologist. A 5-minute parent check-in at the end of each session provided the child and coach with an opportunity to share progress and key concepts. Each session aimed to include approximately 10 minutes of play for each of four training games that differed in their task demands from the assessment battery, as well as time for coaching EF strategies. The games (Pozuelos et al., 2019; Rueda et al., 2005, 2012) were developed for preschoolers and young school-aged children and emphasized EF skills related to visual working memory, set-shifting, and inhibition. Children advanced at their own pace and criteria for progress between levels included the number of consecutive correct responses and overall accuracy. A coaching manual was developed to increase metacognitive awareness, provide psychoeducation about EF to children and their families, support emotion regulation during challenging tasks, and foster generalization. It included procedures for introducing the games and the timing and content of each session. (See Supplemental Materials for additional details.)

Fidelity.

Data confirmed that children played all four training games during each session unless they had already completed the highest level of a game. During each session, children spent 30 to 40min playing the training games (M=36.12min, SD=2.92). All children completed EF and emotion regulation psychoeducation at the first and second training session, respectively, and completed an exercise to consolidate their learning at a final training session. All trainers received formal instruction on how to deliver the manualized content and direct supervision for their initial sessions. Ongoing fidelity to key session elements was reviewed and trainers who did not adhere were retrained. Trainers also received ongoing supervision from a licensed psychologist to consult about optimal strategies for responding to challenging behaviors and obstacles to delivering intervention.

Electrophysiologic methods

EEG data recording, editing, and abstraction followed Faja et al, (2016) and are detailed in Supplemental Materials.

Stimuli and experimental procedure.

Primary Neural Outcome: Flanker.

The flanker portion of the Child Attention Network Task (Rueda et al., 2004) was selected as a primary outcome given its sensitivity to EF training effects among children without ASD (Pozuelos et al., 2019; Rueda et al., 2005, 2012) and discrimination of children with ASD from children without (Faja et al, 2016). It included 12 practice and 108 test trials. Each trial began with a 150ms beep paired with a 450ms fixation cross at the center of the screen. Then, a target and flankers were presented for 2000ms. Congruent trials (50%) consisted of a central target animal flanked by two animals on each side with the same orientation and size as the target. Incongruent trials (50%) were identical except that the target and flankers faced opposite directions. Children pressed a button indicating the direction the target animal faced (50% left, 50% right) and received feedback upon responding. The dependent variable was N2 mean amplitude.

Secondary Neural Outcome: Go-Nogo.

The N2 was also examined with a cued Go/Nogo task. After reaching 80% accuracy on at least 20 practice trials, 200 test trials were presented in four blocks. Each trial was preceded by a 500ms fixation cross followed by a 700 ms stimulus presentation. Go trials (70%) consisted of pressing a button each time a letter appeared on the screen. For Nogo trials (30%), responses were withheld when a specific letter appeared on the screen. To equate frequency across conditions, one Go letter appeared for 30% of trials and responses were analyzed only for that Go stimulus. To control for motor responses on the previous trial, only trials following correct Go responses were analyzed.

Included ERP data.

Subjects with fewer than 10 trials per condition were excluded from analyses to optimize inclusion while maintaining an adequate signal-to-noise ratio (Lamm et al., 2006; Rueda et al., 2004; Todd et al., 2008). For children with adequate data at both timepoints (i.e., ≥10 accurate trials per condition without movement artifacts), groups did not differ in the number of trials included, F(1, 44) = 3.33, p = .08, η_p² = .07, M_Training = 61.8% (SD = 11.3), M_Waitlist: 54.4% (SD = 16.2). Fewer children provided Go-Nogo data because it was always presented after the Flanker task and groups did not differ in the number of trials included, F(1, 29) = 0.32, p = .58, η_p² = .01, M_Training = 49.0% (SD = 10.4), M_Waitlist = 46.4% (SD = 15.2).

Behavioral Measures

Primary behavioral outcomes.

Before and after training, two lab-based computer tasks and a broadband parent questionnaire were administered to evaluate changes in EF behavior. Higher scores for all primary outcomes indicate lower EF.

Change task (De Jong, Coles, & Logan, 1995; Oosterlaan & Sergeant, 1998).

Following practice, four test blocks included Go trials (75%) and Change trials (25%) Change trials consisted of a visual signal to stop the dominant task (i.e., left/right button press) and change to the spacebar. To adjust for individual differences in reaction time (RT), each test block used the mean correct RT from the previous block so stop signals occurred equally at 50, 200, 350, and 500ms before each child’s RT. The dependent variable was the stop signal reaction time (SSRT), which estimates the latency required to inhibit a dominant response when a stop signal was presented (Band, Van Der Molen, & Logan, 2003; Crone & van der Molen, 2004). Higher scores indicated slower inhibition and shifting to the change response.

Stroop task (Perlstein et al., 1998; Stroop, 1935).

Following practice, test trials were presented in pseudorandom order for three conditions: (1) congruent (25%) with a color word written in the same color (e.g., blue written in blue); (2) incongruent (25%) with a color word written in a different color (e.g., blue written in red); and (3) neutral (50%) with a non-color word written in one of the four colors (e.g., bear written in blue). Button presses indicated the color of the text. The dependent variable was the difference between percent correct for congruent and incongruent trials. Higher scores indicated lower ability to suppress interfering information.

Behavior Rating Inventory of Executive Function (BRIEF; Gioia et al., 2000).

Caregiver-report of real-world EF was obtained as a measure of generalization. The dependent variable was the Global Executive Composite.

Secondary behavioral outcomes.

Five additional lab-based computer tasks and one parent questionnaire were administered to explore potential transfer of EF skills and changes in social ability. Higher scores for all secondary outcomes indicate better EF and social functioning.

Digit span.

The Numbers subtest of the Children’s Memory Scale (CMS; Cohen, 1997) measured verbal working memory—an untrained EF subdomain. The dependent variable was the backward scaled score.

Hungry donkey (Crone & Van der Molen, 2004).

Hungry Donkey measures decision-making in response to feedback—another untrained EF subdomain. Children fed a cartoon donkey by opening one of four doors with varying rewards and losses for 100 selections with feedback. Two doors were advantageous and resulted in net gains, two were disadvantageous and resulted in net losses. Doors also varied on the frequency of loss (two high, two low). The dependent variable was the ratio of advantageous to disadvantageous selections for the final 40 trials.

Theory-of-mind test (TOM Test; Muris et al., 1999).

The TOM Test measures social cognition via affective theory of mind, first-order false belief, and second order false-belief questions about drawings and vignettes. Reliability among raters was excellent (r = 0.93). The dependent variable was percent correct.

ToM Video Composite.

Two social cognition videos measured first-order false belief about a location change (Saxe, 2009; Wimmer & Perner, 1983) and unexpected contents (Perner et al., 1987). The dependent variable was percent correct.

Social Attribution Task (SAT; Klin, 2000).

Animated geometric figures (Heider & Simmel, 1944) were presented following the instructions and coding scheme used by Klin (2000). The animation is frequently understood as a social interaction and the task measures the degree to which the information is interpreted as social. The dependent variable was the problem-solving index (inter-rater reliability, r = 0.98), which measured the number of correct responses to explicit questions.

Exploratory behavioral outcome.

Repetitive Behavior Scale-Revised (RBS-R; Lam & Aman, 2007).

Caregiver-report was collected to explore generalization to restricted and repetitive symptoms. The dependent variable was the total score.

Data Analysis

Feasibility and acceptability were evaluated by examining the number of completed training visits and parent feedback. Efficacy data analyses were conducted without knowledge of group assignment and confirmed by an independent statistician. As specified in the a priori analysis plan, neural responses were examined via repeated measures ANOVAs because the group by condition by timepoint interaction was of primary interest. Planned behavioral analyses included examination of differences in baseline behavior via ANOVA and examination of treatment responses via ANCOVA-of-change analyses controlling for baseline. Missing cases were excluded in a pairwise fashion so that for each dependent variable all available participants who contributed data were included. Given the preliminary nature of this investigation, sample size, and measures of multiple levels of analysis, we did not correct for multiple comparisons.

Community Involvement

ASD community members were not directly involved in the development of the research question and outcome measures, design of the study, its implementation, or the interpretation and dissemination of the findings.

Results

Feasibility and acceptability

All families assigned to the training group completed training and returned for follow up; 89% completed all 10 planned training sessions and all families completed at least 7 sessions. Twenty-three families returned the feedback questionnaire of whom 74% (n=17) reported at least some improvement for their children and 83% (n=19) felt it improved their knowledge of EF or their ability to help their children develop EF. Additional qualitative responses are included in the Supplemental Materials.

Neural responses

Primary neural outcome.

Across both time points including baseline the groups had significant differences in overall Flanker N2 amplitude, F(1,44) = 8.45, p = .006, η_p= .16 (Training: M = −0.56, SD = 3.22; Waitlist: M = −3.32, SD = 3.22). Critically, the group (treatment/waitlist) by time (baseline/follow-up) by condition (congruent/ incongruent) interaction was significant, F(1,44) = 7.37, p = .009, η_p² = .14 (Figure 3)¹. No other main effects (condition) or interactions (condition x time, group x time) were significant, Fs < 2.70, ps > 0.10, η_p² < .06.

Figure 3.

(a) ERP Waveforms for frontal electrode cluster (Fz) by time point and Flanker Task condition and (b) Time x flanker condition interaction for N2 mean amplitude

In order to interpret the interaction between group x time x condition, groups were examined separately. The Training group exhibited a significant effect of time, F(1,23) = 9.32, p = .006, η_p² = 0.29, and the change in amplitude over time differed by condition (time x condition), F(1,23) = 5.09, p = .034, η_p² = 0.18. Contrasts to compare changes for each condition indicated a significant increase in incongruent amplitude (i.e., more negative), F(1,23) = 14.58, p = .001, η_p² = 0.39, but not congruent amplitude (p = .23), indicative of normalized differentiation between conditions. The Waitlist group did not exhibit changes overall or by condition, Fs < 2.77, ps > .11.

Secondary neural outcome.

The group by time by condition (go/nogo) interaction was nonsignificant despite a medium effect size, F(1,29) = 2.92, p = .098, η_p² = .09.

Primary behavioral outcomes

At baseline, on average, both groups had EF challenges in the clinical range (BRIEF; Table 1). No group differences were detected in lab-based tasks or parent report of EF (see Table 2 and Figure 2 for all results).

Table 2.

Scores Before and After Training (N = 68)

		Baseline		Post Testing
	Sample	Training	Waitlist	Training	Waitlist	Baseline	Post Testing
Primary Outcomes	NT, NWL	M (SD)	M (SD)	M (SD)	M (SD)	ANOVA, ηp2/d	ANCOVA, ηp2/d
Change SSRT	31, 30	233.8 (87.8)	238.3 (94.3)	213.6 (78.5)	234.7 (98.1)	F(1,59) = .04, 0.001/−0.05	F(1,58) = .86, 0.02/−0.24
Stroop Task Cong-Inc	34, 29	0.038 (0.08)	0.060 (0.12)	0.029 (0.13)	0.038 (0.09)	F(1,61) = .79, 0.013/−0.22	F(1,60) = .07, 0.001/−0.08
BRIEF GEC	33, 30	66.55 (11.51)	68.40 (10.19)	65.85 (9.96)	67.76 (10.42)	F(1,61) = .52, 0.008/−0.17	F(1,60) = .07, 0.001/−0.19
Flanker N2 Con. Mean Amp	24, 22	−0.198 (3.22)	−2.495 (3.26)	−1.050 (3.62)	−3.737 (4.09)
Flanker N2 Inc. Mean Amp		0.825 (3.32)	−3.771 (4.84)	−1.798 (3.68)	−3.285 (4.49)
Secondary Outcomes
Digit Span	33, 30	10.42 (3.45)	9.83 (3.55)	10.15 (3.17)	9.55 (3.55)	F(1,61) = 0.45, 0.007/0.17	F(1,60) = .27, 0.004/0.18
Hungry Donkey	33, 31	1.39 (10.85)	0.16 (10.28)	0.33 (11.67)	1.74 (11.91)	F(1,62) = 0.22, 0.003/0.12	F(1,61) = .25, 0.004/−0.12
SSIS	32, 30	81.63 (12.48)	80.47 (15.66)	81.69 (10.55)	79.33 (14.17)	F(1,60) = 0.10, 0.002/0.08	F(1,59) = .80, 0.01/0.19
TOM Task	34, 33	66.33 (14.34)	62.52 (12.72)	70.98 (12.32)	65.07 (10.49)	F(1,65) = 1.32, 0.02/0.28	F(1,64) = 3.13, 0.05/0.52†
Theory of Mind Composite	31, 29	74.93 (25.49)	70.94 (28.94)	89.52 (20.18)	84.48 (26.23)	F(1,58) = 0.33, 0.006/0.15	F(1,57) = 0.38, 0.007/0.22
Social Attribution Task	24, 21	34.58 (16.41)	33.81 (16.88)	32.92 (16.28)	30.95 (15.13)	F(1,43) = 0.02, 0.001/0.05	F(1,42) = 0.17, 0.004/0.13
Go N2 Mean Amp	16, 15	−2.409 (2.82)	−2.897 (4.06)	−1.090 (2.14)	−2.309 (2.99)
NoGo N2 Mean Amp		−1.572 (3.44)	−3.028 (4.29)	−2.780 (3.57)	−2.525 (4.51)
Exploratory Outcome
RBS-R Total Score	31, 29	16.61 (10.75)	24.14 (14.06)	13.61 (9.04)	27.03 (17.90)	F(1,58) = 5.47, 0.09/−0.60*		F(1,57) = 10.24, 0.15/−0.96**

Note: BRIEF GEC = Behavior Rating Inventory of Executive Function Global Executive Composite; Con = congruent; Inc = incongruent; Amp = amplitude; SSIS = Social Skills Improvement System rating scales; TOM = Theory of Mind; RBS-R = Repetitive Behavior Scale-Revised. η_p² = partial eta squared effect size indicator; d = Cohen’s d effect size estimate.

^†p < .1

^*p < .05

^**p < .01

Figure 2.

Differences between baseline and post training. Scores were computed by calculating the z-scores relative to the combined group means at baseline and are presented so that positive values reflect better functioning following training

Following training, no differences were detected between groups for the lab-based tasks (Change, Stroop) or generalization of EF skills beyond the lab by parent report on the BRIEF Global Executive Composite.

Secondary behavioral outcomes

At baseline, no group differences were detected for secondary outcome measures. After training, no changes were detected for verbal working memory, decision making, social cognition, or social function.

Exploratory behavioral outcome

At baseline, groups differed in the severity of repetitive behaviors, F(1,58) = 5.47, p = .02, η_p² = .09. Critically, when controlling for baseline, the Training group exhibited lower levels of repetitive behavior following training, F(1,57) = 10.24, p = .002, η_p² = .15.

Discussion

This study examined the initial efficacy of a computer-based EF training augmented by in-person metacognition coaching for children with ASD. We hypothesized that a signal would be most likely detected via neural measures of EF and challenging lab-based tasks that most directly related to training (Diamond, 2013). The training group demonstrated significantly increased neural differentiation of incongruent flankers at post-testing, whereas the waitlist group did not. Neural differentiation develops by age six among children without ASD (Buss et al., 2011), suggesting training may lead to more age-appropriate neural responses and enhanced monitoring of conflicting information to support effective EF. Though non-significant, a similar pattern was observed for the Go-Nogo task (i.e., more negative N2 amplitude for Nogo relative to Go trials after training compared to the Waitlist group).

Given these findings, we then examined behavioral outcomes for lab-based measures of related EF subdomains, generalization, and transfer of skills. Lab-based tasks examining response inhibition combined with shifting ability and interference suppression were not sensitive to training. Even with more comprehensive and intensive training, results on lab-based tasks have been mixed for children with ASD. Specifically, Yerys et al. (2019) reported a nonsignificant within group change on a related lab-based task with a medium-large effect size. De Vries et al. (2015) reported a non-significant trend for working memory training on one of two working memory tasks and no effects related to flexibility for either EF training group. Kenworthy et al. (2014) found significant, medium effects on their challenge task on the flexibility domain but not on the planning portion. Unsurprisingly, we found no evidence of significant transfer to social cognition or function or to untrained EF subdomains (verbal working memory, decision-making), which was consistent with other reports that reported non-significant transfer in lab-based social (Kenworthy et al., 2014) or untrained EF tasks (de Vries et al., 2015). Generalization to affect regulation was observed among children without ASD who completed a similar EF training program (Pozuelos et al., 2019; Rueda et al., 2005, 2012) suggesting that this type of training may confer a greater benefit to children without neurodevelopmental disorders. Nonetheless, our findings are consistent with a meta-analysis of EF training programs that indicates most have limited transfer (Kassai et al., 2019).

Following the suggestion that EF training may be helpful in ameliorating restricted and repetitive behavioral symptoms for children with ASD (Kenworthy et al., 2014), we explored this domain and found the training group had reduced parent-reported symptoms when controlling for baseline levels. In contrast, parent report of general real-world EF skills did not change following training, although some parents reported specific training-related changes. Generalization and the overall lack of behavioral EF changes are a concern for children with ASD (Ramdoss et al., 2012). It is possible that children with ASD may require more than 10 hours of intervention in order to demonstrate the improvements made by children without ASD in response to a similar EF training program, which included both N2 changes and behavioral improvements in inhibitory control (Pozuelos et al., 2019; Rueda et al., 2005, 2012). Yet, the computer training with greatest intensity for children with ASD (18.75hours) yielded no significant treatment related effects on real-world EFs, social behavior, or quality of life, despite a trend for slightly improved ADHD-behavior in the working memory training group (De Vries et al , 2015). It may also be the case that subject selection limited our ability to detect more generalized changes. Project EVO selected children with ASD and co-occurring ADHD and reported significant reductions in ADHD symptoms, EF challenges, and social difficulties with large effect sizes after only 8.3hours of training. Computer-based EF training is generally more limited in its benefits for typically developing children than comprehensive, curriculum-based training (Diamond, 2013; Kassai et al., 2019) such that greater intensity of computer-based training may not provide the same benefit of curriculum-based training. Indeed, curriculum-based training resulted in changes on parent and teacher report of set-shifting and planning (Kenworthy et al., 2014) for children with ASD; however, the scalability of such comprehensive interventions is difficult. Computer-based EF training is appealing for service delivery across settings and to individual children. Thus, given an initial “signal” on the primary neural outcome measure, it will be critical to determine whether a larger dose of computer-based EF training or selection of a more impaired group leads to more robust behavioral changes.

Our training differed from previously published computer-delivered EF training for ASD (De Vries et al., 2015; Yerys et al., 2019) because it included coaching, its content was broader (inhibition, set-shifting working, memory, and metacognition), it was relatively less intense, and its subject selection, although our program was similar to these programs in that the games were adaptive and adjusted difficulty according to child performance. Given that EF training is thought to have its greatest impact when task demands exceed a child’s current abilities (Holmes, Gathercole, & Dunning, 2009; Karbach, Strobach, & Schubert, 2015), coaches encouraged children with ASD to continue to play and provided emotional regulation and EF strategies. Thus, computer-based training combined with coaching may confer some of the benefits of a more comprehensive EF curriculum in the context of an inexpensive, individualized format. Prior to the implementation of such programs, it will be critical to determine which aspects of EF training promote generalization from initial neural changes to clinically significant effects such as the reduction of restricted and repetitive behaviors and improved EF behavior.

Limitations and Future Directions

The current investigation demonstrated the initial neural effects of computer-based EF training for ASD but raises additional questions for future research. First, although a variety of computer-based interventions have been used to enhance the EF of typically developing children (Jaeggi et al., 2011; Karbach & Kray, 2009; Karbach & Unger, 2014; Rueda et al., 2005, 2012; Thorell et al., 2009), children with larger initial EF impairments tend to have the largest gains (Diamond, 2013). Likewise, Project EVO reported moderate-large effect sizes within a training group comprised of children with ASD+ADHD (Yerys et al., 2019). The current investigation did not specifically select children with ASD who had initial EF impairments. If targeted interventions such as EF training are to have their greatest benefit, it will be critical to determine which children with ASD are most likely to benefit and respond to training including samples with more diverse backgrounds.

Second, a brief 10 session duration was selected for the initial examination of efficacy based on prior reports of similar training with children without ASD (Pozuelos et al., 2019) while also balancing the demand on child and family time. The intensity of training-either more regular training sessions (Yerys et al., 2019) or more hours of intervention (Kenworthy et al., 2014)–may be especially critical for children with ASD. Embedding comprehensive interventions like Unstuck and On Target in the classroom likely increases the opportunities to practice new EF skills—further increasing the intensity. Future systematic studies are needed to determine the optimal intensity.

Third, it will be important to determine whether training generalizes to more clinically significant changes. It is possible that, as predicted, changes in brain responses may precede or lay the foundation for additional behavioral changes (e.g., Chen, Tsao, & Liu, 2016; McDermott et al., 2018; Tremblay, Kraus, & McGee, 1998) that could be detected via longer-term follow-up. Indeed, conflict monitoring, detected via the N2, may underlie successful inhibitory control, particularly earlier in development (Richardson et al., 2018). However, this study did not include follow up beyond immediate post-testing. Additionally, given our sample size, power estimates indicate that only large behavioral effects could be detected. Replication of initial findings with a larger sample combined with long-term follow up will allow for examination of these possibilities and more rigorous analyses.

Finally, although initial data demonstrate that our training program may be feasibly conducted with fidelity to the delivery of computer games and basic elements of the manualized intervention and that training is acceptable to families, it will be useful to elicit input from other key stakeholders to ensure the implementation and dissemination of programs like ours. Future work that elicits formal acceptability data from children with ASD who receive the training and the input of other stakeholders including community providers and autistic self-advocates will be valuable in refining EF training to best meet the needs of children on the spectrum.

In summary, this study demonstrated that 10 hours of targeted EF training delivered to 7- to 11-year-old children on the autism spectrum via computer combined with coaching led to changes in neural response and parent report of restricted and repetitive behaviors. This work adds to a previous clinical trial of curriculum-based EF intervention and an app for children with ASD+ADHD to show that the significant EF difficulties experienced by many children with ASD may be reduced via intervention. For the substantial subgroup of children with ASD who experience EF difficulties, this represents an important development in identifying more individualized intervention because EF is related to the development of social competence and the expression of ASD symptoms (Faja & Nelson Darling, 2019; Geurts et al., 2014; Pellicano, 2013).

Appendix

Appendix A.

CONSORT Table

Section/Topic	Item No	Checklist item	Reported On page No
Title and abstract
	1a	Identification as a randomized trial in the title	1
	1b	Structured summary of trial design, methods, results and conclusions	2
Introduction
Background and objectives	2a	Scientific background and explanation of rationale	3-6
	2b	Specific objectives or hypotheses	6-7
Methods
Trial design	3a	Description of trial design (such as parallel, factorial) including allocation ratio	8
	3b	Important changes to methods after trial commencement (such as eligibility criteria), with reasons	NA
Participants	4a	Eligibility criteria for participants	7
	4b	Settings and locations where the data were collected	8
Interventions	5	The interventions for each group with sufficient details to allow replication, including how and when they were actually administered	9, Sup.Mat.
Outcomes	6a	Completely defined pre-specified primary and secondary outcome measures, including how and when they were assessed	10-13
	6b	Any changes to trial outcomes after the trial commenced, with reasons	NA
Sample size	7a	How sample size was determined	7
	7b	When applicable, explanation of any interim analyses and stopping guidelines	NA
Randomization Sequence generation	8a	Method used to generate the random allocation sequence	8
	8b	Type of randomization; details of any restriction (such as blocking and block size)	8
Allocation concealment mechanism	9	Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned	8
Implementation	10	Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions	8
Blinding	11a	If done, who was blinded after assignment to interventions (for example, participants, care providers, those CONSORT 2010 checklist Page 2 assessing outcomes) and how	8, 14
	11b	If relevant, description of the similarity of interventions	NA
Statistical methods	12a	Statistical methods used to compare groups for primary and secondary outcomes	14
	12b	Methods for additional analyses, such as subgroup analyses and adjusted analyses	14
Results
Participant flow	13a	For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome	Fig 1,24
	13b	For each group, losses and exclusions after randomization, together with reasons	NA
Recruitment	14a	Dates defining the periods of recruitment and follow-up	7
	14b	Why the trial ended or was stopped	7
Baseline data	15	A table showing baseline demographic and clinical characteristics for each group	23
Numbers analysed	16	For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups	Fig 1, 24
Outcomes and estimation	17a	For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval)	Fig 2, 24
	17b	For binary outcomes, presentation of both absolute and relative effect sizes	24
Ancillary analyses	18	Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory	15
Harms	19	All important harms or unintended effects in each group	8
Discussion
Limitations	20	Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses	19-20
Generalizability	21	Generalizability (external validity, applicability) of the trial findings	19
Interpretation	22	Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence	16-19
Other information
Registration	23	Registration number and name of trial registry	8
Protocol	24	Where the full trial protocol can be accessed, if available	8
Funding	25	Sources of funding and other support (such as supply of drugs), role of funders	Au. Note

Description of the training program

Each session was approximately 1 hour with a target of 40 minutes of computer game play and approximately 20 minutes of coaching. Each session included a brief, 5-minute check-in with a parent or caregiver. The first two sessions began with formal psychoeducation that introduced key concepts about EF and emotion regulation. Scripted content was used to refer to key concepts throughout subsequent sessions. Given that the learning style of many children on the spectrum favors visually presented information, key concepts were presented with consistent language accompanied by visual information. This information was presented on reminder cards that could be referenced throughout the session for prompts, to foster discussion, or as a teaching tool. Parents also received a written handout that provided an overview of key concepts and the training procedures. Finally, some sessions had specific activities (e.g., homework, teaching a family member) to ensure meaningful engagement and promote generalization.

Sample Session Outline:“Intermediate” Training Sessions (From Session 4 to Level 12 of any Game)

WITH CHILD ALONE (50 MINUTES)

1. Review session format: 5 minutes of chat/review 50 minutes of training games with two breaks 5 minutes of debriefing

2. Continue to build rapport with child.

3. (If needed) Revisit psychoeducation reminders. Use visual cue cards.

4. Revise the plan. Now that the child is advancing in levels, ask if there are any specific goals that he/she has for completing certain levels in this session. Review the backup plan concept. Using the written plan from last session as a prompt, ask him/her how well the backup plan worked. Briefly mention any issues from the last session, if applicable, and prompt discussion about why the backup plan didn’t work. Guide child into formulating a new backup plan if necessary.

5. (If needed) Revisit games instructions/reminders.

6. Play training games for approximately 10 minutes each, with two breaks at the child’s choosing. When appropriate, ask child meta-cognitive questions throughout the training games. More scaffolding may be necessary to elicit responses from children (e.g., multiple choice, yes/no questions, asking “what other special skills did you use?”)

WITH PARENTS AND CHILD TOGETHER (5 MINUTES)

Come together to debrief following training session. Highlight child’s successes to parent, and briefly mention what will be targeted for improvement in upcoming sessions. Confirm next session with parent. Thank family for their participation and time. Allow the child to pick a prize.

Description of Computer Games.

The training consisted of four games: Pirates, Oceans, Robots, and Windows. Each game required multiple aspects of executive function. Across the four tasks, the difficulty of levels advanced by increasing the item set or simultaneous dimensions, decreasing the available response time, requiring greater accuracy, adjusting the proportion of distractors or their complexity, and requiring consecutive runs with high accuracy. The parameters that contributed to the difficulty of each level and passing criteria were fixed; however, children progressed from level to level at their own pace. That is, they could re-play games at a lower level throughout the training program if unable to progress beyond that level of difficulty or they could advance to more challenging levels by unlocking them as they successfully achieved the performance criteria of each level. If children did not meet criteria to advance to the next level after eight consecutive failed attempts, the games also returned to a lower level for additional practice before unlocking higher levels. The intensity (i.e., number of sessions and training activities) was determined from Rueda’s prior work with typically developing children and via piloting with children on the autism spectrum.

The training has been developed and used successfully with typically developing preschoolers (as young as 4 years) and early school age children (Pozuelos et al., 2018; Rueda et al., 2005; Rueda et al. 2012 and personal communication with M. Rosario Rueda (2013, 2014) who provided the games for this study and provided input on task selection and calibration). Thus, the games were thought to be appropriate for developmental levels lower than those included and the current study and the training activities were scalable to challenge children who had much higher developmental levels. Our data suggest that the tasks were not too easy for our sample. The number of children who were able to successfully complete all levels for one game or two games was low. Of the 35 children who received the intervention, 16 children completed all levels of at least one (of four) training game, and of those, only five children completed more than two training games.

The Pirate Game was Stroop-like in the sense that it required children to attend to one dimension of information – the quantity of coins contained in each of the pirate’s bags – and inhibit information from a second dimension – the size of the bag. During each trial, children were required to select the more valuable of two bags of coins by clicking on it with a computer mouse. Trials were timed so that children were required to complete their response within a time limit. In addition to inhibition, the Pirate Game also required set-shifting because pirates are sneaky and have fake coins in addition to real ones. Real coins were gold, whereas fake coins were silver. When children encountered gold coins they needed to select the bag with the largest quantity of coins and when they encountered silver coins they needed to select the bag with the smallest quantity of coins. The Pirate Game had 18 levels.

The Oceans Game presented children with a set of sea creatures at the top of the screen. On the bottom of the screen were a different array of creatures that comprised the possible response options. The goal was to either select the creature at the bottom that exactly matched one of the creatures at the top (i.e., same animal and same color as one of animals above) or was the most different (i.e., a different animal and different color than any of the animals above). Selections were made by clicking on the animal with a computer mouse. As such, the game required set-shifting to alternate between the two sorting rules and sorting dimensions. Additionally, because the game was timed, as the number of animals increased the visual working memory load also significantly increased. The Oceans Game included 21 levels.

The Robots Game was a Go/Nogo task wherein children quickly “fed” robots metal nuts if the shape of the nut matched the shape of the robot (i.e., “Go”). They fed the robot the nut by clicking on it with their computer mouse. However, they were not to feed the robot nuts that were a different shape than the robot (i.e., “Nogo”). Children also used set-shifting between two rules during the game because some nuts were rusty and were not to be fed to the robot even when their shapes matched. The Robots Game included 20 levels.

The Windows Game was an n-back task that involved remembering the previous locations of windows that opened and closed in a sequence. Spans ranged from 1-back to 3-back as the task difficulty progressively increased. In addition to the sequence of locations, the task also included sequences with differing colors inside the window for which the sequence of colors must be remembered. These sequences combined both location and color requiring set-shifting. When the location and/or color of the n-back window matched the current window, children indicated the match by clicking the computer mouse. The Windows Game included 21 levels.

Manualized Metacognition Training.

Metacognition Coaching was provided following a manual developed specifically for this study. Consistent with Wood, Bruner & Ross (1976), the metacognition coaching manual provided coaching strategies that: meaningfully engaged children with the tasks, simplified the tasks to make them manageable, supported sustained effort on the tasks, emphasized key EF skills needed for each task, provided coping strategies and sought to reduce frustration associated with challenging aspects of the tasks, and used guided conversations to aid children with solving each task. In order to support the needs of children on the autism spectrum, key concepts were described using consistent language throughout training sessions (e.g., inhibition was described as “stopping myself”) and visual supports were used (i.e., a stop sign).

Metacognitive strategy	Example
Meaningful engagement with the task	Forming a Plan A and Plan B (demonstrating and modeling an approach to the games that is deliberate and draws on metacognitive strategies). Plan B specifically models being flexible and having a backup plan. Examples of some Plan Bs are: Try a different order, skip around after playing 5 minutes of each game, etc. Use of homework sheets with four prompts: I stopped myself when…. I was flexible when…. I made a plan for…. I remembered the details when….. Teach a Family Member Activity – as the child becomes an EF “expert” they lead a family member in a discussion of the EF psychoeducation materials. Trainers ask: • “Who would you like to teach today? ” • “Can you tell__what kinds of things you need your problem solving skills for? ” • “Are there special skills that help you solve tricky problems? ” (Prompt with: Are you forgetting any?) • “Can you tell __ about one of the games and how you use these skills during the game? ” • “Why don’t you tell __ about some things you can do when you get frustrated. ”
Simplification of tasks to make them manageable	Introduction of the tasks to highlight key strategies: In this game you have to pick the bags that have the most treasure inside. Pay close attention to the rules. When you see GOLD coins, pick the bag with the BIGGER number. Pirates are sneaky, so they may put a bigger number of coins in a tiny bag. Don ’t let them trick you!
Support of sustained effort on the task	Normalization that sustained effort is challenging but will lead to positive change: Our brains are kind of like muscles that get stronger when they get exercise.
Emphasis of key EF skills needed	Basic psychoeducation at the beginning: One special skill is being able to stop ourselves from doing or saying things that we don’t want to do. If we have a good plan, we don’t want to do things that will mess up the plan. Like, when we are trying to clear a tricky level in a video game, we don’t want to push all the buttons at the same time. (Can modify examples to be relevant to child’s interests)
Provision of coping strategies to reduce frustration associated with challenging aspects of the task	Psychoeducation and use of strategies from cognitive behavioral therapy (e.g., awareness of feelings) and guided relaxation: Sometimes it’s hard to solve tricky problems and accomplish our goals when we are feeling emotional. Let’s use this meter as a tool to help keep track of where our emotions are so we know if we need to take a break to calm down!
Guided conversations to aid children with solving each task	Use of meta-cognitive questions throughout the training games: “Why do you think you missed that one?” “What can you do differently next time?” “What strategy are you using to beat this level?” “How does your strategy help you?” “Why was this level easier/harder for you?” Try to get the child thinking in terms of the 3 special skills (inhibition, set-shifting, working memory).

To ensure meaningful engagement with tasks and generalization of metacognitive concepts to other settings, children completed homework in the later sessions (after reaching level 12 of any game). The homework sheet included 4 prompts and children demonstrated their understanding of key concepts. For example, one child described “stopping myself” from hitting a peer when upset and another noted “I was flexible when” I wanted to stay home and play video games but had to go out to eat with my family instead. Children reported making plans and “remembering the details” when completing school work and during social interactions with family and peers.

The Metacognition coaching manual is available from the first author.

Details of electrophysiologic methods

Data acquisition.

Neural responses were continuously recorded via a Net Amps 400 (Electrical Geodesics, Inc.) using the 128-channel hydrocel sensor net 2.0 (HSN). Impedances were below 50 kΩ at the start of the session. EEG signals were recorded online using the vertex reference electrode with a 4 KHz antialiasing hardware filter and a sampling rate of 500 Hz. Data were re-filtered off-line using a 0.1 Hz high-pass and 30 Hz low- Kaiser-type FIR filter with 2 Hz rolloff.

Data editing and extraction.

For both tasks, EEG data were segmented with a 200ms baseline period preceding stimulus onset and 800ms after the stimulus onset. Baseline correction used the 200ms baseline period. Trials with incorrect behavioral responses or artifacts were excluded from the averages using the following criteria: (1) presence of an eye blink using the Netstation Eye Blink algorithm set at 220 μV with an 80ms moving average and confirmed by visual inspection, (2) more than 10 channels with fluctuations exceeding 140 μV or less than 1 μV with an 80ms moving average. Data were visually inspected for additional artifacts by a team member who was naïve to treatment assignment and segments were excluded if they contained significant drift, movement artifacts, eye movements, or mechanical artifacts. Channels marked with artifact for >20% of the trials were interpolated using spherical spline interpolation. Data were averaged for each condition, re-referenced to the average of all electrodes minus the four eye channels using the polar average reference effect correction (Junghöfer et al., 1999), and baseline corrected again. N2 amplitude was extracted between 300-400ms over the frontal midline (Fz cluster: HSN electrodes 19, 11, 4; Faja et al., 2016; Lamm, Zelazo, & Lewis, 2006; Samyn et al., 2014).

Results

Fidelity.

Data confirmed that children played all four training games during each session unless they had completed the highest level of a game during a previous training session. Of the 35 children who received the intervention, 16 children completed all levels of at least one training game, and of those, five children completed more than two training games. During each training session, children spent 30 to 40 minutes playing the training games (M=36.12min, SD=2.92). The total number of minutes spent on computer tasks ranged from 209-403 (M=354.59min, SD=39.63). In addition, all children completed both the EF and basic emotional regulation psychoeducation modules at the first and second training session, respectively. These modules were reviewed throughout the remaining visits. Participants set goals (M=7.9/9.8 opportunities, SD=1.63, range=3-10), generated a “Plan A” goals (M=9.3/9.8 opportunities, SD=1.17, range=6-10), and “Plan B” (M=8.4/9.8 opportunities, SD=1.86, range=4-10) for each session (e.g., the order they would play the games), and worked to generalize EF skills by identifying situations in which they used their EF skills at home and at school. At a final training session, all children completed an exercise to consolidate their learning by presenting the EF psychoeducation module to a family member and teaching that person how to play the games. All trainers received formal instruction on how to deliver the content of the training manual and direct supervision for their initial sessions. Ongoing fidelity data were reviewed and trainers who did not adhere to manualized procedures were retrained. Additionally, all trainers received ongoing supervision from a licensed psychologist to consult about optimal strategies for responding to challenging behaviors and obstacles to delivering intervention.

Feasibility and Acceptability.

Prior to beginning the study, the training (including metacognition coaching) had been used with typically developing children from preschool to school age (Pozuelos et al., 2019; Rueda et al., 2005, 2012). Additionally, the training was piloted in a small group of children on the autism spectrum. This piloting led to adaptations including development of the coaching manual, selection of tasks and task difficulty, and the structure and maximum duration of sessions.

Of the 35 families randomized to training, all completed training and returned for follow-up demonstrating that the procedures were feasible. Additionally, within training, there were 10 planned sessions and 31 families completed all 10 sessions (89%). Two families completed 9 sessions, one completed 8, and one completed 7 sessions. Children played all four training games during each session unless they had already completed the highest level of a game; training was discontinued early in some cases when children reached the final level of all games.

To determine whether training was acceptable, family feedback was systematically elicited at the end of each training session about child behaviour during the session. These responses were monitored and children or families who reported difficulty with the training program or sessions were reviewed with a licensed psychologist who oversaw the trainers so that adaptations could be made including additional coaching surrounding emotion regulation, additional structure during sessions, or supports to decrease technical challenges associated with accessing the training games via the internet.

At the final session of the training program, parents/caregivers were asked to provide written responses to three open ended questions about the training program aimed to gather information about possible changes and benefits:

1. Did you notice any changes in your child that you think may be related to our training program? If yes, what were they?

2. If not already described above, do you think there have been any improvements in your child’s executive function since starting training?

3. Did participating in the study improve your understanding of executive function or strategies for helping your child develop executive functioning skills?

Twenty-three families returned the responses to these questions. Of these families, 14 (61%) described a clear and positive change related to training, 3 (13%) reported possible positive changes related to training, 5 (22%) reported no changes, and 1 (4%) reported a negative change. Examples of positive changes included, “Yes, he is more flexible and seems more focused and stays on task and he doesn't get as upset if he is not 100% perfect doing something” and “Yes, he is trying to do difficult tasks by talking them out step by step to himself.” Possible changes included, “Perhaps. His transition to 5th grade has been surprisingly easy thus far” and “He was in a good mood.” Several parents reported no clear child-related changes, “I didn't notice anything specific, but the worksheets gave me more language to use to talk to him about his day” and “No, but he was more aware after an ‘episode’ of what he wants.” One family reported that they did not observe changes, but had already engaged in an extensive executive function program elsewhere. Finally, one family indicated that the additional hour of training each week on top of an already busy schedule caused stress saying, “the only change that can be directly related is that by having an activity every afternoon Monday-Thursday is he was overloaded.”

With regard to specific changes related to executive functioning, seven families reported increased flexibility after training (e.g., “she seems a little more flexible”), three commented on increased inhibition (e.g., “her impulse control seems better”), and one on working memory (e.g., “I find myself not having to repeat things over and over. He gets it! He has a great memory!”). Three families reported increased planning or organization (e.g., “Yes, he seems to keep track of his assignments and things a lot better.”) and eight families reported noticing increased metacognition (“He is more aware of his need to be flexible, stop, etc.”). One family described overall improvements in executive functioning (i.e., “I think he improved overall.”), and five noted improved mood or emotion regulation skills (e.g. “He doesn't get as upset if he is not 100% perfect doing something.”). Nineteen (83%) of parents/caregivers also reported increased knowledge of EF or strategies for helping their children develop EF. Two of the four families who reported not learning about EF from the training reported that they were already very familiar with the concepts. For example, one said, “No, we are well-versed. However, (I was) impressed with games and skills my child was able to use to complete them.” The remaining two parents described either not being engaged with the sessions or struggling with EF in their own functioning. Of the parents who described benefits to themselves, some described using metacognitive language such as, “I have used some new strategies like talking through things so he can see how I got from point A to point B,” while others gained insight about their children, “I understand more about how to improve his understanding, how his mind works.” These responses demonstrate that the training was generally viewed as acceptable and beneficial to the majority of families who participated.

Globally, after the final post-testing session, all families (training, waitlist groups) were asked whether they would recommend the study to other families in general. Thirty-two of the 35 training families responded to this item. All 32 (100%) of the responses were positive. In an open-ended question about positive experiences in the study (i.e., “What did you or your child like about the study?”) 16 of the 32 responders spontaneously noted that their children enjoyed the games or the training sessions. Some examples included, “My son liked playing the games,” “He really liked earning money, the snacks were good. He seemed to enjoy the games as well,” and “My child liked the games. He had lots of fun, didn't want it to end. I love that he was enjoying himself and also that I have more insight on how he is learning new things.”