Do Children with Attention-Deficit/Hyperactivity Disorder (ADHD) Have Set Shifting Deficits?

Abstract

Objective:

Set shifting, or cognitive flexibility, is a core executive function involving the ability to quickly and efficiently shift back-and-forth between mental sets. Meta-analysis suggests medium-magnitude shifting impairments in ADHD. However, this conclusion may be premature because the evidence-base relies exclusively on tasks that have been criticized for poor construct validity and may better reflect general neuropsychological functioning rather than shifting specifically.

Method:

A well-characterized sample of 77 children ages 8-13 (M=10.46, SD=1.54; 32 girls; 66% Caucasian/Non-Hispanic) with ADHD (n=43) and without ADHD (n=34) completed the criterion global-local set shifting task and two counterbalanced control tasks that were identical in all aspects except the key processes.

Results:

The experimental manipulation was successful at evoking set shifting demands during the global-local vs. both non-shift control tasks (p<.001; ω² =.12-.14). Mixed-model ANOVAs revealed that the ADHD group did not demonstrate disproportional decrements in speed shift costs on the shifting vs. non-shift control tasks (p=.30; ω²=.002), suggesting no evidence of impaired set shifting abilities in ADHD. In contrast, the ADHD group made disproportionately more shifting errors than the Non-ADHD group (p=.03; ω²=0.03) that were more parsimoniously attributable to prerequisite (non-shifting) processes necessary for successful performance on the global-local task.

Conclusions:

Children with ADHD’s impaired performance on shifting tasks may be attributable to difficulties maintaining competing rule sets and/or inhibiting currently active rule sets prior to shifting. However, when these higher-order processes are executed successfully, there is no significant evidence to suggest a unique set shifting deficit in ADHD.

Attention-deficit/hyperactivity disorder (ADHD) is a chronic and heterogeneous neurodevelopmental disorder that affects approximately 5% of school-age children (Polanczyk et al., 2007, 2014). It has been proposed that underlying deficits in executive functions(s) may drive ADHD’s phenotypic behavioral presentation for many, if not most, children with ADHD (Barkley, 1997; Rapport et al., 2009; Sonuga-Barke et al., 2010; Kasper et al., 2012; Chacko et al., 2014). Executive functions refer to neurocognitive processes that regulate human behaviors by maintaining problem sets to attain future goals, and include interrelated but separate domains: working memory, inhibition, and set shifting (Miyake et al., 2000; Ven et al., 2013). Working memory refers to the active, top-down manipulation of information held in short-term memory (Baddeley, 2007), and includes interrelated functions of the mid-lateral prefrontal cortex and interconnected networks that involve dual-processing, updating, and reordering (Wager & Smith, 2003). Inhibitory control refers to a set of interrelated cognitive processes that underlie the ability to withhold (action restraint) or stop (action cancellation) an on-going response (Alderson et al., 2007). Set shifting, or cognitive flexibility, refers to the ability to quickly and efficiently switch between mental sets via activation of prefrontal and posterior parietal cortices (Miyake et al., 2000; Pa et al., 2010).

Working memory and inhibition have been given considerable attention in the pediatric ADHD literature (Lijffijt et al., 2005; Alderson et al., 2007; Kasper et al., 2012), whereas relatively few studies have targeted set shifting abilities in these children (Willcutt et al., 2005). This paucity of research is surprising given set shifting’s association with important areas of functioning that are compromised in many individuals with ADHD (Benedetto-Nasho & Tannock, 1999; Kofler et al., 2015). For example, set shifting abilities predict successful academic performance in math, reading, and language acquisition (Bull & Scerif, 2001; Epsy et al., 2004; Yeniad et al., 2013; Roberts et al., 2017). Set shifting also predicts problem-solving abilities (Senn et al., 2004) and teacher-ratings of social competence (Bierman et al., 2008), and impaired set shifting appears to confer risk for internalizing symptoms via increased rumination (Davis & Nolen-Hoeksema, 2000; Whitmer & Banich, 2007).

Set Shifting in ADHD

Overall, meta-analytic estimates suggest medium magnitude set shifting deficits in pediatric ADHD (d = 0.46-0.65; Willcutt et al., 2005), which reflects mixed evidence with several studies finding no evidence of impaired set shifting (Goldberg et al., 2005; Mulas et al., 2006; Biederman et al., 2007) and others reporting that children with ADHD show impaired set shifting based on both decreased accuracy and slower response times (Lawrence et al., 2004; Toplak et al., 2008; O’Brien et al., 2010), decreased accuracy but intact response times (Holmes et al., 2010), or slower response times but intact accuracy (Oades & Christiansen, 2008). However, the tests used to measure set shifting in the majority of previous ADHD studies merit scrutiny for specificity of focus and which neurocognitive processes are actually measured (Snyder et al., 2015). In particular, the majority of ADHD studies to date have estimated set shifting using the Wisconsin Card Sorting Test (WCST; Heaton et al., 1993) or the Trail Making Test-B (TMT-B; Reitan, 1958), which are commonly used neuropsychological tests with well-documented sensitivity for detecting gross neurological impairment (Heaton et al., 1993; Lezak, 1995). Nonetheless, these measures have been criticized for poor specificity, were not developed to assess set shifting, and appear to estimate neuropsychological functioning generally rather than set shifting specifically (Reitan, 1958; Stanczak et al., 1998; Greve et al., 2005; Nyhusa & Barcelób, 2009; Snyder et al., 2015).

Beyond test specificity concerns in the clinical literature’s measurement of set shifting (Snyder et al., 2015), the primary outcome variable used in many of these studies also merits scrutiny. The majority of studies examining set shifting in ADHD report shifting accuracy (i.e., number of errors) either in addition to or in place of the reaction time-based measures used in the cognitive literature (e.g., Miyake et al., 2000). However, while reaction time-based measures indicate children’s ability to quickly and efficiently shift back and forth between mental sets (i.e., set shifting; Miyake et al., 2000) shifting errors can result from failures in multiple executive processes. As shown in Figure 1 and detailed below, accuracy on a set shifting task requires not only set shifting abilities but also, at minimum, successful maintenance of the inactive rule set in working memory and inhibition of the currently active rule set prior to the cognitive shift (Baddeley et al., 1998; Arbuthnott & Frank, 2000). Thus, set shifting may be most directly assessed using reaction time-based measures on correct-response trials in which confounding prerequisite processes are executed successfully to determine whether or not children with ADHD shift as efficiently as their non-ADHD peers when these prerequisite processes are met.

Figure 1.

A theoretical model of the executive and nonexecutive processes required for successful performance on the global-local task. These neurocognitive processes are shown to the right. The global-global and local-local task variants control for these processes as shown (left).

Taken together, a growing body of evidence from the cognitive literature indicates that set shifting can be reliably estimated and differentiated from other core executive functions (working memory, inhibitory control) using criterion tasks designed specifically to assess set shifting abilities via reaction time (speed) to correct trials in both neurotypical (Bialystok, 2010) and clinical child samples (Bellgrove et al., 2003; Yerys et al., 2009). Although several recent developmental studies have examined set shifting using criterion tasks such as the Miyake et al. (2000) global-local task (Hubner, 2000; Bialystok, 2010; Hedden & Gabrieli, 2010), to our knowledge no study to date has assessed set shifting in children with ADHD according to the task parameters considered sufficient to evoke set shifting processes while controlling for executive and non-executive processes as described below (Miyake et al., 2000; St. Clair-Thompson & Gathercole, 2006).

Is Set Shifting a Unique Executive Function in Children?

Understanding the role of set shifting deficits in ADHD is further complicated by mixed evidence regarding the emergence and developmental trajectory of set shifting abilities in the neurotypical population (St. Clair-Thompson & Gathercole, 2006; Ven et al., 2013). For example, replicated evidence indicates that set shifting abilities emerge in children as young as age 4-5 (Espy,1997; Schouten et al., 2000) and reach adult-like levels by age 8-10 (Chelune & Baer, 1986; Luciana & Nelson, 1998). Further, some factor analytic studies have identified a unique set shifting factor in school age children (Huizinga et al., 2006; Van der Sluis et al., 2007). In contrast, other factor analytic results question the extent to which set shifting is a unique executive function in middle childhood, such that set shifting tasks loaded onto working memory and inhibitory control factors (St. Clair-Thompson & Gathercole, 2006) or fit indices supported a unitary model of executive function rather than separable domains such as set shifting (Wu et al., 2011).

These mixed findings may be due to task impurity (Snyder et al., 2015), which has been a particular challenge for measuring set shifting because, as reviewed above, these tasks require multiple higher-order (e.g., working memory and inhibitory control processes) and lower-order abilities for successful execution (Figure 1; Baddeley et al., 1998; Arbuthnott & Frank, 2000). Thus, it is unsurprising that set shifting tasks have cross-loaded with these interrelated executive functions in some developmental studies (St. Clair-Thompson & Gathercole, 2006), despite consistent evidence that they form a unique factor in adult samples (Miyake et al., 2000; Miyake & Friedman, 2012).

Lower-order neurocognitive functions such as basic choice-response and processing speed abilities are also fundamental for successful performance on set shifting tasks (Ridderinkhof et al., 2002; Aron et al., 2004) and may be particularly important to control in ADHD studies because these children have well-documented impairments across a broad range of choice-response tasks (for review see Kofler et al., 2013). For example, meta analytic evidence indicates that ADHD-related impaired performance on tests of inhibition may be attributable to lower-order impairments in choice-response and processing speed rather than impaired inhibition (Alderson et al., 2007; Alderson et al., 2010; Lijffijt et al., 2005). Similarly, lower-order processing speed appears to account for approximately 20%-30% of the difference between ADHD and neurotypical groups on tests of working memory (Karalunas & Huang-Pollock, 2013; Kofler et al., 2014).

Taken together, scoring approaches that estimate within-person decrements in response times evoked by within-task changes in set shifting demands (i.e., shift costs) are necessary to carefully control for lower-order processing speed abilities (Miyake et al., 2000), but may not account for working memory’s role in maintaining multiple rule sets or inhibitory control’s role in stopping children’s prepotent processing to specific stimulus features as detailed below (Poirel et al., 2011). Therefore, carefully-designed control tasks and scoring approaches that estimate performance accuracy are also required to account for these higher-order processes (Hubner, 2000; Bellgrove et al., 2003).

Current Study

Taken together, it seems reasonable to hypothesize that the previously described conflicting findings are due, at least in part, to methodological issues related to task selection and confounding factors associated with both higher-order (other executive functions) and lower-order (e.g., processing speed) abilities that may obfuscate optimal detection of set shifting in children. The current study’s experimental design provides robust control for both lower-order (e.g., processing speed and attention) and higher-order (e.g., working memory and inhibitory control) abilities by using multiple, counterbalanced tasks and comparing within-subject response times and errors during shift versus non-shift trials (i.e., shift costs).

A series of three counterbalanced tasks, which were identical except for the systematic manipulation of shifting and inhibition demands, were adapted from the criterion global-local task (Miyake et al., 2000). We hypothesized that children with ADHD would demonstrate set shifting deficits. Regarding this hypothesis, response times during correct shift and non-shift trials were used to derive the primary estimate of set shifting abilities based on cognitive literature methods (i.e., speed shift costs; Miyake & Friedman, 2012). Evidence for a unique set shifting deficit in ADHD would require disproportionate slowing to correct responses when set shifting demands are increased (i.e., group x task interaction), indicating that children with ADHD take longer than their non-ADHD peers to cognitively shift between rule sets.

We also expected results consistent with the higher-order interference hypothesis that processes beyond set shifting are necessary to complete set shifting tasks and that deficits in these confounding processes may produce deficits on set shifting tasks even in the context of intact set shifting abilities (Hubner, 2000; Bellgrove et al., 2003). Regarding this hypothesis, we expected a similar group x task interaction for the accuracy shift cost data given evidence of deficits in higher-order abilities in children with ADHD (e.g., Alderson et al., 2007; Kasper et al., 2012). However, interpreting errors as evidence of impaired set shifting is limited because, unlike speed shift costs that are computed based on trials in which confounding prerequisite processes are executed successfully (i.e., correct response trials), accuracy shifting errors can reflect a breakdown in one or more of several higher-order processes evoked by the criterion global-local set shifting task as discussed above and shown in Figure 1. That is, we expected the ADHD group to make disproportionately more errors than the non-ADHD group due to difficulty in maintaining competing rule sets and inhibiting their response to the previously activated rule set prior to flexibly shifting to the competing rule set.

Method

Participants

The sample comprised 77 children aged 8 to 13 years (M = 10.46, SD = 1.54; 45 boys, 32 girls) from the Southeastern United States, recruited by or referred to a university-based children’s learning clinic (CLC) through community resources (e.g., pediatricians, community mental health clinics, school system personnel, self-referral) from 2015 to 2017. The CLC is a research-practitioner training clinic known to the surrounding community for conducting developmental and clinical child research and providing pro bono comprehensive diagnostic and psychoeducational services. Its client base consists of children with suspected learning, behavioral or emotional problems, as well as typically developing children (those without a suspected psychological disorder) whose parents agreed to have them participate in developmental/clinical research studies. All parents and children gave informed consent/assent, and the Florida State University Institutional Review Board approved the study prior to the onset of data collection. Sample ethnicity was mixed with 51 Caucasian Non-Hispanic (66.2%), 10 Hispanic English-speaking (13.0%), 9 African American (11.7%), 3 Asian (3.9%), and 4 multiracial children (5.2%).

Group Assignment

All children and caregivers completed a detailed, semi-structured clinical interview using the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Aged Children (K-SADS; Kaufman et al., 1997). The K-SADS (2013 Update) allows differential diagnosis according to symptom onset, course, duration, quantity, severity, and impairment in children and adolescents based on DSM-5 criteria. Its psychometric properties are well established, including inter-rater agreement of .93 to 1.00, test-retest reliability of .63 to 1.00, and concurrent (criterion) validity between the K-SADS and psychometrically established parent rating scales (Kaufman et al., 1997). K-SADS interviews were supplemented with parent and teacher ratings scales from the Behavior Assessment System for Children (BASC-2; Reynolds & Kamphaus, 2004) and Child Symptom Inventory (CSI-IV; Gadow & Sprafkin, 2002). A psychoeducational report was provided to parents.

Forty-three children met all of the following criteria and were included in the ADHD group (n=43, 40% girls): (1) DSM-5 diagnosis of ADHD Combined (n=27), Inattentive (n=12), or Hyperactive/Impulsive Presentation (n=4) by the directing clinical psychologist based on K-SADS; (2) borderline/clinical elevations on at least one parent and one teacher ADHD subscale; and (3) current impairment based on parent report. Children with all ADHD current presentation specifiers were eligible given the instability of ADHD subtypes (Lahey et al., 2005; Valo & Tannock, 2010; Willcutt et al., 2012) and recent evidence from a portion of the current sample indicating that ADHD inattentive and hyperactive/impulsive symptom domains do not covary with impairments in set shifting abilities (Kofler et al., 2018). To improve generalizability (Wilens et al., 2002), children with comorbidities were included. Comorbidities reflect clinical consensus best estimates and included oppositional defiant disorder (16%), depressive disorders (9%), and anxiety disorders (26%). Children with ADHD were screened for specific learning disorders (SLD) in reading (7%), math (14%), and written language (14%) defined by score(s) >1.5 SD below age-norms on one or more subtest(s) in the Academic Skills Battery of the Kaufman Test of Educational Achievement, Third Edition (Kaufman, 2014).

The Non-ADHD group comprised 34 consecutive case-control referrals (14 girls) who did not meet ADHD criteria and included both neurotypical children and children with psychiatric disorders other than ADHD. Neurotypical children (65%) had normal developmental histories and nonclinical parent/teacher ratings, were recruited through community resources, and completed the same evaluation as clinically-referred cases. Clinically referred and evaluated children who did not meet ADHD criteria were also included in the Non-ADHD group. These Non-ADHD disorders were included to control for comorbidities in the ADHD group, and included best estimate diagnoses of anxiety (29%), depressive (6%), and oppositional defiant disorders (6%)¹. One of the clinically-evaluated Non-ADHD cases screened positive for learning disorders in written language. Importantly, the ADHD and Non-ADHD groups did not differ in the proportion of children diagnosed with a clinical disorder other than ADHD (omnibus: p = .15, anxiety: p = .71, depression: p = .58, ODD: p = .16).

Children were excluded from the study if they presented with (a) gross neurological, sensory, or motor impairment, (b) history of a seizure disorder, psychosis, intellectual disability, or autism spectrum disorder, or (c) non-stimulant medications that could not be withheld for testing. Thirteen of the 43 children with ADHD were currently prescribed psychostimulants. Medication was withheld for a minimum of 24 hours prior to both research testing sessions.

Procedures

Children participated in two research sessions (3 hours each) following the baseline psychoeducational assessment. The set shifting and control tasks were administered as part of a larger battery of executive and non-executive laboratory tasks. The tasks were counterbalanced within and across sessions to minimize order effects. Children were seated in a caster-wheel swivel chair approximately 0.66 meters from the computer monitor for all tasks. Performance was monitored at all times by the examiner, who was stationed just out of the child’s view to provide a structured setting while minimizing performance improvements associated with examiner demand characteristics (Gomez & Sanson, 1994). All children received brief (2-3 min) breaks after each task, and preset longer (10-15 min) breaks after every 2-3 tasks to minimize fatigue.

Experimental Manipulation of Set Shifting

The current study adapted the Miyake et al. (2000) global-local set shifting task for use with children. Three task variants were created to be identical in all aspects except our primary dependent variable (set shifting demands). In addition to the global-local set shifting task, we administered both global-global and local-local non-shifting variants to provide more precise control for both higher- and lower-order processes involved in successful performance on the global-local task. These computerized tasks use Navon (1977) figures, which feature a “global” shape (e.g., a circle) constructed using smaller, “local” figures (e.g., squares; Figure 2). Figures were presented one at a time in one of four quadrants (clockwise rotation) on a computer monitor (jittered ISI 800-2000ms). To minimize memory demands, on-screen cues (“big shape”, “small shapes”) were positioned next to each quadrant (Figure 2). Sixty trials were administered following three blocks of 6 to 8 practice trials (100% correct required). Data from the first trial of each task were excluded because they were neither shift nor non-shift trials. Children responded via mouse click. Task duration was approximately 5 minutes per task.

Figure 2.

A sample trial from the global-local task. Children are instructed to click a response button (bottom) based on the presented stimulus (top) and rule set. Navon figures are presented sequentially in each quadrant in clockwise rotation. In this example, the Navon figure is a circle (global feature) comprised of squares (local features). Shift trials require children to inhibit the rule set from the previous trial and cognitively shift to the alternate rule set (top left and bottom right quadrants). Non-shift trials require children to apply to same rule set from the previous trial (top right and bottom left quadrants). The first trial of each task was excluded from analysis because it was neither a shift nor a non-shift trial.

Set shifting condition: Global-local.

As shown in Figure 2, children were required to shift their response between global and local features depending on which quadrant the figures appeared (top quadrants: global; bottom quadrants: local). Trials with stimuli in the top left or bottom right quadrants involved set shifting (shift trials) because responses required a different rule than the previous trial; trials with stimuli in the top right or bottom left quadrants did not require shifting because they featured the same rule as the previous trial (non-shift trials).

Control 1: Global-global.

This control condition was created to control for the well-replicated finding that children with ADHD often make more errors and show slower/more variable reaction times on choice-response tasks, regardless of task content (Kofler et al., 2013; Klein et al., 2006). The global-global task was identical to the global-local task described above except that children always responded to the prepotent global figure (i.e., no explicit shifting or inhibition demands).

Control 2: Local-local.

This control condition was created to control for the inhibition demands required for children in the target age range to ignore a prepotent global figure and respond to the smaller (local) figures (i.e., Stroop effect; Lansbergen et al., 2007). That is, the developmental literature indicates that by age 6, children reliably demonstrate prepotent visual attention to global stimulus features, such that responding based on local features requires them to inhibit their prepotent processing of the global stimulus (Poirel et al., 2011). The local-local task was identical to the global-local and global-global tasks described above except that children always responded to the local features (i.e., no explicit shifting demands).

Dependent variables.

Performance data were recorded separately for ‘shift’ and ‘non-shift’ trials separately for each of the three tasks. Notably, the global-global and local-local control conditions did not require shifting demands; for parsimony we use the terms ‘shift’ and ‘non-shift’ trials to refer to stimuli in the top left/bottom right (shift) and top right/bottom left (non-shift) quadrants for all three conditions (Figure 2).

Reaction time (RT) data was processed following the steps outlined in Miyake et al. (2000) that winsorized the most extreme 2.2% of reaction times. First, all individual trial RTs greater than 9500ms were winsorized to 9500ms. Second, individual trial RTs greater than 3 standard deviations from each child’s mean RT were winsorized relative to that child’s within-task RT distribution, separately for each task. Following Miyake et al. (2000), set shifting abilities were operationalized as speed shift costs, calculated separately for each task condition for each child (Speed shift cost = RT_shift − RT_non-shift for correct trials). Accuracy shift costs were also calculated (Accuracy shift cost = Errors_shift − Errors_non-shift) to parallel previous clinical literature and examine the impact of higher-order processes on set shifting task performance in children with ADHD (i.e., higher-order interference).

Intellectual Functioning (IQ) and Socioeconomic Status

IQ was estimated using the Wechsler Intelligence Scale for Children (WISC-V; Wechsler, 2014) Verbal Comprehension Index. Socioeconomic status (SES) was estimated using the Hollingshead (1975) scoring based on caregiver(s)’ education and occupation.

Data Analysis Overview

A series of two 2 (group: ADHD, Non-ADHD) x 3 (condition: global-global, local-local, global-local) Mixed Models ANOVAs for set shifting speed and accuracy were conducted to examine the study’s hypotheses. For both models, the main effect of task was examined as a manipulation check, such that post hoc tests were expected to indicate higher shift costs for speed and accuracy during the shifting condition relative to both non-shifting conditions. Effect sizes for these models are reported as omega-squared (ω²), which indicates the proportion of variance explained by each effect, as recommended (small = .01, medium = .06, large = .16; Olejnik and Algina, 2000).

For both models, a main effect of group in the absence of an interaction would indicate that impaired performance in the set shifting condition is most parsimoniously explained by lower-order impairments in choice-response processes (i.e., similar deficits across all three tasks; Kofler et al., 2013). Similarly, post hoc tests exploring interaction effects that indicate disproportionate decrements in the ADHD group during both local-local and global-local tasks relative to the global-global task would indicate impaired inhibitory control associated with inhibiting the prepotent global feature to process local features as shown in Figure 1 (Poirel et al., 2011).

Evidence of a unique set shifting deficit in ADHD would require a significant task x group interaction for set shifting speed (Miyake et al., 2000) with post hoc tests indicating disproportionate decrements in the ADHD group during the shift versus both non-shift tasks. Finally, a significant task x group interaction for one but not both models would assist in clarifying the influence of impaired higher-order processes on children’s performance during set shifting tasks. That is, a significant interaction for speed but not accuracy would indicate task-sufficient working memory and inhibitory control abilities but impaired set shifting abilities (i.e., children with ADHD are able to maintain competing rule sets and inhibit prepotent responses but shift slower than their peers). Alternatively, a significant interaction for accuracy but not speed would indicate that poor performance on set shifting tasks is more parsimoniously explained by impaired working memory and/or inhibitory control abilities despite intact set shifting abilities (i.e., children with ADHD have difficulty consistently maintaining competing rule sets and/or inhibiting prepotent responses, but are able to shift as quickly as their peers when these prerequisites are met; Figure 1).

Power Analysis

Power analysis was conducted using GPower v3.1 (Faul et al., 2007) to determine our sensitivity for detecting effects. For power = .80, α = .05, and 2 groups (ADHD, Non-ADHD) with 3 measurements (global-local, local-local, global-global) and our sample size of 77, we are sufficiently powered to detect between-group effects of d = 0.53 and group x task interaction effects of d = 0.29 or larger. These estimates are similar to, or larger than, expected effects based on meta-analysis (d = 0.46-0.65; Boonstra et al., 2005; Willcutt et al., 2005). Thus, the study is sufficiently powered to detect effects of the anticipated magnitude.

Results

Preliminary Analyses

Outliers beyond 3.00 SD were winsorized relative to the within-group distribution (ADHD, Non-ADHD). This process affected 1.6% (ADHD group) to 0.9% (Non-ADHD group) of data points. Global-local data for a subset of the current sample was included in the aggregate estimate of set shifting abilities reported in Kofler et al. (2018). Global-global and local-local data have not been reported previously for any children in the current sample. All parent and teacher ADHD symptom ratings were higher for the ADHD than Non-ADHD group as expected (all p < .05; Table 1). In addition, the groups did not significantly differ in terms of gender (p = .95), ethnicity (p = .09), SES (p = .07), or age (p = .21). There was a small between-group difference in IQ (p = .03). IQ was not included as a covariate based on compelling statistical, methodological, and conceptual rationale against covarying IQ when investigating cognitive processes in ADHD (Dennis et al., 2009). In other words, covarying IQ would preclude conclusions regarding ADHD as a neurodevelopmental disorder by fundamentally changing our grouping variable, and remove significant variance associated with the outcomes of interest (Dennis et al., 2009).

Table 1.

Descriptive Statistics.

Demographics	ADHD (N =43)	Non-ADHD (N = 34)	χ2(4, N=77)	p	Phi
Gender (Boys/Girls)	25/18	20/14	.004	.95	.007
Ethnicity (AA/A/C/H/M)	5/0/32/3/3	4/3/19/7/1	11.13	.09	.38
	M(SD)	M(SD)	t(75)	p	Cohen’s d
Age	10.26(1.51)	10.71(1.56)	1.27	.21	0.29
SES	46.71(11.65)	51.59(11.48)	1.84	.07	0.42
VCI (IQ)	102.67(14.16)	109.21(10.30)	2.26	.03*	0.53
BASC-2/3 Attention Problems
Teacher	63.49(8.69)	52.62(10.76)	−4.91	< .001*	1.13
Parent	65.98(7.10)	56.47(11.46)	−4.46	< .001*	1.03
BASC-2/3 Hyperactive Problems
Teacher	62.62(15.16)	54.15(12.75)	−2.60	.01*	0.60
Parent	68.00(13.46)	54.62(11.40)	−4.63	< .001*	1.06
Mean RT (seconds)
Global-Global	2.20(0.73)	2.09(0.78)	−0.65	.52	0.15
Local-Local	2.26(0.75)	1.84(0.41)	−3.00	.004*	0.69
Global-Local	3.49(1.23)	3.09(0.78)	−1.65	.10	0.38
Mean Errors
Global-Global	0.47(0.62)	0.58(1.00)	0.59	.56	0.13
Local-Local	1.77(1.24)	1.46(0.68)	−1.34	.19	0.31
Global-Local	4.38(3.13)	2.86(1.79)	−2.53	.01*	0.60

Note. AA = African American, A = Asian, BASC = Behavior Assessment System for Children, C = Caucasian Non-Hispanic, H = Hispanic English-speaking, M = Multiracial, SES = Social economic status, VCI (IQ) = Verbal Comprehension Index (Intelligence Quotient). RT = Reaction Time.

Manipulation Check

Results of the manipulation check indicated that we successfully evoked large magnitude increases in set shifting and overall executive demands as intended, as evidenced by significantly higher shift costs during the set shifting task versus both control tasks based on both speed (both p < .001) and accuracy (both p < .05), respectively, as detailed below.

Set Shifting Speed

Results of the 2 (group: ADHD, Non-ADHD) x 3 (task: global-global control, local-local control, global-local shifting) mixed-model ANOVA indicated a main effect of task (F(2,75) = 24.96, p < .001, ω² =.16; Table 2), with Tukey-corrected post hocs indicating that the global-local set shifting task elicited significantly higher speed shift costs than both non-shifting control tasks (both p < .001; ω² = .12, .14), which did not differ (p = .62, ω² = −.002). In contrast, the main effect of group (ADHD/Non-ADHD; F(1,75) = 0.40, p = .53, ω² = −.01) and the group x task interaction (F(2,75) = 1.36, p = .26; ω² = .003) were not significant, indicating that the groups did not differ significantly in terms of overall speed shift costs, and that the ADHD group did not demonstrate disproportional decrements in speed shift costs on the shifting vs. non-shifting control tasks (Figure 3a). In other words, there was no significant evidence to suggest that ADHD is associated with a unique set shifting deficit.

Table 2.

ANOVA models.

	ADHD (N =43)		Non-ADHD (N =34)		Task Main Effect			Group Main Effect			Group*Task Interaction Effect
Dependent Variable	M	SD	M	SD	F(2,75)	p	ω2	F(1,75)	p	ω2	F(2,75)	p	ω2
Speed Shift Costs					24.96	<.001*	.16	.40	.53	−.01	1.36	.26	.003
Global-Global	69.14	288.28	112.03	226.71
Local-Local	137.00	352.14	−7.31	179.60
Global-Local	442.33	549.03	446.98	474.73
Accuracy Shift Costs					9.78	<.001*	.07	2.22	.14	.02	4.99	.01*	.04
Global-Global	−0.20	0.52	0.09	1.03
Local-Local	0.16	1.05	0.26	1.54
Global-Local	1.84	2.76	0.47	2.35

Note. ω² = Omega-squared (small = .01, medium = .06, large = .16).

Figure 3.

Graphs depicting group mean differences in (A) response times (RT_shift – RT_no-shift) and (B) errors (Errors_shift – Errors_no-shift) across the three task conditions. Error bars reflect 95% confidence intervals.

Set Shifting Accuracy

Results of the 2 (group: ADHD, Non-ADHD) x 3 (task: global-global control, local-local control, global-local shifting) mixed-model ANOVA indicated a main effect of task (F(2,75) = 9.78, p < .001; ω² = .07; Table 2) with Tukey-corrected post hocs indicating that the global-local set shifting task elicited significantly higher accuracy shift costs than both non-shifting control tasks (both p < .05; ω² = .08, .03), which did not differ (p = .26; ω² = .01). In contrast, the main effect of group (ADHD/Non-ADHD; F(1,75) = 2.22, p = .14, ω² = .02) was not significant, indicating that the groups did not differ significantly in terms of overall accuracy shift costs.

The group x task interaction was significant (F(2,75) = 4.99, p = .01; ω² = .04; Table 2) with Tukey-corrected post hocs indicating that the ADHD group made more errors than the Non-ADHD group during the global-local task only (p = .02; ω² = .05) whereas no significant group differences emerged during either non-shifting control task (both p > .05; ω² = .02, −.01). Interaction contrasts indicated that the ADHD group had disproportionate decrements in accuracy relative to the Non-ADHD group during the global-local task compared to the global-global task (p = .01; ω² = .04) and the local-local task (p = .04; ω² = .02). In contrast, the groups had proportionate decrements in accuracy during the global-global compared to the local-local task (p = .90, ω² = −.003; Figure 3b).

Taken together, the speed and accuracy findings indicated that children with ADHD’s impaired performance on the criterion global-local set shifting task is attributable to difficulties in maintaining competing rule sets and/or inhibiting the currently active rule set prior to shifting to the competing rule. When these higher-order processes are executed successfully, there is no significant evidence to suggest a unique set shifting deficit in ADHD.

Discussion

The current study was the first to examine set shifting abilities in children with ADHD using the gold standard global-local task (Miyake et al., 2000) within an experimental design that addressed task impurity via the inclusion of multiple, counterbalanced control tasks and examination of within-subject performance decrements evoked by specific neurocognitive demands. The data provided significant evidence that the experimental manipulations were successful, as evidenced by increased speed and accuracy shift costs that occurred exclusively during the set shifting task. The results were generally consistent with developmental evidence that set shifting abilities are developed and measurable in school-aged children (Chelune & Baer, 1986; Luciana & Nelson, 1998), and extend previous findings by way of an experimental manipulation that provided robust control for both lower-order (e.g., processing speed, attention) and higher-order (i.e., working memory, inhibition) confounds. Additional strengths of the experiment include the carefully-phenotyped sample of children with and without ADHD matched in the proportion of non-ADHD disorders (Wilens et al., 2002), and the omnibus finding that both groups demonstrated higher shift costs when set shifting demands were evoked as expected. Overall, there was no significant evidence to suggest set shifting deficits in children with ADHD, with differences in their accuracy being more parsimoniously attributed to deficits in confounding higher-order processes (e.g., working memory, inhibitory control) that are necessary to successfully complete these tasks (Arbuthnott & Frank, 2000; Snyder et al., 2015).

Meta-analytic evidence suggests that pediatric ADHD may be associated with small-to-medium magnitude deficits in set shifting abilities (Willcutt et al., 2005). Additionally, set shifting is associated with important areas of functioning that are compromised in many individuals with ADHD (Benedetto-Nasho & Tannock, 1999; Kofler et al., 2015). Therefore, when designing the experiment we hypothesized that we would find evidence for a unique set shifting deficit in ADHD. However, findings from the speed model produced non-significant group and interaction effects indicating that children with ADHD did not show differential slowing relative to their Non-ADHD peers when set shifting demands were induced. These findings were contrary to our hypothesis and suggest that set shifting abilities are likely intact in ADHD, which adds to mixed literature suggesting children with ADHD either do (Lawrence et al., 2004; Toplak et al., 2008; O’Brien et al., 2010) or do not (Goldberg et al., 2005; Mulas et al., 2006; Biederman et al., 2007; Holmes et al., 2010) have slower set shifting. In light of our carefully controlled experimental findings, these previous mixed results may reflect study-level sampling error around a population effect size of 0.0 and/or the uncontrolled influences of higher-order processes (e.g., working memory and/or inhibitory control) as described below. Alternatively, the incongruence between our a priori hypothesis and our findings may be because the former was developed from meta-analytic estimates that were based entirely on accuracy and completion time data from tasks that have been criticized for poor construct validity and were not developed specifically to assess set shifting (Snyder et al., 2015).

In contrast, findings from the accuracy model produced a significant group x task interaction, supporting our higher-order interference hypothesis, and indicating that children with ADHD made differentially more errors when executive control demands were induced. The difference in results between the accuracy and speed models highlight the task impurity problem (Snyder et al., 2015), such that it appears impossible to develop tasks that purely measure a single higher-order neurocognitive process. This conclusion is consistent with replicated evidence of the confounding influence of higher-order processes (e.g., working memory, inhibition) that bias performance on other neurocognitive tasks in children with ADHD (Alderson et al., 2010; Karalunas & Huang-Pollock, 2013; Kofler et al., 2014; Raiker et al., 2017) and taken together suggest strongly that ADHD is not associated with set shifting deficits. Rather, children with ADHD exhibit impaired performance on these tasks most likely due to deficits in working memory (e.g., difficulties in maintaining competing rule sets) and/or inhibition abilities (e.g., inhibiting the currently active rule set prior to shifting to the competing rule; Hubner, 2000; Bellgrove et al., 2003; Yerys et al., 2009; Bialystok, 2010; Hedden & Gabrieli, 2010). Accordingly, when these higher-order processes are executed successfully, children with ADHD are able to effectively shift between competing rule sets as efficiently as their non-ADHD peers.

Limitations

The current study was the first to assess set shifting in pediatric ADHD using criterion experimental and control tasks, multiple measures of task performance (speed and accuracy) that accounted for confounding higher- and lower-order processes, and a sample of carefully phenotyped children with and without ADHD. Despite these methodological refinements, the following limitations must be considered when interpreting results. Critically, it is difficult to separate higher- and lower-order processes during tasks. We were able to account for attention and processing speed’s influence due to our within-subject manipulation and within-task control for non-shift response times, but were unable to separately assess the influence of higher-order working memory and inhibitory control processes to determine whether one or both contributed to our findings. Further, despite the strong conceptual basis for attributing errors to the overall executive system rather than shifting specifically (Baddeley et al., 1998; Arbuthnott & Frank, 2000), these processes were not directly measured in the current study, and it is likely that shifting errors reflect a combination of executive and non-executive processes that merit scrutiny in future studies.

Moreover, children with all ADHD current presentation specifiers were included based on previous work, which included a subset of the current sample, demonstrating no differences in parent- and teacher-reported inattentive or hyperactive/impulsive symptoms as a function of children’s set shifting abilities (Kofler et al., 2018). At the same time, meta-analytic evidence supports the concurrent, predictive, and discriminant validity of inattentive versus hyperactive/impulsive symptoms and suggests that ADHD symptom domains may differentially relate to other neurocognitive functions implicated in ADHD (Willcutt et al., 2012). Therefore, the extent to which increasing set shifting demands may differentially elicit inattentive versus hyperactive/impulsive symptoms in the moment remains unknown and as such future work is needed to investigate this relation. Finally, independent replications with larger samples and diverse age groups are needed to assess this pattern of results across development. As prior research in the adult ADHD literature demonstrates consistent impairments in set shifting abilities (Boonstra et al., 2005; Müller et al., 2007; Rohlf et al., 2012; Balint et al., 2016), studies are needed to investigate at what point in development persons with ADHD begin to exhibit unique set shifting deficits, and/or whether the adult findings are more parsimoniously explained by higher-order confounds as suggested herein.

Notably, co-occurring conditions are common in ADHD (Wilens et al., 2002). Therefore, inclusion of children with these comorbidities was important to maximize external validity and generalizability of our findings. We recruited a Non-ADHD group with comparable numbers of other psychiatric disorders in an attempt to balance external and internal validity. However, having proportionate numbers of comorbid disorders in each group does not perfectly equate the groups, and the inclusion of non-ADHD disorders in the control group may limit conclusions regarding neurotypical set shifting abilities. Future work is needed to compare more ‘pure’ (non-comorbid) ADHD and typically developing samples.

Clinical and Research Implications

The results of our study suggest that children with ADHD likely do not have a unique set shifting deficit. Children with ADHD flexibly shift between competing rule sets as quickly as children without ADHD, whereas they make more errors related to maintaining competing rule sets and/or inhibiting the prepotent rule set. Future work is needed to disentangle the role of each subcomponent of these higher-order cognitive functions (i.e., working memory, inhibitory control) on set shifting task performance in children with and without ADHD. For example, working memory is comprised, at minimum, of ‘working’ components such as updating, reordering, and dual-processing, as well as storage components including verbal, visual, spatial, and episodic (e.g., Wager & Smith, 2003). With regards to the working memory demands hypothesized to be evoked during set shifting tasks, children must hold the active rule in the internal focus of attention while maintaining the inactive rule set in the verbal short-term storage region of direct access (Baddeley, 2007; Oberauer, 2007). Similarly, inhibitory control processes hypothesized to be evoked during set shifting tasks are likely to relate to interference control (cognitive inhibition) rather than action restraint and action cancellation (motor or behavioral inhibition). Therefore, studies are needed with tasks designed to examine the association between set shifting abilities and interference control (cognitive inhibition) and verbal maintenance in the direct access region (Arbuthnott & Frank, 2000; Baddeley et al., 1998).

Lastly, it remains possible that set shifting is etiologically important in producing the ADHD phenotype for at least a subset of children with ADHD (Kofler et al., 2018) despite evidence from the current study indicating that this ability is likely intact at the group level, particularly given the bidirectional relations between the effects of increasing cognitive demands and increasing gross motor movement in children with and without ADHD (i.e., hyperactivity; Kofler et al., 2016), combined with evidence that increasing gross motor movement may facilitate performance on cognitive tests for children with ADHD (Hartanto et al., 2015; Sarver et al., 2015). That is, it remains possible that set shifting demands may evoke increased gross motor activity in children with ADHD that in turn increases psychophysiological arousal and normalizes set shifting speed for these children. Studies of set shifting’s relation with ADHD phenotypic behaviors are therefore needed prior to conclusively ruling out set shifting as a viable intervention target.

Public Significance Statement.

It has been suggested that ADHD is associated with deficits in the executive ability to flexibly shift between tasks or activities. However, this conclusion may be premature due to problems with the tests used to measure shifting abilities in these children. Using a carefully controlled experimental design, the current study found that set shifting may be intact in pediatric ADHD. In other words, children with ADHD appear to have difficulty maintaining/inhibiting competing rule sets prior to shifting. When these higher-order processes are executed successfully, children with ADHD shift just as quickly as their non-ADHD peers.