Understanding executive control in autism spectrum disorders in the lab and in the real world

Abstract

In this paper, we review the most recent and often conflicting findings on conventional measures of executive control in autism spectrum disorders. We discuss the obstacles to accurate measurement of executive control, such as: its prolonged developmental trajectory; lack of consensus on its definition and if it is a unitary construct; the inherent complexity of executive control; and the difficulty measuring executive control functions in laboratory or clinical settings. We review the potential of an ecological validity framework to address some of these problems, and describe new tests claiming verisimilitude, or close resemblance to “real life” demands. We also review the concept of veridicality, which allows for the measurement of the ecological validity of any task, and discuss the few studies addressing ecological validity in individuals with autism. Our review suggests that a multi-source approach emphasizing veridicality may provide the most comprehensive assessment of executive control in autism.

Executive control is a widely studied construct in the field of autism spectrum disorders, henceforth referred to in this paper as autism. Though the research findings are inconsistent (see below), most clinicians, teachers, and family members agree that individuals with autism have difficulties with various aspects of executive control in their daily lives. This review will explore this discrepancy by addressing the following three questions:

1. What do we know about conventional executive control tasks and autism?

2. Why is the executive control profile in autism so confusing?

3. Where do we go from here?

Our goal is to highlight novel areas of research in this field that could: advance the study of executive control in autism; improve its ability to capture the daily executive control obstacles that impair individuals with autism; and provide new targets for intervention.

What do we know about conventional executive control tasks and autism? Themes and conflicting findings in the literature

Autism was first described as a frontal, executive dysfunction disorder in a seminal paper by Damasio and Maurer (1978), in which they described similarities in behavioral presentation between individuals with autism and individuals with frontal lesions. They argued that atypical functioning of the frontal lobes could drive the lack of social motivation, poor communication, and perseverative behavior commonly observed in autism. In the 30 years that have followed, almost 150 articles and three major reviews (Hill, 2004; Pennington & Ozonoff, 1996; Sergeant et al., 2002) have been published on executive control in autism.

Executive control is an umbrella term that captures a set of cognitive processes that direct behavior regulation and orchestration of attaining a future goal (Welsh & Pennington, 1988). These cognitive processes include working memory, inhibition, cognitive flexibility, monitoring, planning, and generativity (Rogers & Bennetto, 2000). While the neural basis of executive control cognitive processes includes multiple cortico-cortical and cortico-striatal loops (D’Esposito, 2007; Miller & Cohen, 2001; Robbins, 2007), the frontal lobes are thought to play a central role (Miller, 1999; Miller & Cohen, 2001; Stuss & Benson, 1984).

Although the claim of executive dysfunction as a causal factor in autism (Ozonoff et al., 1991; Russell, 1997) is controversial (Dawson et al., 2002; Griffith et al., 1999; Liss et al., 2001; Yerys et al., 2007), it is clear that executive dysfunction plays a role in the social and cognitive deficits observed in individuals with autism (e.g., Hughes, 1996; Lopez et al., 2005). In their extensive review, Pennington and Ozonoff (1996) reported larger effect sizes associated with cognitive inflexibility and poor planning in autism than for any executive dysfunction measures in other developmental disorders (i.e., attention deficit hyperactivity disorder, conduct disorder, and Tourette’s syndrome). Inhibition and working memory, on the other hand, were considered mostly intact in autism. Six years later, Sergeant et al. (2002) reported similar findings in their review. Hill’s (2004) review supported the argument for significant planning and cognitive flexibility deficits in individuals with autism, as well as suggesting difficulties with inhibiting a prepotent response (i.e., response inhibition), and generativity. However, Hill (2004) also highlighted the mixed results of deficits across different age bands and the various executive control cognitive processes. This work suggested that executive control laboratory tasks capture atypical cognition in individuals with autism, but that evidence for a unique executive control profile in autism, or evidence that executive control is the singular cause of autism, is weak.

A literature review entering the terms ‘autism’ with ‘executive function’, ‘working memory’, ‘set-shifting’, ‘planning’, ‘cognitive flexibility’, ‘inhibition’, or ‘cognitive control’ into common search engines revealed more than 40 new articles assessing executive control in individuals with autism in comparison to a control group subsequent to Hill’s (2004) review (see Table 1). Neuropsychological and neuroimaging investigations have yielded mixed results about executive control weaknesses in autism. For high-functioning children and adults, several neuropsychological studies continue to document planning and cognitive flexibility deficits in autism relative to typically developing controls (Geurts et al., 2004; Joseph et al., 2005a; Lopez et al., 2005; Ozonoff et al., 2004; Tsuchiya et al., 2005; Winsler et al., 2007); however, the evidence is not conclusive. Planning was originally considered the most robust deficit in autism with a pooled Cohen’s d greater than 2 (Pennington & Ozonoff, 1996), but Table 1 shows that deficits in planning were observed in five of the 10 studies published since 2004 with two studies showing mixed results and three showing no differences. When analyzed at the task level, however, these results are less contradictory. Early conclusions about planning abilities in autism were predominantly drawn from performance on standard forms of the Tower of London and Tower of Hanoi tasks. The findings of our review confirm deficits in autism groups, as compared to controls, on these and similar tower tasks (Geurts et al., 2004; Joseph et al., 2005a; Lopez et al., 2005; Verté et al., 2005; Pellicano et al., 2006). Recently, however, many investigators have used a computerized version of the tower task (typically the CANTAB Stockings of Cambridge) and have often failed to find deficits in individuals with autism (Ozonoff et al., 2004; Goldberg et al., 2005; Happé et al., 2006; Just et al., 2007; but see Landa & Goldberg, 2005; Sinzig et al., 2008). This difference in performance with computer versus human administration is an important theme that we will return to later in this review.

Table 1.

Summary of all EC investigations published after Hill (2004) review in which an ASD group was compared to a control group.

Reference	Groups/Ages (Years)	Matching Criteria	EC Measures	EC Domains	Impairments?* (TYP/Other)
Geurts et al. (2004)	ASD: 9.4	Age	WCST	Flexibility	Y/Y^
	TYP: 9.1		Verbal Fluency	Generativity	Y/N
	ADHD: 9.3		Change Task	Inhibition/Multiple EF	Y/N
			Circle Drawing	Inhibition	Y/N
			Opposite Worlds	Inhibition	Y/N
			TOL	Planning/Multiple EF	Y/Y
			SOP	Working Memory	Y/N
Ozonoff et al. (2004)	ASD: 15.7	Age, VIQ, PIQ, Gender	ID/ED–CANTAB	Flexibility	Y
	Controls: 16.0		SC-CANTAB	Planning	N
Edgin & Pennington (2005)	ASD: 11.5	VIQ, PIQ, Laterality	ID/ED-CANTAB	Flexibility	N
	TYP: 12.0		SWM– CANTAB	Working Memory	N
Goldberg et al. (2005)	ASD: 10.3	Age, SES	ID/ED–CANTAB	Flexibility	N/N^
	TYP: 10.4		Stroop	Inhibition	N/N
	ADHD: 9.8		SC–CANTAB	Planning/Multiple EF	N/N
			SWM–CANTAB	Working Memory	Y/N
Hala et al. (2005)	ASD: 8.4	Age, VMA, Gender	3 experimental source monitoring tasks	Source Monitoring	Y
	TYP: 6.2
Joseph et al. (2005a)	ASD: 7.9	Age, VIQ, NVIQ	Day-Night Stroop	Inhibition	Y
	TYP: 8.3		KT-NEPSY	Inhibition	Y
			Tower NEPSY	Planning	Y
			Word Span	Working Memory	Y
			Block Span	Working Memory	Y
Joseph et al. (2005b)	ASD: 8.9	Age, VIQ, PIQ, Expressive and Receptive Language	Non-Verbal SOP Verbal SOP	Working Memory	N
	TYP: 8.9			Working Memory	Y
Koshino et al. (2005)**	ASD: 25.7	Age, FSIQ, VIQ, Gender	N-back	Working Memory	N
	TYP: 29.8
Landa & Goldberg (2005)	ASD: 11.0	Age, FSIQ, VIQ, PIQ, SES	ID/ED - CANTAB	Flexibility	N
	TYP: 11.0		SC – CANTAB	Planning	Y
			SWM – CANTAB	Working Memory	Y
Lopez et al. (2005)	ASD: 29.0	Age, PIQ, Gender	WCST	Flexibility	Y
	TYP: 29.0		TM-DKEFS	Flexibility	Y
			Design Fluency	Generativity	Y
			Verbal Fluency	Generativity	N
			Stroop	Inhibition	N
			Tower - DKEFS	Planning/Multiple EF	Y
			LNS	Working Memory	N
Tsuchiya et al. (2005)	ASD: 12.5	Age, IQ, Gender	Computerized WCST	Flexibility	Y/N
	TYP: 12.7
	ADHD: 11.3
Verté et al. (2005)	ASD: 9.1	Age, Gender	WCST	Flexibility	Y/Y/Y^^
	TYP: 9.4		Verbal Fluency	Generativity	Y/Y/Y
	TS: 10.0		Change Task	Inhibition	Y/N/N
	ASD+TS: 10.2		Opposite Worlds	Inhibition	Y/N/N
			TOL	Planning/Multiple EF	Y/N/N
			SOP	Working Memory	Y/N/N
Williams et al. (2005)	ASD (adult): 26.6	Age, IQ, Gender	Finger Windows	Working Memory	Y
	ASD (child): 11.8		Spatial Span	Working Memory	Y
	TYP (adult): 26.8		N-back	Working Memory	N
	TYP (child): 12.4		Letter-Number Seq.	Working Memory	N
			Letter-Number Mem.	Working Memory	N
Ambery et al (2006)	ASD: 37.6	Age, gender, VIQ, PIQ, Education	WCST	Flexibility	Y
	TYP: 33.5		COWAT	Generativity	Y
			Stroop	Inhibition	N
Happé et al. (2006)	ASD: 10.9	Age, FSIQ, PIQ	IDED-CANTAB	Flexibility	N/N^
	TYP: 11.2		CET	Generativity	Y/N
	ADHD: 11.6		Verbal Fluency	Generativity	Y/N
			Design Fluency	Generativity	Y/N
			Go/NoGo	Inhibition	Mixed/N
			SC-CANTAB	Planning/Multiple EF	N/N
			SWM-CANTAB	Working Memory	Y/N
Hill & Bird (2006)	ASD: 31.1	Age, IQ, gender	MCST	Flexibility	N
	TYP: 33.5		Trail Making	Flexibility	N
			Verbal Fluency	Generativity	N
			Stroop	Inhibition	N
Nakahachi et al. (2006)	ASD: 28.0	Age, IQ, Gender	ATMT	Flexibility	N
	TYP: 28.3		Digit Span	Working Memory	N
			Digit Symbol	Processing Speed	Y
Pellicano et al. (2006)	ASD: 5.6	Age, Verbal IQ, Non-Verbal IQ, gender	MCST	Flexibility	Y
	TYP: 5.5		Luria’s Hand game	Inhibition	Y
			Mazes	Planning/Multiple EF	N
			TOL	Planning/Multiple EF	Y
Schmitz et al. (2006)**	ASD: 38.0	Age, IQ	SWITCH	Flexibility	N
	ASD: 39.0		Go/No-Go	Inhibition	N
			Spatial Stroop	Inhibition	N
Silk et al. (2006)**	ASD: 14.7	Age, PIQ	Mental Rotation	Working Memory	N
	TYP: 15.0
Voelbel et al. (2006)**	ASD: 10.2	Age, SES	WCST	Flexibility	Y
	TYP: 10.8		TM-DKEFS	Flexibility
			PFT	Flexibility
Williams et al. (2006)	ASD: 11.7	Age, VIQ, PIQ, FSIQ, Gender, SES, Years of Education	Design Fluency	Generativity	N
	TYP: 12.2		Finger Windows	Working Memory	Y
			Letter Number Mem.	Working Memory	N
Bogte et al. (2007)	ASD: 28	Age, VIQ, PIQ, FSIQ	Variation of Sternberg	Multiple EF	N
	TYP: 28		RT task
Christ et al. (2007)	ASD: 8.2	--------------------	Go/NoGo	Inhibition	Y
	TYP: 11.3		Stroop	Inhibition	N
	Siblings: 10.2		Flanker	Inhibition	Y
Dichter & Belger (2007)**	ASD: 22.9	Age, Education, VIQ, PIQ, FSIQ	WCST	Flexibility	Mixed
	TYP: 24.6		Flanker with Arrows	Inhibition	Y
			Flanker with Faces	Inhibition	Y
Johnson et al. (2007)	ASD: 12.2	Age, IQ	SART – random	Inhibition	Y/N
	TYP: 11.1
	ADHD: 10.5
Just et al. (2007)**	ASD: 27.1	Age, VIQ, FSIQ, Gender	TOL	Planning/Multiple EF	N
	TYP: 24.5	Handedness
Kana et al. (2007)**	ASD: 26.8	AGE, FSIQ, Gender, Ethnicity/Race, SES	Go/NoGo	Inhibition	N
	TYP: 22.5		Go/NoGo with N-back	Multiple EF	N
Kleinhans et al. (2007)**	ASD: 24.5	Age, Handedness	Letter Fluency	Generativity	Y
	TYP: 22.5
Luna et al. (2007)	ASD: 17.0	Age, IQ, Gender	Antisaccade task	Inhibition	Y
	TYP: 17.0		Spatial location task	Working Memory	Y
Pellicano (2007)	ASD: 5.5	Age, VIQ, PIQ	MCST	Flexibility	Y
	TYP: 5.5		Luria’s Hand game	Inhibition	Y
			TOL, Mazes	Planning	Mixed
Steele et al. (2007)	ASD: 14.8	Age, VIQ, PIQ,	SWM– CANTAB	Working Memory	Y
	TYP: 16.9	FSIQ, SES
Winsler et al. (2007)**	ASD: 11.0	Age	WCST	Flexibility	Y
	TYP: 10.3		COWAT	Generativity	N
	ADHD: 11.6		Building Sticks	Planning/Multiple EF	N
Yerys et al. (2007)	ASD: 2.9	Age (except for TYP-MA), VMA and NVMA (except for TYP-CA), SES, Ethnicity/Race	Spatial Reversal	Flexibility	N/N/N^^^
	TYP-MA: 1.9		A-not-B	Multiple EF	N/N/N
	TYP-CC: 2.7		Windows	Multiple EF	N/N/N
	DD: 3.0
Gilbert et al. (2008)**	ASD: 38	Age, IQ, Handedness	Random Generation	Multiple EF	N
	TYP: 32		Alphabet Task	Multiple EF	N
Kaland et al. (2008)	ASD: 16.3	Age, VIQ, PIQ, FSIQ, Parental Education, SES	Computerized WCST	Flexibility	Mixed
	TYP: 15.5
Koshino et al. (2008)**	ASD: 24.5	Age, FSIQ, Gender, Ethnicity/Race, SES	N-back with faces stimuli	Working Memory +Social demands	N
	TYP: 28.7
McGonigle- Chalmers et al. (2008)	ASD: 10.0	Age	Size sequencing	Working Memory	Y
	TYP: 9.5
Shafritz et al. (2008)**	ASD: 22.3	Age, VIQ, PIQ, FSIQ	Behavioral Shift	Flexibility	N
	TYP: 24.3		Cognitive/Behavioral Shift	Multiple EF	N
Sinzig et al. (2008)	ASD: 14.3	Gender	IDED–CANTAB	Flexibility	N/N/N^^^^
	TYP: 31.1		Go/NoGo	Inhibition	N/N/Y
	ADHD: 12.2		SC–CANTAB	Planning/Multiple EF	Y/Y/N
	ASD+ADHD: 10.9		SWM– CANTAB	Working Memory	Y/N/N
Solomon et al. (2008)	ASD: 12.3	Age, FSIQ, VIQ, PIQ	POP	Multiple EF	Y
	TYP: 12.2
Williams et al. (2008)	ASD: 12.25	Age, VIQ, PIQ, VMA	Word Span	Working Memory	N
	ID: 12.1

ADHD= Attention Deficit/Hyperactivity Disorder; ASD= Autism Spectrum Disorders; ATMT= Advanced Trail Making Test; CET= Cognitive Estimation Test; COWAT= Controlled Oral Word Association Test; DD= Group of children with mixture of developmental delays; EC= Executive Control; ID= Individuals with intellectual difficulties; ID/ED-CANTAB= Intradimensional/Extradimensional shift test - CANTAB; KT-NEPSY= Knock-Tap NEPSY; MCST= Modified Card Sorting Test; POP=Preparing to Overcome Prepotency; PFT= Progressive Figures Test; SART- random= Sustained Attention to Response Task– Random Condition; SC-CANTAB=Stockings of Cambridge - CANTAB; SOP= Self-Ordered Pointing; SWM-CANTAB= Spatial Working Memory - CANTAB; TOL= Tower of London; TM-DKEFS = Trail Making –DKEFS; TYP = Typically developing children or neurotypical adults; TYP-MA= mental age matched typically developing controls; TYP-CA; chronologically age matched typically developing controls; WCST= Wisconsin Card Sorting Task

^*This column refers to whether there is an impairment reported for the ASD group relative to the comparison group. The possibilities are: Y= Yes differences are reported for particular domain/EF test; N= No differences are reported for particular domain/EF test; Mixed= Data did not produce a clear result.

^**fMRI study of EF cognitive processes–discussion of impairments is only accounting for behavioral performance.

^{^}First comparison is between ASD and TYP and second comparison is between ASD and ADHD (e.g., Y/Y means ASD group exhibited unique deficits relative to both groups, while Y/N means ASD group exhibited deficits compared to TYP but was not different from ADHD group.

^{^^}First comparison is between ASD and TYP, second comparison is between ASD and TS, third comparison is between ASD and ASD+TS. Comparisons between ASD and ASD+TS indicate worse performance in the ASD group.

^{^^^}First comparison is between ASD and mental age-matched typically developing group, second comparison is between ASD and chronologically age-matched typically developing group, the third comparison is between ASD and DD group.

^{^^^^}First comparison is between ASD and TYP, second comparison is between ASD and ADHD, and third comparison is between ASD and ASD+ADHD. Comparisons between ASD and ASD+ADHD did not yield a consistent direction of differences (e.g., ASD group is always performing better than ASD+ADHD group on EF measures).

Color Codes for EF tasks and corresponding domains

= Flexibility

= Generativity

= Inhibition

= Planning/Multiple EF

= Source Monitoring

= Working Memory

= Processing Speed

The same pattern occurs with cognitive flexibility. A number of recent investigations suggest that high-functioning individuals with autism do not differ from typically developing controls in this domain (Bogte et al., 2007; Goldberg et al., 2005; Landa & Goldberg, 2005; Edgin & Pennington, 2005; Yerys et al., 2007). Again, however, previously reported deficits on the traditional Wisconsin Card Sorting Test (human) administration are upheld. In contrast, the majority of negative findings involve a computerized measure of flexibility, the CANTAB Intra-dimensional/Extradimensional shift test.

Other executive control processes, such as inhibition and working memory, which were initially reported as being spared in autism (Ozonoff & Strayer, 1997; 2001), have been shown as deficient in recent studies with high-functioning children and adults with autism (Christ et al., 2007; Geurts et al., 2004; Joseph et al., 2005a; Luna et al., 2007; Steele et al., 2007; Williams et al., 2005). It is difficult to reconcile different findings regarding inhibition. Hill (2004) postulates that inhibition of prepotent responses is specifically impaired in autism, with sparing of other inhibitory functions. However, functioning even on a simple motor response inhibition task, Go/NoGo, is not consistently intact in recent reports (Christ et al., 2007). Variability in IQ between autism and comparison groups may play a role in these mixed findings. Other studies finding inhibition deficits employ a variety of tasks, each measuring inhibition in the context of disparate domain–specific demands (e.g., oculomotor saccades (Luna et al., 2007)).

Whereas performance on working memory tasks has historically been relatively under-reported in autism studies (for example, it is not addressed in Hill’s 2004 review), there has been an explosion in the number of recent reports on the topic, which generally indicate impaired spatial working memory in autism on four different tasks, with both computer and human administration formats (Geurts et al., 2004; Goldberg et al., 2005; Happé et al., 2006; Joseph et al., 2005a; Landa & Goldberg, 2005; Luna et al., 2007; Sinzig et al., 2008; Steele et al., 2007; Verté et al., 2005; Williams et al., 2005; Williams et al., 2006; but see Edgin & Pennington, 2005). At the same time, several studies indicate spared N-back, as well as digit, letter, and word span performance in autism (Koshino et al., 2005; Koshino et al., 2008; Lopez et al., 2005; Nakahachi, et al., 2006; Williams et al., 2005; Williams et al., 2006; Williams et al., 2008).

Perhaps with the exception of robust spatial working memory deficits in autism, it could be argued that our clarity regarding affected executive control subdomains in autism has actually attenuated as we accumulate more research in this area. There are many factors driving confusion in this literature. Some have to do with the complexity and heterogeneity of intellectual functioning and neuropsychological profiles in autism, which generally complicate assessment in this group. A primary problem involves the difficulty disentangling social cognition and motivation deficits in autism from performance on any socially administered neuropsychological task. An example of this is Ozonoff’s (1995) finding of improved flexibility on the Wisconsin Card Sorting Test when it was administered to individuals with autism using a computer, as opposed to a human examiner. A parallel finding can be inferred from the consistent evidence of autism-related impairment on human-administered tower tasks, but generally intact performance on computer-administered tower tasks (see above). Another problem is that autism is associated with high levels of comorbidity with symptoms of disorders such as attention deficit hyperactivity disorder and obsessive compulsive disorder (Leyfer et al., 2006), which present executive control deficits that can obscure the executive control profile. For example, Sinzig et al. (2008) found greater inhibition problems in children with autism and attention deficit hyperactivity disorder symptoms than in children with autism alone. Finally, as Hill (2004) points out in her review, variability in the intellectual level of individuals with autism affects the expression of executive control deficits on various tasks. There are also many problems in the general measurement of executive control that are interfering with our ability to come to a consensus on the executive control profile in autism. A number of these problems are discussed in the following section.

Why is the executive control profile in autism confusing? Obstacles to the accurate measurement of executive control

We struggle to capture executive control with traditional measures of the construct for a number of reasons, some of which relate to its prolonged developmental trajectory. Other major obstacles to defining the executive control profile arise from: a lack of consensus about the construct itself, such as its definition and whether it is unitary or fractionate; and the inherent complexity of EC, especially its goal setting and plan execution components, which are difficult to capture in traditional laboratory or clinical settings. Each of these issues is discussed below.

The protracted developmental trajectory of executive control through the first decades of life, reflecting the late development of prefrontal brain structures (Bunge et al., 2002; Casey et al., 1995; Durston et al., 2002; Lenroot et al., 2007; for review see Mesulam, 2002), complicates assessment techniques. As Hill (2004) and Jurado and Rosselli (2007) review, discrete executive control processes, such as cognitive flexibility and planning, reach maturity at disparate times in typical development. As a result, investigations of preschoolers with autism (Dawson et al., 1998; 2002; Griffith et al., 1999; Stahl & Pry, 2002; Yerys et al., 2007) may find intact cognitive flexibility and therefore report intact executive control in autism, while an investigation of school age children tapping later developing executive control abilities, such as planning, finds deficits, and therefore reports impaired executive control in autism (Ozonoff et al., 2004). In this case, its prolonged developmental trajectory contributes to conflicting findings, as well as a second important problem in the measurement of executive control: disagreement on its fundamental components.

At the most basic conceptual level, accurate assessment of executive control is challenged by a lack of consensus about the definition of this relatively new construct in neuropsychology (see Jurado & Rosselli, 2007 for review). This leads to problems in autism research when investigators sample different component processes of executive control and draw disparate conclusions about whether or not executive control is impaired. A related problem in the assessment of executive control is disagreement regarding whether it is a unitary or fractionate construct in both typical and atypical development. While some theories of executive control in developmental disorders emphasize impairment in specific, potentially fractionable, processes, such as inhibition (Barkley, 1997) or working memory and inhibition (Pennington & Ozonoff, 1996; Ozonoff & Jensen 1999), others describe executive control as a unitary concept related to goal-directed behavior (e.g., Duncan, 1986). The executive control tasks that have most reliably demonstrated impairments in autism are multi-factorial tasks that combine multiple executive control demands into one, such as the Wisconsin Card Sorting Test and the tower tasks. Even the spatial working memory tasks that have proven difficult for autism groups have strategic demands (Cambridge Cognition, 1996). These findings have traditionally been reported as indication of specific autism-related deficits in “flexibility,” “planning,” and “working memory,” but could in fact represent other weaknesses. In the case of the Wisconsin Card Sorting Test, cognitive flexibility, abstract reasoning, generation of problem-solving ideas, response to socially presented feedback, working memory, and other abilities are required (Gioia & Isquith, 2004). The traditional tower tasks require inhibition of prepotent responses, working memory, planning, generation of problem-solving ideas (Hill & Bird, 2006), as well as adherence to socially presented rules. It is possible that the autism literature is suffering from premature association of complex tasks with specific subdomains of executive control, such as flexibility or planning. In fact, weighing the evidence, the safest conclusion to draw from the plethora of studies may be that individuals with autism perform poorly on tasks that tap multiple executive control processes simultaneously.

This leads to the importance of acknowledging the complexity of the executive control construct. Major theories of executive control describe it as a “central process” that governs responses to complex problems and requires: determining where to focus and switch attention (Baddeley & Logie, 1999); correction of errors, sequencing of actions and multi-step problem-solving (Shallice & Burgess, 1991); and creation and maintenance of goal-related behavior (Burgess et al., 1998). Another aspect of the complexity of the executive control construct is that it is a domain-general function subserved by distributed networks throughout cortical and subcortical structures in the brain (Miller & Cohen, 2001; Royall et al., 2002). Thus, modality-specific demands (i.e., verbal versus visuospatial) of traditional laboratory tasks can complicate interpretation of domain-general executive control deficits (Burgess, 2006; Denckla, 2002). This has been a particular problem in the autism literature in regard to measuring executive control among those with language impairments which can complicate identification of executive control deficits (e.g., Joseph et al., 2005a; Bennetto et al., 1996).

Finally executive control related goal-setting and planning abilities, which include the execution, as well as the development, of plans (Chevignard et al., 2000), and the ability to engage and disengage actions in the service of overarching goals (Hill & Bird, 2006), are difficult to measure in the laboratory. In fact, there is a direct conflict between the parameters of clinical and laboratory assessment settings and the settings which demand higher levels of executive control. Both research and clinical assessments are usually conducted in a quiet room, with one highly supportive adult examiner prompting performance. In this structured arrangement, the examiner provides the plan, organizes the activities, gives cues regarding performance, probes for elaboration, presents tasks one at a time, and generally supports executive control (Gioia & Isquith, 2004; Bernstein & Waber, 1990). Children with autism are known to respond well to highly structured settings (Schopler et al., 1995) and explicit behavioral expectations (Lovaas, 1987). Thus, they often perform better on laboratory-based executive control measures than on executive control related tasks in more natural settings. This discrepancy has led to the investigation of ecologically valid, or “real world,” tasks to tap executive control in the settings that demand it for successful, efficient performance.

Ecological Validity: A complementary approach for assessing executive control

There are many assessment strategies that can help us address limitations in executive control measurement. First, the construction of comprehensive executive control batteries that tap multiple subdomains that can be aggregated into composite scores (e.g., the Behavior Rating Inventory of Executive Functions, (Gioia et al., 2000)) enables researchers to directly test unitary versus fractionate theories of executive control (Gioia et al., 2002c) and encourages measurement of multiple domains in any given study. Second, access to strategic plan execution is improved by techniques designed to reveal the processes involved in learning and problem-solving (e.g., Kaplan’s (1988) process approach; the microgenetic approach (Siegler & Crowley, 1991); and eye tracking techniques). It is further enhanced by the development of standardized strategy scores for neuropsychological tests (e.g., the Rey Osterrieth Developmental Scoring System (Bernstein & Waber, 1996); California Verbal Learning Test, (Delis et al., 2000)). Finally, the problem of capturing a complex process such as executive control is also addressed by an emphasis on ecological validity.

Ecological validity is increasingly valued in the evaluation of neuropsychological tests and research, especially in regard to measures of executive control. In their recent review, Chaytor and Schmitter-Edgecombe (2003) define ecological validity as the degree to which task performance corresponds to real world performance, and they argue that ecological validity does not describe a task but rather the inferences that are drawn from task performance. Following Kvavilashvili and Ellis (2004), Burgess et al. (2006) define ecological validity as a measure of the “representativeness” of the task, or the correspondence between the task and real life situations, and the “generalizability” of the task, or the degree to which task performance predicts problems in real life settings. In this paper, we will use the terminology from Franzen and Wilhem (1996), who refer to the “verisimilitude” of tasks, or their resemblance to demands in the every day environment. They further argue that the ecological validity of any neuropsychological task can be measured by its “veridicality,” or correlation with measures of everyday functioning.

The advent of brain imaging and other sophisticated diagnostic techniques has shifted neuropsychology’s role from diagnosis and lesion location to the definition of functional capacities at home, work, and school, and elevated the importance of ecologically valid measures of neuropsychological constructs (Chaytor & Schmitter-Edgecombe, 2003; Manchester et al., 2004). This has required the development of new tests, as most commonly used executive control tasks, such as the Wisconsin Card Sorting Test (Berg, 1948), were developed many years ago, when the goals of neuropsychology and the understanding of executive control were both quite different than they are today (see Jurado & Rosselli, 2007 for a review; Burgess et al., 2006). This development coincides well with the specific need for increasing the sensitivity of executive control assessment specifically (Goldberg et al., 2005). Tasks striving for verisimilitude get us out of the laboratory, at least figuratively speaking, and more readily contain the demands to integrate multi-dimensional information, determine priorities, set subgoals, and incorporate feedback that are typical of both the real world and executive control functions (Shallice & Burgess, 1991).

Verisimilitude: one approach to ecological validity

Ecologically valid executive control measures fall into two broad categories: task-based approaches or questionnaire-based approaches (see Table 2 for a list of ecologically valid executive control tasks). The most commonly used and comprehensive ecologically valid executive control task is the Behavioral Assessment of the Dysexecutive Syndrome. It is a battery of six measures, which tap planning, organization, shifting, inhibition, novel problem-solving, and temporal judgment by requiring the subject to engage in familiar activities such as searching for lost keys or planning a visit to the zoo. The Behavior Rating Inventory of Executive Function is a widely studied questionnaire that taps up to nine subdomains of executive control through self or informant (parent or teacher) report about executive control abilities in daily life.

Table 2.

A sampling of ecologically valid executive control tasks, including the associated age range and a brief description.

Name	Age Range	Description
Questionnaires
Behavior Rating Inventory of Executive Function (BRIEF) (Gioia et al., 2000)	informant report: 5–18 years self-report: 11–18 years	The BRIEF is an informant (86 items) or self-report (80 items) questionnaire that assesses everyday executive control and produces eight non-overlapping clinical scales, including Inhibit, Shift, Emotional Control, Monitor, Working Memory, Plan/Organize, Organization of Materials, and Initiate (or Task Completion in the self-report version) which can be collapsed into two broader indices, Behavior Regulation and Metacognition.
BRIEF-Preschool version (Gioia et al., 2002a)	2 years-5 years, 11 months	The BRIEF-P is a 63-item informant report of everyday executive control appropriate for preschool age children and producing five clinical scales, including Inhibit, Shift, Emotional Control, Working Memory, and Plan/Organize which can be collapsed into three broader indices, Inhibitory Self-Control, Flexibility, and Emergent Metacognition.
BRIEF-Adult version (Roth et al., 2006)	18–90 years	The BRIEF-A is an informant or self-report (75 items) questionnaire that assesses everyday executive control and produces nine non-overlapping scales, including Inhibit, Shift, Emotional Control, Self-Monitor, Working Memory, Plan/Organize, Organization of Materials, Initiate, and Task Monitor which can be collapsed into two broader indices, Behavior Regulation and Metacognition
Dysexecutive Questionnaire (DEX)* (Wilson et al., 1996)	16–87 years	The DEX is a 20-item questionnaire (in both self and informant rating forms) which provides a measure of disability associated with dysexecutive difficulties. The five item clusters are: inhibition, intentionality, executive memory, positive affect, and negative affect.
DEX for Children (Emslie et al., 2003)	7–16 years	The DEX-C is a 20-item informant report (for completion by parents or teachers), which assesses dysexecutive features across four domains: emotional/personality, motivational, behavioral, and cognitive.
Childhood Executive Functioning Inventory (CHEXI) (Thorell & Nyberg, 2008)	4–7 years	The CHEXI is an everyday informant rating of executive control attempting to avoid as much as possible the potential confound of including ADHD symptoms as item content. It is composed of 24 items from two factors, working memory/planning and inhibitory control.
Frontal Systems Behavior Scale (FrSBe) (Grace & Malloy, 2002)	18–95 years	Formerly known as the Frontal Lobe Personality Scale (FLoPS), the 46-item FrSBe comes in both informant and self-report formats and is composed of three subscales, Apathy, Disinhibition, and Executive Dysfunction.
Behavioral Flexibility Rating Scale-Revised (BFRS-R; Pituch et al., 2007; Peters-Scheffer et al., 2008)	2–19 years	The BFRS-R is designed to measure insistence on sameness or lack of behavioral flexibility in ASD. The BFRS-R is a 16-item scale that asks respondents to rate the child’s response to five situations: (1) An item is unavailable or may have been broken, moved or misplaced; (2) A desirable event or activity is interrupted; (3) The child is subjected to unexpected sensory stimulation (e.g., human contact, noise, or sounds); (4) The child fails at a task; and (5) A task is left unfinished (e.g., a sibling has left some dirty dishes in the sink).
Tasks
Multiple Errands Shopping Test (Regular and Simplified) (Shallice & Burgess, 1991; Alderman et al., 2003)	Adults	Within a shopping center, four sets of subtasks required: purchasing six items, locating and recording four items of information (e.g., closing time), meeting the examiner at a designated spot and stating the time after a designated period, and telling the examiner when s/he was finished. In the simplified version, participants are given paper and allowed to record information (though this strategy was not suggested to them) and given nine (as opposed to six) rules to follow with these rules clearly stated on the instruction sheet.
Test Taking Strategy Task (Kofman et al., 2008)	Children and Adolescents	Participants are given four minutes in which to complete more problems (360 items of indicating if two icons are the same or different by circling ‘S’ or ‘D’) than could be done in that time frame. Some problems were weighted more heavily (resulting in higher point totals) than others requiring strategic planning in approach to problem completion.
Cooking Task (Chevignard et al., 2008)	Adults	Participants are asked to make two recipes (baking a cake and making an omelet for two) while following simple rules in a complex and naturalistic environment. Execution was coded quantitatively for various types of errors and qualitatively for overall ability to achieve the goal and avoid dangerous behaviors.
Modified Six Elements Test* and Modified Six Elements Test-Children’s version	Standard version: 16–87 years	Participants work on six open-ended tasks in a 10-minute time frame while keeping in mind two rules. For the adult version, the six tasks are composed of three tasks (each with two parts): telling a story, completing arithmetic problems, and writing down names of objects presented on cards.
Standard version: (Wilson et al., 1996) Children’s version: (Siklos & Kerns, 2004)	Children’s version: 7–13 years	The child version is comprised of six separate tasks: using instructions to build a catapult from Legos®; finding and circling hidden objects within a series of pictures; using magnetic pieces from a ‘Pirate Scene’, place the pieces on a storyboard and tell a story about the created scene; complete mazes on paper of increasing difficulty; put together pieces to create a dinosaur puzzle; circle what element(s) is (are) ‘wrong’ or ‘silly’ in a series of pictures.
Temporal Judgment* (Wilson et al., 1996)	16–87 years	In this task, the participant is asked four questions to assess how well s/he estimates the duration of various events (e.g., a dental appointment).
Rule Shift Cards* (Wilson et al., 1996)	16–87 years	In the first part of this cognitive flexibility task using familiar materials, a response pattern is established according to a simple rule. The second part of the tasks involves a rule change to which participants must adapt their responses and inhibit their original response pattern.
Action Program* (Wilson et al., 1996)	16–87 years	In this problem-solving task, the participant must remove a cork from a tall tube, but this can only be achieved by planning ahead with the use of various other materials provided.
Key Search* (Wilson et al., 1996)	16–87 years	Participants demonstrate how they would search for lost keys in a field. The proposed strategy is scored for functionality and effectiveness.
Zoo Map* (Wilson et al., 1996)	16–87 years	In this planning task, a participant’s ability to plan a route for visiting six of a possible 12 locations in a zoo is evaluated. In the first part, ability to plan is assessed in a more demanding, open-ended situation with little external structure, while in the second part, the participant needs only to follow a concrete, externally imposed strategy.
BADS for Children (Emslie et al., 2003)	7–16 years	The BADS-C is comprised of five performance-based tests (in addition to the DEX-C as described above) that are downward extensions of the standard BADS subtests: the Playing Cards Test (a card sorting task), the Water Test (a five-step planning step while manipulating test materials), the Key Search Test (similar to the version described above), Zoo Map (see description above), and the Six Part Test (a simpler and shorter [five minutes] analogue of the Six Elements Test described above). The BADS-C was normed on 281 children and its manual provides information on reliability and validity.
Kochanska’s Efforful Control Tasks (snack delay, wrapped gift, and gift in bag) (Kochanska et al., 2000)	22–36 months	These tasks assess early effortful control as in failing to retrieve an easily accessible cookie for 10–30 seconds (snack delay), neither peeking (with back to the examiner) during the wrapping of a gift by an examiner (60 seconds) nor touching the wrapped gift when the examiner goes to retrieve a bow in another room (180 seconds), and not peeking into a bag with a gift inside until a bow is retrieved from another room by the examiner (180 seconds).
Tinkertoy task (Lezak, 1982)	Adults	In Lezak’s original version of this task, the participant is presented with 50 Tinkertoy pieces, and instructed to make whatever they want. Their constriction is coded for number of pieces used and a complexity score based on seven variables (e.g., number of moving parts). This is designed to measure goal setting, planning and effort to reach that goal.
Predicaments Test (Channon et al., 2001)	Adolescents and Adults	The participant is presented with eight brief video scenarios that present awkward social situations, and then asked to offer as many possible solutions as they can in two minutes, then choose their best solution and the one they themselves would be most likely to try, how satisfied the participant feels about those solutions, and then they are presented with additional possible solutions and asked to revaluate those. The participants’ responses are then scored.
Virtual Errands Test (Law et al., 2006)	Adults	The participants are asked to navigate through and complete errands in a virtual environment while also either stating “December” or naming the months randomly. This task is designed to capture the executive control and working memory demands inherent in multitasking.
Test of Everyday Attention (Robertson et al., 1994)	18–80 years	A battery of eight subtests is used to assess selective attention, sustained attention, attentional switching, and divided attention using everyday materials. An example of a task with verisimilitude is the Telephone Search task, in which participants must search in a faux telephone directory for key symbols. The TEA manual provides information on standardization with 154 typically developing adults and on validity and reliability.
Test of Everyday Attention for Children (TEA-Ch) (Manly et al., 1999)	6–16 years	A battery of nine subtests measuring different aspects of attention (selective, sustained, switching, divided, response inhibition). Two of the subtests have verisimilitude: Score DT, which requires the participant to listen to a newscast for a specific word while simultaneously keeping track of ‘scoring’ sounds that resemble video game sounds; and Map Mission, in which the participant finds as many specific symbols (e.g. a fork and knife indicating a restaurant) on a road map as possible in one minute. The TEA-Ch manual reports: norms based on 293 children, and on test- retest reliability and construct validity.
Executive Function Route- Finding Task (Boyd & Sautter, 1993)	Adults	For this planning task, the participant is told that they must find a new office in the building without help from the examiner. The examiner goes with the participant, and scores his/her: understanding of the task, information-seeking, functional memory, self-monitoring, troubleshooting ability and on-task behavior.
Behavioral Assessment Tool for Cognition and Higher Function (BATCH) (Miller et al., 2007)	Adults	This is an observational measure for inpatient settings that measures the frequency of behaviors in 10 domains: Orientation, Attention/concentration, personal responsibility, volition, adaptation, problem-solving/judgment, executive control, memory, language, and visuospatial function.

^*tasks that comprise the Behavioral Assessment of the Dysexecutive Syndrome (BADS); the BADS manual provides information on reliability and validity as well as data from standardization with 216 typically developing adults and 92 patients

Ecologically valid executive control measures vary in the amount of research supporting their validity, reliability, and specificity. Some measures provide normative data and thus have the advantage of being standardized to neurotypical individuals (the Behavioral Assessment of the Dysexecutive Syndrome, the Behavior Rating Inventory of Executive Function, the Dysexecutive Questionnaire, the Frontal Systems Behavior Scale, and Tests of Everyday Attention). Moreover, the Behavioral Assessment of the Dysexecutive Syndrome, the Behavior Rating Inventory of Executive Function, and the Tests of Everyday Attention all have child versions that provide developmental norms for studying ecologically valid executive control and enable the researcher to investigate most of the same executive control domains across much of the life span. This does not discount the potential contribution of other measures such as the Multiple Errands Shopping Test (Shallice & Burgess, 1991; Alderman et al., 2003), the Cooking Task (Chevignard et al., 2008), or the Test Taking Strategy Task (Kofman et al., 2008); however, additional research is needed to validate and standardize these measures.

Research on the validity of the ecologically valid executive control measures, such as the Behavioral Assessment of the Dysexecutive Syndrome, comes with its own set of criticisms. The Behavioral Assessment of the Dysexecutive Syndrome, like some traditional executive control tasks such as the Wisconsin Card Sorting Test, was developed to discriminate between neurotypical adults and adults with frontal lobe brain lesions (Manchester et al., 2004; Jurado & Rosselli, 2007). This logic assumes that all adults with frontal lobe lesions will exhibit executive control deficits, and will therefore perform poorly on executive control measures. However, the Behavioral Assessment of the Dysexecutive Syndrome, like the Wisconsin Card Sorting Test (Shallice & Burgess, 1991) has not always successfully discriminated between adults with frontal lobe lesions and neurotypical adults (Wood & Liossi, 2006). The validity of the Behavioral Assessment of the Dysexecutive Syndrome improves when it is combined with a secondary source of information, such as an informant-based questionnaire like the Dysexecutive Questionnaire (Dywan & Segalowitz, 1996). For example, if a relative completing the Dysexecutive Questionnaire identifies cognitive flexibility as an impaired executive control process for the patient, then the patient is more likely to exhibit deficits on the shifting test from the Behavioral Assessment of the Dysexecutive Syndrome (Adlerman et al., 2003). This type of convergent evidence may appear as a tautology, but it does highlight the need for observing executive control functioning from multiple sources. While many previous studies in autism use multiple lab measures that tap the same construct (Christ et al., 2007; Geurts et al., 2004; Happé et al., 2006; Kenworthy et al., 2005; Yerys et al., 2007), research efforts may be better served by combining lab or ecologically valid executive control measures with standardized informant reports, such as the Dysexecutive Questionnaire, the Frontal Systems Behavior Rating Scale, or Behavior Rating Inventory of Executive Function.

Another problem for ecologically valid executive control tasks is that despite their verisimilitude, or their resemblance to demands of the everyday environment, they sometimes lack veridicality, that is, they do not actually correlate with measures of everyday executive control functioning. Again the Behavioral Assessment of the Dysexecutive Syndrome is the best studied in this regard, with mixed findings. Wilson and colleagues (1998) report significant correlations between the Behavioral Assessment of the Dysexecutive Syndrome and the Dysexecutive Questionnaire, and Norris and Tate (2000) partially replicate these findings. In a study of four Behavioral Assessment of the Dysexecutive Syndrome subtests, Wood and Liossi (2006) found limited ecological validity in patients with severe head trauma, and Evans et al. (1997) report significant Behavioral Assessment of the Dysexecutive Syndrome-Dysexecutive Questionnaire relationships in patients with neurologic impairments, but not those with schizophrenia. Overall, Chaytor and Schmitter-Edgecombe (2003) conclude that there is some evidence of superior veridicality in tasks with verisimilitude as opposed to standard laboratory tasks, but clearly the association cannot be assumed. Further research regarding the relatively new ecologically valid executive control tasks described in Table 2 should address this issue directly.

Ecologically valid executive control questionnaires have a further liability. The validity of self-report for executive control function has been called into question (see Chaytor & Schmitter-Edgecombe, 2003 for a review of this issue), although Obhuda et al. (2005) report accurate self-report from one brain-injured group. Given common problems with self-awareness and self-monitoring in individuals with executive control impairments (Gioia et al., 2000), the consensus is that it is important to obtain informant report on executive control related questionnaires. A related concern is the influence of bias (in either direction) on the part of any informant.

Finally, by virtue of their reliance on everyday activities, ecologically valid executive control tasks frequently tap multiple abilities (e.g., language, processing/motor output speed), and can easily confound domain-general executive control functions with domain-specific functions (Jurado & Rosselli, 2007; Burgess, 1997). This confound may be unavoidable in the executive control domain. As described above, the traditional executive control tasks that are most sensitive in autism tap multiple processes simultaneously. From a neurodevelopmental perspective, Bernstein and Waber (2007) argue that what we call executive control relies not on functional modules but on functional neural networks that develop in the context of experience. This observation appears particularly relevant to autism, which has defied modular explanation at the genetic, neuroanatomical, neurofunctional, and behavioral levels, and is increasingly understood as a disorder of distributed networks in the brain (Müller, 2007). In any case, ecologically valid executive control tasks, even more than their traditional counterparts, must be interpreted cautiously lest we prematurely specify executive control as the culprit in deficient performance on a multi-dimensional task. In autism the problem is most acute when the task occurs in a social context. As noted above, findings of deficient executive control in autism on both the Wisconsin Card Sorting Test and tower tasks are attenuated with computer administration. These cautionary statements apply equally to executive control questionnaires, which are inherently tapping multiply determined behaviors, as they record observations in real life, uncontrolled settings.

Autism performance on new tasks claiming verisimilitude

A review of investigations in autism with tasks that claim ecological validity through verisimilitude reveals only 12 studies (see Table 3). To date, most independent studies of ecologically valid executive control measures in autism have used either the Behavioral Assessment of the Dysexecutive Syndrome or the Behavior Rating Inventory of Executive Function. Four studies using the Behavioral Assessment of the Dysexecutive Syndrome reveal mixed findings. Although Hill and Bird (2006) found executive control deficits on the Behavioral Assessment of the Dysexecutive Syndrome in a group of high-functioning adults, Rajendren and colleagues (2005) and Harris and colleagues (2008) failed to find deficits on the Behavioral Assessment of the Dysexecutive Syndrome in adolescents with autism, and Boucher and colleagues (2005) found intact performance on the Zoo Map subtest of the Behavioral Assessment of the Dysexecutive Syndrome in young adults with autism. Results from the Predicaments Test (Channon et al., 2001), another ecologically valid executive control task, have been reported in a study of adolescents with autism. Adolescents with autism were impaired compared to typically developing controls; however, this task entails significant social demands, which cannot be disentangled from its executive control aspects. It requires the participant to generate multiple possible socially appropriate solutions to awkward social situations.

Table 3.

A list of ecologically valid executive control tasks used in adolescents and adults with autism.

Reference	Groups/Ages (years)	Matching Criteria	EV-EC Task	Impairment?* (TYP/Others)
Channon et al. (2001)	ASD: 13.9	Age, NVMA, Receptive and Expressive Language	PPST	Y
	TYP: 14.4		DEX	Y
Gioia et al. (2002b)	ASD: 10.8	Individual matching between TYP group and six clinical groups on Age, Gender, and SES	BRIEF	Y – unique deficits on Shifting subscale
	TYP: 10.9
	RD: 8.8
	ADHD-I: 8.8
	ADHD-C: 8.9
	Moderate TBI: 14.7
	Severe TBI: 14.9
Gilotty et al. (2002)	ASD: 10.5	Normative Data	BRIEF	Y
Boucher et al. (2005)	ASD: 23.8	Age, VIQ, PIQ, Gender	Brixton test	N
	TYP: 24.1		Hayling test	Y
			BADS – Zoo Map	N
Kenworthy et al. (2005)	ASD: 10.3	Normative Data	BRIEF	Y
Rajendren et al. (2005)	ASD: 16.5	Age, VIQ, PIQ, FSIQ	BADS	N
	TYP: 16.8
Green et al. (2006)	ASD: not recorded	-------------------	BFRS	Y/Y/Y^
	AnS: not recorded
	DS: not recorded
Hill & Bird (2006)	ASD: 31.1	Age, FSIQ, Gender	BADS	Y
	TYP: 33.5		Hayling test	Y
Mackinlay et al. (2006)	ASD: 12.0	Age	BMP	Mixed
	TYP: 11.92		BRIEF	Y – to norms
Winsler et al. (2007)	ASD: 11.0	Age	BRIEF
	TYP: 10.3
	ADHD: 11.6
Didden et al. (2008)	ASD: not recorded	-------------------	BFRS-R	Y/Y/Y^
	AnS: 17.4
	DS: not recorded
	DD: 13.9
Harris et al. (2008)	Autism+LD: 15.9	FSIQ	BADS	Mixed/Mixed
	PDD+LD: 16.4
	LD/DD: 15.3

ADHD= Attention Deficit Hyperactivity Disorder; ASD= Autism Spectrum Disorder; AnS= Angelman Syndrome; Autism+LD= Autism with co-morbid learning disorder; BADS= Behavioral Assessment of the Dysexecutive Syndrome; BFRS= Behavior Flexibility Rating Scale; BFRS-R= Behavior Flexibility Rating Scale–Revised; BMP= Battersea Multitask Paradigm; BRIEF= Behavioral Rating Inventory for Executive Function; DEX= Dysexecutive Questionnaire; DS= Down Syndrome; PDD+LD= Pervasive Developmental Disorder–Not Otherwise Specified with learning disorder; PPST= Predicaments Problem-Solving Task

^*This column refers to whether there is an impairment reported for the ASD group relative to the comparison group. The possibilities are: Y= Yes differences are reported for particular domain/EF test; N= No differences are reported for particular domain/EF test; Mixed= Data did not produce a clear result.

^{^}First comparison is between ASD and AnS, second comparison is between ASD and DS, and third comparison is between ASD and DD (e.g., Y/Y/Y means ASD group exhibited unique deficits relative to all groups).

^{^^}First comparison is between ASD and TYP and second comparison is between ASD and ADHD.

In contrast to real life tasks, Table 3 shows that investigations of questionnaires filled out by primary caretakers provide convergent evidence of unique executive control impairments in children and adolescents with autism. Investigations using the Behavior Rating Inventory of Executive Function revealed executive control deficits in autism when compared to: normative data (Gilotty et al., 2002; Kenworthy et al., 2005; Mackinlay et al., 2006); typical controls (Gioia et al., 2002b; Winsler et al., 2007); and children with ADHD (Gioia et al., 2002b; Winsler et al., 2007). Comparing Behavior Rating Inventory of Executive Function profiles across groups of children with different developmental and acquired disorders, revealed unique deficits in flexibility for children with autism (Gioia et al., 2002b). Other measures, such as the Dysexecutive Questionnaire and the Behavioral Flexibility Rating Scale – Revised (Peters-Scheffer et al., 2008) have also identified difficulties in children with autism (Channon et al., 2001; Didden et al., 2008). While the Behavioral Flexibility Rating Scale–Revised is focused solely on children’s ability to shift from one activity to another, the Dysexecutive Questionnaire assesses a broader set of symptoms associated with the dysexecutive syndrome in adults.

In summary, with the exception of the Behavioral Assessment of the Dysexecutive Syndrome, the use of ecologically valid executive control tasks and questionnaires has yielded convergent findings that children and adults with autism struggle on unstructured tasks (or are rated as such by parents) that they encounter in their daily lives. However, the potential confound of social demands on these measures suggests that more work with other ecologically valid executive control tasks is needed to understand whether social deficits are the driving factor in identifying more ecologically valid deficits in autism.

Veridicality: Another approach to ecological validity

Another approach to improving the ecological validity of executive control assessment in autism is to assess the veridicality of standard executive control tests and favor those tests which show a positive relationship with important everyday outcome variables (Chaytor & Schmitter-Edgecombe, 2003). As the experience with the Behavioral Assessment of the Dysexecutive Syndrome demonstrates, task veridicality can be population specific; thus, it should be established specifically for autism groups of different functioning levels across different ages. A measure’s veridicality is also influenced by the everyday outcome variable selected. Some investigations of veridicality have used general measures, such as job performance (Kibby et al., 1998) or adaptive behavior (Gilotty et al., 2002), while others use measures of behavior more specifically related to executive control, such as the Dysexecutive Questionnaire (Burgess et al., 1998) and the Behavior Rating Inventory of Executive Function (Mackinlay et al., 2006). Burgess and colleagues (1998) present evidence that in adult neurological patients executive control fractionates at the behavioral as well as at the cognitive level. He argues therefore for use of a nuanced executive control related measure such as the Dysexecutive Questionnaire when testing veridicality. The findings of Mackinlay and colleagues (2006) indicating correlations between some subdomains of the Behavior Rating Inventory of Executive Function, but not others, with a laboratory measure of multi-tasking, in a study of autism is consistent with Burgess and colleagues’ (1998) argument. At this point, however, data on the veridicality of specific tasks in any population, not to mention autism, is so sparse that conclusions are premature.

There has been more extensive, although still inconclusive, work in autism assessing the relationship between performance on traditional executive control tasks and number of symptoms. Significant correlations have been reported between autism symptoms and executive control measures, including verbal fluency (Bishop & Norbury, 2005), the CANTAB Intradimensional/Extradimenesional shift task (Yerys et al., in press); a composite score of the Wisconsin Card Sorting Test and Trail Making test (Lopez et al., 2005) and the Wisconsin Card Sorting Test (South et al., 2007).

Where do we go from here? Summary of findings and suggested future directions in autism executive control research

Several conclusions can be drawn from the literature reviewed here. There are robust deficits in autism groups on two multi-dimensional traditional executive control tasks, the Wisconsin Card Sorting Test and tower tests. However, when the human examiner is replaced with computer administration, the magnitude of these findings erodes, indicating perhaps that it is the socially mediated response to feedback and adherence to socially presented arbitrary rules that is difficult for individuals with autism, as opposed to the executive control flexibility and planning processes that are usually associated with the Wisconsin Card Sorting Test and tower tasks. Recent investigations with traditional executive control measures also indicate that spatial working memory tests with strategic task components are consistently difficult for individuals with autism, whether administration is computerized or not.

Confidence in the definitiveness of these findings is limited by heterogeneity of cognitive and behavioral presentation within the autism spectrum and many problems inherent in the measurement of executive control in the laboratory. Regarding the former, one future approach involves studies with large sample sizes that allow for the explicit assessment of the role that intelligence, psychiatric co-morbidities (such as attention deficit hyperactive disorder), and other factors (such as language) play in executive control profiles in autism. Some of the measurement problems in executive control reflect its complex and prolonged developmental trajectory, which can be addressed only through longitudinal studies of executive control in autism, or at the least, consistent use of multiple tasks tapping different aspects of executive control across the life span, with a sensitivity to the typical developmental trajectories of specific component processes of executive control. Another difficulty involves the multifaceted character of executive control, and the confusion which arises when investigations report contradictory findings that are difficult to interpret because different aspects of executive control were measured. The development of comprehensive executive control batteries, such as the Behavioral Assessment of the Dysexecutive Syndrome, which allow interpretation of executive control as both a unitary and fractionated phenomenon, is one potential response to this difficulty. Use of comprehensive batteries across age bands and populations may resolve some of the present confusion in the literature.

Other challenges include the difficulty of capturing complex goal-setting/plan execution abilities in the highly structured laboratory setting. Process sensitive techniques, such as eye tracking and standardized strategy scores, can capture some aspects of these complex functions. In addition, fMRI and other neuroimaging techniques, whose overall contribution to understanding executive control in autism is beyond the scope of this paper, may elucidate issues surrounding goal setting/plan execution abilities in several ways. On the face of it, confining a subject to a narrow, noisy metal tube in which he or she cannot move would appear to be antithetical to tapping real world executive control abilities, but the advent of virtual reality technology improves the capacity of researchers to simulate daily living experiences. By revealing the neurobiological correlates of performance, neuroimaging can demonstrate the extent to which goal/plan tasks depend on the integration of multiple abilities, in the executive control domain or in other neuropsychological domains. This may restrain our tendency to prematurely draw overly precise conclusions about what abilities can be inferred from performance on complex tasks such as the tower. Neuroimaging has also demonstrated that even when the performance of an autism group is intact on a unitary executive control task, such as the inhibitory Go/No-Go test, atypical neural networks are recruited (Kana et al., 2007; Schmitz et al., 2006). This may be useful in both clarifying conflicting findings on inhibition tasks and elucidating the breakdown of performance on complex executive control tasks that require several abilities (e.g., inhibitory control and working memory) combined.

Finally, the development of a new generation of ecologically valid executive control tasks designed to mimic real life problem-solving is encouraging. Vigilance regarding the social and language demands that may be inherent in complex executive control tasks striving for verisimilitude is warranted, however. Disentangling impairments due to reduced social motivation and impaired social cognition from truly deficient executive control abilities on purported measures of executive control is a major challenge for the coming years. In this regard the computerized virtual reality tasks, such as the Virtual Errands Test (Law et al., 2006), are interesting, because they provide a modality for assessing real life goal setting and plan execution without imposing social requirements. Although not computerized, the Battersea Multitasking Paradigm (Mackinlay et al., 2006) was developed specifically for assessment in autism, and it was introduced in a paper that offers a useful model for future investigation of executive control in autism. MacKinaly et al. (2006) combined a rigorously designed laboratory measure of executive control with a comprehensive executive control informant questionnaire, which provides both an ecologically valid measure of executive control and a measure of the veridicality of the task findings. Such inclusive use of the multiple tools that are now available to query executive control function in autism will improve in our understanding of this complex neuropsychological construct in this equally complex population. In the last 30 years almost 150 peer reviewed articles on executive control in autism have been published, yet we are aware of only one peer reviewed report on an intervention to address executive control in autism (Fisher & Happé, 2005). It is our hope that improved understanding and measurement of executive control and its impact on daily functioning will encourage the further development of innovative interventions that target executive control, and enhance the everyday lives and independence of individuals with autism.