The paradox of cognitive flexibility in autism

Abstract

We present an overview of current literature addressing cognitive flexibility in autism spectrum disorders. Based on recent studies at multiple sites, using diverse methods and participants of different autism subtypes, ages and cognitive levels, no consistent evidence for cognitive flexibility deficits was found. Researchers and clinicians assume that inflexible everyday behaviors in autism are directly related to cognitive flexibility deficits as assessed by clinical and experimental measures. However, there is a large gap between the day-to-day behavioral flexibility and that measured with these cognitive flexibility tasks. To advance the field, experimental measures must evolve to reflect mechanistic models of flexibility deficits. Moreover, ecologically valid measures are required to be able to resolve the paradox between cognitive and behavioral inflexibility.

Focus of this review

Both clinicians and researchers widely believe that cognitive flexibility deficits are pathognomonic of autism spectrum disorders. Here, we question this belief. We address why this is important, why cognitive flexibility deficits are considered central to autism spectrum disorders (ASD) and why we are skeptical.

Why is this important?

Autism spectrum disorders, including autistic disorder, high functioning autism (HFA), Asperger syndrome and pervasive developmental disorder not otherwise specified (PDDNOS), are neurodevelopmental disorders involving social and communication impairments combined with restricted, stereotypical patterns of behavior and interests [1,2]. We employ the term autism to refer collectively to these disorders. Clinical observation indicates that pervasive cognitive and behavioral rigidity across functional domains is diagnostic of autism.

An influential cognitive theory of autism [3,4] purports that symptoms arise from executive function deficits (i.e. cognitive control – see Glossary). One component of executive function is cognitive flexibility, which refers to the ability to shift to different thoughts or actions depending on situational demands [5]. So far, none of the other prominent cognitive autism theories gives an explanation for the observed inflexibility in autism. When focusing on everyday behavior it seems that individuals with autism have cognitive flexibility deficits. They encounter difficulties in changing strategy during daily activities or adapting their perspective during social interactions. The idea is that cognitive flexibility deficits are clearly related to this observed rigidity in behavior. Despite the strong face-validity of this relationship (Table 1), it has proven difficult to clearly articulate the links between them [6,7], although some studies could correlate performance on executive functions (EF) tasks with autism characteristics [8–10]. To date, the obtained correlations seem to be aspecific because other executive functions like working memory and inhibition also seem to relate to autism characteristics [9]. The difficulties in gaining insight regarding the link between task performance and autism characteristics might be because of measurement problems (which is the focus of the current review) or to the heterogeneity of the autism spectrum because there are substantial individual differences in the type of difficulties individuals with autism experience.

Table 1.

Autism symptoms and the potential relationship with cognitive flexibility

DSM-IV-TR symptoms of autism per domain	Potential relationship with cognitive flexibility
(a) Qualitative impairment in social interaction
(i) Marked impairments in the use of multiple nonverbal behaviors such as eye-to-eye gaze, facial expression, body posture and gestures to regulate social interaction.	Inability to shift visual attention from eyes to mouth, from one speaker to another speaker.
(ii) Failure to develop peer relationships appropriate to developmental level.	Inflexible in application of social rules (social rigidity). Inability to shift social behavior or conversational topics to meet the changing contextual demands.
(iii) A lack of spontaneous seeking to share enjoyment, interests or achievements with other people (e.g. by a lack of showing, bringing or pointing out objects of interest to other people).	Inability to shift attention to extra-personal space. Difficulty in shifting to another person’s perspective.
(iv) Lack of social or emotional reciprocity (e.g. not actively participating in simple social play or games, preferring solitary activities or involving others in activities only as tools or ‘mechanical’ aids).	Inability to shift attention to extra-personal space. Difficulty in shifting to another person’s perspective.
(b) Qualitative impairments in communication
(i) Delay in, or total lack of, the development of spoken language (not accompanied by an attempt to compensate through alternative modes of communication such as gesture or mime).	Inability to flexibly combine language elements into fluent language. Lack of broadening of complexity level of language.
(ii) In individuals with adequate speech, marked impairment in the ability to initiate or sustain a conversation with others.	Inability to shift to another person’s perspective. Talking about topics of own interests (i.e. inability to shift to other topics) and not knowing when to stop (i.e. perseverating).
(iii) Stereotyped and repetitive use of language or idiosyncratic language.	Perseveration on one specific meaning of words. Impaired flexibility of thought to interpret words in an alternative way. Repetition of words and sentences. Inflexible use of language.
(iv) Lack of varied, spontaneous make-believe play or social imitative play appropriate to developmental level	Perseveration on one type of activity (i.e. inability to shift to different, pretend or unreal view of the world).
(c) Restricted repetitive and stereotyped patterns of behavior, interests and activities
(i) Encompassing preoccupation with one or more stereotyped and restricted patterns of interest that is abnormal either in intensity or focus	Perseveration on a specific topic; cannot move away from one interest, overly focused on one specific aspect.
(ii) Apparently inflexible adherence to specific, nonfunctional routines or rituals	Insistence on routines and rituals.
(iii) Stereotyped and repetitive motor mannerisms (e.g. hand or finger flapping or twisting, or complex whole body movements)	Perseveration expressed in motor movements.
(iv) Persistent preoccupation with parts of objects	Difficulties in shifting attention, disengaging attention from details (i.e. hyperfocus).

Note: Georgiades and colleagues [58] showed that the three categorical DSM-IV ASD domains, social relationships, communication and restrictive repetitive and stereotyped behavior are very heterogeneous. For example, communication includes behavior that regulates social interaction, but also includes flexible use of language. In addition, repetitive behavior consists of both repetitive stereotyped movements and inflexible behavior. They suggested three new factors, (i) social communication, (ii) inflexible language and behavior and (iii) repetitive sensory and motor behavior. Especially the last two might be related to inflexibility, respectively to cognitive and to motor inflexibility.

To understand observed behavioral problems and ultimately to provide targeted treatments for individuals with autism, we need to decompose the behavior into measurable cognitive processes. Therefore, the purpose of the current selective review is to take an experimental and neuroscientific view to determine whether cognitive flexibility deficits are central to autism. Here, we review several recent studies (the main features of these studies appear in Table S1 in the supplementary material), which have employed widely used cognitive flexibility tasks (Table 2). We do not discuss cognitive flexibility in preschoolers because this literature has been reviewed recently by Russo and colleagues [11].

Table 2.

Descriptions of clinical neuropsychological cognitive flexibility tasks, involved neural networks (see for other external validity measures Willcutt [57]), number of included studies and participants and the effect size (Cohen’s d)

Task	Description	Main dependent measures	Potentially involved neural networks [59–65]	Number of studies	Total N	Range effect size (pooled effect sizee)
WCST	A sorting task that requires participants to determine how to sort cards on the basis of unknown categories (color, form and number). The participants need to infer the sorting rule based on the given feedback. Without notice to the participant, the sorting rule changes following a met criterion and the participant must inhibit (i.e. suppress) the previous sorting rule and subsequently discover the new sorting rule.	Number or % pers answers Number or % pers errors Number of categories	Left and right inferior frontal and DLPFC, parietal cortex, premotor area, ACC and cerebellum	12	ASD=520 TYP=479	0.25–1.01 (0.64)
MCST	Similar to WCST, but with less cards and a warning is given when sorting rule changes.	Number of errors	No studies available	2	ASD=42 TYP=45	0.05 (for only 1 study data reported)
CANTAB®a ID/ED	Shifting task that requires rule acquisition and reversal. The sorting rule can change within one dimension (ID shift) or across different dimensions (ED shift). For details see Box 1 and Figure 1 in main text.	Number of trials to criterion Number of errors to criterion	ID: orbito frontal cortex (OFC) and striatal functioning ED: DLPFC and OFC	6	ASD=184 TYP=180	0.02–1.00 (0.35)
TMTb	Timed task that requires the participant to connect a series of letters and numbers in ascending order while alternating between numbers and letters (second part, B). In the first part (A) only numbers need to be connected and no letters are presented.	Time B–Time A Ratio Time B: Time A	DLPFC, supplemental motor areas and dorsal ACC	5	ASD=281 TYP=246	0.5–1.32 (0.89)
D-KEFSc-TMT	This task consists of five conditions that assess visual-motor sequencing, visual scanning, number-letter switching and motor speed. The number-letter switching task requires participants to alternate between connecting numbers and letters.	Time switch condition	Lateral PFC	1	ASD=17 TYP=17	0.56
BADSd Rule shift Cards	To test the ability to shift from one rule to another (first rule is to respond to the color of the shown card and the second rule is to respond to the color of the previous card in comparison to the current card) and to keep track of the color of the previous card and the current rule.	Number of errors	No studies available	1	ASD=22 TYP=22	0.64
D-KEFS Color-Word	A Stroop like task with a fourth condition in which the interference condition is repeated, only now half of the stimulus words are encased in a box. The participant names the dissonant ink color except for the boxed words, in which case the participant must switch sets and read the word itself (and not name the dissonant ink color). Hence, the participants need to switch between four different rules.	Time switch condition Number of errors switch condition	No studies available	2	ASD=35 TYP=35	0.52–1.30 (0.92)

Note: the task-switching paradigm is described in the main text.

^aCANTAB^®, Cambridge Neuropsychological Test Automated Battery.

^bChildren’s version is called the children’s color trail test (CCTT) and uses colors and numbers instead of letters and numbers.

^cD-KEFS, Dellis-Kaplan executive function system.

^dBADS, behavioral assessment of the dysexecutive syndrome.

^eThe studies often differed in the reported dependent measures. This implies that for most tasks a pooled effect was calculated across different measures. This means that the pooled effect size needs to be interpreted with caution. In calculating the pooled effect size, we incorporated the number of participants for each of the effect sizes like Wilcutt [57]. The reported effect sizes are potentially biased because some studies partly included the same participants. Hence, the same participants are sometimes included twice (see Table S1 in supplementary material). Please note that we could not calculate the pooled effect size for the task-switching paradigms as the paradigms were very different from each other and not all studies reported the mean scores and standard deviations. The number of participants in the four task-switch studies was 66 ASD and 71 TYP.

Why are such cognitive flexibility deficits considered central to autism?

First, the cognitive flexibility construct seems to map easily onto the observed behavior (Table 1). Parents and clinicians alike will see inflexibility as one of the most troubling, consistent and difficult-to-intervene characteristics of the disorder. Second, many cognitive flexibility autism studies using clinical neuropsychological measures indicate that there are cognitive flexibility deficits. However, we question that failure on these measures is indeed because of cognitive inflexibility because performance on these measures draws upon a broad range of cognitive processes.

The majority of the studies that reported cognitive flexibility deficits in autism included a clinical neuropsychological measure, the wisconsin card sorting task (WCST, Table 2). Willcutt and colleagues [12] demonstrated, in their meta-analysis, a large effect size (Cohen’s d between 1.0 and 1.5) for the difference in ‘cognitive flexibility’ between individuals with autism compared to typically developing groups. Recent studies using this measure in autism also demonstrate deficits in changing sorting strategy across populations of various autism subtypes, ages and cognitive levels ([9,13–22], but see Ref. [23]). Hence, we do not dispute that people with autism encounter difficulties when performing on this task. The question is whether these difficulties are indeed related to cognitive inflexibility.

Failure on the Wisconsin Card Sorting Task can be because of deficits in various cognitive processes including learning from feedback, keeping the goal of the task in mind, noticing that a change in strategy is necessary, inhibiting a previous motor response, switching to another response and sustaining responding over time. None of these processes can be distinguished by the traditional task scoring system [24–26]. Hence, we cannot determine why individuals with autism fail on this task.

Illustrative in this respect is that two studies that used an adjusted version of this task, the modified card sorting task (MCST) could not differentiate between adults with autism (low [27] and high functioning [8]) and typically developing adults. The MCST includes a warning that the sorting rule needs to be changed, however this does not ensure the participants can do so or ascertain the new sorting rule [28]. The findings indicate that being provided with knowledge of a change facilitates performance in individuals with autism. However, also in this modified task, multiple cognitive processes come into play.

Studies using another clinical neuropsychological measure, the trail making test (TMT), report mixed conclusions [8,9,15–17,29]. This task consists of two parts. In the first part, a series of numbers must be located and connected in ascending order, whereas in the second part a series of both numbers and letters need to be connected in ascending order while alternating between these two categories. Interpretation of results of this test involves examining differences in completion time between the two parts of the task (Table 2). Two studies showed that children and adults with autism had no difficulties on the TMT [9,29]. By contrast, four studies reported that participants with autism did have difficulties with this task [8,15–17], but it is not clear whether this was because of cognitive inflexibility because only times for each part of the task, as opposed to difference scores, were reported. Arbuthnott and Frank [30] showed that a ratio score of the first part of the task in relation to the second part that is higher than three is indicative of cognitive flexibility difficulties. If we calculate this ratio (autism groups 2.0 [15,16] or 2.1 [8], schizophrenia groups >3 [16]), none of the studies actually showed evidence for difficulties with cognitive flexibility in autism.

Two other recent studies [9,29] included a hybrid clinical neuropsychological/experimental measure, the Dellis-Kaplan executive function system (D-KEFS) color word task [31]. One study of adults with autism reported no cognitive flexibility deficits [9], but another study including children with autism reported deficits [29]. Because a part of this task requires switching across four conditions, whereas most tasks involved switching between fewer rules, the task might be more difficult. The D-KEFS probably has greater discriminating power than the other tasks, but would also be susceptible to confounds related to generalized deficits. Despite clear rules presented throughout the task, color-word switching in this task might be too difficult for children with autism. These findings also indicate that in interpreting cognitive flexibility studies, developmental factors need to be taken into account [11,32].

In summary, with respect to clinical neuropsychological measures, only studies using the WCST clearly report deficits (i.e. has high discriminating power), whereas the findings of studies using other measures are inconsistent, but generally do not support these findings.

Why the skepticism?

First, as described in the previous paragraph, there are serious issues with the WCST that makes it impossible to conclude that failure is indeed because of cognitive inflexibility, although failure might be because of generalized performance deficits (i.e. executive functioning deficits). Second, as described later, studies in autism using experimental cognitive psychology measures that are developed to examine the function of specific cognitive systems, often do not report cognitive flexibility deficits in autism.

An increasing number of studies have applied a hybrid neuropsychological/experimental cognitive flexibility paradigm, the intra-dimensional/extra-dimensional shift task (ID/ED; Box 1 and Figure 1 [33]). This task consists of shifts within one dimension (ID) and between different dimensions (ED). As in the WCST, participants are not provided with a warning about when a shift will occur, but are warned (and can be reminded later on) that change of sorting rules will occur during the task. Similar to the MCST, the participants do not know what the sorting rule will be. However, because of the stepwise task design (Box 1) it can be assumed that failure on the ID or ED-shift cannot be attributed to impairments in learning from feedback, maintaining a response over time or keeping a future goal in mind because this has been tested at the beginning of the task. Nonetheless, there are two potential confounds in this task. First, participants failing to reach the final stages of the task are automatically given the highest error score, which might obscure findings (Box 1). Sustaining attention difficulties, as opposed to cognitive flexibility, can decrease performance on this task, which is the second potential confound.

Box 1. Description of the Intra Dimensional–Extra Dimensional shift task.

The Intradimensional–Extradimensional Shift task (ID/ED) from the CANTAB^® [33], has been broadly used in several research domains as a measure of cognitive flexibility and consists of colored shapes and white lines that increase in complexity across nine different levels (see Figure 1 in main text). The first five levels determine whether the participant is able to discriminate between stimuli and benefit from feedback because at these levels, the participant is presented with a series of multidimensional stimuli and the participant must learn to respond selectively to one specific shape while ignoring the other shapes and lines. The next four levels are the crucial levels and the primary variables of the ID/ED task are the number of trials to achieve criterion and the number of errors committed for level 6 through to level 9. At level 6, the ID-shift, new shapes and lines are introduced, but the participant needs to keep responding to shape. At level 7, the ID-reversal, the previously ignored shape now becomes the correct target. At level 8, during the ED-shift, the correct rule now changes to the lines instead of the shapes. At level 9, the ED-reversal, the participant must respond to the previously ignored line.

The ID/ED task seems to be a more specific measure of cognitive control than the WCST because distinctions can be made between relevant cognitive processes. Furthermore, monkey studies [66] indicate that different regions of the PFC are recruited in the ID and ED-shifts (see Table 2 in main text). The ED-shift is regarded as a more demanding shift than an ID-shift because it included a perceptual shift in addition to a shift from one dimension to another (i.e. shifting cognitive set). Despite advantages of the ID/ED task compared to the WCST, there are also some difficulties with this task.

First, because of the step-wise design, the ED-shift always appears at the end. When participants fail on earlier levels of the task, the ED level will not be administrated. Subsequently, the scoring protocol of the task requires that 25 errors must be added to the total score. However, because many of the participants never reach the crucial level, it cannot be unequivocally determined whether individuals with ASD demonstrate problems with cognitive flexibility. Second, to our knowledge the sequence of trial types involved in the ID/ED is not based on a validated neural network model of cognitive control.

Figure 1.

ID/ED task of the Cambridge Neuropsychological Test Automated Battery (CANTAB^®). The correct choice for each stage is marked with a green box. During the initial ID discrimination stages, the participants learn via trial and error to respond selectively to a specific shape while simultaneously ignoring another shape and lines. When criterion is reached (six correct responses), the next level requires the participant to reverse from a previously rewarded response to a new choice (ID reversal learning). During stages (vi) and (vii) (ID-shift), new shapes and lines are presented, but shape is still the relevant response dimension. The crucial ED-shift occurs at stage (viii), as the correct rule changes to the ‘extra’ dimension as the lines become the relevant response dimension. During stage (xi), the participant must reverse to the previously non-reinforced line (ED-reversal). Reproduced, with permission, from (CANTAB), © Copyright 2008 Cambridge Cognition, Ltd, Cambridge, UK), Ref [33].

The majority of recent studies using this task (including participants of various ages and cognitive levels) could not differentiate between individuals with autism and typically developing individuals [29,32,34,35]. However, Ozonoff and colleagues [36] reported difficulties with ED-shifts and not with ID-shifts in the largest autism study to date, which included a very well-characterized national sample of children with high functioning autism and Asperger syndrome. By contrast, Landa and colleagues [7] reported that children with high functioning autism had difficulties with ID-shifts, but outperformed typically developing children on ED-shifts. Some argue that attention difficulties are central to autism [37], and there is a striking co-occurrence of autism with attention deficit hyperactivity disorder (ADHD) [38,39]. So, the reported difficulties with this task [7,36] might be because of the presence of comorbid ADHD in the participants with autism.

Recent studies reporting a lack of difficulty with the ID/ED task [29,32,34,35] all included direct comparisons of autism to both typically developing individuals and individuals with ADHD. In these studies, neither the children with autism nor the children with ADHD failed the task (see also Box 2). The findings of Sinzig and colleagues [35] suggest that children with autism with comorbid ADHD show difficulties with the task, whereas children with autism without any comorbid ADHD do not. However, other studies which included children with autism that had comorbid symptoms of ADHD [29,34] did not show a group difference on the ID/ED task. Thus, it remains unclear whether comorbidity of ADHD in autism is a valid explanation for mixed findings (Box 3).

Box 2. ADHD and cognitive flexibility.

It is well established that people with ADHD encounter various severe cognitive control difficulties [57], but cognitive flexibility does not seem to be a key deficit [4,12,26,57,67]. However, according to a recent meta-analysis of Willcutt and colleagues [12], people with ADHD do experience mild difficulties in this domain (effect size [Cohen’s d] was .01–0.3, which is small). Hence, both ASD and ADHD have been associated with cognitive control difficulties, but ostensibly diverge when we focus on cognitive flexibility.

In direct comparisons between ADHD and ASD on the WCST, the findings are inconsistent. ADHD children showed difficulties with the WCST in two studies [20,68], but not in two other studies [14,69]. Difficulties with this task in ADHD might arise from the presence of comorbid ASD as the two studies that did not report problems on the WCST for the ADHD group were those that carefully excluded children with ASD characteristics. However, the discrepancies in cognitive flexibility findings might not be so straight forward because ADHD children seem to be primarily challenged on switch paradigms [70,71] but not on the ID/ED task [29,32,34,35]. Because Happé [32] only included children with ADHD of the combined subtype, differences related to subtype do not provide a sufficient explanation for these mixed findings. What the data indicate is that these tasks are not equivalent in regards to measuring the same construct. Stated simply, success on the ID/ED task does not predict success on a task-switch paradigm because other factors, such as the level of attention shifting (i.e. feature versus dimension) probably have a role in determining performance (Box 1).

An important avenue for future research is to determine whether ADHD symptoms seen in people with primary ADHD without ASD characteristics are similar to those seen in people with primary ASD with ADHD. Findings from two recent studies of children with ASD indicate that 40% to 50% meet symptom criteria for at least one of the three ADHD subtypes [38,39]. It might be that people with symptoms of both ADHD and ASD are more impaired functionally, and subsequently, might respond differentially to treatment [29,72]. Detailed comparisons across the various combinations to include ADHD, ASD and ASD with ADHD are needed to more fully understand the differences and overlap between these disorders and their neurobiological substrates. Such careful diagnostic comparisons might delineate the now vague boundaries of higher order processes such as cognitive flexibility.

Box 3. What can we learn from neuroimaging findings?

The clinical neuropsychology tasks were designed to be sensitive to lesions of the PFC before it was possible to verify actual brain region(s) and neural circuits recruited during their completion using functional magnetic resonance imaging (fMRI). Several of these tasks, including WCST, ID/ED and TMT, have been adapted for neuroimaging. Please note that as in designing fMRI experiments, multiple adaptations from the original tasks needed to be made, one needs to consider differences in task design for each of these neuropsychological measures. To only examine aggregate findings might, therefore, be an oversimplification and be misleading. As shown in Table 1 (in the main text), regions commonly activated during these tasks are related to fronto-parietal and fronto-striatal brain regions [59–65].

There are a handful of fMRI studies examining EFs in ASD, but only two of these have focused on cognitive flexibility. In a relatively small study with a wide age range, Schmitz and colleagues [40] reported that adults with ASD, who performed behavioral tasks comparably to typically developing (TYP) adults, showed increased activation in the right inferior and parietal areas associated with performance on a task-switching paradigm in ASD. Shafritz and colleagues [41] showed in a larger study that adult with ASD performed worse than TYP controls in inhibiting responses to rare stimuli. There were no differences between the groups in behavioral performance for switching, but there was reduced activation in frontal, striatal and parietal regions in the ASD group. Task-related activation for individuals with ASD was limited to the ventrolateral PFC when a response shift or shift in cognitive set was required, whereas many other areas including dorsolateral PFC, anterior cingulate cortex, premotor cortex and the basal ganglia were activated in the TYP group. This suggests that the groups might have employed different strategies. This very brief overview of recent neuroimaging findings indicates that more studies based on mechanistically plausible hypotheses using well-controlled experimental paradigms, including large samples of children with ASD, will be required before it is possible to draw conclusions about cognitive flexibility from imaging studies.

In cognitive neuroscience it is more common to use task-switching paradigms, which employ a more controlled experimental design, to measure cognitive flexibility. These paradigms include both repetition (the sorting rule stays the same) and switch (the sorting rule changes) trials. The difference in response time between these two trial types (switch cost) is the main dependent variable used to assess cognitive flexibility [5]. In most task-switching paradigms a cue shows which sorting rule should be applied, moreover the switch trials are presented throughout the task and not solely at the end of the task like in the ID/ED task. Hence, sustained attention difficulties do not confound interpretation of findings. This more mechanistic approach probably is the best way to study cognitive flexibility. To our knowledge, only four studies have investigated task-switching in people with autism and none of them reported performance deficits [40–43].

Conclusions and future directions

This qualitative literature overview shows that the findings regarding cognitive flexibility deficits in individuals with autism as assessed by either clinical neuropsychology (other than the WCST) or experimental cognitive psychology paradigms, are rather inconsistent, despite exhibiting behavioral inflexibility in many respects. Besides the differences in the applied cognitive flexibility tasks among the studies, there is considerable heterogeneity in age, cognitive level and autism subtype of the participants. Although one might assume that this also partly explains the inconsistent findings [3,11,32], we believe that these factors do not provide a sufficient explanation for the current findings. Grouping the studies (Table S1 in supplementary material) according to each of these three factors instead of according to task type did not reveal more consistent results. The current findings provide a clear and relevant ‘call to action’ for future autism research (Box 4). To this end, we discuss four issues to enhance research in this area; (i) which comparison group should be included, (ii) the sample sizes, (iii) which tasks should be used and, (iv) what other processes need to be taken into account.

Box 4. Questions for future research.

• How can we continue to develop more mechanistic tasks in which we can control and manipulate difficulty level to measure cognitive flexibility that map on to neural circuits? How can the autism field make better use of knowledge derived from computational and animal models?

• How can we translate findings with the aforementioned mechanistic tasks to the observed behavior and vice versa?

• How can the behavioral rigidity observed in individuals with autism be measured in an ecologically valid way?

• How do psychobiological factors (bottom up), like stress, relate to cognitive flexibility in individuals with autism?

• How can we use more sophisticated statistical modeling techniques in the field of autism to help us understand the relationship between variables related to cognition and those related to observable behavior?

• How do we need to organize the DSM-IV autism symptoms to be able to link performance on cognitive flexibility measures to observable behavior?

• How might clear profiles of cognitive control deficits in autism support the differential diagnosis of autism with other neurodevelopmental disorders associated with cognitive control deficits, such as ADHD?

• How do individuals with autism with comorbid ADHD differ from individuals with autism without comorbid ADHD, and from individuals with ADHD without comorbid autism?

• How do these flexibility findings relate to those found in other neurodevelopmental disorders? For example, should we conceptualize disorders such as autism, ADHD and schizophrenia as discrete or related with respect to their endophenotypes?

• How can we use the information regarding cognitive flexibility to develop interventions that target the multitude of behavioral rigidity that is present in individuals with autism?

First, instead of just including a typically developing control group, comparison between individuals with autism and other neurodevelopmental disorders can elucidate the difficulties of individuals with autism. For example, further investigation of the relationship between comorbid ADHD symptoms and cognitive flexibility is warranted. In the current diagnostic statistical manual (DSM)-IV-text revision [1], both diagnoses cannot be given simultaneously. However, there is considerable evidence that ADHD characteristics need to be carefully considered when studying autism (Box 2), because attention difficulties might interact with deficits in other developmental domains where people with autism are challenged. Moreover, neurodevelopmental disorders might differ in terms of onset of apparent deficits in addition to the neurobiological substrate level in which these deficits arise. Hence, these comparisons between clinical groups might help us to unravel the etiology of each of these disorders and will provide insight into the differences and similarities among various neurodevelopmental disorders [44].

Second, the discussed studies differ enormously in the number of participants. It seems that the better the task (i.e. more mechanistic), the smaller the groups. Moreover, non-significant group differences are not indicative of fully intact performance, and in these group comparisons individual differences might not become apparent. Therefore, in consideration of the sample size, we calculated pooled effect sizes of the clinical neuropsychological measures. These results indicated some quantitative evidence for cognitive inflexibility in autism (Table 2). Therefore, firm conclusions that there is no evidence for failure on these tasks might be premature. Thus, to determine whether the more mechanistic tasks indeed fail to reveal any cognitive flexibility deficits in autism, studies with larger samples are crucial.

Third, the currently used ‘tool box’ to study questions related to cognitive flexibility requires improvement and expansion. The field lacks clear mechanistic hypotheses about the autism related deficits that can be tested and validated. Clinically relevant mechanistic tasks are needed to understand the specific cognitive processes that are related to autism symptoms. Such tasks should have a foundation in the extant literature and be able to distinguish a specific deficit in the mechanism from a generalized cognitive or motivational deficit [45–47] because we know that autism is associated with deficits in both domains. This enables us to tease apart discrete cognitive processes related to cognitive flexibility. One benefit of task-switching paradigms is that they have been extensively studied in typical development, providing a context for use in atypically developing populations. For example, Wager and colleagues [48] reported that in typically developing adults, good and poor performance on a cognitive flexibility task were associated with activation of different neural circuits (ventral medial prefrontal cortex [MPFC] and rostral anterior cingulate cortex [ACC], and dorsolateral PFC [DLPFC], MPFC and parietal cortex, respectively). Moreover, Ravizza and Carter [49] have argued that all switches are not created equal, and that the mixture of shifts of visuospatial attention (perceptual shifting) and contextual rules (rule shifting) are potential sources of conflicting reports in the literature. These two shift types are associated with different brain activation patterns. Hence, the examination of neural correlates of different shift types will be meaningful (see also Box 3). Interestingly, perceptual shifting has shown to be deficient in individuals with autism [50] and these deficits could be linked to difficulties with joint attention. Difficulties initiating and responding to joint attention might be because of problems with perceptual shifting that are attributable more to the parietal neural circuitry, whereas inflexible application of social rules might be a more prefrontal mediated problem. Both circuitries seem to be implicated in the etiology of autism [51].

However, measures that are more refined probably will not provide all the answers. It has proven challenging to link performances on the current cognitive flexibility measures to actual behavior [6–10]. In Table 1 we related DSM-IV-TR [1] symptoms of autism with hypotheses about their relationship with cognitive flexibility. Current measures might be unable to capture the multitude or complexity of environmental factors that impinge on an individual with autism in the real world. More ecologically valid measures might help in forging associations between the observed day-to-day behavior and what we are measuring. For example, Hill and Bird [8] showed clear correlations between autism characteristics (i.e. mainly in the communication domain) with a more ecologically valid task. Also South and colleagues [10] showed that WCST performance correlated with the amount of repetitive behavior (see also Ref. [9]). However, increasing ecological validity always comes at the cost of reducing the mechanistic purity of the task. Therefore, the correlations obtained in the given examples [8–10] might be because of a general executive functioning deficit, but we cannot determine whether this correlation can be explained by a specific cognitive flexibility deficit. Hence, we believe that we first need detailed measures that are derived from well-developed theoretical frameworks to determine which processes are deficient in autism, and next we need ecologically valid measures that can more reliably link actual behavior to task performance. To go directly from the mechanistic tasks to observable behavior might be too large a gap to bridge. However, future studies using these more mechanistic tasks should test whether a link with everyday behavior can be made because we should be careful in measuring constructs without any relevance to real life.

Fourth, in addition to improving and expanding the cognitive flexibility ‘tool box,’ a paradigm shift might be required to capture the inflexibility observed in individuals with autism. Other top down factors and/or mediating variables (bottom up) might better explain inflexibility than the measures discussed so far. For example, failure on the WCST might be because of social-motivational factors [25,26] or uncertainty regarding the expectations of the task which might evoke stress. Psychobiological factors, such as stress, might underlie the poor response to novelty and changes in routine in autism [52]. It is well-established that a certain level of arousal and/or stress is conducive for optimal performance; however, excessive elevation can result in reduced performance [53]. Elevations in stress related hormones (i.e. cortisol) have been associated with poor performance on measures of cognitive flexibility in typically developing adults [54]. Although unstudied, over or under arousal might provide a plausible connection between psychobiological factors and performance on cognitive flexibility tasks in autism.

Although we cannot conclude whether cognitive inflexibility as currently measured is central to autism, isolating the crucial cognitive processes while considering influential bottom up processes will aid in ultimately resolving the paradox between behavioral and cognitive inflexibility in autism.

Glossary

Clinical neuropsychological measures

standardized tests with well-established psychometric properties originally developed to test patients with specific lesions. Scores are often combined to generate composite scores. Such measures have been criticized for complexity, failure to isolate specific functions and lack of sensitivity to effects on precise cognitive systems [45,46].

Cognitive control

refers to what previously have been thought of as executive functions. Evolved in the field of cognitive neuroscience and refers to the ability to maintain task relevant information to suppress inappropriate behaviors, and to flexibly adjust behavior according to environmental contingencies [45,47,55]. Cognitive control (CC) is required to guide action in novel, difficult and rapidly changing situations. Failures of CC cause perseveration on over-learned behaviors. CC models provide a parsimonious and mechanistic account that maps to specific, and not necessarily frontal, neural regions and circuits [45,47,55]. It is possible to isolate and manipulate various aspects of CC during experiments.

Discriminating power

sensitivity to performance differences, which is a function of a test’s difficulty and reliability. Tests with high discriminating power are especially susceptible to confounding effects of nuisance factors such as low motivation, sedation or fatigue, general inattentiveness and poor test-taking abilities [47].

Executive functions

is an umbrella term used to describe various problems in complex, goal directed actions, an incapability in planning future actions and difficulties with overcoming habitual responses found in patients with focal frontal lobe lesions. The term ‘executive functions’ (EF) refers to the multiple skills including planning, inhibition, organization, self-monitoring, cognitive flexibility and mental representation of tasks and goals, which are required to prepare for and execute goal directed behavior [4]. EF has been criticized for conceptual under-specification and there has been considerable debate about whether EF is a single ‘central executive’ process or whether EF consists of multiple component process [56,57]. CC frameworks have been developed to understand the specific nature of EF deficits encountered in various populations and to be able to link these to underlying brain mechanisms [45].

Experimental cognitive psychology measures

developed to examine the function of specific cognitive systems, which are validated experimentally by varying task parameters to test predictions from cognitive models. Given their frequent modification, they generally lack standardization. They can be validated using neuroscience methods [45–47].

Generalized performance deficits

performance deficit that seems to be caused by a specific cognitive process, but really is because of low motivation, sedation or fatigue, general inattentiveness and/or poor test-taking abilities.