The Neural Underpinnings of Cognitive Flexibility and their Disruption in Psychotic Illness

Abstract

Schizophrenia has long been associated with a variety of cognitive deficits, including reduced cognitive flexibility. More recent findings, however, point to tremendous inter-individual variability among patients on measures of cognitive flexibility/set-shifting. With an eye toward shedding light on potential sources of variability in set-shifting abilities among schizophrenia patients, I examine the neural substrates of underlying probabilistic reversal learning (PRL) – a paradigmatic measure of cognitive flexibility – as well as neuromodulatory influences upon these systems. Finally, I report on behavioral and neuroimaging studies of PRL in schizophrenia patients, discussing the potentially influences of illness profile and antipsychotic medications on cognitive flexibility in schizophrenia.

2. Introduction

Cognitive flexibility, or the ability to appropriately adjust one's behavior according to a changing environment (Dajani and Uddin, 2015), has long been studied in patients with schizophrenia (SZ), using a variety of measures. Considerable early evidence from studies of SZ pointed to the presence of deficits in cognitive executive functions, such as attentional set-shifting and task switching (Kehagia et al., 2010), similar to those observed individuals with frontal lobe lesions (Elliott et al., 1995). Accordingly, these deficits were often accompanied by evidence of frontal lobe hypometabolism (Weinberger, 1988). The relative inability to shift attentional set – called “stuck-in-set behavior”, or “perseveration” – became the paradigm case of a cognitive consequence of frontal lobe dysfunction based on the results of early studies with the Wisconsin Card Sort Test (WCST; Milner, 1963). In performing the WCST, the test-taker is presented with a set of cards, with varying numbers of colored shapes, that he/she must assign to piles based on one of the of the dimensions (color, shape, or number). At first, the participant does not know the dimension by which he/she is supposed to sort, learning only through trial-and-error, when the examiner provides the feedback of “correct” and “incorrect”. After the participant determines the appropriate sorting criterion and sorts by it a set number of times, the criterion is then changed, unbeknownst to the participant. Thus, the participant begins the process of trial-and-error learning again, and, in this way, achieves as many categories as possible, before the deck is exhausted. Early studies revealed that patients with frontal lobe lesions frequently exhibited a characteristic behavior on the WCST: they showed particularly difficulty in shifting from one sorting criterion to another, in the face of negative feedback (Milner, 1963, Nelson, 1976, Stuss et al., 2000). These kinds of errors were called “perseverative errors”, to distinguish them from other types of incorrect responses on the test, and these perseverative errors became the foremost exemplar of stuck-in-set behavior.

In the interim, numerous lesion and imaging studies have shown that prefrontal cortex (PFC) is far from a unitary structure, and that the cognitive consequences of frontal lobe dysfunction are more complicated and variable than originally thought (Fuster, 2001). In addition to the cognitive consequences of frontal lobe dysfunction depending on which particular subfield of PFC has been affected, it has become clear that complex forms of learning and executive function are not localized to frontal cortex, but, rather, depend on interactions among a number of cortical and subcortical brain regions. In the following sections, I will describe the evolution of our understanding of the nature of deficits in executive function in schizophrenia. I will survey the literature on set-shifting in SZ and discuss possible predictors of different patterns of behavior, with regard to set-shifting. Finally, I will discuss probabilistic reversal learning (PRL) as a probe of set-shifting and the neural processes that have been linked to different kinds of reversal learning impairments.

3. Set-shifting in schizophrenia: Is perseveration a major factor?

The WCST has long been used in neuropsychological investigations of schizophrenia, with considerable evidence indicating that SZ patients make a significantly higher number of perseverative responses than do normal control subjects and patients with other psychiatric disorders (Bellini et al., 1991, Braff et al., 1991, Abbruzzese et al., 1995, Cavallaro et al., 2003). What is also apparent, however, is that SZ patients are not characterized by high numbers of perseverative errors to the same degree as individuals with frontal lobe lesions (Heaton et al., 1979), and that SZ patients make many non-perseverative errors, as well, such that SZ patients do not differ significantly from controls in terms of the ratio of perseverative to non-perseverative errors (Li, 2004). Importantly, many SZ patients achieve no categories at all, on the WCST (Prentice et al., 2008), a fact suggestive of a more general problem of reinforcement learning, not limited to set-shifting. Findings also appear to indicate that impairments in both the formation and overriding of prepotent responses could stem from working memory deficits (Gold et al., 1997, Glahn et al., 2000, Hartman et al., 2003), which are prevalent in schizophrenia (Heinrichs and Zakzanis, 1998, Lee and Park, 2005), but not specific to the condition. Finally, performance on the WCST, among patients with schizophrenia, may vary with symptom profile, with paranoid patients making a higher number of perseverative errors than nonparanoid patients (Abbruzzese et al., 1996).

A second measure commonly used to probe executive function and set-shifting in neuropsychiatric illness is the Intra-dimensional/Extra-dimensional (ID/ED) task from the Cambridge Neuropsychological Test Automated Battery (CANTAB; Downes et al., 1989, Owen et al., 1991). In this task, each subject is required to learn a series of discriminations in which one of two stimulus dimensions (purple-filled shapes or white lines) is relevant and the other is not, using feedback provided automatically by the computer. Four boxes are presented on the computer screen, two of which contain different exemplars of one of the dimensions, either shapes or lines. Initially, individuals are given a simple discrimination (SD), in which they have to identify which exemplar is “correct”. Feedback is both auditory and visual, with the word “CORRECT” appearing in green letters or the word “WRONG” appearing in red. Following eight consecutive correct responses, the task moves on to the next set-shifting stage: a reversal of the simple discrimination just acquired (SDR). The same feedback and criteria are used in each subsequent stage. In the SDR stage, the previously incorrect choice becomes the correct one. In the third stage (compound discrimination, or C_D) the second dimension (purple shapes) is introduced with one exemplar of each dimension paired together to form a compound stimulus in two of the response boxes. To succeed, a subject has to continue to respond to the correct exemplar of the previous stage. For this and subsequent stages, exemplars of different dimensions are paired in a pseudo-random fashion so that all four combinations are used. However, no more than three trials with the same pairings are allowed. The stimuli for the fourth stage (CD) and subsequent stages are also compounds, but the two exemplars from the different dimensions are superimposed, with the white line always in the foreground. The contingencies are again unchanged from the previous two stages. A reversal then occurs at the fifth stage (CDR). New exemplars for both dimensions are introduced at the sixth stage, the intra-dimensional shift (IDS), but the relevant dimension for a correct response is unchanged from stage 1 (i.e. if lines were the correct dimension in stage 1, lines continue to be correct). This is followed by a further reversal at the seventh stage (IDR). In the next stage, the extra-dimensional shift (EDS), new exemplars are again introduced, and subjects are now required to respond to the previously irrelevant dimension (e.g. shapes rather than lines). In the final stage there is again a reversal (EDR) so that response to the previously irrelevant exemplar of the new dimension is required for a correct response. The main measure of performance on this task is the highest stage successfully attained. Additional performance measures from the ID/ED task include trials to criterion and number of errors at each stage.

The ID/ED has been used on numerous occasions to examine set-shifting deficits in schizophrenia. Early studies (Elliott et al., 1995, Pantelis et al., 1999) appeared to support the idea that patients with schizophrenia frequently exhibit stuck-in-set behavior. As with the WCST, however, later studies with the ID/ED task have revealed a high degree of variability in the performance of SZ patients. As with the WCST, the performance of SZ patients is not characterized exclusively by perseveration; rather, many patients show difficulty in acquiring stages of the task not involving reversals (Pantelis et al., 1999). Furthermore, performance on the ID/ED task appears to be heavily influenced by working memory capacity (Pantelis et al., 2004). Interestingly, set-shifting deficits do not appear to be highly heritable (Ceaser et al., 2008), suggesting that set-shifting deficits in SZ may be closely-related to illness state, and not a trait. Data on the effects of illness course are mixed, as set-shifting deficits have been found to be present at initial psychotic episodes (Joyce et al., 2002, Murray et al., 2008), and the severity of these deficits has been found to correlate with illness duration (Pantelis et al., 2004). Additional data, however, point to the stability of ID/ED performance over the first six years of illness (Leeson et al., 2009).

Several other results bear on the issue of set-shifting behavior in schizophrenia, even if they did not emerge from studies or tasks specifically designed to examine set-shifting behavior. The Iowa Gambling Task (Bechara et al., 1994, Bechara et al., 1997) is a reinforcement learning paradigm that was designed to focus on punishment sensitivity, with early studies showing dramatic impairment in individuals with frontal lobe (Bechara et al., 1994) or amygdala (Bechara et al., 1999) lesions. More recent analyses (Maia and McClelland, 2004) have emphasized the complexity of the task, noting that the need to integrate both punishments and rewards to estimate the values of decks, as well as the need to rapidly update estimates of value based on fluctuations in the magnitudes of punishments (amounting to a kind of reversal). While studies using this paradigm have produced mixed results in individuals with schizophrenia (Wilder et al. 1998; Ritter et al. 2004; Shurman et al. 2005; Sevy et al. 2007), findings are relatively clear that, while some SZ patients show performance impairments on the task, they rarely exhibit the stark insensitivity to punishment that individuals with frontal lobe (Bechara et al., 1994) or amygdala (Bechara et al., 1999) lesions do. Rather, poor performance in SZ patients appears to stem from a more general difficulty in integrating the frequency and magnitudes of gains and losses over time, in order to estimate the values of choices (Brown et al., 2015).

Finally, a small number of studies (Waltz and Gold, 2007, Waltz et al., 2013, Schlagenhauf et al., 2014, Culbreth et al., 2015, Reddy et al., In press) have examined probabilistic reversal learning (PRL) in patients with schizophrenia, in order to probe set-shifting. Most of these studies (Waltz et al., 2013, Schlagenhauf et al., 2014, Culbreth et al., 2015) were done in conjunction with functional magnetic resonance imaging (fMRI), and thus were more designed to look at differences in neural signals than differences in behavior. Two studies of PRL performance in patients outside of the scanner (Waltz and Gold, 2007, Reddy et al., In press) used an identical paradigm, which presented individuals with three separate discriminations to learn. If the participant reached a criterion of nine choices of the best stimulus in a run of ten trials on a given discrimination, then the contingencies were reversed, and the subject was required to learn which stimulus was now the best. If the participant reached a criterion of nine choices of the best stimulus in a run of ten trials in that stage, then the contingencies were reversed again (back to the original contingencies), and the subject was again required to learn which stimulus was the best. In the Waltz and Gold (2007) study, schizophrenia patients achieved fewer reversals than controls, even when controlling for the number of discriminations learned. With regard to error rates, these authors observed a group × stage interaction, such that the groups did not differ in their error rates during the discrimination phase, but differed greatly in their error rates in reversal phases. While this result was interpreted as evidence of a particular deficit in rapid/reversal learning, a recent study by Reddy et al. (In press), with a much larger sample, suggests that the reversal learning deficits in schizophrenia may be an outgrowth of a more general deficit in reinforcement learning. In this study, no group × stage interaction was observed, as the groups differed in their error rates during both discrimination and reversal phases. Thus, the question of whether SZ patients have a specific deficit in set-shifting vs. a more general deficit in set acquisition remains unanswered.

While data from various measures do not align perfectly, results from studies of set learning and set-shifting in SZ patients converge in several respects:

• They indicate that patients are not simply insensitive to punishments in the manner that individuals with frontal lobe or amygdala lesions often appear to be;

• They point to a more general deficit in learning about value, even when rapid reversals are not required; and

• They support the idea that this deficit in learning about value may stem from working memory impairments, frequently associated with SZ.

The remainder of the manuscript will focus on studies using probabilistic reversal learning, as a paradigm case, to investigate rapid reinforcement learning. Given their extensive application in both the human and nonhuman studies of cognitive flexibility, the functional anatomy of processes involved in probabilistic reversal learning have been relatively well mapped-out, potentially enabling one to make detailed statements about the neural substrates of both adaptive and pathological value-based learning and decision making.

4. Reversal learning as a paradigm case – difference neural substrates for different cognitive processes

4.1. Frontostriatal systems and reversal learning

Due to their potential as translational paradigms for the study of reinforcement learning and set-shifting, Probabilistic Reversal Learning tasks have frequently been used in cognitive neuroscience studies over the last several decades. Both lesion studies (Hornak et al., 2004) and neuroimaging studies (Cools et al., 2002) have confirmed a role for frontostriatal circuits in successful reversal learning. As with other paradigms, however, more recent studies using PRL paradigms have highlighted their complexity and have focused on isolating the component processes. Ventrolateral prefrontal cortex (vlPFC; Brodmann areas 47/12) is an area that has been consistently implicated in set-shifting behavior, both in imaging studies of the WCST and related tasks (Konishi et al., 1999, Monchi et al., 2001), and in imaging studies of probabilistic reversal learning (Cools et al., 2002). The idea that this region is specifically involved in “switching” behavior is supported by evidence that vlPFC is selectively activated by negative feedback that leads to switching (Cools et al., 2002). Additional evidence indicates that the vlPFC is essential for the integration of negative feedback in rule-learning, as the act of switching, itself, activates vlPFC to a much lesser degree (Konishi et al., 1999, Monchi et al., 2001).

Several other subregions of PFC have been linked to specific set-shifting processes, as well. Switching to the alternative response in a PRL task depends on at least two processes: inhibiting a previously reinforced choice and overcoming the avoidance of a previously-punished choice. Using a clever design, Greening and colleagues (2011) mapped these two subprocesses to ventromedial prefrontal cortex (vmPFC) and dorsomedial prefrontal cortex (dmPFC), respectively. Specifically, a suppression of activity in vmPFC was observed in conjunction with the inhibiting of a previously reinforced choice, whereas an increase in activity in dmPFC was observed in conjunction with overcoming the avoidance of a previously-punished choice (Greening et al., 2011). Finally, given the dependence of rule learning and set-shifting on working memory (Gold et al., 1997, Pantelis et al., 2004), it should come as no surprise that the mid-dorsolateral prefrontal cortex (dlPFC; area 9/46) generally shows increased activity in association with the performance of set-shifting and reversal learning tasks.

The PFC regions implicated in PRL have extensive connections with numerous subcortical regions known to be involved in feedback processing and reinforcement learning. These areas include both the ventral striatum/nucleus accumbens (VS/NAcc; Cools et al., 2002) and habenular complex (or epithalamus; Ullsperger and von Cramon, 2003), two areas thought to primarily signal mismatches between expected and obtained outcomes, called reward prediction errors (RPEs; Seymour et al., 2004, Matsumoto and Hikosaka, 2007). Robust signaling of these mismatches in the face of contingency reversals is necessary to motivate set-shifts. Robinson and colleagues (2010a) have specifically demonstrated simultaneous valence-specific and valence-nonspecific signals in the striatum, in the context of a reversal learning task, with the posterior dorsal striatum responding only to unexpected reward, and the anterior ventral striatum responding to both unexpected punishment as well as unexpected reward.

The established involvement of prefrontal cortical regions in PRL fits with the large body of evidence pointing to impairments in set-shifting – including reversal learning – in patients with neuropsychiatric illness. The fact that these prefrontal regions are richly innervated by various neuromodulator systems – including those for dopamine (DA), serotonin (5-hydroxytryptamine, or 5HT), and acetylcholine – suggests that these systems have the ability to influence set-shifting, as well. This idea is strongly supported by available evidence, as both dopamine and serotonin systems are clearly implicated in both neuropsychiatric illness and set-shifting impairments. Furthermore, there is considerable evidence that set-shifting behavior can be influenced by genes coding for aspects of dopamine and serotonin system function, as well as drugs targeting these systems. I will recount some of this evidence below.

4.2. Dopamine and reversal learning

Evidence that dopamine system function modulates behavior and neural signals associated with reversal learning comes from studies in both human and nonhuman subjects (Dodds et al., 2008, Boulougouris et al., 2009, Clatworthy et al., 2009, Cools et al., 2009, Rutledge et al., 2009). These studies have also been carried out using genotyping, pharmacological, and molecular imaging techniques, and combinations thereof. Studies of the role of dopamine transmission in reversal learning have focused on three sites: presynaptic dopamine transporters (DATs), post-synaptic dopamine receptors, and the catechol-O-methyltransferase (COMT) enzyme, which acts mainly to metabolize dopamine in the PFC (Egan et al., 2001). Other studies have looked more generally at the role of dopamine transmission in reversal learning by either depleting or enhancing the concentration of dopamine precursor available for synthesis into dopamine, or by assessing reversal learning in medicated and/or unmedicated patients with Parkinson’s disease (PD).

Pharmacological studies of reversal learning in human subjects

What these studies have tended to reveal is that PD patients on DA-enhancing medication (L-dopa, DA-receptor agonists) exhibited impaired reversal learning relative to patients off medication, chiefly because they were less sensitive to unexpected punishments, and therefore achieved fewer reversals signaled by unexpected punishments (Cools et al., 2006). Rutledge et al. (2009) produced a complementary result, in a study of PD patients on and off medications enhancing DA transmission. These authors fit choice behavior from a dynamic foraging task with RL models, with separate learning rates for gains and losses. In RL models, learning rate (noted as α) determines the impact of positive and negative prediction errors on changes in association strength. These authors found that dopaminergic medications enhanced learning rates for rewards only, showing no effect of dopaminergic drugs on learning from negative outcomes.

Several studies involving the administration of methylphenidate have produced results consistent with these findings. In a study combining pharmacological challenge with fMRI, Dodds et al. (2008) found that methylphenidate attenuated the BOLD signal in the ventral striatum during response switching after negative feedback but modulated activity in PFC when subjects maintained their current response set. In a study combining pharmacological challenge and molecular imaging, Clatworthy et al. (2009) administered therapeutic doses of oral methylphenidate to young healthy subjects, measuring D2/D3 receptor availability using [(11)C]-raclopride radioligand PET imaging. These authors found that performance on a reversal learning task was predicted by the drug-induced change in D2/D3 receptor availability in the caudate, such that the greatest degree of raclopride displacement was associated with the worst task performance. That is, the greatest amount of striatal DA release was associated with the least cognitive flexibility, in terms of the ability to reverse learned associations. Cools et al. (2009) found the effects of the D2 receptor agonist bromocriptine to be dependent on the baseline dopamine synthesis capacity of individuals. That is, D2 agonism improved reward-based relative to punishment-based reversal learning in subjects with low baseline dopamine synthesis capacity in the striatum, while impairing it in subjects with high baseline dopamine synthesis capacity.

Of note, studies involving either dopamine-precursor depletion or DA-receptor blockade have been shown to either disrupt reward processing or improve punishment processing, or both, in the context of RL tasks. For example, Janssen et al. (2015) investigated the effects of the dopamine D2-receptor antagonist sulpiride on reward- and punishment-based reversal learning in 18 pathological gamblers and 22 healthy controls. This group found that the blockade of D2 receptors with sulpiride impaired reward-driven reversal learning in controls, leaving punishment-driven reversal learning unaffected. Robinson et al. (2010b) showed that acute tyrosine and phenylalanine depletion (ATPD) significantly improved punishment processing, but not reward processing, in the context of a PRL task.

Effects of dopamine genes on reversal learning in human subjects

Largely consistent with the results of pharmacological studies, these studies have tended to show that higher dopamine concentrations in both the striatum and PFC are associated with less cognitive flexibility, likely due to an increase in reward sensitivity at the expense of punishment sensitivity. Several studies have examined reward responsivity in the striatum as a function of inter-individual variation in the 40 bp variable number of tandem repeats polymorphism in the 3′ untranslated region of the DAT gene (DAT1, SLC6A3). The 10-repeat (10R) allele of this gene has been associated with increased gene expression and presumably lower levels of synaptic dopamine in the striatum, relative to the 9-repeat (9R) allele (Heinz et al., 2000, Fuke et al., 2001, Mill et al., 2002). Variants of the dopamine transporter (DAT) gene have been observed to predict reward-related activity in the ventral striatum (Forbes et al., 2009), with 9R carriers showing stronger responses to reward predicting cues than 10R homozygotes (Dreher et al., 2009). In a related study, Aarts et al. (2010) assessed dopamine-dependent effects of reward on task switching, in an event-related fMRI study. These authors found that 9R carriers exhibited a greater influence of anticipated reward on switch costs, as well as greater activity in the dorsomedial striatum during task switching in anticipation of high reward relative to low reward. In a subsequent study, den Ouden et al. (2013) observed that an increasing number of 9R-alleles resulted in a stronger reliance on reward history, and a corresponding reduced influence of less-frequent punishments, leading to a reluctance to update learned associations. Thus, while a higher number of 9R-alleles appears to be associated with increased reward sensitivity, the relative reduction in punishment sensitivity seems to lead to relatively less cognitive flexibility, as measured by successful reversals.

With regard to SNPs affecting the activity of the COMT enzyme, several studies indicate that more COMT Met alleles are associated with less cognitive flexibility. The Met/Met genotype of the Val¹⁵⁸Met polymorphism has been linked to reduced activity in the COMT enzyme, and thus higher concentrations of dopamine in the PFC. In a study involving the performance of a probabilistic RL task in conjunction with event-related MRI, Krugel et al. (2009) found that the Val/Val genotype of the Val¹⁵⁸Met polymorphism was associated with a higher and more flexible learning rate. That is, more rapid updating (increased cognitive flexibility) was tied to lower concentrations of dopamine in the PFC, probably resulting in greater relative sensitivity to punishments. Furthermore, fMRI analyses revealed that participants with the Val/Val genotype (who showed more dynamic modification of learning rate) also exhibited more differentiated striatal fMRI responses to prediction errors during task performance (Krugel et al., 2009). Furthermore, participants with the Val/Val genotype showed greater learning rate-dependent changes in effective connectivity between the striatum and prefrontal cortex than participants with the Met/Met genotype, suggesting that the relatively greater dynamic range in learning rate may result from more flexible modulation of dopaminergic activity in the striatum by PFC.

Researchers have also observed an effect of genes coding for dopamine D2 receptors on cognitive flexibility. Specifically, Jocham et al. (2009) investigated the effects of variations in the DRD2/ANKK1-Taqla polymorphism on reversal learning. The A1 allele of this gene has been associated with a reduction in striatal D2 receptor density of up to 30% (Thompson et al., 1997, Pohjalainen et al., 1998), most markedly in the ventral striatum. Jocham et al. (2009) observed several specific differences in reversal learning performance and associated signals between carriers and noncarriers of the A1 allele. First, A1 carriers showed a reduced tendency to stick with a rewarded response. Second, A1 carriers failed to show successive increases in responses in the rostral cingulate zone with successive instances of negative feedback. Finally, A1 carriers showed reduced activity in both the VS and VLPFC, relative to noncarriers of the A1 allele, in association with final reversal errors (instances of negative feedback that prompted shifts to the alternate stimulus). Thus, specific D2 genotypes impacted behavior as well as responses to both rewards and punishments in multiple brain regions critical for successful set-shifting.

In conclusion, variations in dopamine release and availability, through either pharmacological challenge or genetic polymorphisms have been associated with differences in cognitive flexibility, in the context of PRL tasks, within and across individuals. In general, lower availability of synaptic dopamine, either as a consequence of ATPD or a higher number of 10R alleles at the DAT1/SLC6A3 gene, has been associated with greater punishment sensitivity and reduced reward sensitivity, whereas higher availability of synaptic dopamine, either as a consequence of L-dopa administration or a higher number of 9R alleles at the DAT1/SLC6A3 gene, has been linked to the opposite.

Numerous questions remain, however, regarding dopaminergic influences on cognitive flexibility, in general. For example, it is important to note that, while individuals with the Val/Val genotype of the Val¹⁵⁸Met COMT polymorphism appear to exhibit a more flexible learning rate, and, thus, more rapid updating, in the context of a PRL-like task (Krugel et al., 2009), Val/Val individuals also, by and large, show worse WCST performance than Met/Met carriers (Egan et al., 2001, Malhotra et al., 2002, Barnett et al., 2007). Why would Val/Val individuals appear to show elevated cognitive flexibility in one context, but reduced cognitive flexibility in another? This apparent discrepancy in findings may be explained, at least in part, but two factors: 1) different levels of cognitive flexibility, and 2) interactions among multiple influences on cognitive flexibility. Specifically, “shiftiness”, in the context of PRL texts, depends to a large extent on relative sensitivity to rewards vs. punishments, whereas attentional set-shifting (in the context of the WCST) relies, to a large degree, on the ability to maintain and manipulate information in working memory (Gold et al., 1997). While the Val/Val type of the COMT gene may increase the tendency to reverse (in the context of a PRL task) by increasing sensitivity to (and/or boosting learning rates associated with) losses, the Met/Met genotype is associated with enhanced working memory performance (Goldberg et al., 2003). It is likely this benefit to the ability to manipulate information in working memory that contributes to the superior WCST performance often exhibited by Met/Met carriers (Bruder et al., 2005). Thus, the specific effects of dopamine modulations on cognitive flexibility depend on the site of the manipulation (PFC or striatum or both), as well as the experimental context calling for cognitive flexibility.

4.3. Serotonin and reversal learning

A role for serotonin transmission in cognitive flexibility and reversal learning makes sense in light of established evidence that serotonin figures prominently in the inhibition of punished behaviors (Soubrie et al., 1986, Deakin and Graeff, 1991). The influence of serotonin system function on reversal learning has been studied using a variety of methods, including acute tryptophan depletion (ATD), the assessment of the acute and chronic effects of the administration of selective serotonin reuptake inhibitors (SSRIs, which block presynaptic serotonin transporters), and the assessment of the acute and chronic effects of antagonists at postsynaptic serotonin receptors. Early studies of the effects of ATD on set-shifting and reinforcement learning (Rogers et al., 1999, 2003) appeared to point to a detrimental effect of ATD on set-shifting behavior, with ATD leading to deficits in reversal learning (Rogers et al., 1999) and the processing of reward magnitude (Rogers et al., 2003). These findings have been supported by the results of studies involving the focal depletion of serotonin in the marmoset PFC, using the selective neurotoxin 5,7-dihydroxytryptamine (Clarke et al., 2004, Clarke et al., 2005). However, Evers and colleagues (2005) found that acute tryptophan depletion enhanced punishment-related signals in dmPFC. Evidence that ATD enhances sensitivity to negative feedback is consistent with findings from studies involving the acute administration of SSRIs, which has been shown to increase punishment-sensitivity, such that healthy subjects shift away more frequently from a stimulus that resulted in a loss. Chamberlain et al. (2006) specifically found that an acute low dose of citalopram, which most likely reduces 5-HT function, enhanced sensitivity to negative feedback in human subjects. In the context of a PRL task, this increase in punishment-sensitivity was maladaptive, in that it led to increased rates of shifting away from the correct stimulus, when presented with misleading (probabilistic) feedback (Chamberlain et al., 2006).

Beside reducing sensitivity to reward magnitude and enhancing sensitivity to punishment, ATD has been found to have several other effects, in the context of reinforcement learning tasks. First, ATD has been shown to enhance prediction of punishment (Cools et al., 2008). Second, ATD in healthy volunteers had the effect of reducing punishment-induced inhibition of responding (Crockett et al., 2009). Thus, on the face of it, reduced serotonin transmission appears to have the effect of both increasing and reducing punishment sensitivity. Rygula and colleagues (2015) have argued that different behavioral effects of serotonin depletion may depend on the actual site of serotonin depletion (amygdala vs. dmPFC vs. vlPFC). Several other integrative theories have been proposed, including: 1) serotonin provides an anticipatory signal for both rewarding and punishing motivational outcomes (Miyazaki et al., 2011); and 2) that the nature of the behavioral deficit may depend on what use is made of the anticipatory signal: for example, to inhibit responding (conditioned suppression), to maintain a particular choice (ignoring misleading feedback, as in the probabilistic discrimination task), or to increase response vigor as a function of reward magnitude (Cools et al., 2005).

Studies with 5HT receptor agonists and antagonists have been more rare, but there is some evidence that buspirone, a partial agonist at the 5HT1 receptor, reduces sensitivity to aversive stimuli in conditioning paradigms (Hellewell et al., 1999, Bond et al., 2003). By contrast, the administration of 5HT2 receptor antagonists in rats has been shown to increase sensitivity to aversive outcomes, and thus reduce the number of errors during the performance of a reversal learning paradigm, including perseverative errors (Boulougouris et al., 2008, Boulougouris and Robbins, 2010). Consistent with this finding, at least one study involving the administration of a 5HT2 receptor antagonist (cyproheptadine) to human subjects points to increased cognitive flexibility, in the form of improved performance on the Stroop color word task and an antisaccade task, e.g. (Chaudhry et al., 2002; but see Hensman et al., 1991). Importantly, Alsiö et al. (2015) have shown that the 5-HT2C antagonist, SB 242084, improved early performance on a 3-stimulus reversal task, ostensibly through reducing the tendency to perseverate, but failed to improved overall performance, due to a detrimental effect on new learning later in sessions.

The vast majority of studies of the behavioral effects of serotonin genotypes have examined genes coding for serotonin transporter morphology, with strong evidence now linking the short (s) allele of the serotonin transporter gene (SLC6A4; also known as 5-HTT) linked to increased reactivity (especially in the amygdala) to aversive or threatening stimuli, relative to the long (l) form of the allele (Hariri and Holmes, 2006, Caspi et al., 2010). A limited number of studies have examined the role of 5-HTT genotypes in reinforcement learning and decision making, and these studies (Crisan et al., 2009, Roiser et al., 2009) tend to support the idea that the short allele of the 5-HTT gene is associated with increased loss aversion. Roiser and colleagues (2009) administered participants, selected as homozygous for either the long or short allele, a decision-making task where they made choices between receiving an amount of money for certain and taking a gamble. These authors found that short allele homozygotes showed a particularly strong bias, relative to long allele homozygotes, toward choosing the certain option when the option was phrased in terms of gains and toward gambling when the decision was phrased in terms of losses (the framing effect). In simultaneously acquired functional magnetic resonance imaging data, the ss group showed greater amygdala activation during choices made in accord with, compared with those made counter to the frame, an effect not seen in the ll group. These differences were also mirrored by differences in anterior cingulate-amygdala coupling between the genotype groups during decision making. Specifically, ll participants showed increased coupling during choices made counter to, relative to those made in accord with, the frame, with no such effect evident in ss participants.

In a similar study, Crisan et al. (2009) also found that short allele carriers showed an exaggerated framing effect, in that they showed a greater tendency to gamble in loss frames than long allele homozygotes. Short allele carriers also demonstrated enhanced loss aversion in the context of a common decision making task called the Balloon Analog Risk Task (BART; Lejuez et al., 2003), in which a virtual balloon pops after a random number of presses. Sort-allele carriers were found to make fewer presses per unexploded balloon, indicative of greater risk aversion in decision making. While these results point to increased sensitivity to losses, a recent study by den Ouden et al. (2013) found that that the 5HTT polymorphism altered behavioral adaptation after losses, in the context of a probabilistic reversal learning paradigm. In this particular study, the long allele was associated with increased lose-shifting behavior, and thus enhanced sensitivity to aversive outcomes. This result appears to be more consistent with the finding from Crockett et al. (2009) that ATD had the effect of reducing punishment-induced inhibition of responding in healthy volunteers.

While the results of investigations of serotonin’s role in cognitive flexibility, in the context of reinforcement learning tasks, allow a limited number of clear generalizations, several tentative conclusions can be drawn. First, the results of a large number of studies suggest that increased serotonin transmission, by way of either 5HT receptor agonists or a less efficient form of serotonin transporters, leads to a reduced sensitivity to aversive outcomes. By contrast, evidence suggests that reduced serotonin transmission, by way of either acute tryptophan depletion, post-synaptic receptor blockade, or upregulation of serotonin transporters, leads to an increased sensitivity to aversive outcomes and perhaps increased cognitive flexibility (as measured, e.g., by lose-shift rates; Chamberlain et al., 2006, but see den Ouden et al., 2013). Curiously, however, the clinical picture suggests that increasing available serotonin (through SSRI administration), can be used to successfully reduce compulsivity (Goddard et al., 2008), whereas reducing 5HT transmission, with 5HT receptor antagonists, can induce compulsivity (Schirmbeck et al., 2013, Fonseka et al., 2014).

Why might reductions in serotonin concentrations lead to increased “shiftiness”, in an experimental context, but an increased tendency to engage in real-world compulsive behaviors? As is the case with dopamine, understanding the role of serotonin in cognitive flexibility requires consideration of multiple kinds of cognitive flexibility, as well as interactions between striatal habit learning systems and frontal systems for goal-directed action selection (Gillan et al., 2011). For example, the fact that serotonin depletion appears to increase sensitive to both negative feedback (Evers et al., 2005) and threats (Hariri and Holmes, 2006, Fisher and Hariri, 2013) could have the effect of either increasing impulsive shifting in the face of misleading punishments (Harrison et al., 1997, Chamberlain et al., 2006) or indirectly driving compulsive behavior by increasing anxiety (Deakin, 1998, Stein and Stahl, 2000, Maron et al., 2012) and/or interacting with other neuromodulatory pathways, such as dopamine systems (Perani et al., 2008). By contrast, SSRIs may reduce compulsivity by reducing threat sensitivity (Fisher and Hariri, 2013, Williams et al., 2015) and/or stress-related dopamine release (Vaessen et al., 2015). Theorists (Daw et al., 2002, Boureau and Dayan, 2011) have proposed that dopaminergic and serotoninergic systems interact in systematic and possibly oppositional ways. The habenula has been identified as a potential hub of said opposition (Luo et al., 2015), in that its stimulation has been associated with both the stimulation of 5HT cells in the midbrain Raphe (Morris et al., 1999, Amat et al., 2001, Pobbe and Zangrossi, 2010) and the cessation of dopamine cell firing in the SN/VTA (Ji and Shepard, 2007, Matsumoto and Hikosaka, 2007). A clearer picture of the role of serotonin in cognitive flexibility will require a better understanding of these sorts of interactions.

5. Probabilistic reversal learning deficits in SZ and associated neural signals

Our initial study of PRL in patients with schizophrenia (Waltz and Gold, 2007) revealed that SZ patients and controls performed similarly on the initial acquisition of probabilistic contingencies, whereas patients showed substantial learning impairments when reinforcement contingencies were reversed. Patients achieved far fewer reversals, even when analyses were limited to subjects who acquired all probabilistic contingencies initially. While it was clear that patients with schizophrenia achieved fewer reversals than healthy volunteers (Waltz and Gold, 2007), it was not clear why. Based on the results of earlier lesion (Hornak et al., 2004) and neuroimaging (Cools et al., 2002) studies, reversal learning deficits were taken as an indicator of ventral PFC dysfunction (Waltz and Gold, 2007).

As recounted above, however, successful probabilistic reversal learning involves the integration of multiple cognitive processes in order to use surprising negative feedback in the service of set-shifting. It is thus important, for example, to distinguish reduced punishment sensitivity from a reduced tendency to act (shift) upon the receipt of negative feedback. Individuals might exhibit reduced lose-shifting behavior that stems from abnormal in-the-moment sensitivity to punishments. Individuals may also show intact in-the-moment sensitivity to punishments, but reduced lose-shifting behavior. In the process of integrating these various cognitive processes, components of multiple brain networks come into play (Greening et al., 2011). For example, the signaling of salient feedback by the anterior insula and ventral striatum has the consequence of both activating executive control network (ECN) structures, in the service of goal-directed behavior, and suppressing activity in default mode network (DMN) structures that are most active when the brain is “idling” (Seeley et al., 2007, Sridharan et al., 2008, Menon and Uddin, 2010).

At the current time, results of neuroimaging studies of reversal learning are mixed, regarding both the abnormalities that characterize schizophrenia patients as a group and the brain signals showing close relationships with clinical symptoms. In conjunction with lose-shifting behavior (essential for achieving reversal), SZ patients have exhibited abnormal signals in frontal, striatal, and parietal regions (Waltz et al., 2013, Culbreth et al., 2015). While Culbreth et al. (2015) observed between-group differences in the activation of anterior cingulate, dlPFC, and areas of parietal cortex, our group (Waltz et al., 2013) found that patients failed to deactivate right vmPFC in response to surprising negative feedback, a signal shown to contribute to response inhibition, by Greening et al. (2011). Furthermore, our lab observed between-group differences in components of default/task-negative networks. What appears to happen, in the case of patients with schizophrenia, is that the activation of task-positive networks is not accompanied by the same degree of DMN suppression as is observed in the case of healthy volunteers exhibiting adaptive behavior. That is: even if surprising, salient feedback is recognized as such, and task positive networks are activated, adaptive set-shifting may be slowed/prevented by interference from an insufficiently-suppressed DMN.

Based on the behavioral findings described above, it would not be surprising if illness profile accounted for some of the variance in brain activity associated with reversal learning in SZ patients. There is, in fact, evidence for this, as unmedicated (generally first-episode) patients exhibit striatal dysfunctional in conjunction with RPE-signaling, in the context of a PRL task (Schlagenhauf et al., 2014). In our own data, correlation analyses revealed a systematic relationship between punishment-evoked changes in activity in both the striatum and the VMPFC and Avolition/Anhedonia scores from the SANS (Waltz et al., 2013). New analyses of the behavioral data from this study have revealed that patients with the highest ratings for positive symptoms showed the highest rates of lose-shifting. In order to do this analysis, my colleagues and I divided patients into subgroups by performing a median-split on average scores from psychosis/reality distortion items from the Brief Psychiatric Rating Scale (after McMahon et al., 2002; median average score = 2.25). In order to quantify the rapidity with which subjects shifted after a contingency reversal, we extracted two new variables from our data: 1) the numbers of trials required, after each contingency reversal, to switch to the alternate stimulus; and 2) the number of stages in which a subject switched back to the previously-better stimulus, before completing the stage. These values were then converted into percentages: the percentage of switches occurring × number of trials after a contingency reversal (denoted trial n+x, where x = 1, 2, 3, 4, 5, >5); and the percentage of stages in which a subject switched back to the previously-better stimulus, before completing the stage.

When we performed an ANOVA on switch rates, by GROUP and TRIAL, we observed a GROUP × TRIAL interaction [F(4,92) = 6.098, p < 0.001]. Post-hoc LSD tests confirmed that patients with the highest ratings for psychotic symptoms showed the highest rates of shifting 1 or 2 trials after a contingency reversal, and the lowest rates of shifting 5 or more trials after a contingency reversal (Figure 1A). Further analyses revealed that rates of shifting in the first two trials after a contingency reversal correlated significantly with patients’ mean psychosis scores (Figure 1B; r = 0.448, p = 0.015). When we examined the relationship between positive symptom severity and rates of (prematurely) switching to the previously-better stimulus, we found that patients in the high positive symptom group switched back much more often (mean = 43.6%) than both low-positive-symptom patients [mean = 24.3%; p = 0.014] and controls [mean = 19.2%; p = 0.001]. Importantly, low-positive-symptom patients and controls did not differ in their rates of prematurely switching to the previously-better stimulus (p = 0.465). In short, patients with the highest ratings for psychotic symptoms required fewer experiences with negative feedback in order to switch to the alternative response than less-psychotic patients, often shifting too readily. As described above, shifting too readily to negative feedback, in the context of a probabilistic learning task, can be maladaptive if subjects do not settle on a response option, and frequently alternate between response options in the face of probabilistic feedback, eventually causing individuals to achieve fewer reversals. That was also the case, in the context of this experiment, as patients in the high positive symptom group achieved fewer reversals (mean = 5.6) than both low-positive-symptom patients [mean = 9.8; p = 0.05] and controls [mean = 14.2; p < 0.001].

Figure 1.

(A) Mean percentages of switches occurring × number of trials after a contingency reversal (denoted trial n+x, where x = 1, 2, 3, 4, 5, >5). (B) Scatter plot illustrating systematic relationship, in schizophrenia patients, between psychosis severity ratings and the percentages of stages in which a subject switched back to the previously-better stimulus, before completing the stage. Abbreviations: NC, normal controls; LPS, low-positive-symptom patients; HPS, high-positive-symptom patients, BPRS, Brief Psychiatric Rating Scale.

How do we understand our finding of elevated rates of lose-shifting in the most psychotic individuals in our sample? Acute psychosis has often been associated with excessive striatal dopamine (Laruelle and Abi-Dargham, 1999, Howes et al., 2012), which has, in turn, been associated with reduced sensitivity to punishments (Dodds et al., 2008) and, consequently, reduced cognitive flexibility (lose-shifting; Cools et al., 2006). Several other pieces of evidence, however, might lead one to associate increased psychosis with elevated cognitive flexibility. For example, reduced metabolic activity in prefrontal cortex is often seen as a hallmark of schizophrenia (Weinberger, 1988). Importantly, hypofrontality has been associated with both an increased number of val alleles in the Val¹⁵⁸Met COMT polymorphism (Egan et al., 2001) and excessive striatal dopamine (Meyer-Lindenberg et al., 2002). As noted above, both an increased number of val alleles in the Val¹⁵⁸Met COMT polymorphism and lower concentrations of dopamine in the PFC have been associated with increased lose-shifting on PRL-like tasks (though not better WCST performance). In a study specifically examining the role of COMT genotypes in cognitive flexibility in schizophrenia, Nolan and colleagues (2004) found that Met homozygotes showed better acquisition of an imitation rule but greater impairment in reversing the initial rule, whereas val carriers showed greater cognitive flexibility, as measured by the costs of task-switching. This fits well with the result of Krugel et al. (2009), described above.

The rapidity with which psychotic individuals switch in the context of a PRL task may also be seen as consistent with the observation that SZ patients require less evidence to draw conclusions based on observed patterns. These findings come largely from the results of studies using the “Beads” task (Garety et al., 1991, Moritz and Woodward, 2005). Increased rates of shifting to the alternative in a PRL task might be seen as another example of “jumping to conclusions”.

What is important to note is that the fact that some SZ patients exhibit stuck-in-set behavior on reversal learning tasks, while other shift more readily than is adaptive, suggests that SZ patients may be either compulsive, or impulsive, depending on their symptom profiles or genotypes, with respect to SNPs relevant to reinforcement learning. For example, while some findings point to the COMT val158met polymorphism conferring risk for the diagnosis of schizophrenia, an alternate possibility is that particular allele counts are closely associated with subtypes of the disorder. The same may be true of other genotypes coding for aspects of dopamine or serotonin system function. The influence of dopamine and serotonin on cognitive flexibility, as assessed by probabilistic reversal learning tasks, for example, is of particular interest, from the standpoint of schizophrenia research, due to the suspected involvement of these neurochemicals (especially dopamine) in the positive and negative symptoms of the disorder (Laruelle and Abi-Dargham, 1999, Abi-Dargham, 2007, Toda and Abi-Dargham, 2007). Furthermore, all effective antipsychotic medications modulate activity at dopamine (especially D2) receptors, and most second-generation drug block serotonin receptors, as well (Meltzer et al., 2003). The fact that antipsychotics vary in their affinity for D2 and 5HT receptors (Kusumi et al., 2015) has led to possible differences between subgroups of SZ patients on different antipsychotic medications, as I describe below.

6. Antipsychotic drugs and cognitive flexibility schizophrenia

Given that all effective antipsychotic drugs (APDs) occupy dopamine D2-type receptors (almost all being antagonists; Seeman, 1987, Seeman and Tallerico, 1998) and that many also occupy postsynaptic serotonin receptors (Meltzer et al., 2003), it is reasonable to think that APDs might modulate reinforcement learning and cognitive flexibility. Despite clear evidence that both reinforcement learning and cognitive flexibility are influenced by activity in dopaminergic and serotonergic pathways, however, there are only a limited number of findings from the human cognitive neuroscience literature regarding whether APDs alter cognitive flexibility in schizophrenia patients. In evaluating the potential effects of APDs on learning and decision making, it is first important to distinguish between antipsychotics that are relatively selective for D2 receptors (more common among first-generation antipsychotics, like haloperidol) and “dirtier” compounds, with affinity for a variety of receptors (more common among second-generation antipsychotics, like clozapine). It is also important to distinguish direct effects of APDs on cognition from indirect effects on cognition, such as improvements resulting from the amelioration of symptoms.

Findings reported above indicate that limiting dopamine transmission at D2 receptors, either through D2 receptor blockade or acute tyrosine and phenylalanine depletion (Robinson et al., 2010b, Janssen et al., 2015), improves punishment processing at the expense of reward processing in healthy human subjects. The administration of 5HT2 receptor antagonists has also been shown to increase sensitivity to aversive outcomes, at the expense of rewarding ones, leading to a reduction in the number of perseverative errors during the performance of a reversal learning paradigm (Boulougouris et al., 2008, Boulougouris and Robbins, 2010). Based on these findings, one might expect APD administration to improve cognitive flexibility in schizophrenia patients. In fact, there have been several reports of improvements in cognitive flexibility associated with APD administration to schizophrenia patients, such as improvements in WCST performance (Sumiyoshi et al., 2003) and performance on an anti-saccade task (Harris et al., 2006). A number of other studies, however, report that APD administration has either no effect on cognitive flexibility in schizophrenia (Verdoux et al., 1995, Purdon et al., 2001), or no effect independent of that due to symptom amelioration (Borkowska et al., 2002, Daban et al., 2005).

As noted above, there is evidence that atypical anti-psychotics such as clozapine can cause symptoms in schizophrenia similar to those observed in obsessive-compulsive disorder (Schirmbeck et al., 2013, Fonseka et al., 2014). At least one study (Beninger et al., 2003) has found that the administration of second-generation antipsychotics is associated with impaired performance on the Iowa Gambling Task, which is thought to depend heavily on sensitivity to the frequency and magnitude of monetary losses (Bechara et al., 1994; Brown et al., 2016). Furthermore, Leeson et al (2009) showed that atypical anti-psychotics were associated with greater deficits in extra-dimensional shifting on the ID/ED task than typical anti-psychotics. By contrast, SGAs do not appear to impact performance on more procedural/BG-dependent learning tasks (Beninger et al., 2003, Kumari et al., 2015). Having a more precise sense of the effects of antipsychotic drugs on cognitive flexibility in schizophrenia will require the administration of refined measures of cognitive flexibility in the context of randomized controlled clinical trials.

7. General Conclusions

In short, due to the heterogeneity of the disorder, the question of whether SZ patients showed pathologically reduced or increased cognitive flexibility is too simple. Instead, subgroups of SZ patients appear to show excessive rigidity, while other subgroups show excessive instability of learning. The fact that these subgroups of patients exist points to heterogeneity in the functioning of frontostriatal circuits in patients with the disorder, likely themselves the product of three factors: 1) genetic make-up; 2) illness state/profile; and 3) medication type and dose. The circuits implicated in deficits in cognitive flexibility in schizophrenia are the circuits responsible for carrying out goal-directed behavior, including executive control structures (which are activated in the service of accomplishing the goal), default, or task-negative, structures (which are deactivated while goal-directed behavior is being carried out), and salience network structures (which respond to biologically-important events and trigger the activation and deactivation of the other networks).

With regard to the question of whether impairments in cognitive flexibility are trait markers, or markers of the product of illness state: one possibility is that they are both. In other words, there are clear genetic effects on cognitive flexibility, but cognitive flexibility may also change with illness state. For example, if acute psychosis is, in fact, associated with striatal hyperdopaminergia, then one would expect acutely psychotic individuals to show increased reward sensitivity, reduced punishment sensitivity, and reduced cognitive flexibility, as people with more 9R alleles of the DAT1 gene appear to (Aarts et al., 2010, den Ouden et al., 2013). If striatal hyperdopaminergia is not characteristic of individuals with schizophrenia between episodes of acute and severe psychosis, then one might expect remitted psychotic individuals to show normative reward and punishment sensitivity and normative cognitive flexibility. One must also consider the potential influences of different types of antipsychotic drugs on cognitive flexibility in schizophrenia, especially with regard to whether an antipsychotic is relative selective for D2-type dopamine receptors (like haloperidol), or if it is known to block D1-type dopamine receptors and/or serotonin receptors.

Additionally, just as there is variability in the severity of frontostriatal dysfunction among individual with psychotic illness, there is great variability in the genotypes concerning the specific dopamine and serotonin SNPs mentioned above. While there is evidence that some genes that code for aspects of dopamine system function may be schizophrenia risk genes (Talkowski et al., 2008), there is still tremendous variation among individuals with psychotic illness, and associations of specific genotypes with diagnoses may be mediated by associations with intermediate phenotypes (such as deficits in reinforcement learning or cognitive flexibility; e.g., Nolan et al., 2004).

Finally, while there is ample evidence of dysfunction in frontostriatal systems in schizophrenia, frontostriatal dysfunction is, by no means, unique to schizophrenia, and there is tremendous variability in the extent of frontostriatal dysfunction among people with psychotic illness. Importantly, Culbreth et al. (2015) found that the between-group differences they observed in reversal learning performance were mediated by inter-individual variability in the activity of cognitive control regions. That is: their results indicated that poor reversal learning performance was driven by PFC dysfunction in both patients and controls, with patients showing relatively worse performance due to a higher prevalence of PFC dysfunction. In short: frontostriatal circuit dynamics may track variability in cognitive flexibility not just in individuals with severe mental illness, but in the general population, as well.

Furthermore, while frontostriatal dysfunction is common among individuals with psychotic illness, it is also prevalent in numerous other psychiatric and neurological conditions (Clark et al., 2009, Klanker et al., 2013). Not surprisingly, cognitive flexibility has been a subject of intense study in other patient samples, with some patient samples (obsessive-compulsive disorder, Parkinson’s disease) being characterized, to some extent, by excessive rigidity in the selection of cognitive and motor plans (Swainson et al., 2000, Cools et al., 2001, Cools et al., 2006, Robbins et al., 2012). Major depressive disorder (MDD) is another example of a condition where cognitive flexibility has been a focus of study (Eshel and Roiser, 2010). Results in MDD have been mixed, as well (Chen et al., 2015), but it is interesting to note that “excessive” cognitive flexibility, as quantified by the tendency to shift prematurely following probabilistic negative feedback, has also been observed in individuals diagnosed with MDD (Murphy et al., 2003, Dombrovski et al., 2013, Dombrovski et al., 2015). This enhanced tendency to switch in response to negative feedback, even when it might be invalid, is consistent with evidence of either an oversensitivity to punishment in MDD, or impaired control over negative feedback, behaviorally and/or neurally (Taylor Tavares et al., 2008, Mueller et al., 2015).

These findings further support the idea that the extent to which habitual or goal-directed action dominates is determined by frontostriatal interactions (Gillan et al., 2011), and that frontostriatal dysfunction can lead to either pathological set-shifting or set-maintenance, depending on the nature of the dysfunction. Cognitive inflexibility can arise either because frontal systems involved in goal-representation and action selection fail to override representations of habits, or because frontal or striatal systems are insufficiently sensitive to punishments. Inordinate switching can occur if frontal representations of goals are degraded, and/or because frontal or striatal systems are overly-sensitive to punishments, and insufficiently sensitive to rewards. In the context of probabilistic reversal learning tasks, individuals can achieve learning in multiple ways: by either integrating reward prediction errors in model-free way, or by inducing a rule in a model-based way (Daw et al., 2005, Chen et al., 2015, Costa et al., 2015). Accordingly, neither excessive perseveration nor excessive “shiftiness” is specific to a complex disorder like schizophrenia or major depression, as both rigidity and flexibility can arise from multiple neural mechanisms, likely to be characteristic of subgroups of patients, possibly with specific symptom profiles. In sum, our ability to improve cognitive flexibility in individuals with psychiatric illness likely depends on our ability to identify the underpinnings of particular deficits at the levels of genes and neural circuits.

Highlights for Review.

1. Different subgroups of patients show deficits in set-shifting vs. set-maintenance.

2. There is heterogeneity in the functioning of frontostriatal circuits in patients.

3. Cognitive flexibility is influenced by activity in dopamine and serotonin systems.

4. Cognitive flexibility is likely influenced by illness state and profile.

5. Cognitive flexibility is likely influenced by antipsychotic drug type and dose.