Regressing to Prior Response Preference After Set Switching Implicates Striatal Dysfunction Across Psychotic Disorders: Findings From the B-SNIP Study

Abstract

Difficulty switching behavioral response sets is established in psychotic disorders. In rodent models, prefrontal lesions cause difficulty initially switching to new response sets (perseverative errors) while striatal lesions cause difficulty suppressing responses to previous choice preferences (regressive errors). Studies of psychotic disorders have not previously assessed these 2 error types. Bipolar and Schizophrenia Network on Intermediate Phenotypes (B-SNIP) participants included probands with schizophrenia (N = 212), psychotic bipolar (N = 192), and schizoaffective disorder (N = 131), their first-degree relatives (N = 267,226,165 respectively), and healthy controls (N = 258). Participants completed the Penn Conditional Exclusion Test (PCET) to assess cognitive set switching and the Brief Assessment of Cognition in Schizophrenia (BACS) to assess generalized neuropsychological dysfunction. All proband groups displayed elevated rates of perseverative and regressive errors compared to controls. After correcting for generalized cognitive deficits to identify specific deficits in set shifting and maintenance, there were no significant group differences for perseverative errors, while the increased rate of regressive errors remained significant. Level of regressive errors was similar across proband groups with minimal correlations with antipsychotic medication dose, clinical ratings, and demographic characteristics. Relatives of schizophrenia patients showed increased rates of regressive errors, but familiality of this trait was significant only in bipolar pedigrees. Regressive errors were partially independent of generalized cognitive deficits, suggesting a potentially informative and specific cognitive deficit across psychotic disorders. Preclinical data indicate that this deficit could be related to altered function in a neural system that may include the dorsal striatum or other elements of frontostriatal systems.

Introduction

Executive dysfunction is a hallmark of schizophrenia^1–4 and has been reported in first-degree relatives of schizophrenia patients.^5–7 In bipolar disorder, especially those with a history of psychosis, executive disturbances have been reported even after acute episodes of illness.^3,4,8–11 Most studies of relatives of individuals with bipolar disorder have indicated executive deficits.^12–15

The Wisconsin Card Sorting Test (WCST), and its computerized analog the Penn Conditional Exclusion Test (PCET), have been widely used to assess cognitive flexibility in psychotic disorders. Impairments on these measures are typically interpreted to indicate prefrontal dysfunction. However, there is growing recognition of striatal involvement in executive functions^16,17 and of its potential role in neurocognitive deficits associated with psychotic disorders¹⁸ based on frontostriatal loop anatomy and neurophysiological findings in primates¹⁹ and rodents.^20,21

In set shifting tests such as the WCST and PCET at least 2 separate processes are required for optimal performance: (1) the initial inhibition of the previously correct response set after a category shift so that an alternative choice can be made, and (2) a reliable preference of a new response set relative to the previously learned response preference. Traditional scoring conventions for neuropsychological tests do not separate errors related to these 2 cognitive demands.²² There is evidence from nonhuman primates and human studies that the initial inhibition of a previously established response set (to allow a new alternative choice to be selected) is mediated in prefrontal cortex, and that the subsequent maintenance of a new response set is more dependent upon the basal ganglia.^23–25 Set-shifting tests have been developed for rodents and are conceptually similar to those used in human subject. For example, 2 different complex stimuli are presented in a trial and only one choice is correct and followed by positive reinforcement (eg, food reward). Each complex stimulus can vary on several dimensions (eg, odor, texture, and/or spatial location) and each rodent must learn which attribute is relevant. After learning the initial discrimination, a set-shift occurs in which a different stimulus attribute is correct and each rodent must now learn the new contingency. In rodent models, when a learned response set no longer leads to positive reinforcement, regions in prefrontal cortex have been shown to support the initial inhibition of the learned response set and selection of a new one.^20,26 Disruption of prefrontal function leads to an increase in perseverative errors in which a previously learned response set continues to be used despite a switch in the category rule and termination of positive feedback for that response in both animal models^20,27–31 and in clinical studies.^32–35 In contrast, the striatum plays a greater role in maintaining consistent performance of a newly learned response set in lieu of a previously established response set vis-à-vis the previous response preference. Temporary inactivation or pharmacologic alterations in glutamatergic signaling in rat dorsomedial striatum does not affect the initial shift away from a learned response set to a new choice preference, but it reduces the stable selection of the new response set. This is reflected in increased rates of regressive errors, when a previously learned response rule is chosen instead of the new correct response after the initial shift to the new correct rule.^36–39 Extensive experimentation involving temporary inactivation of different rodent frontal cortical and striatal areas has led to a model in which specific rodent prefrontal and striatal subregions support distinct, but complementary processes in support of behavioral flexibility. Specifically, orbitalfrontal and medial frontal subregions support the initial inhibition of a previous strategy and the generation of a new strategy, while the dorsal striatum enables selection of the new correct strategy.²⁰ To our knowledge differentiation of perseverative and regressive errors has yet to be considered in studies of cognitive flexibility in psychotic disorders.

The present study was designed to: (1) characterize the ability to make an initial shift in response set and to utilize that new response set without intrusions from a previously learned response preference (reflected in rates of perseverative and regressive errors, respectively) across psychotic disorders in a set shifting task, (2) examine these error rates and their familiality in first-degree relatives of probands with psychotic diagnoses, and (3) determine whether set shifting errors represent a specific cognitive deficit or a nonspecific manifestation of the generalized cognitive deficit associated with psychotic disorders.⁴⁰

Methods

Participants

The 5-site B-SNIP consortium⁴¹ was organized to address questions about phenotypic boundaries of psychotic disorders and the familiality of traits that define these disorders. Identical inclusion criteria and testing procedures were employed across sites as described previously. Briefly, individuals with a history of psychotic symptoms were recruited from local communities if they had at least 1 available first-degree relative aged 15–65 willing to participate in the study. Probands were required to have a lifetime diagnosis of a psychotic disorder across the schizophrenia, schizoaffective disorder, bipolar disorder continuum based on the Structured Clinical Interview for DSM Disorders (SCID)⁴² as determined at consensus diagnostic meetings. Probands were clinically stable and on consistent medication regimens for at least 1 month before testing. Clinical symptoms were assessed using the Positive and Negative Symptom Scale (PANSS),⁴³ Montgomery Asberg Depression Rating Scale (MADRS),⁴⁴ and Young Mania Rating Scale.⁴⁵ Healthy participants, recruited from the community, were required to have no personal history of a psychotic disorder or recurrent depression and no known immediate family history of these disorders. Demographic and clinical sample characteristics are presented in tables 1 and 2.

Table 1.

Demographic and Clinical Data for Probands With a History of Psychosis and Healthy Controls

	Healthy Controls		Schizophrenia		Schizoaffective		Bipolar With Psychosis		Findings
	n = 258		n = 212		n = 131		n = 192
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Age (years)	37.45	(12.63)	34.66	(12.31)	35.61	(11.64)	35.93	(12.89)	F = 2.02ns
Education (years)	15.04	(2.56)	12.93	(2.28)	13.10	(2.29)	14.43	(2.34)	F = 36.56a,**
Wide Range Achievement Test–IV: Reading test (SS)	103.64	(13.83)	95.71	(15.51)	97.88	(14.62)	102.74	(13.31)	F = 15.00b,**
	N	%	n	%	n	%	n	%
Sex
Male	107	41.5	141	66.53	52	39.7	69	35.9%	χ2 = 47.79c,**
Female	151	58.5	71	33.5	79	60.3	123	64.1%
Race
Caucasian	165	64.2	98	46.2	68	51.9	145	75.5	χ2 = 50.07d,**
African-American	69	26.8	101	47.6	55	42.0	37	19.3%
Other	23	8.9	13	6.1	8	6.1	10	5.2
			Mean	SD	Mean	SD	Mean	SD
Clinical variables
PANSS total			65.4	16.9	67.6	15.4	53.7	14.0	F = 45.35e,**
PANSS positive			16.7	5.6	17.7	4.8	12.9	4.5	F = 49.46e,**
PANSS negative			16.6	5.9	15.5	4.9	12.1	3.9	F = 47.61e,**
YMRS			5.3	5.9	7.1	6.5	5.8	6.6	F = 3.80f,*
MADRS			8.5	7.8	14.9	10.1	10.7	9.2	F = 24.08g,**

Note: ^aControls > schizophrenia and schizoaffective; bipolar > schizophrenia and schizoaffective.

^bControls > schizophrenia and schizoaffective; bipolar > schizophrenia and schizoaffective.

^cDisproportionate number of males in schizophrenia group.

^dDisproportionate number of African-Americans in schizophrenia group.

^eBipolar < schizophrenia and schizoaffective.

^fSchizoaffective > schizophrenia.

^gSchizoaffective > bipolar and schizophrenia.

*P < .05, **P ≤ .001.

Table 2.

Demographic Data, History of Psychosis, and Psychosis Spectrum Personality Traits for First-Degree Relatives

	Healthy Controls		Relatives of Schizophrenia Probands		Relatives of Schizoaffective Probands		Relatives of Bipolar With Psychosis Probands
	n = 258		n = 267		n = 165		n = 226
	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Findings
Age (years)	37.45	(12.63)	42.81	14.99	40.13	16.40	40.04	16.03	F = 5.65a,**
Education (years)	15.04	(2.56)	14.13	2.38	14.14	2.99	14.68	2.73	F = 6.29b,**
Wide Range Achievement Test–IV: Reading test (SS)	103.64	(13.83)	99.03	14.01	101.60	15.60	103.78	13.90	F = 6.19c,**
	N	%	n	%	n	%	n	%
Sex
Male	107	41.5	79	29.6	48	29.1	75	33.2	χ2 = 10.59d,*
Female	151	58.5	188	70.4	117	70.9	151	66.8
Race
Caucasian	165	64.2	150	56.4	105	64.0	178	79.1	χ2 = 33.79e,**
African-American	69	26.6	99	37.2	58	30.5	41	18.2
Other	23	8.9	17	6.4	9	5.5	6	2.7
Positive psychosis history			37	12.3	25	14.3	22	9.8
Relatives with no psychosis history
Elevated cluster A traits			41	13.7	24	13.7	30	12.9
Elevated cluster B traits			17	5.7	11	6.3	15	6.9
Not elevated			205	68.3	115	65.7	165	71.1

Note: ^aControls < relatives of schizophrenia.

^bControls > relatives of schizophrenia and schizoaffective.

^cRelatives of schizophrenia < controls and relatives of bipolar.

^dDisproportionate number of males in control group.

^eDisproportionate number of Caucasians in relatives of bipolar.

*P < .05, **P ≤ .001.

Procedures

Participants were administered the PCET and the Brief Assessment of Cognition in Schizophrenia (BACS). BACS findings in the B-SNIP sample have been reported previously and include significant generalized cognitive deficits in all proband groups with increasing deficits from bipolar to schizoaffective to schizophrenia probands with considerable overlap across disorders.⁴⁰

Antipsychotic dose (converted to chlorpromazine equivalents),⁴⁶ benzatropine (anticholinergic) dose, the presence (vs absence) of current antipsychotic treatments, mood stabilizers, antidepressants, clinical ratings, age, sex, race, and site all accounted for <5% of the variance in perseverative and regressive errors across all proband groups. Therefore, these parameters were not used as covariates in the analyses reported below. In secondary analyses including significant demographic variables as covariates, the findings reported below remained significant.

Measures

The PCET is a computerized test that evaluates cognitive set shifting processes and facilitates standardized administration in multi-site studies.^5,47,48 Participants are presented 4 objects and instructed to select the one that does not belong. The objects vary by size, shape, and line thickness (figure 1). Feedback is provided after each response (“correct” or “wrong”) so participants can determine the correct sorting principle. When 10 consecutive trials are sorted correctly, the sorting principle changes unbeknownst to the participant. The test ends if the first sorting principle is not learned to criterion within 48 trials, otherwise 2 more categories are administered.

Fig. 1.

PCET trial with 3 possible sorting principles (shape, size, and line thickness).

Error Types. According to the WCST manual there are 3 ways for a response to be considered perseverative. Thus, perseverative errors are heterogeneous and capture more than 1 type of error.²² The present report parses into 2 types, errors that occur after the first category shift: (1) Perseverative errors: persistent use of the prior response rule in the face of negative feedback until the first, correct choice for the new category rule has been selected, and (2) Regressive errors: a return to the prior response rule after the first unambiguous switch to the new, correct response rule. This definition of regressive errors is conceptually similar to the WCST failure to maintain set indicator but is less restrictive in defining set acquisition and more specific in that regressive errors must revert to the prior response set. In secondary analyses, we observed that a limited number of participants (13%–25%) “failed to maintain set” (according to the WCST manual criteria).⁴⁹ Using a Spearman approach, the correlation between regressive errors and failure to maintain set was low (r = −.19 to .13) indicating a lack of redundancy in the 2 ways of measuring errors in cognitive flexibility and greater variance in regressive error scores for differentiating subjects (a psychometric advantage). Furthermore, the rate of subsequent regressive errors was similar regardless of the number of unambiguously correct sorts. Analysis of regressive errors was restricted to performance when learning Category 2 because regressive errors to an earlier sorting category are more difficult to assess after 2 sorting principles have been learned and a third category is introduced. To allow a more direct comparison of perseverative and regressive errors, analyses of perseverative errors were also restricted to trials when the second response set was learned Category 2. Likewise, these error types were largely independent of the BACS Tower subtest (regressive errors: r = −.13 to −.20; perseverative errors: r = −.08 to −.09). To provide a direct comparison to prior PCET studies of psychotic disorders, effect sizes were computed for previously reported indicators of PCET accuracy^5,47,48 as well as for our measures of perseverative and regressive errors (table 4).

Table 4.

Effect Sizes for a Previously Reported PCET Indicator (Percent Correct) and for Perseverative and Regressive Errors

	SZ	SzAff	BP	SZ-Rel	SzAff- Rel	BP-Rel
Percent correct (all trials)	.46	.35	.16	.30	.17	.14
Category 2: regressive errors	.56	.68	.32	.38	.17	.13
Category 2: perseverative errors	.27	.15	.09	.06	.02	.00

Statistical Analyses

As is common with set shifting measures, PCET scores were not normally distributed and were resistant to conventional normalization transformations. Therefore, primary analyses were conducted with a nonparametric approach paralleling Spearman’s rank test in which rank order of performance across participants was analyzed with parametric statistics. Hierarchical linear modeling (HLM) was used to test for group differences in performance with participants nested within families. For relatives, analyses were conducted with all available participants and significant findings were followed by analyses excluding relatives with a history of psychosis or psychosis spectrum traits.

The composite score from the BACS was used as a covariate in analyses described below as an index of generalized deficit to examine the cognitive specificity of PCET performance abnormalities. This was done to determine whether effects of interest were uniquely informative (ie, still significant after correcting for BACS scores) rather than merely 1 manifestation of global neuropsychological impairment. As noted in our prior report, BACS scores were computed using age and sex stratified normative data⁵⁰ and adjusted for race.⁴⁰ Planned comparisons tested for differences among proband groups and between controls and each proband and relative group.

Familiality.

A heritability analysis to estimate familiality of cognitive flexibility impairments was performed using Sequential Oligogenic Linkage Analysis Routine software (SOLAR).⁵¹ In a design such as ours, an estimate of familiality (h ²) represents the portion of phenotypic variance accounted for by family membership. To test for the significance of familiality, a maximum likelihood ratio test compared phenotypic variation explained by family membership to a model assuming that no variation is explained by familial factors. A correction was applied to account for ascertainment bias as families were recruited through the identification of a psychotic proband and not a representative community sample.⁵² Given the smaller number of schizoaffective patients, and potential heterogeneity of depressed and bipolar subtypes, these analyses were restricted to schizophrenia and bipolar pedigrees.

Results

Deficits in Psychotic Probands

Schizophrenia probands more often failed to complete the first category than controls or the other probands [χ²(1,3) = 25.04, P < .001; table 3]. HLM analysis of Category 2 performance revealed higher rates of perseverative errors than controls for schizophrenia probands [F(1, 521) = 5.90, P = .01] but not for schizoaffective [F(1, 521) = 0.59, P = .44] or bipolar [F(1, 521) = 1.21, P = .27] probands. For regressive errors there were significant impairments across all proband groups [Schizophrenia: F(1, 521) = 24.47, P < .0001; Schizoaffective: F(1, 521) = 25.05, P < .0001; Bipolar: F(1, 521) = 12.76, P ≤ .0001]. The magnitude of this impairment did not differ across disorders.

Table 3.

Performance on Categories 1 and 2 on the Penn Conditional Exclusion Test (PCET)

	Probands					Relatives
		HC	SZ	SzAff	BP	SZ	SzAff	BP
Category 1
Percent completing category		89.6	75.4	84.0	88.1	86.7	87.3	91.5
Category 2
General (never reinforced) errors (in Category 2)	Mean (SD)	0.64 (1.74)	1.02 (2.32)	0.64 (1.29)	0.83 (2.12)	0.81 (2.06)	0.68 (2.86)	0.48 (1.12)
Prior to set acquisition: perseverative errors	Mean (SD)	2.16 (3.28)	3.05 (3.56)	2.66 (3.33)	2.46 (2.92)	2.34 (2.35)	2.21 (2.23)	2.17 (2.06)
After set acquisition: regressive errors	Mean (SD)	1.73 (3.78)	3.86 (6.42)	4.30 (6.47)	2.93 (4.79)	3.18 (5.04)	2.39 (4.35)	2.22 (4.01)

Note: Regressive errors in Category 2 were elevated in all proband groups and relatives of schizophrenia patients. Note that rate of perseverative errors was not different from controls in any proband or first-degree relative group.

Effects in First-Degree Relatives

Relatives of bipolar probands showed an increased rate of perseverative errors compared to controls [F(1, 521) = 3.72, P = .05]. Relatives of schizophrenia patients displayed a trend for a similar finding [F(1, 521) = 3.33, P = .07], while schizoaffective relatives did not [F(1, 521) = 1.42, P = .23]. Relatives of schizophrenia [F(1, 521) = 14.16, P < .001] and schizoaffective probands [F(1, 521) = 3.97, P < .05] had more regressive errors than controls (regardless of personal history of psychosis), but not relatives of bipolar probands [F(1, 521) = 1.03, P = .31]. Relatives of schizophrenia patients who had elevated Cluster A (psychosis spectrum) personality traits (within 1 trait of meeting criteria for any psychosis spectrum personality disorder) did not have more regressive errors than those without elevated traits [F(2, 232) = 0.17, P = .85].

Comparison With Previously Reported PCET Measures

Table 4 presents effect sizes for PCET measures used in the COGS schizophrenia study^5,53 alongside perseverative and regressive error measures found in the present study. In schizophrenia probands, the effects for total error scores across all categories were modestly lower than regressive errors in Category 2. The effect sizes for the rate of regressive errors in schizoaffective and bipolar patients were similar to those of schizophrenia probands and, for schizoaffective and bipolar patients, were approximately double the effect size seen with the total error measure. In relatives, effect sizes for raw total and regressive errors were similar for all 3 disorders.

Specific and Generalized Deficits

Probands.

When included in the model as a covariate, BACS scores were associated with both perseverative [F(1, 520) = 19.91, P < .0001] and regressive error rates [F(1, 521) = 35.63, P < .0001] (figure 2). When covarying for the BACS estimate of generalized deficit, none of the proband groups still had increased perseverative error rates compared to controls [Schizophrenia: F(1, 520) = 0.41, P = .52; Schizoaffective: F(1, 520) = 0.23, P = .63; Bipolar: F(1, 520) = 0.01, P = .92]. In contrast, after co-varying for generalized cognitive deficits, significant impairments were still evident for regressive errors in all proband groups [Schizophrenia: F(1, 520) = 6.15, P = .01; Schizoaffective: F(1, 520) = 10.51, P < .01; Bipolar: F(1, 520) = 4.98, P = .03] (figure 3). Thus, regressing back to a previously established response set rather than maintaining the new correct one appeared to be a specific deficit beyond the impairment expected based on generalized cognitive deficits in probands, while perseverative deficits were not specific deficits.

Fig. 2.

The propensity to perseverate on the previously established sorting principle after category shift did not vary across proband or relative groups. Statistical testing included BACS composite scores as covariates to assess the magnitude of group differences after controlling for the generalized cognitive deficit.

Fig. 3.

Despite positive feedback for correctly sorting to the new principle, all proband groups and relatives of schizophrenia patients showed significant difficulty maintaining the new sorting strategy. Instead, they were more likely than controls to revert to the first sorting principle learned.

Relatives.

Perseverative errors, after covarying for the generalized deficit, were not increased in any relative group [Schizophrenia: F(1, 520) = 1.67, P = .20; Schizoaffective: F(1, 520) = 0.55, P = .46; Bipolar: F(1, 520) = 2.55, P = .11]. After covarying the BACS composite from regressive errors, only relatives of schizophrenia participants showed higher error rates than controls [F(1, 520) = 9.37, P < .01]; relatives of bipolar and schizoaffective probands did not. The relatives of schizophrenia patients with no personal history of psychosis also showed elevated regressive errors compared to controls [F(2, 490) = 8.86, P < .01].

Familiality

Familiality of cognitive flexibility measures from the PCET were computed for schizophrenia and psychotic bipolar pedigrees (the 2 larger proband groups). Age and race were included in the model as covariates. Familiality estimates were nonsignificant in schizophrenia families for total errors (h ² = .00, P = .50), perseverative errors (h ² = .01, P = .48), and regressive errors (h ² = .00, P = .50). In contrast, statistically significant familiality was observed in psychotic bipolar pedigrees for total errors (h ² = .42, P < .001) and regressive errors (h ² = .35, P < .01) but not perseverative errors (h ² = .06, P = .31).

Comment

To our knowledge, this is the first cognitive flexibility study to distinguish heretofore heterogeneous errors by differentiating switching from and reverting to a previously established response preference. The key findings included: (1) Similarly increased rates of regressing to a previously established response set across psychotic disorders. This impairment was significant after controlling for the generalized cognitive deficit using BACS scores, indicating that regressive errors in a set switching test may represent an informative specific cognitive deficit across psychotic disorders. Deficits in perseverative error rate were not significant after controlling for BACS performance. (2) Elevated regressive errors were seen in first-degree relatives of schizophrenia patients even after controlling for generalized deficits; however, familiality estimates were significant only in bipolar pedigrees. Thus, while regressive error rates may be a useful strategy for tracking a common abnormality across psychotic disorders, the abnormality was not present and familial in any relative group so it may have limited utility for tracking familial risk.

In the present study, an error to the previously correct choice or set was separated into perseverative and regressive errors. One possibility is that regressive errors simply reflect a continual impairment of the same process with regressive errors indicating a more severe form of the deficit. However, depending on when errors occur in a set-shift, we argue that this error distinction engages different cognitive processes. In particular, a perseverative error only occurs during the initial trials of a set-shift during which the previously correct set is persistently chosen despite negative feedback. This reflects an inability to initially inhibit a previously correct choice or generate a new choice. In contrast, a regressive error only occurs after one “acquires” the new choice set by generating a new (correct) choice followed by positive feedback. In this manner, one only reaches the regressive stage by generating a new, correct response choice. At this point one has initially completed the necessary steps for a successful set-shift and must now learn to reliably execute this new response set. The degree to which these regressive type errors are independence from generalized cognitive dysfunction in response contingency learning remains to be established, but the present findings suggest that the ability to reliably execute a newly acquired strategy is selectively disrupted psychotic disorders and certain first-degree relatives.

General and Specific Cognitive Deficits

Probands showed some difficulty initiating new response choices when old response preferences were no longer reinforced, but this abnormality appeared to be a manifestation of the generalized cognitive deficit rather than a specific deficit because the effect was not significant after co-varying BACS scores. In contrast, the present findings provide evidence for a specific executive dysfunction in psychotic disorders, not previously reported, characterized by difficulties utilizing feedback to maintain a new correct response in favor of regressing to a previously learned choice preference, despite having received positive feedback for correctly using the new sorting principle. Importantly, unlike the case with perseverative errors, the increased prevalence of regressive errors was significant after controlling for generalized cognitive deficits. This suggests that errors regressing back to previous choice preferences may represent an informative specific deficit in psychotic disorders. Furthermore, although BACS composite scores indicated a declining severity of generalized cognitive impairments from schizophrenia to schizoaffective to psychotic bipolar disorder,⁴⁰ regressive errors were present at a similar level across proband groups both before and after correcting for generalized deficits. Thus, difficulty maintaining new choice preferences and inhibiting responses based on old choice patterns may represent a psychosis-related trait, unrelated to the specific diagnosis. One intriguing speculation is that this type of rigidity of cognitive sets in the face of disconfirming environmental feedback could be a mechanism for the persistence of delusional thinking across psychotic disorders.

Regressive Errors and the Striatum

In animal models of reversal learning and set shifting, regressive errors have been linked to the integrity of the dorsal striatum, while perseverative errors are associated more with frontal lobe dysfunction.^20,27,29 Moreover, event-related fMRI data during the WCST revealed increased striatal activation during negative feedback and when matching a new response choice following negative feedback.³³ When considered in conjunction with findings in the rodent literature, this pattern of activation suggests that the dorsal striatum is one brain area that plays a central role in reducing regressive errors and maintaining a new response set. Thus, the present findings may implicate dorsal striatal dysfunction as a psychosis-related trait manifest in a reduced ability to maintain new adaptive choice preferences. Within frontostriatal loops, the striatum is thought to link response sets with feedback and signal when the information being maintained in working memory stores needs to be refreshed.⁵⁴

Dopamine modulated striatal activity in humans has been associated with manipulating information in working memory and switching response preferences following a change in feedback for response choices.^55–57 Prefrontal dysfunction has been associated with a failure to switch response preferences despite feedback that a previous response rule is no longer correct.⁵⁸ The present findings suggest that either newly learned response sets are insufficiently linked to positive feedback, which could implicate an abnormal integration of dorsal and ventral striatal function, or that striatal signaling is insufficient to displace the old response preferences and/or update the contents of prefrontal working memory storage buffers. Either abnormality, or a combination of the 2, could account for why action selection fails to discard the outdated response preference and instead reverts to it despite consistent negative feedback for such choices.^59–61

Striatal Dopamine–Acetylcholine Interactions and Regressive Errors

Increased rates of regressive errors could be related to alterations in dopamine signaling¹⁸ or another dysfunction in the dorsal striatum.^62–64 Altered striatal dopamine signaling has long been associated with schizophrenia.^65–67 In rodent models, altered phasic dopamine signaling in dorsomedial striatum and nucleus accumbens has been associated with behavioral flexibility deficits when reward-related cues indicate a change in task contingencies.^68,69 Specifically, amphetamine that increases phasic dopamine signaling appears to interfere with the ability to learn that behavioral responses will no longer lead to reward.⁶⁹ Furthermore, set-shifting and reversal learning impairments have been induced by activation of dopamine D2 receptors in the rat medial striatum.⁷⁰ Interestingly, dopamine D2 receptors are found, in part, on striatal cholinergic interneurons⁷¹ and have inhibitory actions on these interneurons.⁷² Acetylcholine also plays a critical role in behavioral flexibility as enhanced release of acetylcholine in the rodent medial striatum reduces regressions to the previous response set while decreasing striatal acetylcholine release selectively increases regressive errors.⁷³ While this preclinical literature suggests possible neurochemical substrates for the increased regressive error rate across psychotic disorders, it also raises questions about a potential role of psychotropic medications in this deficit. While correlations with drug therapies were not significant, future studies are needed to determine whether treatments contribute to or reduce this specific cognitive impairment.

Familial Patterns

Studies of family members provide information about a potential role for elevated rates of regressive errors as an endophenotype for psychosis disorders. In general, while some positive findings were observed, findings from the present study provided only inconsistent support for the use of cognitive flexibility measures for advancing gene discovery. Elevated regressive errors were significant after controlling for the generalized deficit in schizophrenia patients, but this effect was neither familial nor related to psychosis spectrum personality traits. Regressive errors were familial in bipolar families, but their rate was not increased in relatives of psychotic bipolar patients. This discrepancy between deficits in the schizophrenia relatives and familiality in bipolar pedigrees might be accounted for by positive findings in only a subset of families. But, at present, our findings do not offer strong or consistent support for set switching measures as an intermediate phenotype for family genetic research into psychotic disorders.

Limitations

Certain aspects of the present study limit the interpretation and generalizability of the findings. First, switching response sets could not be evaluated in participants whose cognitive impairments were so pronounced that they could not learn the first response set (11.9%–24.6% of probands; 8.5%–13.3% of relatives). Second, probands qualified for the B-SNIP study only if at least one first-degree relative was willing and able to participate. This may have limited recruitment of more severely disturbed or socially isolated patients. At the same time, the small family pedigree size limits analyses of the heritability analyses. Third, the majority of patients were treated with psychotropic medications, most commonly antipsychotics. Although associations with antipsychotic dose and treatment were modest, there is also the possibility that the findings in probands reflect a threshold rather than dose-related effect of antipsychotic medication. Fourth, there were more opportunities to commit regressive than perseverative errors and reduced variance for perseverative errors, which may confer a psychometric advantage enhancing the statistical differences between the 2 error types. Further, the primary analyses focused on the second category and may have restricted the range of the 2 error types. Lastly, although the scoring method for classifying error types on a category set shifting paradigm is rooted in the preclinical literature, and the degree to which the neural basis of set switching in rodents and humans are comparable remains unclear. In particular, although the animal literature has linked dorsal striatal and prefrontal regions with regressive and perseverative errors, respectively, more work is needed to establish this translational bridge and interpretation of localization must be viewed in this context. It will be important to establish this homology in future studies and evaluate striatal activation in the context of set-shifting using event related fMRI. Antipsychotic treatment studies assessing set-shifting in this manner may also provide unique insights regarding striatal dopamine in the context of cognitive flexibility. Fortunately, the scoring method used to separate perseverative and regressive errors can be applied retrospectively to existing WCST and PCET data sets, and thus confirmation of this approach and the present findings can be conducted relatively quickly with existing data sets.

Funding

This study was supported in part by NIMH grants MH078113, MH077945, MH077852, MH077851, MH077862, MH072767, and MH083888.