Probabilistic Reversal Learning Impairments in Schizophrenia

Abstract

Impairments in feedback processing and reinforcement learning appear to be prominent aspects of schizophrenia (SZ), which may relate to symptoms of the disorder. Evidence from cognitive neuroscience investigations indicates that disparate brain systems may underlie different kinds of feedback-driven learning. The ability to rapidly shift response tendencies in the face of negative feedback, when reinforcement contingencies are reversed, is an important type of learning thought to depend on ventral prefrontal cortex (PFC). Schizophrenia has long been associated with dysfunction in dorsolateral areas of PFC, but evidence for ventral PFC impairment in more mixed. In order to assess whether SZ patients experience particular difficulty in carrying out a cognitive function commonly linked to ventral PFC function, we administered to 34 patients and 26 controls a modified version of an established probabilistic reversal learning task from the experimental literature (Cools et al. 2002). Although SZ patients and controls performed similarly on the initial acquisition of probabilistic contingencies, patients showed substantial learning impairments when reinforcement contingencies were reversed, achieving significantly fewer reversals [Χ^2(6)=15.717, p=0.008]. Even when analyses were limited to subjects who acquired all probabilistic contingencies initially (22 patients and 20 controls), patients achieved significantly fewer reversals [Χ^2(3)=9.408, p=0.024]. These results support the idea that ventral PFC dysfunction is a prevalent aspect of schizophrenic pathophysiology, which may contribute to deficits in reinforcement learning exhibited by patients. Further studies are required to investigate the roles of dopaminergic systems in these impairments.

One of the most common neuropsychological findings in the schizophrenia (SZ) literature is that of impaired attentional set-shifting, as evidenced by studies using tasks like the Wisconsin Card Sort Test (WCST) and the intradimensional/extradimensional (ID/ED) attentional set-shifting task. This deficit has often been linked to dysfunction of dorsolateral prefrontal cortex (DLPFC), one of the most frequently-observed neural correlates of schizophrenia (Weinberger et al. 1986; Berman et al. 1988). One possible source of set-shifting deficits in patients may be set-learning impairments related to limitations in DLPFC-dependent attentional and working memory resources. Another possible source of set-shifting difficulties, however, may be a specific impairment in reversing learned associations. The reversal of learned associations is known to depend on ventral and medial areas of prefrontal cortex (PFC) from a variety of studies involving both human and nonhuman animal subjects, including both lesion studies (Dias et al. 1996; Fellows and Farah 2003; Hornak et al. 2004) and those using physiological data acquisition (Rolls et al. 1996; Cools et al. 2002; Evers et al. 2005). It is thought that ventral PFC contributes to the rapid reversal of learned associations through the integration and online representation of the reinforcement value of stimuli and actions (Rolls 1996; Roesch and Olson 2005; Schoenbaum and Roesch 2005), while areas of medial prefrontal cortex, such as anterior cingulate cortex (ACC), figure critically in the monitoring of performance and detection of errors, processes which lead to behavioral modifications (Carter et al. 1998; Paulus et al. 2002; Holroyd et al. 2004).

The idea that dysfunction in ventral and medial areas of PFC might be a prominent feature of SZ is supported by an increasing amount of evidence pointing to both structural (Goldstein et al. 1999; Crespo-Facorro et al. 2000; Pantelis et al. 2003; Davatzikos et al. 2005) and functional (Bertollo et al. 1996; Carter et al. 2001; Quintana et al. 2003) abnormalities in these brain regions in schizophrenia. Behaviorally, schizophrenia patients have shown impairment on a number of tasks thought to be dependent on intact ventral PFC, including delayed alternation/object alternation (Seidman et al. 1995) and the Iowa Gambling Task (IGT; Ritter et al. 2004; Shurman et al. 2005), although these results have not been unequivocal (see, e.g., Wilder et al. 1998). Up to this point, the possibility that schizophrenia involves a specific impairment in reversal learning has been most directly addressed by studies using the ID/ED attentional set-shifting task (Elliott et al. 1995; Pantelis et al. 1999). Studies involving schizophrenia patients using this task have found that, even when compared with patients with PFC lesions, significantly more patients with SZ fail to reach criterion on reversals of relatively simple rules not requiring the consideration of multiple stimulus dimensions (Elliott et al. 1995; Pantelis et al. 1999).

While the above results suggest that patients with schizophrenia have particular difficulty using feedback to guide future choices, none of the paradigms mentioned above was designed to test the ability of subjects to adjust to sudden shifts in reinforcement contingencies in a sensitive way. In the ID/ED task, for example, the stimulus-response (S-R) rules are easily acquired by most subjects, and the difficulty in the task comes from the need to associate responses with increasingly complex stimuli. In probabilistic reversal learning tasks, the choice of a particular stimulus is reinforced most, but not all of the time, and correct responses are occasionally followed by negative feedback. Thus, the difficulty in performing these tasks comes from the need to integrate feedback over a number of trials (Cools et al. 2002). Because not all instances of negative feedback signal a shift in reinforcement contingencies, subjects need to consider each single instance of feedback in the context of the recent history of reinforcement. Thus, participants typically make more errors, and require more trials to reach the learning criterion than when performing reversal learning when guided by probabilistic, as opposed to fully-reliable, feedback.

Neuroimaging studies (Cools et al. 2001; Cools et al. 2002; Cools et al. 2006) have demonstrated the dependence of probabilistic reversal learning on ventral PFC (especially the lateral aspect), as well as dopaminergic systems in the brain. In addition, several studies (Huettel et al. 2002; Paulus et al. 2004) support a role for this same region in monitoring reinforcement trends during learning tasks. By contrast, there is evidence that the gradual acquisition of probabilistic contingencies depends more heavily on subcortical structures in the basal ganglia, and less on cortical structures (Knowlton et al. 1996; Seger and Cincotta 2005). The results of multiple studies (Keri et al. 2000; Weickert et al. 2002; Beninger et al. 2003; Keri et al. 2005) suggest that the acquisition of probabilistic contingencies in schizophrenia patients may be relatively unimpaired, perhaps indicative of a relatively intact basal ganglia function in schizophrenia. Thus, patients with SZ could show intact initial discrimination learning, but impaired reversal learning.

Adopting a modified version of the paradigm used by Cools et al. (2002), we performed a direct test of probabilistic reversal learning performance in patients and controls. In order to assess whether patients showed particular impairment in the reversal of learned discriminations, we compared the proportions of discriminations achieved and reversed, and the proportions of errors made in discrimination and reversal stages by subjects. We predicted that patients would perform as well as controls on the acquisition of initial discriminations, but show substantial impairment, relative to controls, in the reversal of learned discriminations. Such a dissociation, we argue, would reflect a particular dysfunction in schizophrenia of ventral PFC circuits involved in the detection of sudden shifts of reward contingencies.

Method

Patients

Thirty-four outpatients with a diagnosis of schizophrenia, based on the Structured Clinical Interview for DSM-IV (SCID-I; First et al. 1997), were recruited from the Maryland Psychiatric Research Center (MPRC; Table 1). All patients were clinically stable, as determined by their treating clinician. All patients were tested while receiving stable medication regimens (no changes in type or dose within 4 weeks of study; see Table 1).

Table 1.

Characterizing information for patients and controls.

Measure	Control Mean (SD)	Patient Mean (SD)	p-value
Age	45.01 (11.30)	45.47 (8.40)	0.860
Age at Illness Onset	-	23.28 (7.854)
Education (years)	15.15 (2.31)	12.94 (2.36)	0.001
Paternal Education (years)	14.95 (3.48)	13.91 (4.37)	0.334
Gender (M:F)	14:12	23:11	0.298
Race			0.239
African American	5	12
Caucasian	21	21
Other	0	1
Medication Regimens
Single First-generation Med		7
Haloperidol	-	4
Fluphenazine		3
Single Second-generation Med		16
Clozapine		7
Risperidone	-	4
Olanzapine		4
Aripiprazole		1
Multiple Antipsychotics		11
Clozapine + Risperidone	-	10
Clozapine + Fluphenazine		1
Clinical Ratings
BPRS	-	35.97 (10.51)
SANS	-	33.18 (16.82)
Calgary Depression Scale	-	3.30 (3.63)
Standard Neuropsychology
Wechsler Test of Adult Reading	110.38 (11.62)	98.24 (17.15)	=0.002
Hopkins Verbal Learning Score	28.04 (4.96)	21.76 (5.53)	<0.001
Spatial Span Scaled Score	11.46 (2.16)	7.97 (3.03)	<0.001
Letter-number Sequencing	16.12 (3.01)	11.74 (3.23)	<0.001

Control subjects

Twenty-six healthy control subjects participated in the study. They were recruited through a combination of newspaper advertisements and random phone number dialing and were extensively screened for Axis I and II disorders using the SCID-I (First et al. 1997) and the Structured Interview for DSM-III-R Personality Disorders (SIDP-R; Pfohl et al. 1989). Subjects were also screened for family history of psychosis and medical conditions that might impact cognitive performance, including drug use. All control subjects were free of any significant personal psychiatric and medical history, had no history of severe mental illness in first-degree relatives, and did not meet criteria for current substance abuse or dependence.

General Procedures

After explanation of study procedures, all subjects provided written informed consent. Before signing consent documents, patients had to demonstrate adequate understanding of study demands, risks, and means of withdrawing from participation in response to structured probe questions. All subjects were compensated for study participation.

Data collection occurred through a battery of standard and experimental neuropsychological tests. Tests included measures of word reading (the Wechsler Test of Adult Reading, or WTAR), word list learning, and working memory. Patients were also characterized using the Brief Psychiatric Ratings Scale (BPRS; Overall and Gorman 1962), the Scales for the Assessment of Negative Symptoms (SANS; Andreasen 1984), and the Calgary Depression Scale (CDS; Addington et al. 1992).

Cognitive Task

Participants performed a simple probabilistic discrimination learning task, involving the presentation of one discrimination to learn at a time. Participants attempted three discriminations in total, each involving a different pair of gray-scale fractal stimuli. At the beginning of the session, subjects were told that one stimulus from each pair was the “correct” stimulus, and that they had to figure out which one it was and choose it, with a left or right button press, depending upon which side of the screen it appeared. Participants were told, however, that no stimulus was correct all the time, and furthermore, that the “correct” stimulus changed occasionally. Subjects were then told that, once they decided which stimulus was “correct”, they should “stick with it” until they felt that the correct stimulus had changed. The choice of one of the stimuli was reinforced 80% of the time (a choice of the other stimulus was reinforced the remaining 20% of the time). If subjects reached a criterion of 9 choices of the more-frequently reinforced stimulus in a block of ten trials (in 50 or fewer total trials), the reinforcement contingencies for the stimuli were reversed: the stimulus that was reinforced 80% of the time previously was now reinforced only 20% of the time. In order to reach criterion in this phase, subjects need to detect the shift in reinforcement contingencies and learn to choose the stimulus now reinforced 80% of the time. If subjects succeeded in reaching criterion in this phase, the reinforcement contingencies were reversed one more time, and subjects needed to learn to choose the stimulus that was originally correct, in order to reach criterion. Each subject thus completed up to 2 reversal stages with each stimulus pair (or up to 6 total, along with the 3 initial discrimination stages).

Statistical Analyses

In order to assess differences in the numbers of discrimination and reversal stages achieved by participants in the two groups, we determined the numbers of subjects in each group achieving given numbers of discriminations and reversals and performed chi-square tests. Subjects were said to have “achieved” a stage if they had reached the learning criterion within 50 trials. We also quantified the number of “first reversals” achieved by subjects, which could range from 0 to 3. This term was used to designate the initial reversal of a learned discrimination. We compared the subjects from each group who achieved all possible discriminations on the numbers of first reversals achieved, in order to determine if subjects who showed relatively intact discrimination learning still show impairment in the reversal of acquired discriminations. We also compared the subjects from each group who achieved all possible discriminations on the proportion of error trials during each stage type. An “error” reflected the choice of the less-frequently reinforced stimulus, regardless of the type of feedback given on that trial. We used an Analysis of Covariance (ANCOVA) to compare average error rates during discrimination stages and reversal stages. This ANCOVA used factors of group and learning stage (discrimination versus reversal) and used performance on the WTAR as a covariate to control for the effects of more between-group global differences in intellectual functioning.

Results

Initial Discrimination Learning Performance

We first examined the ability of both groups to learn the initial discriminations. As shown in Figure 1A, the groups performed similarly. This impression was confirmed statistically by the results of a chi-square test [Χ²(3)=2.084, p=0.555]. Greater than 60% of subjects in each group achieved all 3 discriminations, while another 15% of subjects in each group achieved 2 discriminations.

Figure 1.

Task performance by controls (gray bars) and patients with schizophrenia (black bars). (A) Percentages of subjects in each group achieving possible numbers of initial discriminations. (B) Percentages of subjects in each group achieving different numbers of reversals (6 were possible). (C) Numbers of initial discriminations reversed at least once by participants in each group (maximum of 3). Twenty controls and 22 patients achieved all three initial discriminations. (D) Average percentages of error trials on initial discrimination and first reversal trials by participants.

Reversal Learning Performance

We next examined the ability of both groups to reverse learned discriminations. When we compared the number of reversals achieved by subjects in each group, we found a significant group difference [Χ²(6)=15.717, p=0.008]. As shown in Figure 1B, while over 60% of controls achieved 5 or more reversals, only 24% of patients did. While only 25% of controls achieved fewer than 3 reversals, fully 56% of patients did. Even when we compared only the (22) patients and (20) controls who achieved all three discriminations on the number of discriminations reversed at least once, we found we found a significant group difference [Χ²(3)=9.408, p=0.024; see Fig. 1C]. Even among patients who achieved all 3 discriminations, 36% reversed 0 or 1 of those discriminations, compared with 6% of controls. Almost 90% of controls reversed all 3 discriminations at least once, whereas only 45% of patients did.

Finally, we examined percentages of error trials in discrimination and reversal stages for the 22 patients and 20 controls who achieved all three discriminations (Figure 1D). An ANCOVA, using WTAR performance as a covariate, revealed an interaction between the factors of group and learning stage [initial discrimination versus reversal; F(1,39)=4.12; p<0.05], indicating that the effect of group on learning performance was modulated by test type. Post-hoc least-squared means tests confirmed the presence a group difference in reversal error rates [t(39)=2.12, p<0.05], as well as the lack of a group difference in the percentage of discrimination errors [t(39)=0.80].

Correlation Analyses

Pearson correlation analyses revealed weak associations between experimental measures and several negative symptom ratings. There was a trend toward a significant correlation between reversal errors and total scores on the SANS, as well as scores on the affective blunting subscale of the SANS (see Table 2). There was a trend toward a significant correlation between discrimination errors and scores on the alogia subscale of the SANS. There was no evidence of systematic associations between experimental measures and ratings of positive symptoms or depression.

Table 2.

Correlations between Symptom Ratings and Standard Neuropsychological Measures and Error Percentages during Discrimination and Reversal Phases.

	Discrimination Phase		1st Reversal Phase
Symptom Rating Scale	Correlation with Error %	p value of correlation	Correlation with Error %	p value of correlation
CDS Total	-0.04	0.82	-0.20	0.29
SANS Total	0.24	0.19	0.34	0.06
Affective Blunting Subscale	0.18	0.32	0.34	0.06
Alogia Subscale	0.33	0.06	0.21	0.28
Avolition Subscale	0.23	0.19	0.20	0.28
Anhedonia Subscale	-0.03	0.85	0.15	0.42
BPRS Total	0.22	0.23	-0.01	0.97
Psychosis Cluster	0.25	0.15	-0.21	0.28
Disorganization Cluster	0.18	0.33	0.01	0.95
Neg. Symptom Cluster	0.21	0.23	0.28	0.14
Depression Cluster	-0.17	0.34	-0.12	0.52
	Discrimination Phase		1st Reversal Phase
Neuropsychol. Measure	Correlation with Error %	p value of correlation	Correlation with Error %	p value of correlation
HVLT Total	-0.37	0.03	-0.02	0.93
WTAR - Scaled Score	-0.41	0.02	-0.39	0.03
Letter-Number Sequencing	-0.45	0.01	-0.38	0.03
Spatial Span - Scaled Score	-0.22	0.22	-0.05	0.80

Abbreviations: CDS = Calgary Depression Scale; SANS = Scale for the Assessment of Negative Symptoms; BPRS = Brief Psychiatric Rating Scale; HVLT = Hopkins Verbal Learning Test; WTAR = Wechsler Test of Adult Reading.

Correlation analyses revealed systematic associations between experimental measures and two standard neuropsychological measures: the WTAR and letter-number sequencing. The fact that correlations with both measures were significant for both discrimination errors and reversal errors suggests that both probabilistic learning and reversal capture some aspects of executive and general intellectual functioning. Finally, verbal learning scores correlated significantly with discrimination errors, but not reversal errors.

Discussion

The above results provide support for the idea that patients with schizophrenia have a particular impairment in the reversal of learned associations, according with the previous reports of Pantelis et al. (1999) and Elliot et al. (1995). These results are consistent with other evidence of ventral PFC dysfunction in SZ, in that a considerable body of work indicates that intact ventral PFC function is required for efficient reversal learning (Cools et al. 2002; Fellows and Farah 2003). In fact, the performance of our patients shows a striking resemblance to that of the orbitofrontal lesion patients of Fellows and Farah (2003). Ventral PFC is thought to contribute to the successful performance of tasks like reversal learning by participating in a kind of working memory - namely, the online retention of outcomes associated with certain actions - which enables the detection of sudden shifts in reward contingencies (Rolls 1996; Schoenbaum and Roesch 2005). One can potentially think of other supposedly OFC-dependent experimental tasks, such as the Iowa Gambling Task, in this way (Frank and Claus 2006). Findings of impairments on tasks like the IGT suggest that a prominent cognitive deficit in schizophrenia may involve difficulty in using changing feedback to rapidly learn associations.

Whereas most evidence of OFC dysfunction in SZ has emerged in the last decade or so, findings of dorsolateral PFC dysfunction in SZ are long-established (Weinberger et al. 1986; Berman et al. 1988). In fact, it is likely that dysfunction in multiple PFC subregions contributes to learning impairments in SZ. It is now apparent from neuroimaging studies that successful performance on feedback-driven learning tasks like the WCST (Monchi et al. 2001) and conditional associative learning (Boettiger and D’Esposito 2005) depends upon interactions among multiple subregions of PFC, with each involved in the online representation of a kind of information (such as reward expectancies, in the case of ventral PFC).

While patients and controls in our study achieved significantly different numbers of reversals, they achieved similar numbers of initial discriminations, and did not differ significantly in the proportion of errors they made in discrimination learning phases. This result is consistent with the finding of several studies (Keri et al. 2000; Weickert et al. 2002; Beninger et al. 2003; Keri et al. 2005) indicating that the gradual acquisition of probabilistic contingencies in schizophrenia patients may be relatively intact, as well as the idea that different neural systems may support rapid and gradual feedback integration. The gradual integration of probabilistic feedback over many trials seems to rely more on the basal ganglia.

Future studies of probabilistic reversal learning in schizophrenia are needed to elucidate the precise neural underpinnings of reversal learning impairments in schizophrenia. While numerous studies of reversal learning have implicated ventral PFC, recent studies also point to a role for medial prefrontal cortex (including anterior cingulate cortex), the ventral striatum (nucleus accumbens), and multiple neuromodulatory systems (especially those involving dopamine and serotonin; Cools et al. 2002; Evers et al. 2005; Cools et al. 2006). Cools and colleagues (2001; 2006), for example, have shown that increases in tonic dopamine levels in the basal ganglia, following the administration of a dopamine precursor, lead to decrements in reversal learning performance in patients with Parkinson’s disease. Schizophrenia has also been associated with excessive dopaminergic tone in the basal ganglia (Breier et al. 1997; Abi-Dargham et al. 1998), along with reduced stimulus-evoked activity in the ventral striatum (Juckel et al. 2006). Thus, multiple systems could contribute to poor reversal learning in SZ patients.

Because all effective antipsychotic medications exert their therapeutic effect by acting at D2 dopamine receptors (Kapur and Mamo 2003), and thus necessarily interact with the brain system most-frequently implicated in reward processing (Montague et al. 2004), future studies of probabilistic reversal learning in schizophrenia should also consider the role that antipsychotic medications possibly play in modulating the effects of the disease. Furthermore, because antipsychotic medications vary in their affinities for diverse receptor types (Remington and Kapur 2000), there is reason to believe that different classes of antipsychotics may affect reversal learning in somewhat different ways. The results of several studies suggest that high doses of potent D2-blocking medications (first-generation antipsychotics) have a relatively detrimental effect on basal ganglia-driven probabilistic learning, whereas atypical antipsychotic medications, which bind more strongly to D1 and serotonin receptors in the cortex, may have a more adverse effect on the performance of reinforcement learning tasks dependent on ventral PFC (Beninger et al. 2003; Keri et al. 2005). Unfortunately, we were not able to study medication effects in this study in a systematic way, as subgroups of patients on similar medication regimens in our study were not well-matched in terms of demographics or symptom profiles. The unique effects that individual antipsychotic medications have on aspects of reinforcement learning need to be studied in the context of controlled clinical trials.