Probabilistic Reversal Learning in Schizophrenia: Stability of Deficits and Potential Causal Mechanisms

Abstract

Although individuals with schizophrenia show impaired feedback-driven learning on probabilistic reversal learning (PRL) tasks, the specific factors that contribute to these deficits remain unknown. Recent work has suggested several potential causes including neurocognitive impairments, clinical symptoms, and specific types of feedback-related errors. To examine this issue, we administered a PRL task to 126 stable schizophrenia outpatients and 72 matched controls, and patients were retested 4 weeks later. The task involved an initial probabilistic discrimination learning phase and subsequent reversal phases in which subjects had to adjust their responses to sudden shifts in the reinforcement contingencies. Patients showed poorer performance than controls for both the initial discrimination and reversal learning phases of the task, and performance overall showed good test–retest reliability among patients. A subgroup analysis of patients (n = 64) and controls (n = 49) with good initial discrimination learning revealed no between-group differences in reversal learning, indicating that the patients who were able to achieve all of the initial probabilistic discriminations were not impaired in reversal learning. Regarding potential contributors to impaired discrimination learning, several factors were associated with poor PRL, including higher levels of neurocognitive impairment, poor learning from both positive and negative feedback, and higher levels of indiscriminate response shifting. The results suggest that poor PRL performance in schizophrenia can be the product of multiple mechanisms.

Introduction

Individuals with schizophrenia show marked disturbances in motivation, decision making, and performance of daily life activities. The ability to learn from positive and negative feedback is central to functioning in each of these domains. It has been suggested that abnormalities in motivation and goal-oriented decision making may arise from maladaptive reward learning.¹ Probabilistic reversal learning (PRL) tasks are frequently used to assess these deficits. Although individuals with schizophrenia often show performance deficits on PRL tasks (for a review see Waltz and Gold²), our understanding of the reliability and clinical significance of these impairments, as well as the underlying factors that contribute to them, is incomplete.

In PRL paradigms, choices of 2 visual stimuli are probabilistically rewarded, with the choice of one stimulus being more frequently rewarded than the other. Individuals learn to select the more-frequently-rewarded (“correct”) stimulus, and, once a learning criterion is reached, the reward contingencies reverse, and the previously “incorrect” stimulus becomes the “correct” stimulus. Thus, there are 2 distinct phases within PRL tasks. The initial “discrimination phase” involves the capacity to learn probabilistic contingencies, which relies on cumulative learning associated with dopaminergic activity and subcortical structures in the basal ganglia.³ The “reversal phase” involves modifying value representations after the contingencies are reversed, using explicit feedback, and relies on prefrontal cortical functioning (especially orbitofrontal).^4,5 Successful learning during the reversal phase involves the ability to accurately detect errors, inhibit responding to formerly rewarded stimuli, and overcome avoidance of previously punished stimuli.⁶

Most studies in schizophrenia patients have found significant impairments during both the initial discrimination and reversal phases.^4,5,7–9 One PRL study found impaired reversal learning and intact discrimination learning.¹⁰ Examinations of whether poor PRL relates to particular types of clinical symptoms have yielded mixed results. Negative symptoms have been linked to impaired probabilistic reward learning^11,12 and PRL.^10,13 However, others have failed to replicate such associations.⁴ These inconsistencies may reflect the use of a variety of PRL paradigms with diverse task parameters and relatively small, heterogeneous samples. Furthermore, nearly all studies have been cross-sectional, and inconsistencies in symptom correlates may reflect poor psychometric stability. Recently, studies have begun to move beyond the question of whether patients show impaired PRL to the question of why they show such deficits. For example, Gold et al¹ posited that impaired reward learning may reflect difficulty with the maintenance of value representations rather than insensitivity to reward and punishment. Consistent with this proposal, impaired PRL has been associated with diminished working memory.^4,5,14,15 Other factors that may contribute to poor PRL have been identified through analyses of the types of errors patients make. Some studies, but not others, have found that feedback valence affects learning (ie, patients are selectively impaired for learning from positive, but not negative, feedback)^1,16–20 (but see Culbreth et al⁴, Cicero et al¹⁴, and Fervaha et al²¹). Others found that patients showed impaired response selection strategies; whereas controls tend to stay with the same selection following positive feedback and shift following negative feedback, patients have sometimes been found to shift more frequently and indiscriminately, regardless of the preceding feedback valence^5,7,8,10 (but see Waltz et al¹⁹). Further consideration of these factors may help to more precisely explain why patients demonstrate PRL impairments.

This study had 4 aims. The first was to assess PRL in a large sample of stabilized outpatients with schizophrenia and matched healthy individuals. Based on previous findings, we expected that the schizophrenia group would show impaired performance on both phases of the PRL task. Second, we looked for potential contributors to PRL impairments by examining associations with external measures of cognition (particularly working memory) and specific types of errors related to feedback valence and response selection strategies. Third, we examined whether PRL was associated with clinical symptoms (particularly negative symptoms) or antipsychotic medication. Fourth, given the paucity of psychometric data on PRL, we evaluated the test–retest reliability of PRL in the patient sample.

Method

Participants

The sample included 126 individuals with schizophrenia and 72 demographically-matched healthy controls. Patients were recruited from outpatient clinics at University of California, Los Angeles (UCLA), the Veterans Affairs Greater Los Angeles Healthcare System (GLA), and from local clinics and housing facilities. Selection criteria for patients included (1) Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) diagnosis of schizophrenia, determined with the Structured Clinical Interview for DSM-IV (SCID-I/P; First et al²²), (2) age 18–60 years, (3) no clinically significant neurological disease, (4) no history of serious head injury, (5) no evidence of substance dependence in the past 6 months and no evidence of substance abuse in past month, (6) no history of mental retardation or developmental disability, and (7) clinically stable (ie, no inpatient hospitalizations for 3 months prior to enrollment, no changes in antipsychotic medication type in the 4 weeks prior to enrollment). Diagnostic assessments were conducted by interviewers trained according to established procedures.²³ Patients were taking medications at clinically determined dosages.

Control participants were recruited through advertisements posted on websites. Selection criteria for healthy controls included (1) no psychiatric history involving schizophrenia spectrum disorder (including avoidant, paranoid, schizotypal, or schizoid personality disorders), or other psychotic or recurrent mood axis I disorder according to the SCID-I and SCID-II, (2) no family history of a psychotic disorder among first-degree relatives, and (3) no history of substance or alcohol dependence/abuse disorder. Criteria concerning age, neurological disease, and head trauma were the same as listed above. For all participants, written informed consent was obtained in accordance with procedures approved by the Institutional Review Board at GLA.

As shown in table 1, the groups did not significantly differ in sex, age, or ethnicity. The patients had lower personal education levels than controls but the groups did not differ in parental education. The controls had significantly higher scores on all neurocognitive domains with the exception of reasoning and problem solving.

Table 1.

Demographic, Cognitive, and Symptom Characteristics for Patients and Controls

	Patients	Controls	Group Comparisons
Sex (% male)	68%	55%	X 2 = 3.2, P > .05
Ethnicity (% Hispanic)	19%	23%	X 2 = 0.4, P > .05
Race (%)
American Indian/Alaskan	0.1%	0%	X 2 = 4.2, P > .05
Asian	5%	3%
Hawaiian/Pacific Islander	1 %	5%
Black/African American	32%	27%
White	55%	58%
More than 1 race	5%	6%
	M (SD)	M (SD)
Age	48.8 (11.2)	46.7 (8.1)	t = 1.4, P > .05
Education	13.1 (1.9)	14.6 (1.8)	t = −4.9, P < .01
Father education	13.5 (3.6)	12.8 (3.8)	t = 1.1, P > .05
Mother education	12.6 (3.1)	13.6 (2.8)	t = −1.9, P > .05
Neurocognition (MCCB)
Processing speed	38.9 (13.1)	49.5 (8.0)	t = −6.2, P < .001
Attention/vigilance	39.4 (12.7)	51.5 (9.7)	t = −7.1, P < .001
Working memory	37.8 (9.7)	48.3 (8.7)	t = −7.6, P < .001
Verbal learning	39.9 (9.1)	47.9 (9.6)	t = −5.8, P < .001
Visual learning	39.6 (12.1)	47.1 (11.7)	t = −4.3, P < .001
Reasoning and problem solving	45.9 (10.5)	46.7 (8.9)	t = −0.5, ns
Social cognition	35.4 (11.2)	47.6 (9.8)	t = −7.7, P < .001
Overall composite	32.9 (12.1)	47.0 (8.9)	t = −8.7, P < .001
Symptoms
CAINS experiential	16.1 (7.1)
CAINS expressive	5.0 (4.1)
CAINS total	21.2 (9.5)
PANSS positive	18.5 (7.7)
PANSS negative	15.7 (6.9)
PANSS disorganized	12.5 (4.5)
Medications	N (%)
Atypical antipsychotics	104 (83%)
Typical antipsychotics	13 (10%)
Atypical and typical antipsychotics	3 (2%)
Mood stabilizer/antidepressant only	4 (3%)
No medication	2 (2%)
Chlorpromazine 100-mg equivalent dose (SD)	381.69 (291.9)

Note: MCCB, MATRICS Consensus Cognitive Battery; CAINS, Clinical Assessment Interview for Negative Symptoms; PANSS, Positive and Negative Syndrome Scale.

Procedures

Participants with schizophrenia were administered the PRL paradigm twice (baseline, 4-week retest); controls received it once. At baseline, both groups completed a neurocognitive battery, and patients also received assessments of clinical symptoms.

PRL Task

We employed an adaptation of a well-established PRL task developed by Cools et al.²⁴ Subjects were presented with 3 blocks of trials, each with 2 unique gray-scale fractal pattern stimuli that appeared side-by-side on each trial (left/right positioning of the stimuli was randomized; see figure 1). Each block was comprised of up to 3 phases, and each phase had a maximum of 50 trials. The first phase was always a discrimination phase in which subjects had a chance to learn which of the 2 stimuli was more likely to lead to reward. For each trial, subjects were instructed to select which of the 2 stimuli (which remained on the screen for up to 6s) they believed was “correct” based on the feedback they received. The choice of one stimulus resulted in a win 80% of the time and a loss 20% of the time, while the choice of the other resulted in a win 20% of the time and a loss 80% of the time. The criterion for successful learning was nine choices of the better stimulus in a run of 10 trials. If this criterion was not achieved within 50 trials, the block ended. If the criterion was achieved, the discrimination phase ended and the first reversal phase began. In this phase, the reward contingencies were reversed: the previously infrequently rewarded stimulus became the better choice and the previously frequently rewarded stimulus became the poorer choice. If the same learning criterion was achieved (9 out of 10 choices of the better stimulus), the first reversal phase ended, and a second reversal phase began, in which the better and worse stimuli were again reversed and the same criterion for successful learning was applied.

Fig. 1.

Example probabilistic reversal learning (PRL) block.

Notes: Blue squares indicate the “correct” stimuli for that phase. If the Discrimination Phase is achieved in a given block, the participant advances to the first Reversal Phase. If the Discrimination Phase is not achieved, the participant moves on to the next block. After the third phase, all participants begin the Discrimination Phase of the next block. If the acquisition criterion is not achieved in the Discrimination or Reversal phases of a block, the participant moves on to the next block, and is presented with a new set of stimuli, and a new probabilistic discrimination to learn. The 3 blocks are identical in format, only the fractal pattern stimuli change between blocks.

A practice block, in which participants were presented with one discrimination phase, was followed by the test blocks. After subjects completed the first block (or if they failed to reach criterion in any phase of the block), they were presented with a new pair of fractal stimuli for the next block, and used the same sequence of phases and reinforcement probabilities. Upon completing block 2, the third block was initiated. It should be noted that this task is not designed to test for a differential deficit in reversal learning, as the discrimination and reversal components of the task are not matched on the psychometric characteristics required to demonstrate such a deficit.^25,26 Participants were paid an hourly rate for participation in the study.

Neurocognition

The MATRICS Consensus Cognitive Battery (MCCB; Nuechterlein et al²⁷) was used to assess neurocognitive functioning. The MCCB includes 10 tests to measure 7 domains of cognition: speed of processing, attention/vigilance, working memory, verbal memory, visual memory, reasoning and problem solving, and social cognition.

Symptom Assessments

Symptoms were evaluated using the Positive and Negative Syndrome Scale (PANSS; Kay et al²⁸) and the Clinical Assessment Interview for Negative Symptoms (CAINS; Horan et al²⁹). The PANSS is a 30-item structured interview that yields symptom factor scores; 3 factors were included in the analyses: positive, negative, and disorganized.³⁰ The CAINS is a 13-item instrument that yields two subscales which comprehensively measure the two primary negative symptom factors: Experiential and Expression. Symptom raters were trained to a minimum intra-class correlation coefficient (ICC) of 0.80 using established procedures.³¹

Statistical Analyses

For the PRL task, the main dependent variables (collapsed across blocks) were total number of discrimination phases completed (out of 3), total number of reversals (out of 6), and the proportion of errors (ie, choice of the less frequently reinforced stimulus, regardless of feedback) within the discrimination and reversal phases. We also evaluated response selection strategies, ie, whether subjects “stay” or “shift” their selections following various types of feedback received on the preceding trial. The types of selection strategies were categorized according to the valence of the feedback (positive, negative) and the validity of the feedback (feedback was considered “valid” if the subject was rewarded for a “correct” choice or punished for an “incorrect” choice). Based on a prior study,⁸ we focused on the following feedback-choice sequences: (1) Valid Lose-stay: when a subject was not rewarded for choosing the poorer stimulus and subsequently stayed with the same stimulus; (2) Valid Lose-shift: when a subject was not rewarded for choosing the poorer stimulus and subsequently shifted; (3) Invalid Lose-shift: when a subject was not rewarded for choosing the better stimulus and subsequently shifted to the alternate stimulus; and (4) Valid Win-shift: when a subject was rewarded for choosing the better stimulus and subsequently shifted to the alternate stimulus. To examine learning based on feedback valence, we focused on Valid Lose-stay and Valid Win-shift events (lower rates of these events indicate better learning from negative and positive feedback, respectively). To examine the rates of shifting, we focused on Valid Lose-shift, Valid Win-shift, and Invalid Lose-shift events (Invalid Win-shift events were extremely rare). Quantifying these types of events provided a measure of how prone a subject was to switch between response alternatives, regardless of feedback.

Distributions and skewness/kurtosis for the PRL measures indicated that total number of discriminations and reversals were non-normally distributed, and nonparametric statistics were used for these variables (Mann-Whitney U tests for between comparisons, Spearman correlation coefficients for correlational analyses). The distributions for all other variables were normal and analyzed using parametric methods.

Analyses of the baseline data were as follows. First, we conducted group comparisons for the discrimination and reversal phases in terms of total achieved and proportion of errors. Second, we examined between-group differences for positive and negative feedback using a repeated-measures ANOVA with group as the between-subject factor and feedback-choice sequence (ie, Valid Lose-stay, Valid Win-shift) as the within-subject factor. We then tested for between-group differences in shifting rates using a repeated-measures ANOVA with group as the between-subjects factor and feedback-choice sequence (ie, Valid Lose-shift, Valid Win-shifts; Invalid Lose-shift) as the within-subject factor. Third, we conducted a subgroup analysis that included only those participants (in each group) who achieved all three discriminations, and compared the groups on total reversals, errors, and correlations with external variables. This subgroup provides a direct analysis of deficits in reversal learning, distinct from deficits in basic probabilistic reinforcement learning. Fourth, correlational analyses within the entire patient group examined whether total discriminations, total reversals, and error rates for each phase were associated with neurocognition, symptoms, and chlorpromazine equivalents for antipsychotic medication dose (based on Andreasen et al³² and Leucht et al³³).

To evaluate test–retest of PRL task performance within the patient group, we computed ICCs for the total number of discriminations and reversals achieved and error rates for the discrimination and reversal phases across the two assessments. Practice effects were examined with paired-samples t tests; within-group effect sizes were calculated by dividing the mean difference score by its SD.

Results

Group Comparisons on the Primary PRL Task Variables

Discrimination Phases.

Patients (M = 2.2; SD = 1.0) achieved significantly fewer discriminations than controls (M = 2.5; SD = 0.9; Z = −2.4, P = .02). In terms of error rates, patients (M = 0.45; SD = 0.11) had a significantly higher proportion of error trials than controls (M = 0.40; SD = 0.11; t = 2.93, P < .01).

Reversal Phases.

Patients (M = 2.9; SD = 2.3) achieved significantly fewer reversals than controls (M = 3.6; SD = 2.3; Z = −2.2, P = .03). In terms of error rates, patients (M = 0.53; SD = 0.10) also had a significantly higher proportion of error trials than controls (M = 0.49; SD = 0.10; t = 2.0, P < .05).

Group Comparisons of Response Selection Types

Discrimination Phase.

The feedback valence (positive vs negative) RM-ANOVA revealed significant main effects of group (F = 8.2, P < .01) and feedback type (F = 5.0, P = .03), but no significant interaction. Patients made more of both types of errors (stayed more when they lost and shifted more when they won) and both groups had a higher rate of Valid Lose-stays than Valid Win-shifts. The RM-ANOVA for shifting revealed significant main effects of group (F = 6.0, P = .02) and feedback type (F = 193.4, P < .001), as well as a significant interaction (F = 6.1, P < .01). As shown in figure 2, patients shifted significantly more than controls for Valid Lose and Valid Win events, but there was no group difference for Invalid Lose-shifts. The observation that patients actually shifted more frequently than controls, both overall, and in response to valid negative feedback, argues against the idea that SZ patients, as a group, are characterized by stuck-in-set behavior.

Fig. 2.

Proportion of feedback events in discrimination phases for patients and controls. Note: *P < .05.

Reversal Phase.

The feedback valence (positive vs negative) RM-ANOVA revealed significant main effects of group (F = 4.1, P < .05) and feedback type (F = 125.3, P < .01), but no significant interaction. Patients made more of both types of errors (stayed more when they lost and shifted more when they won) and both groups had a higher rate of Valid Lose-stays than Valid Win-shifts. The shift-rate RM-ANOVA revealed a significant main effect of feedback type (F = 193.4, P < .001) and an interaction that was significant at a trend-level (F = 2.6, P = .07), but no main effect of group (F < 1). As shown in figure 3, patients and controls shifted significantly more in response to valid losses than valid wins or invalid losses.

Fig. 3.

Proportion of feedback events in reversal phases for patients and controls.

Subgroup Analyses

When we looked only at participants that achieved all 3 discriminations (patients = 64; controls = 49), there were no longer any significant group differences in total reversals achieved or error rates in the reversal phases. When we compared the patients who did and did not achieve the 3 discriminations, the 2 groups did not differ in terms of demographic variables, symptom levels, medication type, or medication dose. However the 2 groups did differ on 3 measures of neurocognitive functioning, with the poor performers showing worse speed of processing (good discriminators M = 42.0, SD = 12.8; poor discriminators M = 36.1, SD = 12.9, F = 5.5, P = .02), working memory (good discriminators M = 40.3, SD = 9.2; poor discriminators M = 35.1, SD = 9.5, F = 9.8, P < .01), and overall composite (good discriminators M = 35.8, SD = 11.1; poor discriminators M = 29.9, SD = 12.4, F = 8.0, P < .01).

Correlational Analyses Within Schizophrenia Sample

As shown in table 2, we observed significant correlations, with medium effect size, between MCCB working memory scores and multiple measures of PRL performance, including discriminations and reversals achieved, error rates, and win-shift and lose-stay response selection types. Associations with PRL measures were not limited to working memory scores, as similar patterns were also seen for all MCCB domains, with the exceptions of reasoning and problem solving and social cognition.

Table 2.

Spearman’s and Pearson’s Correlations Between PRL Dependent Measures and External Variables (Patients)

	Discriminations Achieveda	Discrimination Error Rateb	Reversals Achieveda	Reversal Error Rateb	Total Valid Win Shiftb	Total Valid Lose Stayb
Neurocognition
Processing speed	0.18*	−0.28**	0.25**	−0.22*	−0.19*	−0.27**
Attention/vigilance	0.20*	−0.27**	0.20*	−0.12	−0.16	−0.29**
Working memory	0.33**	−0.35**	0.28**	−0.18	−0.26**	−0.33**
Verbal learning	0.14	−0.23**	0.17	−0.20*	−0.17	−0.29**
Visual learning	0.22*	−0.25**	0.19*	−0.08	−0.20*	−0.19*
Reasoning and problem solving	0.11	−0.06	0.08	0.01	−0.04	−0.08
Social cognition	0.09	−0.09	0.10	−0.08	−0.07	−0.10
Overall composite	0.27**	−0.33**	0.28**	−0.17	−0.23*	−0.33**
Symptoms
CAINS experiential	−0.11	0.22*	−0.01	−0.05	0.08	0.12
CAINS expressive	−0.05	0.10	−0.02	−0.11	0.05	0.02
CAINS total	−0.14	0.21*	−0.01	−0.08	0.08	0.10
PANSS positive	−0.04	0.09	0.04	−0.05	−0.002	−0.08
PANSS negative	−0.12	0.23**	−0.07	−0.07	0.14	0.06
PANSS disorganized	−0.13	0.20*	−0.13	0.01	0.06	0.06
Chlorpromazine 100-mg equivalent dose	0.04	−0.10	−0.02	−0.07	−0.03	−0.11

Note: ^aSpearman’s correlations.

^bPearson’s correlations.

*P < .05; **P < .01.

Regarding relationships between PRL performance and symptoms, there were significant (though modest) correlations between patients’ discrimination phase error rates and total negative symptom scores from both the CAINS and PANSS (table 2). Higher overall error rates in the discrimination phase were also associated with higher CAINS experiential negative symptom ratings. There was also an association between disorganized symptoms on the PANSS and discrimination error rates.

Stability of PRL Performance Within the Schizophrenia Group

We examined test–retest reliability and practice effects in 112 patients. The primary indices had acceptable test–retest reliability (ie, discriminations achieved ICC = 0.57, reversals achieved ICC = 0.68, discrimination error rate ICC = 0.49, reversal error rate ICC = 0.54). Practice effects had relatively small effect sizes (discriminations achieved d = 0.01, reversals achieved d = 0.05).

Discussion

Consistent with previous studies, the schizophrenia group showed poorer performance than controls for both the initial discrimination and reversal learning phases of the PRL task. There was notable heterogeneity within schizophrenia, such that individual patients show a variety of patterns of intact or impaired performance on the discrimination vs reversal learning phases. However, there was a large subgroup of patients who had intact initial discrimination learning and were not impaired in reversal learning. This overall pattern suggests that schizophrenia patients, as a group, do not appear to have impairment in reversal learning, over and above deficits in the acquisition in the discrimination phase.

The fact that no reversal learning impairment, relative to controls, was observed in the subgroup of patients with intact initial discrimination learning contradicts the results of 2 previous PRL studies that had much smaller samples. One study used the same PRL task and found impairments in reversal learning among those patients (n = 22) who achieved all initial probabilistic discrimination phases.¹⁰ The second study used a different task and found that a subgroup of patients (n = 7) who could learn the initial discriminations showed impaired reversal learning, but only with low rewards, not for higher rewards.⁹

Although there was substantial variability in probabilistic learning among patients in this study, our examination of characteristics that distinguished the patient subgroups revealed few differences. Patients with worse discrimination learning had poorer neurocognition in the domains of processing speed, working memory, and overall composite, relative to patients with better discrimination learning. However, the 2-patient subgroups did not differ in demographic features, clinical symptoms/characteristics, or medication type/dosages.

Not all aspects of reinforcement learning appear to be impaired in schizophrenia. Simple/deterministic discrimination learning is generally not impaired in schizophrenia (for a review see Barch et al³⁴). Furthermore, studies using electrophysiology and functional magnetic resonance imaging (fMRI) methods suggest that neural responses to positive and negative feedback are largely intact.^35–37 However, problems in schizophrenia arise on tasks such as PRL tasks, in which participants are required to learn complex (eg, probabilistic) reinforcement contingencies by maintaining and updating representations of the stimulus and action values over time. Our findings are consistent with recent proposals^1–4 that individuals with schizophrenia have a deficit in the ability to use feedback valence and prediction errors to update value representations and guide choice. These proposals are supported by recent fMRI findings^4,8 that poor PRL performance in schizophrenia is associated with disturbances that extend beyond the striatum to cognitive control network regions, including dorsolateral prefrontal cortex, anterior cingulate cortex, and dorsal parietal cortex.

The current study considered 3 possible contributors to impaired PRL performance: working memory impairments, a selective impairment in the ability to learn from positive (rather than negative) feedback, and elevated rates of indiscriminate shifting. All 3 of these factors were associated with impaired discrimination learning by patients on the PRL. Of the cognitive variables, working memory had the numerically largest correlation with performance, although several additional domains were associated at comparable magnitudes. These findings are consistent with previous studies that show reinforcement learning and working memory abilities are closely linked.^15,38–40 There were somewhat weaker relationships between the cognitive and PRL variables in the reversal than the discrimination phases, which may reflect more restricted range in the neurocognitive variables. Notably, we found that working memory and processing speed were the only subdomains that separated the patient subgroups. Thus, our findings support the notion that impaired reward learning may partly reflect difficulty with maintaining representations of choice value.¹

We next evaluated the possibility of valence-specific learning impairments. Consistent with prior studies, patients made more errors following valid positive feedback.^5,20,41–43 However, in contrast to the above-noted studies, patients also responded sub-optimally following negative feedback by more often staying with the “incorrect” stimulus. A similar pattern was observed in recent studies in which patients inadequately used both positive and negative feedback for subsequent decision-making.^4,8

We then evaluated response strategies, finding that patients shifted more than controls in response to both valid negative and valid positive feedback. That is, schizophrenia patients had elevated rates of shifting both appropriately and inappropriately. This finding is consistent with the few prior studies that examined this issue^4,8 and supports the argument that patients are more prone to indiscriminate shifting. This finding is consistent with the few prior studies that examined this issue and found that patients were more prone to indiscriminant shifting.^4,8 Further, higher levels of shifting were associated with lower levels of working memory and other aspects of neurocognition, consistent with the notion that unstable value representations contribute to poor PRL performance.²⁴ It has been hypothesized that such unstable value representations may result from a failure to integrate information between the cognitive control network and reward processing regions.⁴ Overall, our analyses of error types suggested that poor PRL performance in schizophrenia patients stemmed from abnormalities in multiple aspects of reinforcement learning. Further, they argue against the idea the schizophrenia patients are especially prone to making perseverative errors in the context of reinforcement learning, as suggested in past reports using the Wisconsin Card Sort Test.⁴⁴

We found modest support for an association in feedback-based learning and negative symptoms.¹³ There were multiple small-to-medium correlations between performance measures and negative symptom scores, consistent with the results of several prior studies of probabilistic and deterministic reversal learning tasks in schizophrenia.^7,10,19

Patients’ performance on the PRL had relatively good test–retest reliability, with ICCs ranged from 0.49 to 0.68. Although these ICC’s fall below acceptable levels for use in clinical trials,⁴⁵ they suggest that scores are relatively stable over a 1-month period. Furthermore, there were minimal practice effects associated with repeated administration. With further development, PRL tasks could provide useful measures of reward processing for clinical trials beyond currently available interview-based methods (see Green et al⁴⁶).

This study had several limitations. First, the patients were receiving antipsychotic medications at clinically-determined dosages, which could have impacted performance on the PRL task.^47,48 However, we did not find any association between PRL task performance and dose or type of antipsychotic medication dose (expressed in oral-chlorpromazine-equivalent units). Further, reversal learning deficits and reduced reward-related striatal activation have been found in unmedicated and first-episode schizophrenia patients.^5,49 Nevertheless, additional research in unmedicated samples is required to unequivocally determine the impact of this confound. Second, the sample was chronically ill. Although a few studies have found discrimination and reversal impairments on different PRL tasks in recent-onset patients,^5,7 further research in PRL performance in early course and prodromal subjects would be useful. Third, our results should be interpreted with caution because we did not correct for multiple comparisons.

In summary, our results provide support for the idea that individuals with schizophrenia perform poorly on PRL tasks because they have significant difficulty learning initial probabilistic discriminations, and this difficulty is related to neurocognitive impairments, failure to adequately incorporate feedback information, and a tendency to indiscriminately shift. Given that numerous studies have pointed to impaired probabilistic discrimination learning, future studies might attempt to advance our understanding of the causal mechanisms by examining the pathway that prevents feedback information from adequately informing behavior. Such a fine-grained analysis would help to link findings related to neurocognitive deficits and feedback learning, and may offer insights into deficient motivation and goal-oriented behavior.

Funding

This project was supported by a VA MERIT Award to W.P.H.