Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Sep 1.
Published in final edited form as: Biol Psychiatry Cogn Neurosci Neuroimaging. 2016 Sep;1(5):460–473. doi: 10.1016/j.bpsc.2016.05.005

Probabilistic Reinforcement Learning in Patients With Schizophrenia: Relationships to Anhedonia and Avolition

Erin C Dowd a, Michael J Frank b, Anne Collins c, James M Gold d, Deanna M Barch e
PMCID: PMC5098503  NIHMSID: NIHMS796111  PMID: 27833939

Abstract

Background

Anhedonia (a reduced experience of pleasure) and avolition (a reduction in goal-directed activity) are common features of schizophrenia that have substantial effects on functional outcome, but are poorly understood and treated. Here, we examined whether alterations in reinforcement learning may contribute to these symptoms in schizophrenia by impairing the translation of reward information into goal-directed action.

Methods

38 stable outpatients with schizophrenia or schizoaffective disorder and 37 healthy controls underwent fMRI during a probabilistic stimulus selection reinforcement learning task with dissociated choice- and feedback-related activation, followed by a behavioral transfer task allowing separate assessment of learning from positive versus negative outcomes. A Q-learning algorithm was used to examine functional activation relating to prediction error at the time of feedback and to expected value at the time of choice.

Results

Behavioral results suggested a reduction in learning from positive feedback in patients; however, this reduction was unrelated to anhedonia/avolition severity. On fMRI analysis, prediction error-related activation at the time of feedback was highly similar between patients and controls. During early learning, patients activated regions in the cognitive control network to a lesser extent than controls. Correlation analyses revealed reduced responses to positive feedback in dorsolateral prefrontal cortex and caudate among those patients higher in anhedonia/avolition.

Conclusions

Together, these results suggest that anhedonia/avolition are as strongly related to cortical learning or higher-level processes involved in goal-directed behavior such as effort computation and planning as to striatally mediated learning mechanisms.

Keywords: Motivation, schizophrenia, reinforcement learning, prediction error, anhedonia, striatum

Negative symptoms are major contributors to disability and poor quality of life among individuals with schizophrenia, but are poorly understood and treated (1, 2). Anhedonia (reduced ability to experience pleasure) and avolition (reduced motivation to initiate or persist in goal-directed activity) together comprise a dissociable factor of negative symptomatology (3) that has garnered increasing attention for a possible association with abnormalities in reward processing. In previous work, we have described several processes required for the translation of reward information into goal-directed behavior, disruption of any of which could lead to anhedonia and/or avolition (4). The work described here examines one of these processes, reinforcement learning, and its relationship to anhedonia and avolition in schizophrenia.

Numerous behavioral studies have suggested that reinforcement learning is intact in schizophrenia when learning is fairly implicit (though see (5) for evidence of impaired Serial Reaction Time task learning), but more impaired when learning tasks require explicit representations of stimulus-reward contingencies (see (4, 6)). This pattern has given rise to the theory that the striatally mediated gradual reinforcement learning system may be intact in schizophrenia, while more rapid, on-line, cortically-mediated learning systems are impaired (6, 7). Support for this theory is drawn from probabilistic reversal learning studies that show intact acquisition of probabilistic reward contingencies (thought to be striatally mediated) coupled with impaired reversal learning (thought to be cortically mediated) (8, 9). Similarly, several studies using the Weather Prediction task have shown a relatively intact learning rate, but impaired asymptotic performance, which provides mixed evidence for striatal learning impairments (7, 10–12). However, a study with a larger sample size found lower learning rates in patients than controls, suggesting possible impairments in striatally mediated learning (13). The behavioral literature therefore provides a mixed picture on whether striatally mediated learning is intact in schizophrenia patients as a group.

Another approach to studying reinforcement learning is to ask whether the pattern of functional activation in regions receiving dopaminergic projections is consistent with a prediction error signal. Prediction errors are thought to be coded by dopaminergic projections to the basal ganglia, which signal the difference between predicted and received rewards and drive learning by iteratively updating reward predictions (14). In the schizophrenia literature, this approach has revealed some evidence for altered striatal prediction error activity among individuals with schizophrenia using both Pavlovian and instrumental reward-learning tasks, and for both monetary and liquid rewards (15–18), with some suggestion that positive prediction errors may be more affected than negative prediction errors (19, 20) and some suggestion that the effects may be more apparent in unmedicated (21) as compared to medicated patients.

The findings reviewed above suggest the hypothesis that impairments in learning from positive outcomes, related to reductions in striatal signaling of positive prediction errors and/or impaired cortical learning systems, may contribute to motivational deficits in schizophrenia. Here, we test these hypotheses by examining brain activity during a probabilistic reinforcement learning paradigm allowing examination of activation during both choice execution and feedback, as well as the separate assessment of learning from positive versus negative outcomes. We used a model of the role of DA in RL proposed by Frank et al. (22–24), which emphasizes the separate contributions of D1 and D2 receptors in the striatum to “Go” and “NoGo” learning respectively. Two prior studies have used this framework to examine Go learning (learning from rewarding outcomes) and NoGo learning (learning from nonrewarding outcomes) in medicated patients with schizophrenia, and found evidence of impaired Go learning but intact NoGo learning (25, 26), though one other study found impairments in both Go and NoGo learning (27). These findings are consistent with the hypothesis that the effectiveness of phasic DA signals in response to positive feedback is reduced in schizophrenia, thereby impairing Go learning. These studies also examined the relationship between negative symptoms and reinforcement learning impairments, and showed correlations between negative symptom severity and measures of rapid explicit learning, suggesting a role for deficits in cortical learning systems in negative symptomatology. Further, in a modeling study by Gold and colleagues, the behavior of patients with high negative symptoms was best captured by a computational model of striatal learning only, while a model with both striatal and cortical components best captured the behavior of patients lower in negative symptoms (28).

Methods & Materials

Participants

Participants were 49 stable outpatients with DSM-IV schizophrenia or schizoaffective disorder and 41 healthy controls with no personal or family history of psychosis. Both medicated and unmedicated patients were recruited from the community, and medication status and dose was required to have been stable for at least two weeks. Participants were group matched on sex, age, race, parental education, handedness (29), and smoking status. Inclusion criteria were 1) age 18–50 years and 2) ability to give informed consent. See Supplemental Materials for exclusions, which included exclusion for DSM-IV major depressive disorder or dysthymia in the past year. Ten individuals with schizophrenia and 4 healthy controls were excluded for excessive movement (described below), and an additional patient was excluded for having more than 50% nonresponse trials, yielding a final sample size of 37 controls and 38 patients (29 schizophrenia, 9 schizoaffective). All procedures were approved by the Washington University Human Research Protection Office.

Diagnosis and Clinical Assessment

Participant diagnoses were based on a Structured Clinical Interview for DSM-IV-TR (30) conducted by a Masters-level clinician. See Supplemental Materials for details on clinical assessments and measures, which generated both clinician rated and self-reported measures of anhedonia/amotivation.

Task

The experimental paradigm was a modified version of the Probabilistic Stimulus Selection Task (Figure 1) (22), consisting of an acquisition phase, during which fMRI scanning took place, and a test phase that was completed outside the scanner. During acquisition, participants were presented on each trial with one of three pairs of stimuli (“AB”, “CD”, or “EF”), in pseudorandomized order, and were instructed to choose the stimulus that they believe is ”correct” based on feedback received over time. Stimuli were displayed for 2000ms, during which the participant was required to choose one of the stimuli via button press. After a jittered inter-stimulus interval ranging from 2000–6000ms, feedback consisting of the words “Correct! +$” in green text, “Incorrect $0” in red text, or “Too Slow!” were presented on screen for 2000ms. Subjects were told that for each “Correct” choice, they would win money, up to $20 (in actuality, all subjects were paid an additional $20 upon completion). For stimulus pair AB, choice of A was rewarded 80% of the time, while B was rewarded 20% of the time; pair CD was 70:30, and pair EF was 60:40. Feedback was followed by an inter-trial interval jittered from 2000–6000ms. For additional details, see Supplemental Materials.

Figure 1.

Figure 1

Open in a new tab

Experimental paradigm. Shown are the trial types and timing of the acquisition phase of the Probabilistic Stimulus Selection task. Both inter-stimulus intervals and inter-trial intervals were jittered to allow reconstruction of the BOLD response at the time of both choice and feedback.

Image Acquisition and Processing

Imaging was performed on a 3T Siemens TIM TRIO system with a 12-channel head coil. High-resolution structural images were acquired using a sagittal magnetization-prepared rapid acquisition gradient echo (MP-RAGE) sequence (TR=2.4s, TE=3.08ms, inversion time=1s, flip=8°, 176 slices, 1 mm3 voxels). Functional images were collected in 10 runs of 213 frames each using a gradient echo echo-planar sequence (TR=2030ms, TE=27ms, flip=90°, 36 slices). For details, see supplementary materials.

fMRI Analysis

Statistical analysis of fMRI data used two complementary approaches – a more traditional analysis approach categorizing events in terms of specific choices (e.g., AB/CD/EF) to make contact with the existing literature using such approaches, and a computational model-based approach.

Traditional GLM Based Analyses

For these analyses, at the time of stimulus presentation, 6 choice types were modeled (A, B, C, D, E, and F), and at the time of feedback, 12 feedback types were modeled (positive and negative feedback for each choice). Non-response trials were coded as a variable of no interest. The analyses was conducted based on the general linear model (GLM) (31) using in-house software (32). The GLM for each subject included time as a 9-level regressor, made up of the 9 MR frames following each event (a finite-impulse response function approach - FIR). An FIR approach was used rather than a canonical HRF approach because of the mixed evidence in the literature for the integrity of the HRF function in schizophrenia (33, 34). With an HRF approach, changes in the timing of responses can lead to artifactually altered magnitudes estimates (35, 36), while an FIR allows examination of the nature and pattern of the BOLD response patterns across groups (see (35, 36)for comparison of approaches). Activation at the time of stimulus presentation and at the time of feedback were modeled separately. Parameter estimates from the GLMs for each subject, including time (the 9 time points of the response), were entered into ANOVAs using subject as a random factor. In these analyses, a significant main effect of time for a voxel or region indicates activation or deactivation, and a significant interaction of any other factor with time indicates that the hemodynamic response varies across that factor. Analyses using this approach for Choice related activation are presented below, and additional details and results of these analyses are presented in Supplemental Materials.

Model-based fMRI Analyses

Behavioral data was modeled using a Q-learning algorithm with separate learning rates from positive feedback (“gains”; ∝G) and negative feedback (“losses”, ∝L) (24). This algorithm models subjects’ choices by calculating a Q value, which is an estimate of expected reward value, for each stimulus (A–F). This value is modified on each trial according to the reward r(t) received, where r(t) = 1 for positive feedback and r(t) = 0 for negative feedback. For additional details, see Supplemental Materials. Model-based fMRI analyses were conducted by including trial-by-trial, subject specific, values of Q (expected value) and prediction error as parametric regressors in a GLM that included choice and feedback events (collapsed across stimulus type) with the (Q value) modulate the choice events and predictions error modulating the feedback events. As with the traditional analyses described above, parameter estimates from the GLMs for each subject, including time (the 9 time points of the response for either the choice or feedback event), were entered into ANOVAs using subject as a random factor. In these analyses, a significant main effect of time for a voxel or region indicates activation or deactivation as a function of that regressor, and a significant interaction of any other regressor with time indicates that the hemodynamic response varies across that regressor. Whole-brain analyses were corrected for multiple comparisons using a p-value/cluster size threshold of p<.003 (two-tailed) and 13 voxels, as determined by Monte Carlo simulations to provide a whole-brain false-positive rate of p<.05 (37, 38). ROI analyses were conducted using mean activation within 6 regions including bilateral caudate, putamen, and nucleus accumbens. These regions were defined anatomically (39) and were applied at the group level in Talairach atlas space. Significance levels in ROI analyses were False Discovery Rate corrected for multiple comparisons using the Benjamini-Hochberg procedure (40) to yield an alpha of .05 across all 6 regions.

We also conducted correlation analyses within the patient group between BOLD signal and anhedonia/avolition scores. Activation at time points 4 and 5 (the peak of the hemodynamic response) was averaged and correlated with clinical and questionnaire-based anhedonia/avolition scores. These correlations were conducted using the same voxelwise whole-brain and regionwise ROI procedures described above. Analyses as a function of antipsychotic dose are also in Supplemental Materials.

Results

Demographic and Clinical Characteristics

Participant demographic and clinical characteristics are shown in Table 1.

Table 1.

Demographic and clinical characteristics

CON (N = 37) SCZ (N = 38)
Mean SD Mean SD
Age 36.43 8.44 35.00 9.25
Education (years) 14.14 2.1 12.95 2.3
Highest Parental Education (years) 13.78 1.65 14.00 4.3
Sex (% Male) 43.2 63.2
Race (% Caucasian) 29.7 42.1
Smoking Status (% Smokers) 43.2 57.9
Fagerstrom Test for Nicotine Dependence 7.4 5.5 9.2 4.0
Handedness 4.31 0.95 4.97 0.19
SAPS/SANS Positive 0.03 .16 3.5 2.64
SAPS/SANS Negative 1.49 2.09 7.92 2.97
SAPS/SANS Disorganization 2.38 1.30 2.63 2.65
BNSS Total Anhedonia 0.68 2.15 1.05 1.91
BNSS Total Avolition 5.76 3.48 5.38 2.44
Chapman Social Anhedonia 8.92 6.26 15.39 7.98
Chapman Physical Anhedonia 11.78 6.12 18.53 10.13
Snaith-Hamilton Pleasure Scale 52.38 3.21 48.45 8.21
TEPS Anticipatory Pleasure 48.22 5.69 46.05 8.31
TEPS Consummatory Pleasure 38.41 5.64 35.03 7.37
Apathy Scale 22.19 3.57 26.11 7.26
Past Major Depressive Disorder (#)(not in past year) 2 7 7
Past Substance Dependence (#) 2 7
Antipsychotic Medication (# Taking) -- 35a
Antipsychotic Dose (CPZ equivalent) -- 717.78 474.25
Antidepressant Medication (# Taking) -- 17
Antianxiety Medication (# Taking) -- 6
Mood Stabilizer Medication (# Taking) -- 9
Anticholinergic Medication (# Taking) -- 12
Open in a new tab

Notes: There were no significant differences between patients and controls in age, sex, race, or handedness. Personal education was higher among controls than patients, an expected finding given the effects of schizophrenia on function, but parental education (a surrogate for premorbid socioeconomic status) was similar between groups. Smoking status also did not differ significantly between groups, both in terms of the number of participants who smoke, and the Fagerstrom nicotine dependence scores among smokers. SAPS/SANS scores for positive and negative symptoms were higher among patients than controls, though disorganization scores were low in patients and did not differ between groups. Anhedonia and avolition scores were higher among patients than controls on all clinical and self-report measures except the TEPS anticipatory pleasure score.

a

Typical only n = 4; atypical only n = 29; typical + atypical n = 2; clozapine only n = 2; clozapine + other n = 3.

Behavioral Analysis

Acquisition Phase

The acquisition phase data was divided into 5 blocks of 72 trials (24 per stimulus pair), which were used in a repeated measures ANOVA with Block and Pair (AB, CD, EF) as within-subjects factors and Group (CON, SCZ) as a between-subjects factor (Figure 2a). This analysis revealed significant main effects of Block (F(4,292) = 8.74, p<.001), with increasing proportions of high-probability choices over time, and Pair (F(2,146) = 22.00, p<.001), with the greatest proportion of high-probability choices for AB pairs, followed by CD, then EF. The main effect of group was not significant, nor were any interactions with group (p values >.1). However, planned simple effects tests revealed trend-level group differences within the AB (F(1,73) = 3.45, p=.07 and EF (F(1,73) = 3.22, p=.08) pairs, but not the CD pair (p>.8). As shown in Figure 2a, performance for the CD pair was similar between patients and controls, whereas patients performed more poorly than controls on the AB and EF pairs.

Figure 2.

Figure 2

Open in a new tab

Behavioral results. (A) Acquisition phase performance. Proportion of high-probability choices (A/C/E) per 24-trial block. (B) Test-phase performance for original AB, CD, and EF pairs. (C) Test-phase performance for ChooseA/AvoidB and ChooseC/AvoidD transfer measures.

Test Phase

Test phase choice data is shown in Figure 2b. For the original AB, CD, and EF pairs that had been presented during acquisition, a Pair X Group ANOVA revealed a main effect of pair (F(2,144) = 13.81, p<.001), but no main effect of group (F(1,72) = 2.67, p>.1) or Pair X Group interaction (F(2,144) = 0.86, p>.42). However, planned simple effects tests revealed a significant group difference for the AB pair only (F(1,72) = 4.44, p=.04), in which patients performed more poorly than controls. To examine whether learning from positive versus negative feedback differed between groups, we compared ChooseA (learn positive) and AvoidB (avoid negative) using a repeated measures ANOVA with Group as a between subjects factor, which revealed only a trend-level main effect of Group (F(1.72) = 3.61, p=.07) (Figure 2b). However, while the ChooseA/AvoidB measure has been the transfer measure of interest in previously published versions of this task, its appropriateness in this sample is called into question by the fact that patients performed more poorly on the AB pair than controls. To avoid this problem, an equivalent transfer measure was created that relies on the CD pair, performance on which was very closely matched between groups: ChooseC (CE, CF) vs. AvoidD (DE, DF). ANOVA analysis of this measure revealed a significant ChooseC/AvoidD X Group interaction (F(1,72) = 5.21, p=.03), with no significant main effects (p values>.2). As shown in Figure 2c, ChooseC performance was significantly lower in patients than controls (t(72) = 2.40, p=.02), while performance on the AvoidD measure did not differ significantly between groups (p>.5).

Modeling Results

Model fits as measured by LLH did not differ significantly between groups (Table 2). However, there were a number of subjects who showed no appreciable learning and for whom model fits were poor. To restrict the modeling analysis to those subjects whose choices were well-described by the model, a subset of “nonlearners” was excluded (Table 2). We used AIC to verify that the model with gain and loss learning rates fit better than a single learning rate model (this was true for 54 subjects; 28 controls and 26 patients). The learners were higher than the non-learners on self-reported anhedonia/amotivation (t(1,36) = −3.07, p = .004), lower on WAIS Vocabulary scores (t(1,36) = −2.42, p = .02), but did not differ significantly on clinician-rated anhedonia/amotivation (t(1,36) = −0.48, p = .64), positive symptoms ((t(1,36) = 0.13, p = .90) and chlorpromazine equivalents (t(1,36) = 0.77, p = .44). Model simulations showed that fit parameters were able to appropriately predict subjects’ choices (Figure 3). Gain and loss learning rates are shown in Table 2. Independent sample t-tests indicated a trend for patients to show lower gain learning rates (t(52) = 1.63, p = .055, 1-tailed), but not loss learning rates (t(52) = 0.56, p = .96, 1-tailed).

Table 2.

Model Fit and Parameter Data

Sample Criterion/Value CON SCZ
Full Sample (38 CON, 37 SCZ) LLH −173.13 (57.6) −184.68 (52.9)
BIC 363.83 (115.1) 386.89 (105.9)
Gain Learning Rate .279 (.30) .173 (.24)
Loss Learning Rate .275 (.33) .215 (.33)
Learners Only (28 CON, 26 SCZ) LLH −154.6 (53.0) −171.62 (53.8)
BIC 326.78 (106.0) 360.83 (107.6)
Gain Learning Rate .268 (.25) .165 (21)
Loss Learning Rate .218 (.28) .213 (.31)
Open in a new tab

Figure 3.

Figure 3

Open in a new tab

Modeling results. Comparison between empirical behavior and model-predicted learning curves.

fMRI Prediction Error Analysis

Within the striatal ROIs, all regions demonstrated significant prediction error effects (all ps < .007) with positive modulation (greater activation with greater prediction error), with no significant group differences. These regions were also identified by the whole-brain analysis, (Tables 3/4 and Figure 4). A set of regions demonstrating activation that was positively modulated (Table 3, Figure 4A,B) included bilateral ventral striatum and amygdala. As can be seen in graphs to the right in Figure 4B, in these regions, activity in response to outcomes was more positive with high prediction errors. Further, as can be seen in graphs to the left in Figure 4B, both patients and controls showed greater activation when they received positive feedback, regardless of whether it was for a high probability choice or a low probability choice. A second subset of regions demonstrated activation with negative modulation, such that activation was greater for smaller positive (or larger negative) prediction errors (Table 4). These regions included cognitive control regions such as bilateral DLPFC, PPC, anterior insula, and preSMA. Examples of each type of activation pattern are shown in Figure 4B. Another third set of regions, including rostral ACC and medial frontal gyrus, demonstrated deactivation that was positively modulated (i.e., less deactivation for more positive prediction errors).

Table 3.

Regions demonstrating positive modulation of activity for prediction errors

Region BA X Y Z # Voxels Z Activation PE Modulation
L Ventral Striatum -- −11 9 2 167 5.94 Activation Positive
L Cingulate Gyrus 24 −11 −19 40 27 4.16 Activation Positive
R Ventral Striatum -- 14 6 2 198 6.71 Activation Positive
R Hippocampus -- 29 −15 −9 60 4.87 Activation Positive
L Inferior Parietal Lobule 40 −42 −32 41 102 4.53 Activation Positive
L Middle Occipital Gyrus 18 −26 −95 4 77 5.17 Activation Positive
L Middle Occipital Gyrus 18 −22 −88 22 45 4.10 Activation Positive
L Middle Temporal Gyrus 21 −61 −19 −03 15 4.23 Activation Positive
R Amygdala -- 18 −28 −9 46 5.57 Activation Positive
L Amygdala -- −15 −27 −9 57 5.19 Activation Positive
R Postcentral Gryus 3 29 −34 55 107 4.58 Activation Positive
R Postcentral Gyrus 3 18 −37 69 43 4.13 Activation Positive
L Precuneus 7 −13 −48 54 100 4.87 Activation Positive
L Anterior Cingulate 32 −2 43 −1 254 8.12 Deactivation Positive
R Anterior Cingulate 32 18 34 14 224 4.62 Deactivation Positive
R Anterior Cingulate 25 6 19 −5 120 558 Deactivation Positive
R Anterior Cingulate 24 2 25 14 224 4.62 Deactivation Positive
R Anterior Cingulate 32 −19 33 23 32 3.90 Deactivation Positive
L Medial Frontal Gyrus 9 −3 53 15 177 6.38 Deactivation Positive
L Middle Frontal Gyrus 8 −25 12 39 53 4.56 Deactivation Positive
L Middle Temporal Gyrus 39 −44 −72 25 77 4.29 Deactivation Positive
L Superior Frontal Gyrus 8 −21 28 46 148 6.00 Deactivation Positive
L Superior Frontal Gyrus 8 −33 23 54 28 4.26 Deactivation Positive
Open in a new tab

Table 4.

Regions demonstrating negative modulation of activity for prediction errors

Region BA X Y Z # Voxels Z Activation PE Modulation
R Cingulate Gyrus 32 5 18 41 214 5.67 Activation Negative
L Cingulate Gyrus 32 −10 19 33 82 4.79 Activation Negative
L Inferior Frontal Gyrus 9 047 7 32 201 4.50 Activation Negative
L Anterior Insula 13 −34 17 6 171 5.69 Activation Negative
R Anterior Insula 13 40 16 6 185 6.23 Activation Negative
R Middle Frontal Gyrus 9 43 23 29 143 5.18 Activation Negative
R Middle Frontal Gyrus 6 37 8 53 72 4.43 Activation Negative
L Precentral Gyrus 4 −40 −11 52 192 5.10 Activation Negative
R Precentral Gyrus 6 40 2 35 130 4.77 Activation Negative
L Precentral Gyrus 9 −42 25 37 95 4.68 Activation Negative
R Precuneus 7 7 −72 38 94 4.02 Activation Negative
L Precuneus 7 −20 −71 38 94 4.02 Activation Negative
L Superior Frontal Gyrus 6 −3 10 51 250 7.22 Activation Negative
R Superior Frontal Gyrus 6 5 01 64 157 6.25 Activation Negative
L Superior Frontal Gyrus 6 −14 −1 66 78 5.23 Activation Negative
R Superior Frontal Gyrus 6 12 14 62 77 4.83 Activation Negative
L Superior Frontal Gyrus 10 −34 49 21 32 4.37 Activation Negative
R Superior Parietal Lobule 7 31 −55 43 142 5.21 Activation Negative
L Superior Parietal Lobule 40 −35 −52 48 159 4.50 Activation Negative
L Superior Temporal Gyrus 22 −49 10 2 127 6.05 Activation Negative
R Superior Temporal Gyrus 38 46 12 −8 79 6.38 Activation Negative
L Anterior Cingulate 32 −14 29 8 97 4.79 Deactivation Negative
Open in a new tab

Figure 4.

Figure 4

Open in a new tab

Prediction error analysis: Prediction Error x Time interactions. (A) Regions with significant prediction error effects in both groups. Red = activation with positive modulation; blue = activation with negative modulation; green = deactivation with positive modulation; orange = deactivation with negative modulation. (B) Example timecourses for each activation pattern. ACE: High-probability choice (A, C, or E), BDF: low-probability choice (B, D, or F), Pos: Positive feedback, Neg: Negative feedback, CON: Control, SCZ: Schizophrenia, PE: Prediction Error. Regions shown were significant at whole brain threshold of Z = 3.0, N = 13 voxels.

There were two regions that demonstrated an interaction between prediction error and group: right inferior frontal gyrus (+47,+15,+7; 26 voxels) and right superior temporal gyrus (+60,−60,+26; 20 voxels). Right inferior frontal gyrus showed activation that was negatively modulated in controls (reduced activation with higher prediction errors), but no significant modulation in patients. Superior temporal gyrus showed activation that was decreased for larger positive prediction errors in controls, but increased for larger positive prediction errors in patients. No significant group differences in prediction error activity were found in striatal regions.

fMRI Q-value Analysis

Striatal ROI analysis of Q value-related activity revealed robust positive modulation in bilateral nucleus accumbens (L: F(8,416) = 6.92, p<10−7) , R: F(8,416) = 5.89, p<10−8), indicating greater activation when choosing stimuli with higher expected value. These effects did not interact with group, and were strongly present within each group. Similar regions were identified in the whole-brain analysis (Table 5), and example timecourses are shown in Figure 5. Similar to the ROI analysis results, in the whole brain analyses, right ventral striatum demonstrated activation that was modulated positively by Q value, as did regions in right caudate, right postcentral gyrus, and left inferior frontal gyrus (Figure 5B). Cognitive control network regions including bilateral DLPFC, posterior parietal cortex, anterior insula, and ACC/preSMA, showed activation with negative modulation, meaning that the lower the Q value of the chosen stimulus, the greater the activation in these regions. The activation patterns and magnitudes were highly similar between patients and controls for all of these regions (Figure 5C).

Table 5.

Regions demonstrating a significant parametric effect of Q values

Activation Pattern
Effect Region BA X Y Z # Voxels Z Choice vs Baseline Q Value Modulation
Q R Postcentral Gyrus 3 42 −25 54 82 4.79 Activation Positive
R Ventral Striatum -- 16 3 −9 220 6.98 Activation Positive
R Caudate -- 11 24 3 116 5.69 Activation Positive
L Inferior Frontal Gyrus 47 −17 29 −5 60 5.11 Activation Positive
R ACC/pSMA 8 1 19 44 570 7.38 Activation Negative
L Anterior Insula 13 −32 18 3 251 6.87 Activation Negative
R Anterior Insula 13 43 18 7 376 7.38 Activation Negative
L Fusiform Gyrus 37 −34 −56 −6 70 4.57 Activation Negative
R Cerebellar Crus I -- 12 −85 −17 57 5.18 Activation Negative
R Cerebellar Vermis -- 3 −72 −37 14 4.66 Activation Negative
R Lingual Gyrus 19 29 −59 −3 65 4.87 Activation Negative
R Middle Frontal Gyrus 8 42 11 41 598 6.28 Activation Negative
L Middle Frontal Gyrus 9 −45 8 34 232 5.03 Activation Negative
R Middle Frontal Gyrus 10 39 49 7 155 7.41 Activation Negative
L Middle Frontal Gyrus 10 −42 50 5 74 6.56 Activation Negative
L Middle Frontal Gyrus 9 −47 31 27 92 4.54 Activation Negative
L Middle Occipital Gyrus 18 −30 −87 6 85 4.54 Activation Negative
L Posterior Cingulate 23 −1 −26 32 82 5.43 Activation Negative
L Posterior Parietal Cortex 40 −40 −53 47 236 6.06 Activation Negative
L Posterior Parietal Cortex 7 32 −61 49 192 5.27 Activation Negative
L Cerebellar Crus II -- −10 −87 −31 147 7.26 Activation Negative
L Superior Frontal Gyrus 6 −2 10 68 85 5.01 Activation Negative
L Middle Temporal Gyrus 21 −53 −17 −9 109 5.21 Deactivation Positive
L Middle Temporal Gyrus 39 −40 −70 28 129 4.52 Deactivation Positive
R Middle Temporal Gyrus 21 53 −8 −14 28 4.71 Deactivation Positive
L Parahippocampal Gyrus 28 −21 −13 −18 27 3.44 Deactivation Positive
R Anterior Cingulate 24 1 33 −2 129 6.60 Deactivation Positive
R Inferior Parietal Lobule 40 49 −54 35 127 4.80 Deactivation Negative
Q X Group R Inferior Parietal Lobule 7 30 −65 57 13 4.01 Activation CON: Neg; SCZ: None
R Brainstem -- 1 −32 −38 26 4.13 Activation CON: Neg; SCZ: Pos
L Cerebellum -- −28 −48 −47 71 4.84 Activation CON: Neg; SCZ: Pos
R Cerebellum -- 28 −60 −42 57 4.53 Activation CON: Neg; SCZ: Pos
L Superior Frontal Gyrus 10 −11 63 18 22 4.93 Activation CON: Neg; SCZ: Pos
Cingulate Gyrus 23 0 −20 28 14 3.82 Activation CON: None; SCZ: Neg
L Precuneus 7 −5 −70 47 18 3.57 Activation CON: None; SCZ: Neg
L Parahippocampal Gyrus 36 −22 −40 −15 32 4.8 Activation CON: None; SCZ: Pos
Precuneus 7 0 −53 38 15 3.94 Deactivation CON: None; SCZ: Pos
Open in a new tab

Figure 5.

Figure 5

Open in a new tab

Q value effects. (A) Regions demonstrating significant Q-value modulation. Red = activation with positive modulation; blue = activation with negative modulation; green = deactivation with positive modulation; orange = deactivation with negative modulation. (B) Timecourse for right ventral striatum showing activation with positive modulation. (C) Timecourse for DLPFC showing activation with negative modulation. (D) Timecourse for rACC showing deactivation with positive modulation. ACE: high-probability choice (A, C, or E), BDF: low-probability choice (B, D, or F), CON: control, SCZ: schizophrenia, Q = Q-value (expected value). Regions shown were significant at whole brain threshold of Z = 3.0, N = 13 voxels.

Several regions showed a significant Time X Group interaction in the Q value effect (Table 5). Regions in brainstem, bilateral cerebellum, and left superior frontal gyrus displayed activation that was negatively modulated in controls but positively modulated in patients. Similarly, left parahippocampal gyrus was modulated positively in patients, but not in controls, and right superior parietal lobule was modulated negatively in controls, but not in patients. Finally, precuneus showed deactivation that was negatively modulated in patients, but not controls. No group differences in Q value effects were seen in striatal regions.

ANOVA Analysis: Choice-Related Activity Early in Learning

As shown in Figure 2a, group-level performance began to plateau during Block 3. Thus, we conducted analyses only with early learning (Blocks 1–3). On ROI analysis, a Choice X Time (i.e., time within trial for a FIR analysis) X Group ANOVA again revealed no significant effects of choice or group in striatal regions. On whole-brain analysis, the patterns of activation for high- versus low-probability (of reward) choices within early learning showed overlap with the Q value analysis described above. Several regions demonstrated Choice X Time interactions in which activity was greater for high than low-probability choices in both groups, including regions in bilateral caudate, left inferior frontal gyrus, and right anterior insula (Supplemental Table S6 and Figure S6). Similar to the full Q value analysis, greater activation for low- than high-probability choices was seen in left DLPFC and precentral gyrus. A Time x Group interaction was present in a number of cognitive control regions including bilateral posterior parietal cortex, right DLPFC, preSMA, thalamus, and right anterior insula/inferior frontal gyrus (Table S6 and Figure S6a). All of these regions activated more strongly in controls than patients during early learning at the time of choice, regardless of which stimulus was ultimately chosen. Choice x Time x Group interactions were also seen in a few regions; among these were midbrain and right cerebellar crus I, which activated more strongly for low than high-probability choices among controls, with the opposite pattern among patients (Table S6).

Individual Differences Analyses

There were no significant relationships between higher clinician rated or self-reported anhedonia/amotivation scores and impairments in either Choose C performance (go learning), gain learning rates, activity during high-probability responses to positive or negative feedback or positive or negative prediction errors in the striatum using the ROI analyses (all ps > .1, see Table S7). However, whole-brain analysis revealed significant negative relationships between the self-reported anhedonia/avolition scores and responses to positive feedback in several regions including left caudate and bilateral posterior DLPFC (Brodmann areas 44/6) (Figure 6 and Table S8). These regions showed reduced activation in response to positive feedback in those patients who were higher in self-reported anhedonia/avolition. We did not find any regions that survived whole brain correction for the relationship between greater positive feedback related activity and either lower clinician rated anhedonia/amotivation scores or better performance in either Choose C performance (go learning) or gain learning rates for either prediction error or positive feedback related activity.

Figure 6.

Figure 6

Open in a new tab

Correlation analyses: Regions with significant negative correlations between responses to positive feedback and self-reported anhedonia/avolition among patients. Regions shown were significant at whole-brain threshold of Z = 3.0, N = 13 voxels.

Discussion

Behavioral results using transfer measures sensitive to Go vs. NoGo learning suggested some evidence for impairments in learning from positive, but not negative, feedback in patients as compared to controls. However, while the behavioral results demonstrated some impairment in Go learning, we found little evidence in the neuroimaging results for reduced striatal responses among patients as compared to controls. We had hypothesized that Go learning impairments would be associated with reduced positive prediction error activity, and with reduced anticipatory reward responses at the time of choice in striatal regions among patients. Instead, we found that striatal activity was intact in patients at the time of both choice and feedback. We found no group differences in striatal activity for positive versus negative choices or feedback, expected values, or prediction errors when examining the full acquisition phase.

Our findings of significant striatal activation at the group level in schizophrenia during learning, with no significant differences from controls, contrast with the findings of several studies in the literature demonstrating reduced striatal prediction errors in patients (41–43). However, other studies have found intact striatal prediction errors in schizophrenia patients (44–48). A possible source of these differences across studies are clinical differences in the populations examined (medicated versus unmedicated); among other possible differences, our patients had lower positive symptom severity than many published reports, and there is evidence in the literature that aberrant prediction error activity is related to positive symptoms in schizophrenia (49). Further, we also excluded patients and controls with evidence of major depression in the past year to help de-confound depression versus psychosis effects on reinforcement learning. Given that depression is also associated with altered striatal activation in response to reward (50) and to prediction error (51), it is possible that this difference from many previous studies reduced the evidence for altered prediction error signals in the striatum among our sample of individuals with schizophrenia.

The lack of significant relationships between anhedonia/avolition scores and striatal responses specifically to prediction error differs from some reports in the literature (17, 52), although at least one other study did not find such relationships (16) and we did see a relationship of anhedonia/avolition to positive feedback responses in the caudate. Behavioral studies have also reported relationships between reinforcement learning and negative symptom severity (26, 28), which we did not find. We speculate that these differences may have been influenced by the specifics of the experimental design. One important difference between our task and many in the literature is that patients received additional practice on the task, with different stimuli, before the scanning session. This was done to avoid an influence of confusion about task procedures, which was common among patients but not controls. This procedure introduced a practice mismatch between groups, but mismatches in the amount of practice given are not unusual in prediction error studies, which sometimes have subjects train to criterion before entering the scanner. It is possible that this additional practice in patients contributed to the relative lack of group differences in our study as compared to others in the literature, which has implication for our understanding of the mechanisms driving such impairments in patients. However, it is also possible that the lack of significant correlation with prediction error responses reflected in part the small sample of participants with good model fits, though the magnitudes of the correlations between prediction error activity in the striatum and amotivation/anhedonia scores were low and we did not see additional significant associations if we included subjects who did not have good model fits.

Regions such as orbitofrontal cortex, prefrontal cortex, and medial temporal lobe are commonly associated with more rapid, explicit forms of learning. This task was designed to rely heavily on the basal ganglia slow learning system, but it is likely that these explicit systems contributed as well. Our behavioral data provides some evidence for this. At both acquisition and test, we found reduced performance on the AB pair in patients as compared to controls, while performance on the CD and EF pairs (at test) did not differ between groups, though the group differences in the EF pair might have achieved significance with sample sizes larger than 38 and 37. We speculate that this finding may be related to impairments in explicit learning among patients, given that the AB pair had the highest probability ratio and was therefore the easiest to learn via explicit mechanisms. Higher ratios require fewer trials to be held in working memory for explicit representations of reward contingencies to be formed, while lower ratios require integration over many more trials and are better suited to the gradual, implicit learning system of the basal ganglia. This interpretation is consistent with the hypothesis in the literature that cortical learning is impaired in patients (6).

There were also some imaging results consistent with the hypothesis of impaired cortical learning in this group. In both the prediction error and Q-value analyses, we saw evidence for altered activation in frontal cortex, with right inferior frontal modulation in patients in response to prediction errors, and altered superior frontal gyrus modulation in patients in the Q value analysis. Further, during the early learning phase, several regions involved in cognitive control demonstrated reduced overall choice-related activation in patients as compared to controls, which is consistent with a reduction in explicit learning during the early learning phase. As noted above, analyses of both positive versus negative feedback and prediction error analyses revealed intact activity in patients. However, despite the largely intact group activation, activation in response to positive feedback correlated with anhedonia/avolition in the patient group in both striatal and cortical regions. This finding is consistent with the hypothesis that deficits in responses to positive feedback in both cortical and striatal regions may contribute to these symptoms.

The conclusions from this study are limited by the fact that one cannot prove the null hypothesis (i.e., that patients and controls as a group do not differ in striatal prediction error activity), and we could potentially have seen differences from controls if the sample sizes were larger. However, Figure 4B shows that the patients with schizophrenia did show strong PE responses in the striatum and the effect sizes of any difference from controls at the group level were very small. We did have to exclude more non-learners from the patients than the controls, which could have biased the results in favor of seeing strong PE responses in the striatum in patients. However, Figure S1 illustrates strong striatal responses to positive feedback even in the whole sample of patients in the traditional GLM analyses. Another limitation was that the majority of patients examined here were taking antipsychotics. As presented in Supplemental Materials, correlations with dose equivalents revealed increased NoGo learning in patients with higher medication doses, but no significant relationships between dose and brain activity. Interestingly, increased NoGo learning is what the Frank model predicts for greater levels of D2R antagonism, meaning that this relationship is actually consistent with the model (23, 53). Perhaps surprisingly, studies examining striatal activation in schizophrenia tend to find reduced striatal activation for unmedicated patients, with intact activation for patients taking atypical antipsychotics, including some direct evidence of a normalizing effect of starting these medications (54). The present study lends further support to these findings by demonstrating intact striatal activation in a population of patients primarily taking atypical antipsychotics.

In conclusion, this study demonstrated some behavioral evidence of impaired learning from positive versus negative feedback, as well as impaired learning of stimuli with high versus low reinforcement ratios, among medicated patients with chronic schizophrenia. Striatal activation was intact in the patient group at the time of choice and feedback, including intact prediction error activity. At the time of choice, patients failed to recruit cognitive control regions to the same extent as controls during early learning. These findings are suggestive of alterations in cortical, but not basal ganglia, reinforcement learning mechanisms in the patient group as a whole. However, severity of anhedonia and avolition in patients were associated with reduced responses to positive feedback in caudate and bilateral DLPFC, suggesting a relationship between these symptoms and altered processing of positive feedback in patients in both cortical and striatal regions.

Supplementary Material

supplement

Acknowledgments

The authors would like to thank the participants in this study, who gave generously of their time.

Footnotes

Financial Disclosures

Funding for this study was provided by NIMH MH066031. DMB is a consultant for Pfizer, Amgen and Takeda on studies related to the treatment of negative symptoms in schizophrenia. JMG is a consultant for Amgen, Roche, Pfizer, Merck, Astra Zenaca, Solvay and Glaxo Smith Kline on studies related to the treatment of negative symptoms in schizophrenia. JMG also receives royalty payments from the Brief Assessment of Cognition in Schizophrenia (BACS). MJF and AC consult for Roche on studies related to the treatment of negative symptoms in schizophrenia. Dr. Dowd reported no biomedical financial interests or potential conflicts of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • 1.Milev P, Ho BC, Arndt S, Andreasen NC. Predictive values of neurocognition and negative symptoms on functional outcome in schizophrenia: a longitudinal first-episode study with 7-year follow-up. Am J Psychiatry. 2005;162:495–506. doi: 10.1176/appi.ajp.162.3.495. [DOI] [PubMed] [Google Scholar]
  • 2.Kirkpatrick B, Fenton WS, Carpenter WT, Jr, Marder SR. The NIMH-MATRICS consensus statement on negative symptoms. Schizophr Bull. 2006;32:214–219. doi: 10.1093/schbul/sbj053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Messinger JW, Tremeau F, Antonius D, Mendelsohn E, Prudent V, Stanford AD, et al. Avolition and expressive deficits capture negative symptom phenomenology: implications for DSM-5 and schizophrenia research. Clin Psychol Rev. 2011;31:161–168. doi: 10.1016/j.cpr.2010.09.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Barch DM, Dowd EC. Goal representations and motivational drive in schizophrenia: the role of prefrontal-striatal interactions. Schizophr Bull. 2010;36:919–934. doi: 10.1093/schbul/sbq068. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Siegert RJ, Weatherall M, Bell EM. Is implicit sequence learning impaired in schizophrenia? A meta-analysis. Brain Cogn. 2008;67:351–359. doi: 10.1016/j.bandc.2008.02.005. [DOI] [PubMed] [Google Scholar]
  • 6.Gold JM, Waltz JA, Prentice KJ, Morris SE, Heerey EA. Reward processing in schizophrenia: a deficit in the representation of value. Schizophr Bull. 2008;34:835–847. doi: 10.1093/schbul/sbn068. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Weickert TW, Terrazas A, Bigelow LB, Malley JD, Hyde T, Egan MF, et al. Habit and skill learning in schizophrenia: evidence of normal striatal processing with abnormal cortical input. Learn Mem. 2002;9:430–442. doi: 10.1101/lm.49102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Waltz JA, Gold JM. Probabilistic reversal learning impairments in schizophrenia: further evidence of orbitofrontal dysfunction. Schizophr Res. 2007;93:296–303. doi: 10.1016/j.schres.2007.03.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Weiler JA, Bellebaum C, Brune M, Juckel G, Daum I. Impairment of probabilistic reward-based learning in schizophrenia. Neuropsychology. 2009;23:571–580. doi: 10.1037/a0016166. [DOI] [PubMed] [Google Scholar]
  • 10.Beninger RJ, Wasserman J, Zanibbi K, Charbonneau D, Mangels J, Beninger BV. Typical and atypical antipsychotic medications differentially affect two nondeclarative memory tasks in schizophrenic patients: a double dissociation. Schizophr Res. 2003;61:281–292. doi: 10.1016/s0920-9964(02)00315-8. [DOI] [PubMed] [Google Scholar]
  • 11.Keri S, Kelemen O, Szekeres G, Bagoczky N, Erdelyi R, Antal A, et al. Schizophrenics know more than they can tell: probabilistic classification learning in schizophrenia. Psychol Med. 2000;30:149–155. doi: 10.1017/s0033291799001403. [DOI] [PubMed] [Google Scholar]
  • 12.Keri S, Nagy O, Kelemen O, Myers CE, Gluck MA. Dissociation between medial temporal lobe and basal ganglia memory systems in schizophrenia. Schizophr Res. 2005;77:321–328. doi: 10.1016/j.schres.2005.03.024. [DOI] [PubMed] [Google Scholar]
  • 13.Weickert TW, Goldberg TE, Egan MF, Apud JA, Meeter M, Myers CE, et al. Relative risk of probabilistic category learning deficits in patients with schizophrenia and their siblings. Biol Psychiatry. 2010;67:948–955. doi: 10.1016/j.biopsych.2009.12.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Schultz W. Multiple dopamine functions at different time courses. Annu Rev Neurosci. 2007;30:259–288. doi: 10.1146/annurev.neuro.28.061604.135722. [DOI] [PubMed] [Google Scholar]
  • 15.Murray GK, Corlett PR, Clark L, Pessiglione M, Blackwell AD, Honey G, et al. Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Mol Psychiatry. 2008;13:239, 267–276. doi: 10.1038/sj.mp.4002058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Gradin VB, Kumar P, Waiter G, Ahearn T, Stickle C, Milders M, et al. Expected value and prediction error abnormalities in depression and schizophrenia. Brain. 2011;134:1751–1764. doi: 10.1093/brain/awr059. [DOI] [PubMed] [Google Scholar]
  • 17.Morris RW, Vercammen A, Lenroot R, Moore L, Langton JM, Short B, et al. Disambiguating ventral striatum fMRI-related bold signal during reward prediction in schizophrenia. Mol Psychiatry. 2012;17:235, 280–239. doi: 10.1038/mp.2011.75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Segarra N, Metastasio A, Ziauddeen H, Spencer J, Reinders NR, Dudas RB, et al. Abnormal Frontostriatal Activity During Unexpected Reward Receipt in Depression and Schizophrenia: Relationship to Anhedonia. Neuropsychopharmacology. 2015 doi: 10.1038/npp.2015.370. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Walter H, Kammerer H, Frasch K, Spitzer M, Abler B. Altered reward functions in patients on atypical antipsychotic medication in line with the revised dopamine hypothesis of schizophrenia. Psychopharmacology (Berl) 2009;206:121–132. doi: 10.1007/s00213-009-1586-4. [DOI] [PubMed] [Google Scholar]
  • 20.Waltz JA, Schweitzer JB, Ross TJ, Kurup PK, Salmeron BJ, Rose EJ, et al. Abnormal responses to monetary outcomes in cortex, but not in the basal ganglia, in schizophrenia. Neuropsychopharmacology. 2010;35:2427–2439. doi: 10.1038/npp.2010.126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Schlagenhauf F, Huys QJ, Deserno L, Rapp MA, Beck A, Heinze HJ, et al. Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. Neuroimage. 2014;89:171–180. doi: 10.1016/j.neuroimage.2013.11.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Frank M, Seeberger LC, O'reilly RC. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science. 2004;306:1940–1943. doi: 10.1126/science.1102941. [DOI] [PubMed] [Google Scholar]
  • 23.Frank MJ, O'Reilly RC. A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci. 2006;120:497–517. doi: 10.1037/0735-7044.120.3.497. [DOI] [PubMed] [Google Scholar]
  • 24.Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE. Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A. 2007;104:16311–16316. doi: 10.1073/pnas.0706111104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Waltz J, Frank M, Robinson B, Gold J. Selective Reinforcement Learning Deficits in Schizophrenia Support Predictions from Computational Models of Striatal-Cortical Dysfunction. Biol Psychiatry. 2007;62:756–764. doi: 10.1016/j.biopsych.2006.09.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Waltz JA, Frank MJ, Wiecki TV, Gold JM. Altered probabilistic learning and response biases in schizophrenia: behavioral evidence and neurocomputational modeling. Neuropsychology. 2011;25:86–97. doi: 10.1037/a0020882. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Fervaha G, Agid O, Foussias G, Remington G. Impairments in both reward and punishment guided reinforcement learning in schizophrenia. Schizophr Res. 2013;150:592–593. doi: 10.1016/j.schres.2013.08.012. [DOI] [PubMed] [Google Scholar]
  • 28.Gold JM, Waltz JA, Matveeva TM, Kasanova Z, Strauss GP, Herbener ES, et al. Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence. Arch Gen Psychiatry. 2012;69:129–138. doi: 10.1001/archgenpsychiatry.2011.1269. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Oldfield RC. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia. 1971;9:97–113. doi: 10.1016/0028-3932(71)90067-4. [DOI] [PubMed] [Google Scholar]
  • 30.First M, Spitzer R, Gibbon M, Williams J. Structured clinical interview for DSM-IV-TR axis I disorders-patient edition. Biometrics Research Department, New York State Psychiatric Institute; 2001. [Google Scholar]
  • 31.Friston KJ, Holmes AP, Worsley KJ, Poline J-P, Frith CD, Frackowiak RSJ. Statistical parametric maps in functional imaging: a general linear approach. Human Brain Mapping. 1995;2:189–210. [Google Scholar]
  • 32.Miezin FM, Maccotta L, Ollinger JM, Petersen SE, Buckner RL. Characterizing the hemodynamic response: Effects of presentation rate, sampling procedure, and the possibility of ordering brain activity based on relative timing. Neuroimage. 2000;11:735–759. doi: 10.1006/nimg.2000.0568. [DOI] [PubMed] [Google Scholar]
  • 33.Barch DM, Mathews JR, Buckner RL, Maccotta L, Csernansky JG, Snyder AZ. Hemodynamic responses in visual, motor, and somatosensory cortices in schizophrenia. Neuroimage. 2003;20:1884–1893. doi: 10.1016/s1053-8119(03)00449-x. [DOI] [PubMed] [Google Scholar]
  • 34.Ford JM, Johnson MB, Whitfield SL, Faustman WO, Mathalon DH. Delayed hemodynamic responses in schizophrenia. Neuroimage. 2005;26:922–931. doi: 10.1016/j.neuroimage.2005.03.001. [DOI] [PubMed] [Google Scholar]
  • 35.Lindquist MA, Meng Loh J, Atlas LY, Wager TD. Modeling the hemodynamic response function in fMRI: efficiency, bias and mis-modeling. Neuroimage. 2009;45:S187–198. doi: 10.1016/j.neuroimage.2008.10.065. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Lindquist MA, Wager TD. Validity and power in hemodynamic response modeling: a comparison study and a new approach. Hum Brain Mapp. 2007;28:764–784. doi: 10.1002/hbm.20310. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.McAvoy MP, Ollinger JM, Buckner RL. Cluster size thresholds for assessment of significant activation in fMRI. NeuroImage. 2001;13:S198. [Google Scholar]
  • 38.Forman SD, Cohen JD, Fitzgerald M, Eddy WF, Mintun MA, Noll DC. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI) Use of a cluster-size threshold. Magn Reson Med. 1995;33:636–647. doi: 10.1002/mrm.1910330508. [DOI] [PubMed] [Google Scholar]
  • 39.Mamah D, Wang L, Barch D, de Erausquin GA, Gado M, Csernansky JG. Structural analysis of the basal ganglia in schizophrenia. Schizophr Res. 2007;89:59–71. doi: 10.1016/j.schres.2006.08.031. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Benjamini Y, Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. 1995;57:289–300. [Google Scholar]
  • 41.Morris R, Vercammen A, Lenroot R, Moore L, Langton J, Short B, et al. Disambiguating ventral striatum fMRI-related bold signal during reward prediction in schizophrenia. Molecular psychiatry. 2012;17:280–289. doi: 10.1038/mp.2011.75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Murray G, Corlett P, Clark L, Pessiglione M, Blackwell A, Honey G, et al. Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Molecular psychiatry. 2007;13:267–276. doi: 10.1038/sj.mp.4002058. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Schlagenhauf F, Huys QJ, Deserno L, Rapp MA, Beck A, Heinze H-J, et al. Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. Neuroimage. 2014;89:171–180. doi: 10.1016/j.neuroimage.2013.11.034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Gradin VB, Kumar P, Waiter G, Ahearn T, Stickle C, Milders M, et al. Expected value and prediction error abnormalities in depression and schizophrenia. Brain. 2011:awr059. doi: 10.1093/brain/awr059. [DOI] [PubMed] [Google Scholar]
  • 45.Koch K, Schachtzabel C, Wagner G, Schikora J, Schultz C, Reichenbach JR, et al. Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia. Neuroimage. 2010;50:223–232. doi: 10.1016/j.neuroimage.2009.12.031. [DOI] [PubMed] [Google Scholar]
  • 46.Walter H, Kammerer H, Frasch K, Spitzer M, Abler B. Altered reward functions in patients on atypical antipsychotic medication in line with the revised dopamine hypothesis of schizophrenia. Psychopharmacology. 2009;206:121–132. doi: 10.1007/s00213-009-1586-4. [DOI] [PubMed] [Google Scholar]
  • 47.Waltz JA, Schweitzer JB, Gold JM, Kurup PK, Ross TJ, Salmeron BJ, et al. Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers. Neuropsychopharmacology. 2009;34:1567–1577. doi: 10.1038/npp.2008.214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Wolf DH, Satterthwaite TD, Kantrowitz JJ, Katchmar N, Vandekar L, Elliott MA, et al. Amotivation in schizophrenia: integrated assessment with behavioral, clinical, and imaging measures. Schizophrenia bulletin. 2014;40:1328–1337. doi: 10.1093/schbul/sbu026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Kapur S. Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. Am J Psychiatry. 2003;160:13–23. doi: 10.1176/appi.ajp.160.1.13. [DOI] [PubMed] [Google Scholar]
  • 50.Zhang WN, Chang SH, Guo LY, Zhang KL, Wang J. The neural correlates of reward-related processing in major depressive disorder: a meta-analysis of functional magnetic resonance imaging studies. J Affect Disord. 2013;151:531–539. doi: 10.1016/j.jad.2013.06.039. [DOI] [PubMed] [Google Scholar]
  • 51.Chen C, Takahashi T, Nakagawa S, Inoue T, Kusumi I. Reinforcement learning in depression: A review of computational research. Neurosci Biobehav Rev. 2015;55:247–267. doi: 10.1016/j.neubiorev.2015.05.005. [DOI] [PubMed] [Google Scholar]
  • 52.Waltz JA, Schweitzer JB, Gold JM, Kurup PK, Ross TJ, Salmeron BJ, et al. Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers. Neuropsychopharmacology. 2009;34:1567–1577. doi: 10.1038/npp.2008.214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Waltz JA, Frank MJ, Robinson BM, Gold JM. Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction. Biol Psychiatry. 2007;62:756–764. doi: 10.1016/j.biopsych.2006.09.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Nielsen MO, Rostrup E, Wulff S, Bak N, Broberg BV, Lublin H, et al. Improvement of Brain Reward Abnormalities by Antipsychotic Monotherapy in Schizophrenia. Arch Gen Psychiatry. 2012:1–10. doi: 10.1001/archgenpsychiatry.2012.847. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement
NIHMS796111-supplement.pdf (1.2MB, pdf)

RESOURCES