Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence

Abstract

Fluid intelligence represents the capacity for flexible problem solving and rapid behavioral adaptation. Rewards drive flexible behavioral adaptation, in part via a teaching signal expressed as reward prediction errors in the ventral striatum, which has been associated with phasic dopamine release in animal studies. We examined a sample of 28 healthy male adults using multimodal imaging and biological parametric mapping with 1) functional magnetic resonance imaging during a reversal learning task and 2) in a subsample of 17 subjects also with positron emission tomography using 6-[¹⁸F]fluoro-L-DOPA to assess dopamine synthesis capacity. Fluid intelligence was measured using a battery of nine standard neuropsychological tests. Ventral striatal BOLD correlates of reward prediction errors were positively correlated with fluid intelligence and, in the right ventral striatum, also inversely correlated with dopamine synthesis capacity (FDOPA K_in^app). When exploring aspects of fluid intelligence, we observed that prediction error signaling correlates with complex attention and reasoning. These findings indicate that individual differences in the capacity for flexible problem solving may be driven by ventral striatal activation during reward-related learning, which in turn proved to be inversely associated with ventral striatal dopamine synthesis capacity.

Introduction

The fluid intelligence quotient (fluid IQ) [Horn and Cattell, 1966;Sternberg, 2000] represents the capacity of an individual for interpreting novel stimuli and flexible behavioral adaptation, whereas crystallized IQ reflects learning over the lifespan and has been associated with a neurobiological signature expressed in cortical structure [Choi et al., 2008]. Fluid IQ is a general factor comprising attributes such as attention, cognitive speed, working memory, reasoning, and episodic memory, which have been linked to activation in brain areas such as the dorsolateral prefrontal cortex (dlPFC) [Fuster, 2000;Goldman-Rakic et al., 2000] and striatum [Cools et al., 2008;Landau et al., 2009]. Fluid IQ declines over the human lifespan [Salthouse, 1992], and it has been suggested that individual differences in cognitive functions such as cognitive speed and working memory performance are associated with alterations in dopaminergic neurotransmission in the striatum [Cools et al., 2008;Landau et al., 2009;Salthouse, 1992].

In animal studies of reward-related learning, the firing properties of dopaminergic midbrain neurons correlate with trial-by-trial changes in errors of reward prediction (PEs), which reflect the difference between the expected reward and the reinforcement that was actually received [Schultz et al., 1997]. This finding was paralleled in human neuroimaging studies, which showed that functional activity in ventral striatum (VS), a target area of dopaminergic midbrain neurons, is correlated with PEs [O'Doherty, 2004;Pessiglione et al., 2006]. PE signals can be used to update the reward values of stimuli or actions in striatum [e.g., Frank and Claus, 2006] and hence play a key role in trial-and-error learning. As such, they are a key force driving flexible adaptation [Frank and Claus, 2006], and are intimately related to broad aspects of decision making in disease [Murray et al., 2008;Park et al., 2010] and across the life span [Lindenberger and Baltes, 1997].

Despite these findings, a relationship between individual differences in fluid IQ, reward-related learning and dopamine neurotransmission has not yet been formally demonstrated. In this study, our main aim was to relate blood oxygen level-dependent (BOLD) VS PE signals derived from a model-based analysis of learning performance during a reversal learning task [Park et al., 2010] with 1) individual differences in a composite measure of fluid IQ [Lindenberger and Baltes, 1997] and 2) VS dopamine synthesis capacity as measured with positron emission tomography (PET) with 6-[¹⁸F]fluoro-L-DOPA [Kienast et al., 2008]. We used biological parametric mapping to test for an association between functional magnetic resonance imaging (fMRI) and PET data in a voxel-wise manner while controlling for grey matter volume differences.

Materials and Methods

Subjects and screening instruments

A previously unpublished sample of 28 right-handed healthy men with a mean age of 36.9 years (SD = 12.4, range 22–61) underwent fMRI and neuropsychological testing. A subgroup of 17 participants (39.4 years SD = 12.1, range 19–61 years) was also investigated with FDOPA PET. Subjects with axis I and II psychiatric disorders according to DSM IV were excluded through the Structured Clinical Interview [First et al., 2001;First et al., 1997] and drug abuse was further excluded with urine tests. The study was approved by the local Ethics Committee of the Charité – Universitätsmedizin Berlin according to the Declaration of Helsinki, and written informed consent was obtained from all participants.

Neuropsychological assessment and intelligence measures

A neuropsychological battery was given during an initial session within two months of PET and fMRI measurements. Components of fluid and crystallized IQ were measured with an adaptation of the standard battery used in the Berlin Aging Study [Lindenberger and Baltes, 1997]. Fluid IQ was measured with a battery of nine tests comprising cognitive speed (i.e, the Digit Symbol Substitution test [Wechsler, 1955]; Reitan Trailmaking test, part A [REITAN, 1955]), attention and executive function (Reitan Trailmaking test, part B [REITAN, 1955]; Stroop color-word interference test [Stroop and ., 1935]), working memory (forward and backward digit span tests [Wechsler, 1955], episodic memory (Rivermead Behavioral Memory Test [Beckers et al., 1992]; auditory verbal learning test [Helmstaedter et al., 2001]), and reasoning (figural analogies [Horn, 1983]: for details, see Supplementary Methods S1).

Fluid IQ scores were derived from a factorial analysis of the raw scores of each of these tests using SPSS 11.0 for Macintosh (SPSS, Inc.). Specifically, we used a Varimax rotation with an Eigenvalue cutoff set to 1.0; the final (single factor) solution accounted for 74.3% of the variance in the nine tests provided). The final composite measure of fluid IQ hence reflects z-scores of aggregates of the nine tests provided derived from factorial analysis for the measured sample.

Furthermore, in order to allow for correlations within subdomains of fluid IQ we averaged z-scores of each test for each domain (i.e., for cognitive speed, DSST and Trailmaking A; for complex attention, Trailmaking B and Stroop; for working memory, forward and backward digit span; for episodic memory, RBMT and AVLT, for reasoning, LPS-3 scores were used). Correlations between mean standardized scores in each domain (cognitive speed, attention, working memory, episodic memory, and reasoning) and the composite fluid IQ score ranged from .54 to .97 (all p-values <.001).

Crystallized IQ was estimated using a verbal knowledge test [Schmidt and Metzler, 1992], during which subjects are required to identify each one meaningful word from a total of 42 lists of five words (of which four are nonsense words), which are ordered in increasing difficulty as reflected by the frequency of word use, and raw scores were used as the variable of interest.

Reversal learning task

During fMRI acquisition, subjects performed a reversal learning task [Cools et al., 2002;Kahnt et al., 2009;Park et al., 2010] known to evoke a BOLD PE signal in the striatum [O'Doherty et al., 2004;O'Doherty, 2004]. In each of 200 trials (100 per session), subjects first saw two abstract targets on the screen and were asked to choose one of them as quickly as possible by pressing the left or right button with the left or right thumb on a response box (maximum response time: 2s). A blue box surrounding their chosen target and feedback (either a green smiley face for positive feedback or a red frowning face for negative feedback) were simultaneously shown for 1s. The trials were separated with a jittered interval of 1–6.5s.

Participants went through a random sequence of three types of blocks. In block type 1, a reward was delivered for choosing the right stimulus if less than 80% of the recent right choices had been rewarded, and a punishment delivered otherwise. Conversely, a punishment was delivered for choosing the left stimulus if less than 80% of the recent left choices had been punished, and a reward delivered otherwise. In block type 2 the contingencies were simply reversed for left and right. In block type 3, the probabilities were 50/50 instead of 80/20. Switches between blocks happened always after 16 trials, or any time after 10 trials if subjects reached 70% correct choices.

Computational modeling of reinforcement learning

The trial-by-trial sequence of choices for each subject was fit by a simple Rescorla-Wagner (RW) model [Sutton and Barto, 1998]. This model assumes that the likelihood of a subject choosing action a on trial t is proportional to a value Q_t(a) and given by the softmax

$$\mathrm{p}(\mathrm{a}|{\mathrm{Q}}_{\mathrm{t}})=\text{exp}({\mathrm{Q}}_{\mathrm{t}}(\mathrm{a}))/({\Sigma }_{\mathrm{a}\text{'}}\text{exp}({\mathrm{Q}}_{\mathrm{t}}(\mathrm{a}\prime ))).$$

The value Q_t(a) in turn is the expected value of that action, i.e. the expected reinforcement conditional on taking the action. It is updated iteratively

$${\mathrm{Q}}_{\mathrm{t}}(\mathrm{a})={\mathrm{Q}}_{\mathrm{t}-1}(\mathrm{a})+\mathrm{\epsilon }({\mathrm{R}}_{\mathrm{t}}-{\mathrm{Q}}_{\mathrm{t}-1}(\mathrm{a}))$$

where ε is the learning rate. The variable R_t represents the effective reinforcement sensitivity as expressed by the effect of the reinforcement on the subject’s choice behavior. This variable took on value R_t = β_rew if a reward was obtained and -β_pun if a punishment was obtained. To fit the models, parameters are transformed to lie on the real line vector of parameters. Letting θ=[ε’, β_pun’, β_rew’] denote the vector of transformed parameters, we report the maximum a posteriori estimates of these parameters using a Gaussian prior with mean and variance parameters µ and ∑:

$${\mathrm{\theta }}_{\mathrm{e}\mathrm{s}\mathrm{t}}^{\mathrm{i}}={\text{argmax}}_{\mathrm{\theta }}\text{log}\mathrm{p}({\mathrm{A}}^{\mathrm{i}}|\mathrm{\theta })\mathrm{p}(\mathrm{\theta }|\mathrm{\mu },\Sigma )={\text{argmax}}_{\mathrm{\theta }}[{\Sigma }_{\mathrm{t}}\text{log}\mathrm{p}({\mathrm{a}}_{\mathrm{t}}^{\mathrm{i}}|{\mathrm{Q}}_{\mathrm{t}},\mathrm{\theta })]\mathrm{p}(\mathrm{\theta }|\mathrm{\mu },\Sigma ).$$

where Aⁱ represents all the actions by subject i and where the dependence of each individual action probability on the parameters θ determining the Q value was emphasized. Importantly, we set the prior parameters empirically using Expectation Maximization to find the maximum likelihood estimates of µ and ∑ given all the data by all the subjects. Separate parameters were fitted to each subject. An empirical Bayesian approach was used to constrain the individual parameters and fit the prior distribution to the data directly.

Based on the individually fitted parameters θⁱ for each of the subjects, a temporal sequence of PEs was computed for each subject i as

$$\mathrm{P}{\mathrm{E}}_{\mathrm{t}}^{\mathrm{i}}={\mathrm{R}}^{\mathrm{i}}(\mathrm{t})-{\mathrm{Q}}_{\mathrm{t}}^{\mathrm{i}}({\mathrm{a}}_{\mathrm{t}}).$$

FMRI protocol

fMRI acquisition

Functional imaging was conducted using a 3.0 Tesla GE Signa scanner with an 8 channel phase array head coil to acquire gradient echo T2*-weighted echo-planar images as previously described [Kahnt et al., 2009;Park et al., 2010]. For each of the two sessions, 310 EPI volumes (~12 min) containing 29 slices (4mm thick) were acquired (repetition time (TR) = 2300 ms, echo time (TE) = 27 ms, matrix size 128×128 and a field of view (FOV) 256 × 256 mm², thus yielding an in-plane voxel resolution of 2.7 mm², flip angle α=90 degree). A 3D anatomical image of the entire brain was obtained by using a T1-weighted 3D spoiled-gradient echo pulse sequence with (TR = 7.8 ms, TE=3.2 ms, matrix size 256 × 256, FOV 256 × 256 mm², 1 mm slice thickness, flip angle α =20 degree, voxel size 1 mm × 1 mm × 1mm).

fMRI data preprocessing

Functional imaging data were analyzed using SPM8 (Wellcome Department of Imaging Neuroscience, Institute of Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm/). ArtRepair was used to apply to remove noise spikes and bad slices within one scan and bad slices were repair by interpolation (“Noise Filtering”, http://cibsr.stanford.edu/tools/ArtRepair/ArtRepair.htm). After that the following preprocessing steps were performed: acquisition time and motion correction, coregistration of the mean EPI to the anatomical T1 image, spatial normalization and segmentation into tissue classes of the T1 image using the unified segmentation approach as implemented in SPM8 [Ashburner and Friston, 2005], application of the normalization parameters to all functional images, and spatial smoothing with an isotropic Gaussian kernel of 8mm full width at half maximum (FWHM) kernel.

Statistical analysis

The images were analyzed in an event related manner using the general linear model approach (GLM) as implemented in SPM8, Neuronal activity was modeled for win and loss trials separately by stick functions at the onsets of the feedback. We used a parametric design [Buchel et al., 1998;O'Doherty et al., 2007], in which the trial-by-trial PE values from the Rescorla-Wagner (RW) model modulated the amplitude of the trial related stick. Regressors of interest for the BOLD-responses corresponding to the trial-wise PEs were generated by convolving the modulated stimulus functions with the canonical hemodynamic response function (HRF), provided by SPM8. To account for signal fluctuations associated to the movement by susceptibility interaction, the six movement parameters from the realignment preprocessing step were included in the model as additional regressors. The individual contrast images for the contrast of the PE modulated feedback (combining win and loss feedback) were then taken to a random effects group-level analysis using a one sample t-test. To test for associations with measures of IQ, these measures were entered as covariates into additional random effects analyses. To control for age and individual fit of the Rescorla-Wagner model, these variables were added as an additional covariate in the SPM analyses. Correlations were plotted using the mean PE-related signal in VS VOIs (described below) and fluid IQ.

Correction for multiple comparisons

Small volume correction was used within the VS volume of interest (VOI). The VS VOI was constructed based on coordinates of previous findings using an in house tool provided by one of the authors (TW) to create an fMRI-literature based probabilistic VOI for the VS. To this end, we selected 16 recent papers containing data from 325 subjects [Bray and O'Doherty, 2007;Cohen, 2007;Cohen and Ranganath, 2005;D'Ardenne et al., 2008;Gershman et al., 2009;Kahnt et al., 2009;Krugel et al., 2009;Murray et al., 2008;O'Doherty et al., 2004;O'Doherty et al., 2003;Palminteri et al., 2009;Pessiglione et al., 2006;Rodriguez et al., 2006;Schonberg et al., 2010;Tobler et al., 2006;Valentin and O'Doherty, 2009]. From each study, the coordinates of PE related activation for right and the left VS were extracted (see Supplementary Methods S2).

PET protocol

PET acquisition

We used PET with FDOPA [Heinz et al., 2005;Kienast et al., 2008;Kumakura et al., 2007;Meyer-Lindenberg et al., 2002] to define the magnitude of the net blood brain clearance of FDOPA, designated as K_in^app, which has units of cerebral blood flow (ml g⁻¹ min⁻¹). Subjects reclined on the scanning bed and their head positioned within the aperture of the PET/CT (Siemens Biograph 16) scanner in 3-D mode. After a low dose CT-scan, a dynamic „list-mode“ emission recording lasting 124 minutes started immediately after intravenous bolus administration of 200 MBq FDOPA. After CT-based tissue attenuation correction and scatter correction, listmode data were iteratively reconstructed (OSEM, 16 iterations with 6 subsets) and framed (30 frames: 3 × 20 s, 3 × 1 min, 3 × 2 min, 3 × 3 min, 15 × 5 min, 3 × 10 min). Arterial blood samples were collected during the emission recording (in the first 6 min continuous measuring using a blood sampler, then manually at intervals), and the total radioactivity concentration in plasma samples was measured using a well-counter cross-calibrated to the PET. The fractions of untransformed FDOPA and the main metabolite O-methyl-[¹⁸F]-fluoro-L-DOPA (OMFD) were measured in plasma extracts from blood collected at 5, 15, 30, 45, and 60 post injection by reversed phase HPLC, and the continuous arterial input functions were calculated by bi-exponential fitting of the measured fractions [Gillings et al., 2001]. Initially we also planned to calculate V_d similar to [Kienast et al., 2008]. Due to sever motion artifacts in 9 out of 17 participants during the second scanning hour Kinapp was used for the first scanning hour (frame 1 to 20).

PET data preprocessing

PET data were analyzed using SPM8 (Wellcome Department of Imaging Neuroscience, Institute of Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm/). The emission recording frames and the individual T1 image were coregistered to frame 12. The individual anatomical T1 image was spatially normalized using the unified segmentation approach of SPM8 [Ashburner and Friston, 2005], and the computed normalization parameters were applied to all frames.

Voxel-wise quantification of net blood brain clearance of FDOPA

The net blood-brain clearance of FDOPA from plasma to brain (Kinapp, ml g-1 min-1) was calculated voxel-wise by Gjedde-Patlak Linear Graphic analysis [Patlak and Blasberg, 1985] after subtracting the radioactivity measured in the cerebellum (using the mean activity within a standard cerebellum mask as defined in the WFU Pick Atlas), and using the frames of the first 60min of the emission recording for the linear analysis [Cumming and Gjedde, 1998;Kumakura and Cumming, 2009]. The K_in^app images were spatially smoothed with a Gaussian kernel of 8 mm full width at half maximum (FWHM). This net blood-brain clearance is a macroparameter defined as K1*k3/(k2 + k3), where K1 and k2 describe the partitioning across the blood brain barrier, and k3 is the relative activity of DOPA decarboxylase with respect to exogenous FDOPA. As such FDOPA Kinapp describes the capacity to synthesize the dopamine from exogenous dopamine, as distinct from the dopamine synthesis rate, which depends on the unknown brain activity of tyrosine hydroxylase.

Biological Parametric Mapping (BPM)

To test for association of local K_in^app and local PE related BOLD response Biological Parametric Mapping [BPM; Casanova et al., 2007] was used. Because both values appear to depend on local neuronal structures as reflected in grey matter density [Goense and Logothetis, 2008;Woodward et al., 2009], data should be corrected for this unspecific – usually age related – proportion of variance. Group models containing locally specific variable values can take advantage of this multimodal information and can best preserve the physiological meaningful local relationships. However, the algorithms currently implemented in SPM are not able to fit such locally different models to the data. To overcome this limitation, we used the BPM-Toolbox [Casanova et al., 2007]. The conceptual difference between this approach and a conventional SPM style group analysis is in the use of other images as covariates. To evaluate the impact of dopamine synthesis capacity on the fMRI PE signal a BPM ANCOVA design was used with fMRI as primary modality, FDOPA K_in^app maps and grey matter density from the structural MRI as imaging covariate and age as non-imaging covariate. Modulated gray matter density images of each participant were calculate from the structural high-resolution T1-weigthed MRI images using the unified segmentation approach [Ashburner and Friston, 2005] as implemented in SPM8 and smoothed with a 8mm full width at half maximum Gaussian kernel.

Results

Participants made on average 72.8% +/− 6.3% correct responses and reached criterion on 5.5 +/− 1.3 conditions with a learning speed of 6.8 +/− 1.0 trials. In the fMRI group of 28 healthy controls, we observed a significant PE signal in VS (right VS: t=3.113; x=20, y=6, z=−5; p_{FWE-corrected for VS VOI} =0.019; left VS: t=3.230 x=−11, y=8, z=−3; p_{FWE-corrected for VS VOI} =0.019) (Figure 1).

Figure 1. Prediction error signal in the bilateral ventral striatum.

Coronal slice at MNI coordinate y=6, statistical threshold t>3.0, minimum cluster size = 20 voxels).

Fluid IQ was significantly and positively correlated with the BOLD PE signal in bilateral VS (right VS: t=3.617; x=20, y=3, z=−8; p_{FWE-corrected for VS VOI}=0.007; and left VS: t=2.994; x=−11, y=8, z=−3; p_{FWE-corrected for VS VOI}=0.031), while crystalline IQ displayed no association, even when applying a much lower statistical threshold (p_{FWE-corrected for VS VOI}>0.2). There was no correlation between fluid IQ and correct responses on the reversal learning task (r=−.015, p>.9).

Fluid IQ declined with age (r=−.647; p<0.001). Upon controlling for age by introducing age as an additional covariate into the SPM analysis, VS PE signal remained associated with fluid IQ (right VS: t=3.959; x=20, y=6, z=−8; p_{FWE-corrected for VS VOI}=0.004; left VS: t=3.970; x=−11, y=11, z=−3; p_{FWE-corrected for VS VOI}=0.004), suggesting that this association was not simply explained by an age-related decline in fluid IQ (Figure 2). The positive correlation between VS PE signal and fluid IQ also remained significant when individual model fit (predictive probability) was introduced as an additional covariate (right VS: t=3.47; x=17, y=6, z=−8; p_{FWE-corrected for VS VOI}=0.010; left VS: t=3.32; x=−11, y=11, z=−3; p_{FWE-corrected for VS VOI}=0.001).

Figure 2. Positive Correlation between prediction error signal and fluid intelligence.

Mean prediction error signal extracted from the bilateral ventral striatal VOI (shown on the right panel) plotted against the z-value of the fluid intelligence score (Z_{fluid IQ}).

Regarding exploratory analyses of associations between components of fluid IQ (attention, reasoning, working memory, episodic memory, and cognitive speed) while controlling for age, we observed a significant positive correlation of VS PE-related signals with attention (left VS: t=3.117; x=−11, y=11, z=−3; p_{FWE-corrected for VS VOI} =0.026) and reasoning performance (right VS: t=2.735; x=15, y=8, z=−5; p_{FWE-corrected for VS VOI} =0.043; left VS: t=3.024; x=−16, y=11, z=−3; p_{FWE-corrected for VS VOI} =0.030).

A subgroup of the participants (n=17; 39.4 years SD = 12.1, range 19–61 years) also underwent PET with 6-[¹⁸F]fluoro-L-DOPA (FDOPA) to directly measure individual differences in dopamine metabolism. Dopamine synthesis capacity (K_in^app; ml g⁻¹ min⁻¹), which represents the net influx of FDOPA to the brain relative to the metabolite-corrected plasma input, was computed on a voxel-wise basis [Kienast et al., 2008].

The association between local BOLD response elicited by PE and local dopamine synthesis capacity K_in^app was tested on a voxel-by-voxel basis to best take advantage of the multimodal imaging information. The Biological Parametric Mapping Toolbox (BPM) was applied [Casanova et al., 2007], which in contrast to standard SPM analyses allows the use of other parametric images such as K_in^app as covariates. This analysis revealed an inverse correlation between local FDOPA K_in^app and BOLD PE signal in right VS (t=4.40; x=15, y=13, z=−8; p_{FWE-corrected for VS VOI}=0.011), but not on the left side (p_{FWE-corrected for VS VOI}.>0.8; uncorrected p=0.020).

Discussion

We learn from making mistakes and need to adapt our predictions in the face of changing circumstances; iterative learning via PEs plays a major role in such learning processes [O'Doherty et al., 2004;Park et al., 2010;Pessiglione et al., 2006]. Our data show for the first time that individual neuronal signatures of PEs were directly related to individual differences in fluid IQ, even when correcting for the decline of fluid IQ with age: The age-independent association between fluid IQ and the functional correlate of a VS PE signal suggests that the VS contributed to individual differences in cognitive flexibility.

This interpretation was given credence by two further analyses. First, more detailed analyses of the sub-components of fluid IQ, which revealed that VS PE signaling was associated particularly with capacities such as complex attention and reasoning performance that are intimately related to error detection (20). Second, the correlation between fluid IQ and the VS BOLD correlate of PEs remained significant when we controlled for how well the PEs accounted for behavior (i.e. when controlling for the model fit). This latter result is critical. If the VS BOLD correlates strongly with PEs in one subject but not another one, then that might just be because one subject used PEs to guide their behavior, while the other subject did not. Had the correlation with fluid IQ not survived correction for model fit, then the conclusion would have had to be that subjects with higher IQ were more likely to learn using PEs than subjects with lower IQ, and thus that the VS correlate was a reflection of this strategic difference. Furthermore, no direct relationship was observed between fluid IQ and performance on the reversal learning task. Our data do not support that conclusion.

An additional consideration of importance is that several types of learning can co-exist, and be expressed at different times. In rats, lesions of different parts of the prefrontal cortex can reveal ‘latent’ habitually learned responses when goal-directed decisions are normally expressed [Killcross and Coutureau, 2003]. In this paper, the use of PEs is explicitly motivated by their neurobiological face value in terms of DA phasic responses [Pessiglione et al., 2006;Schultz et al., 1997]. However, such iterative PE learning forfeits much of the intricate structure of reversal learning tasks (for instance Hidden Markov Models; [for instance Hidden Markov Models; Hampton et al., 2006]. As such, the BOLD correlates of PE learning we observe is best taken as an index of PE learning that is ongoing even though it may not be fully expressed (and thus not be correlated with behavioral fit). Thus, subjects with higher fluid IQ show a higher subcortical, ongoing, RW learning even when this is not in charge. This suggests an interpretation whereby high fluid IQ subjects show a more varied, multifaceted approach to learning: rather than only exploiting one aspect of the reinforcement feedback, they exploit multiple interpretations of reinforcements, possibly allowing for more flexible future deployment of a larger variety of behavioral strategies. Our data point to VS PE signals as one key ingredient of such fluid flexibility.

The PE signal closely matches the temporal properties of a phasic dopamine response [Schultz et al., 1997] and appears to be modulated by dopamine agonists [Pessiglione et al., 2006]. While acute striatal dopamine release correlated with functional activation of the substantia nigra/ventral tegmental area during reward anticipation [Schott et al., 2008], it has to date not been explored whether individual differences in dopaminergic neurotransmission as assessed in vivo with PET are directly correlated with the PE related BOLD response [O'Doherty, 2004]. Like functional activation during reward anticipation, the PE signal may well be triggered by phasic alterations in firing rates of dopaminergic neurons and their respective effect on striatal neurotransmission [Pessiglione et al., 2006], while the magnitude of dopamine synthesis capacity K_in^app likely reflects the local parenchymal brain capacity to form [¹⁸F]fluorodopamine from plasma FDOPA, and to retain that product within vesicles, mainly located in dopaminergic nerve terminals. FDOPA K_in^app is thus widely accepted as a surrogate marker for the activity of DOPA-decarboxylase and is interpreted as an index of dopamine synthesis capacity [Kumakura and Cumming, 2009]. Studies in Parkinson’ disease patients with a loss of nigrostriatal nerve terminals found reduced FDOPA K_in^app [Kumakura and Cumming, 2009] and therefore this macroparameter may reflect the density of dopaminergic innervations [Pate et al., 1993]. Therefore, one may have expected to find a positive correlation between VS PE signaling and dopamine synthesis capacity.

Indeed, dopamine depletion via blockade of synthesis reduces electrically evoked dopamine release [Venton et al., 2006]; however, such a rather profound stimulation of dopamine release rapidly depletes presynaptic dopamine storage pools and may impair continuous tonic as well as stimulus-driven phasic [Goto et al., 2007] dopamine release. In our study, we observed a negative correlation in right VS between dopamine synthesis capacity and the functional PE signal. One explanation for this comes from animal and human studies with DA agonists and antagonists, which suggest that dopamine synthesis capacity is regulated by presynaptic autoreceptors [Cumming et al., 1997;Vernaleken et al., 2006]. Hassoun [Hassoun et al., 2005] examined the Kin from 60 to 90 minutes after injection and compared the baseline measurement with a Kin calculated during sensory stimulation and found that increased dopamine release induced by the sensory stimulation was associated with a decreased striatal synthesis capacity. Using a different approach compared to our study, which only assessed baseline dopamine synthesis capacity, the observation of Hassoun et al. thus suggests inverse correlations between dopamine synthesis capacity and release. Interestingly, a recent study using PET with [¹¹C]raclopride and L-[β-¹¹C]DOPA found a negative correlation between the binding potential of dopamine D2 receptors and endogenous dopamine synthesis rate in the striatum [Ito et al., 2011]. In line with our findings this negative correlation points towards a compensative relation between presynaptic and postsynaptic dopaminergic functions may mediated by D2 autoreceptors, which may indicate an inverse relationship between synthesis capacity and dopamine release, hypothetically mediated by D2 autoreceptors.

Therefore, our finding of an inverse relationship between synthesis capacity and functional PE signal may indicate a homeostatic regulation of dopamine synthesis, under the control of autoreceptors. Indeed, dopaminergic autoreceptor availability in human mesencephalon was inversely correlated with striatal dopamine release evoked by amphetamine [Buckholtz et al., 2010]. Animal research has also suggested that a rather continuous, tonic mode of dopamine release is negatively correlated with a phasic dopamine signal [Goto et al., 2007;Grace, 2000]. The FDOPA K_in^app is a steady-state parameter measured during an hour, presumably reflecting the long-lasting rather than phasic aspects of dopaminergic neurotransmission, although the exact mechanism of synthesis capacity and dopamine release remains to be elucidated. Therefore, one possible interpretation of the observed inverse correlation between the VS PE signal and local dopamine synthesis capacity is that these two measures reflect a phasic versus tonic component of dopamine-related neurotransmission.

Higher dopamine synthesis capacity in some subjects does not necessarily reflect hereditary differences in DA neurotransmission; instead, elevated DA levels have, for instance, been found in socially stressed primates [Morgan et al., 2002;Nader et al., 2006]. In subjects with rather high levels of DA synthesis capacity and putatively higher stress levels (such as patients with psychosis [such as patients with psychosis van Os et al., 2010], both impaired cognitive performance and a reduced VS PE signal has been reported [Heinz and Schlagenhauf, 2010;Murray et al., 2008]. Further studies are needed to assess acute and chronic stress effects on DA neurotransmission and their respective effects on cognitive performance.

Limitations of our study include the cross-sectional nature of our data, the limitation to male subjects, and the relatively limited sample size of n=17 for the multimodal imaging group, which nevertheless yielded significant correlations between VS PE signaling and dopamine synthesis capacity in VS. We only observed a significant correlation between FDOPA K_in^app and the functional PE signal in the right but not left VS, and further studies may be required to clarify whether there are indeed lateralization differences in the mediation of PE signaling by striatal dopamine.

Altogether, our data reveal a potential mechanism directly linking individual differences in functional brain activation associated with reward PEs to flexible cognitive performance. Indeed, phasic alterations in striatal PE signaling appear to preferentially facilitate complex reasoning and staying on task [Aalto et al., 2005;Goto and Grace, 2005]. It has been suggested that cognitive performance can be targeted pharmacologically by dopaminergic agents such as modafinil [Grady et al., 2010]; however, the inverse correlation observed in this study between right VS dopamine synthesis and PE signaling in VS suggests that pharmacological intervention seeking to optimize cognitive abilities may face unexpected side-effects due to the apparently complex nature of striatal dopaminergic signaling.

Figure 3. Negative correlation between dopamine synthesis capacity as assessed in vivo with FDOPA PET and the BOLD prediction error signal in the ventral striatum.

Left panel: Voxel-by-voxel association between FDOPA K_in^app and BOLD prediction error signal from the Biological Parametric Mapping analysis. Coronal slice at MNI coordinate y=12, statistical threshold t>3.0, minimum cluster size =20 voxels). Right panel: Plot of z-standardized mean K_in^app value derived from the right VS VOI and mean BOLD prediction error signal derived from the right ventral striatal VOI.