Ventral striatal prediction error signaling is associated with dopamine synthesis capacity and fluid intelligence

Abstract

Fluid intelligence represents the capacity for flexible problem solving and rapid behavioral adaptation. Rewards drive flexible behavioral adaptation, in part via a teaching signal expressed as reward prediction errors in the ventral striatum, which has been associated with phasic dopamine release in animal studies. We examined a sample of 28 healthy male adults using multimodal imaging and biological parametric mapping with (1) functional magnetic resonance imaging during a reversal learning task and (2) in a subsample of 17 subjects also with positron emission tomography using 6‐[¹⁸F]fluoro‐L‐DOPA to assess dopamine synthesis capacity. Fluid intelligence was measured using a battery of nine standard neuropsychological tests. Ventral striatal BOLD correlates of reward prediction errors were positively correlated with fluid intelligence and, in the right ventral striatum, also inversely correlated with dopamine synthesis capacity (FDOPA K ${}_{\text{in}}^{\text{app}}$). When exploring aspects of fluid intelligence, we observed that prediction error signaling correlates with complex attention and reasoning. These findings indicate that individual differences in the capacity for flexible problem solving relate to ventral striatal activation during reward‐related learning, which in turn proved to be inversely associated with ventral striatal dopamine synthesis capacity. Hum Brain Mapp, 2013. © 2012 Wiley Periodicals, Inc.

INTRODUCTION

The fluid intelligence quotient (fluid IQ) [Horn and Cattell, 1966; Sternberg, 2000] represents the capacity of an individual for interpreting novel stimuli and flexible behavioral adaptation, whereas crystallized IQ reflects learning over the lifespan and has been associated with a neurobiological signature expressed in cortical structure [Choi et al., 2008]. Fluid IQ is a general factor comprising attributes such as attention, cognitive speed, working memory, reasoning, and episodic memory, which have been linked to activation in brain areas such as the dorsolateral prefrontal cortex (dlPFC) [Fuster, 2000; Goldman‐Rakic et al., 2000] and striatum [Cools et al., 2008; Landau et al., 2009]. Fluid IQ declines over the human lifespan [Salthouse, 1992], and it has been suggested that individual differences in cognitive functions such as cognitive speed and working memory performance are associated with alterations in dopaminergic neurotransmission in the striatum [Cools et al., 2008; Landau et al., 2009; Salthouse, 1992].

In animal studies of reward‐related learning, the firing properties of dopaminergic midbrain neurons correlate with trial‐by‐trial changes in errors of reward prediction (PEs), which reflect the difference between the expected reward and the reinforcement that was actually received [Schultz et al., 1997]. This finding was paralleled in human neuroimaging studies, which showed that functional activity in ventral striatum (VS), a target area of dopaminergic midbrain neurons, is correlated with PEs [O'Doherty, 2004;Pessiglione et al., 2006]. PE signals can be used to update the reward values of stimuli or actions in striatum [e.g., Frank and Claus, 2006] and hence play a key role in trial‐and‐error learning. As such, they are a key force driving flexible adaptation [Frank and Claus, 2006], and are intimately related to broad aspects of decision making in disease [Murray et al., 2008;Park et al., 2010] and across the life span [Lindenberger and Baltes, 1997].

Despite these findings, a relationship between individual differences in fluid IQ, reward‐related learning and dopamine neurotransmission has not yet been formally demonstrated. In this study, our main aim was to relate blood oxygen level‐dependent (BOLD) VS PE signals derived from a model‐based analysis of learning performance during a reversal learning task [Park et al., 2010] with (1) individual differences in a composite measure of fluid IQ [Lindenberger and Baltes, 1997] and (2) VS dopamine synthesis capacity as measured with positron emission tomography (PET) with 6‐[¹⁸F]fluoro‐L‐DOPA [Kienast et al., 2008]. We used biological parametric mapping to test for an association between functional magnetic resonance imaging (fMRI) and PET data in a voxel‐wise manner while controlling for gray matter volume differences.

MATERIALS AND METHODS

Subjects and Screening Instruments

A previously unpublished sample of 28 right‐handed healthy men with a mean age of 36.9 years (SD = 12.4; range, 22–61) underwent fMRI and neuropsychological testing. A subgroup of 17 participants (39.4 years SD = 12.1; range, 19–61 years) was also investigated with FDOPA PET. Subjects with Axis I and II psychiatric disorders according to DSM IV were excluded through the Structured Clinical Interview [First et al., 2001;First et al., 1997] and drug abuse was further excluded with urine tests. The study was approved by the local Ethics Committee of the Charité – Universitätsmedizin Berlin according to the Declaration of Helsinki, and written informed consent was obtained from all participants.

Neuropsychological Assessment and Intelligence Measures

A neuropsychological battery was given during an initial session within two months of PET and fMRI measurements. Components of fluid and crystallized IQ were measured with an adaptation of the standard battery used in the Berlin Aging Study [Lindenberger and Baltes, 1997]. Fluid IQ was measured with a battery of nine tests comprising cognitive speed (i.e., the Digit Symbol Substitution test [Wechsler, 1955]; Reitan Trailmaking test, part A [REITAN, 1955]), attention and executive function (Reitan Trailmaking test, part B [Reitan, 1955]; Stroop color‐word interference test [Stroop, 1935]), working memory (forward and backward digit span tests [Wechsler, 1955], episodic memory (Rivermead Behavioral Memory Test [Beckers et al., 1992]; auditory verbal learning test [Helmstaedter et al., 2001]), and reasoning [figural analogies [Horn, 1983]: for details, see Supporting Information S1].

Fluid IQ scores were derived from a factorial analysis of the raw scores of each of these tests using SPSS 11.0 for Macintosh (SPSS, Inc). Specifically, we used a Varimax rotation with an Eigenvalue cutoff set to 1.0; the final (single factor) solution accounted for 74.3% of the variance in the nine tests provided). The final composite measure of fluid IQ hence reflects z‐scores of aggregates of the nine tests provided derived from factorial analysis for the measured sample.

Furthermore, in order to allow for correlations within subdomains of fluid IQ we averaged z‐scores of each test for each domain (i.e., for cognitive speed, DSST, and Trailmaking A; for complex attention, Trailmaking B and Stroop; for working memory, forward and backward digit span; for episodic memory, RBMT and AVLT, for reasoning, LPS‐3 scores were used). Correlations between mean standardized scores in each domain (cognitive speed, attention, working memory, episodic memory, and reasoning) and the composite fluid IQ score ranged from 0.54 to 0.97 (all P‐values < 0.001).

Crystallized IQ was estimated using a verbal knowledge test [Schmidt and Metzler, 1992], during which subjects are required to identify each one meaningful word from a total of 42 lists of five words (of which four are nonsense words), which are ordered in increasing difficulty as reflected by the frequency of word use, and raw scores were used as the variable of interest.

Reversal Learning Task

During fMRI acquisition, subjects performed a reversal learning task [Cools et al., 2002; Kahnt et al., 2009; Park et al., 2010] known to evoke a BOLD PE signal in the striatum [O'Doherty et al., 2004; O'Doherty, 2004]. In each of 200 trials (100 per session), subjects first saw two abstract targets on the screen and were asked to choose one of them as quickly as possible by pressing the left or right button with the left or right thumb on a response box (maximum response time: 2 s). A blue box surrounding their chosen target and feedback (either a green smiley face for positive feedback or a red frowning face for negative feedback) were simultaneously shown for 1 s. The trials were separated with a jittered interval of 1–6.5 s.

Participants went through a random sequence of three types of blocks. In block Type 1, a reward was delivered for choosing the right stimulus if less than 80% of the recent right choices had been rewarded, and a punishment delivered otherwise. Conversely, a punishment was delivered for choosing the left stimulus if less than 80% of the recent left choices had been punished, and a reward delivered otherwise. In block Type 2 the contingencies were simply reversed for left and right. In block Type 3, the probabilities were 50/50 instead of 80/20. Switches between blocks happened always after 16 trials, or any time after 10 trials if subjects reached 70% correct choices.

Computational Modeling of Reinforcement Learning

The trial‐by‐trial sequence of choices for each subject was fit by a simple Rescorla‐Wagner (RW) model [Sutton and Barto, 1998]. This model assumes that the likelihood of a subject choosing action a on trial t is proportional to a value Q _t(a) and given by the softmax

The value Q _t(a) in turn is the expected value of that action, i.e., the expected reinforcement conditional on taking the action. It is updated iteratively

where ε is the learning rate. The variable R _t represents the effective reinforcement sensitivity as expressed by the effect of the reinforcement on the subject's choice behavior. This variable took on value R _t = β_rew if a reward was obtained and −β_pun if a punishment was obtained. To fit the models, parameters are transformed to lie on the real line vector of parameters. Letting θ = [ε′, β′_pun, β′_rew] denote the vector of transformed parameters, we report the maximum a posteriori estimates of these parameters using a Gaussian prior with mean and variance parameters μ and Σ:

where A ⁱ represents all the actions by subject i and where the dependence of each individual action probability on the parameters θ determining the Q value was emphasized. Importantly, we set the prior parameters empirically using Expectation Maximization to find the maximum likelihood estimates of μ and Σ given all the data by all the subjects. Separate parameters were fitted to each subject. An empirical Bayesian approach was used to constrain the individual parameters and fit the prior distribution to the data directly [Huys et al., 2011].

On the basis of the individually fitted parameters θⁱ for each of the subjects, a temporal sequence of PEs was computed for each subject i as

FMRI Protocol

fMRI acquisition

Functional imaging was conducted using a 3.0 Tesla GE Signa scanner with an 8 channel phase array head coil to acquire gradient echo T2*‐weighted echo‐planar images as previously described [Kahnt et al., 2009; Park et al., 2010]. For each of the two sessions, 310 EPI volumes (∼12 min) containing 29 slices (4 mm thick) were acquired (repetition time (TR) = 2,300 ms, echo time (TE) = 27 ms, matrix size 128 × 128 and a field of view (FOV) 256 × 256 mm², thus yielding an in‐plane voxel resolution of 2.7 mm², flip angle α = 90°). A 3D anatomical image of the entire brain was obtained by using a T1‐weighted 3D spoiled‐gradient echo pulse sequence with (TR = 7.8 ms, TE = 3.2 ms, matrix size 256 × 256, FOV 256 × 256 mm², 1 mm slice thickness, flip angle α =20°, voxel size 1 mm × 1 mm × 1 mm).

fMRI data preprocessing

Functional imaging data were analyzed using SPM8 (Wellcome Department of Imaging Neuroscience, Institute of Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm/). ArtRepair was used to remove noise spikes and to repair bad slices within a particular scan and bad slices were repaired by interpolation between adjacent slices (“Noise Filtering”, http://cibsr.stanford.edu/tools/ArtRepair/ArtRepair.htm). After that the following preprocessing steps were performed: acquisition time and motion correction, coregistration of the mean EPI to the anatomical T1 image, spatial normalization, and segmentation into tissue classes of the T1 image using the unified segmentation approach as implemented in SPM8 [Ashburner and Friston, 2005], application of the normalization parameters to all functional images, and spatial smoothing with an isotropic Gaussian kernel of 8 mm full width at half maximum (FWHM) kernel.

Statistical analysis

The images were analyzed in an event‐related manner using the general linear model approach (GLM) as implemented in SPM8, Neuronal activity was modeled for win and loss trials separately by stick functions at the onsets of the feedback. We used a parametric design [Buchel et al., 1998; O'Doherty et al., 2007], in which the trial‐by‐trial PE values from the Rescorla‐Wagner (RW) model modulated the amplitude of the trial related stick. Regressors of interest for the BOLD‐responses corresponding to the trial‐wise PEs were generated by convolving the modulated stimulus functions with the canonical hemodynamic response function (HRF), provided by SPM8. To account for signal fluctuations associated to the movement by susceptibility interaction, the six movement parameters from the realignment preprocessing step were included in the model as additional regressors. The individual contrast images for the contrast of the PE modulated feedback (combining win and loss feedback) were then taken to a random effects group‐level analysis using a one sample t‐test. To test for associations with measures of IQ, these measures were entered as covariates into additional random effects analyses. To control for age and individual fit of the Rescorla‐Wagner model, these variables were added as an additional covariate in the SPM analyses. Correlations were plotted using the mean PE‐related signal in VS VOIs (described below) and fluid IQ.

Correction for multiple comparisons

Small volume correction was used within the VS volume of interest (VOI). The VS VOI was constructed based on coordinates of previous findings using an in house tool provided by one of the authors (TW) to create an fMRI‐literature based probabilistic VOI for the VS. To this end, we selected 16 recent papers containing data from 325 subjects [Bray and O'Doherty, 2007; Cohen, 2007; Cohen and Ranganath, 2005; D'Ardenne et al., 2008; Gershman et al., 2009; Kahnt et al., 2009; Krugel et al., 2009; Murray et al., 2008; O'Doherty et al., 2004; O'Doherty et al., 2003; Palminteri et al., 2009; Pessiglione et al., 2006; Rodriguez et al., 2006; Schonberg et al., 2010; Tobler et al., 2006; Valentin and O'Doherty, 2009]. From each study, the coordinates of PE‐related activation for right and the left VS were extracted (see Supporting Information S2).

PET Protocol

PET acquisition

We used PET with FDOPA [Heinz et al., 2005;Kienast et al., 2008;Kumakura et al., 2007;Meyer‐Lindenberg et al., 2002] to define the magnitude of the net blood brain clearance of FDOPA, designated as K ${}_{\text{in}}^{\text{app}}$), which has units of cerebral blood flow (ml g⁻¹ min⁻¹). Subjects reclined on the scanning bed and their head positioned within the aperture of the PET/CT (Siemens Biograph 16) scanner in 3D mode. After a low dose CT‐scan, a dynamic ”list‐mode“ emission recording lasting 124 minutes started immediately after intravenous bolus administration of 200 MBq FDOPA. After CT‐based tissue attenuation correction and scatter correction, listmode data were iteratively reconstructed (OSEM, 16 iterations with six subsets) and framed (30 frames: 3 × 20 s, 3 × 1 min, 3 × 2 min, 3 × 3 min, 15 × 5 min, 3 × 10 min). Arterial blood samples were collected during the emission recording (in the first 6 min continuous measuring using a blood sampler, then manually at intervals), and the total radioactivity concentration in plasma samples was measured using a well‐counter cross‐calibrated to the PET. The fractions of untransformed FDOPA and the main metabolite O‐methyl‐[¹⁸F]‐fluoro‐L‐DOPA (OMFD) were measured in plasma extracts from blood collected at 5, 15, 30, 45, and 60 postinjection by reversed phase HPLC, and the continuous arterial input functions were calculated by bi‐exponential fitting of the measured fractions [Gillings et al., 2001]. We had initially intended to calculate V _d, the steady‐state storage capacity for FDOPA as described in Kienast et al. [2008] and elsewhere. However, the occurrence of extensive head motion during the second scanning hour precluded this analysis in 9 out of 17 participants. Therefore we confined our analysis to the conventional net blood brain clearance of FDOPA (K ${}_{\text{in}}^{\text{app}}$) during the first scanning hour.

PET data preprocessing

PET data were analyzed using SPM8 (Wellcome Department of Imaging Neuroscience, Institute of Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm/). The emission recording frames and the individual T1 image were coregistered to frame 12. The individual anatomical T1 image was spatially normalized using the unified segmentation approach of SPM8 [Ashburner and Friston, 2005], and the computed normalization parameters were applied to all frames.

Voxel‐wise quantification of net blood brain clearance of FDOPA

The net blood‐brain clearance of FDOPA from plasma to brain (K ${}_{\text{in}}^{\text{app}}$, ml g⁻¹ min⁻¹) was calculated voxel‐wise by Gjedde‐Patlak Linear Graphic analysis [Patlak and Blasberg, 1985] after subtracting the radioactivity measured in the cerebellum (using the mean activity within a standard cerebellum mask as defined in the WFU Pick Atlas), and using the frames of the first 60 min of the emission recording for the linear analysis [Cumming and Gjedde, 1998; Kumakura and Cumming, 2009]. The K ${}_{\text{in}}^{\text{app}}$ images were spatially smoothed with a Gaussian kernel of 8 mm full width at half maximum (FWHM). This net blood‐brain clearance is a macroparameter defined as K1 × k3/(k2 + k3), where K1 and k2 describe the partitioning across the blood brain barrier, and k3 is the relative activity of DOPA decarboxylase with respect to exogenous FDOPA. As such FDOPA K ${}_{\text{in}}^{\text{app}}$ describes the capacity to synthesize the dopamine from exogenous dopamine, as distinct from the dopamine synthesis rate, which depends on the unknown brain activity of tyrosine hydroxylase.

Biological parametric mapping (BPM)

To test for association of local K ${}_{\text{in}}^{\text{app}}$ and local PE related BOLD response Biological Parametric Mapping [BPM; Casanova et al., 2007] was used. Because both values appear to depend on local neuronal structures as reflected in gray matter density [Goense and Logothetis, 2008; Woodward et al., 2009], data should be corrected for this unspecific—usually age related—proportion of variance. Group models containing locally specific variable values can take advantage of this multimodal information and can best preserve the physiological meaningful local relationships. However, the algorithms currently implemented in SPM are not able to fit such locally different models to the data. To overcome this limitation, we used the BPM‐Toolbox [Casanova et al., 2007]. The conceptual difference between this approach and a conventional SPM style group analysis is in the use of other images as covariates. To evaluate the impact of dopamine synthesis capacity on the fMRI PE signal a BPM ANCOVA design was used with fMRI as primary modality, FDOPA K ${}_{\text{in}}^{\text{app}}$ maps and gray matter density from the structural MRI as imaging covariate and age as nonimaging covariate. Modulated gray matter density images of each participant were calculate from the structural high‐resolution T1‐weigthed MRI images using the unified segmentation approach [Ashburner and Friston, 2005] as implemented in SPM8 and smoothed with a 8 mm full width at half maximum Gaussian kernel.

RESULTS

Participants made on average 72.8% ± 6.3% correct responses and reached criterion on 5.5 ± 1.3 conditions with a learning speed of 6.8 ± 1.0 trials. In the fMRI group of 28 healthy controls, we observed a significant PE signal in VS (right VS: t = 3.113; x = 20, y = 6, z = −5; P _{FWE‐corrected for VS VOI} =0.019; left VS: t = 3.230 x = −11, y = 8, z = −3; P _{FWE‐corrected for VS VOI} =0.019) (Fig. 1).

Figure 1.

Prediction error signal in the bilateral ventral striatum. Coronal slice at MNI coordinate y = 6, statistical threshold t > 3.0, minimum cluster size = 20 voxels). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Fluid IQ was significantly and positively correlated with the BOLD PE signal in bilateral VS (right VS: t = 3.617; x = 20, y = 3, z = −8; P _{FWE‐corrected for VS VOI} = 0.007; and left VS: t = 2.994; x = −11, y = 8, z = −3; P _{FWE‐corrected for VS VOI} = 0.031), while crystalline IQ displayed no association, even when applying a much lower statistical threshold (P _{FWE‐corrected for VS VOI} > 0.2). There was no correlation between fluid IQ and correct responses on the reversal learning task (r = −0.015, P > 0.9).

Fluid IQ declined with age (r = −0.647; P < 0.001). Upon controlling for age by introducing age as an additional covariate into the SPM analysis, VS PE signal remained associated with fluid IQ (right VS: t = 3.959; x = 20, y = 6, z = −8; P _{FWE‐corrected for VS VOI} = 0.004; left VS: t = 3.970; x = −11, y = 11, z = −3; P _{FWE‐corrected for VS VOI} = 0.004), suggesting that this association was not simply explained by an age‐related decline in fluid IQ (Fig. 2). The positive correlation between VS PE signal and fluid IQ also remained significant when individual model fit (predictive probability) was introduced as an additional covariate (right VS: t = 3.47; x = 17, y = 6, z = −8; P _{FWE‐corrected for VS VOI} = 0.010; left VS: t = 3.32; x = −11, y = 11, z = −3; P _{FWE‐corrected for VS VOI} = 0.001).

Figure 2.

Positive correlation between prediction error signal and fluid intelligence. Mean prediction error signal extracted from the bilateral ventral striatal VOI (shown on the right panel) plotted against the z‐value of the fluid intelligence score (Z _{fluid IQ}). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Regarding exploratory analyses of associations between components of fluid IQ (attention, reasoning, working memory, episodic memory, and cognitive speed) while controlling for age, we observed a significant positive correlation of VS PE‐related signals with attention (left VS: t = 3.117; x = −11, y = 11, z = −3; P _{FWE‐corrected for VS VOI} = 0.026) and reasoning performance (right VS: t = 2.735; x = 15, y = 8, z = −5; P _{FWE‐corrected for VS VOI} = 0.043; left VS: t = 3.024; x = −16, y = 11, z = −3; P _{FWE‐corrected for VS VOI} = 0.030).

A subgroup of the participants (n = 17; 39.4 years SD = 12.1; range, 19–61 years) also underwent PET with 6‐[¹⁸F]fluoro‐l‐DOPA (FDOPA) to directly measure individual differences in dopamine metabolism. Dopamine synthesis capacity (K ${}_{\text{in}}^{\text{app}}$; ml g⁻¹ min⁻¹), which represents the net influx of FDOPA to the brain relative to the metabolite‐corrected plasma input, was computed on a voxel‐wise basis [Kienast et al., 2008].

The association between local BOLD response elicited by PE and local dopamine synthesis capacity K ${}_{\text{in}}^{\text{app}}$ was tested on a voxel‐by‐voxel basis to best take advantage of the multimodal imaging information. The Biological Parametric Mapping Toolbox (BPM) was applied [Casanova et al., 2007], which in contrast to standard SPM analyses allows the use of other parametric images such as K ${}_{\text{in}}^{\text{app}}$ as covariates. This analysis revealed an inverse correlation between local FDOPA K ${}_{\text{in}}^{\text{app}}$ and BOLD PE signal in right VS (t = 4.40; x = 15, y = 13, z = −8; P _{FWE‐corrected for VS VOI} = 0.011), but not on the left side (P _{fwe‐corrected for VS VOI}. > 0.8; uncorrected P = 0.020) (Fig. 3).

Figure 3.

Negative correlation between dopamine synthesis capacity as assessed in vivo with FDOPA PET and the BOLD prediction error signal in the ventral striatum. Left panel: Voxel‐by‐voxel association between FDOPA K ${}_{\text{in}}^{\text{app}}$ and BOLD prediction error signal from the Biological Parametric Mapping analysis. Coronal slice at MNI coordinate y = 12, statistical threshold t > 3.0, minimum cluster size =20 voxels). Right panel: Plot of z‐standardized mean K ${}_{\text{in}}^{\text{app}}$ value derived from the right VS VOI and mean BOLD prediction error signal derived from the right ventral striatal VOI. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

DISCUSSION

We learn from making mistakes and need to adapt our predictions in the face of changing circumstances; iterative learning via PEs plays a major role in such learning processes [Friston, 2010; O'Doherty et al., 2004; Park et al., 2010; Pessiglione et al., 2006; Sutton and Barto, 1998]. Our data show for the first time that individual neuronal signatures of PEs were directly related to individual differences in fluid IQ, even when correcting for the decline of fluid IQ with age: The age‐independent association between fluid IQ and the functional correlate of a VS PE signal suggests that the VS contributed to individual differences in cognitive flexibility.

This interpretation was given credence by two further analyses. First, more detailed analyses of the subcomponents of fluid IQ, which revealed that VS PE signaling was associated particularly with capacities such as complex attention and reasoning performance that are intimately related to error detection [Fuster, 2000]. Second, the correlation between fluid IQ and the VS BOLD correlate of PEs remained significant when we controlled for how well the PEs accounted for behavior (i.e., when controlling for the posterior likelihood the model assigned the data). This latter result is critical and supports our interpretation. If the VS BOLD correlates strongly with PEs in one subject but not another one, then that might just be because one subject used PEs to guide their behavior, while the other subject did not. Had the correlation with fluid IQ not survived correction for model fit, then the conclusion would have had to be that subjects with higher IQ were more likely to learn using PEs than subjects with lower IQ, and thus that the VS correlate was a reflection of this strategic difference. Our data do not support that conclusion. Indeed, such a conclusion would have been counterintuitive. If anything, subjects with higher fluid IQ could have been more inclined to extract and use the complex hidden Markovian structure of the task instead of approximating it with PE learning and this would have improved performance overall. However, we did not find any association between overall performance and fluid IQ. Furthermore, displacement of the model‐free by model‐based decision making should, if anything, have led to a negative correlation between PE signaling and fluid IQ.

To interpret these results, we turn to the finding that several types of learning can co‐exist, and be expressed at different times. In rats, lesions of different parts of the prefrontal cortex can reveal “latent” habitually learned responses when goal‐directed decisions are normally expressed [Killcross and Coutureau, 2003]. In this article, the use of PEs is explicitly motivated by their neurobiological face value in terms of DA phasic responses [Pessiglione et al., 2006; Schultz et al., 1997]. However, such iterative PE learning forfeits much of the intricate structure of reversal learning tasks (for instance Hidden Markov Models; e.g. [Hampton et al., 2006]. As such, the BOLD correlates of PE learning we observe is best taken as an index of PE learning that is ongoing even though it may not be fully expressed (and thus not be correlated with behavioral fit). Thus, subjects with higher fluid IQ show a higher subcortical, ongoing, RW learning even when this is not in charge. This suggests an interpretation whereby high fluid IQ subjects show a more varied, multifaceted approach to learning: rather than only exploiting one aspect of the reinforcement feedback, they exploit multiple interpretations of reinforcements, possibly allowing for more flexible future deployment of a larger variety of behavioral strategies. Our data point to VS PE signals as one key ingredient of such fluid flexibility.

The PE signal closely matches the temporal properties of a phasic dopamine response [Schultz et al., 1997] and appears to be modulated by dopamine agonists [Pessiglione et al., 2006]. While acute striatal dopamine release correlated with functional activation of the substantia nigra/ventral tegmental area during reward anticipation [Schott et al., 2008], it has to date not been explored whether individual differences in dopaminergic neurotransmission as assessed in vivo with PET are directly correlated with the PE related BOLD response [O'Doherty, 2004]. Like functional activation during reward anticipation, the PE signal may well be triggered by phasic alterations in firing rates of dopaminergic neurons and their respective effect on striatal neurotransmission [Pessiglione et al., 2006], while the magnitude of dopamine synthesis capacity K ${}_{\text{in}}^{\text{app}}$ likely reflects the local parenchymal brain capacity to form [¹⁸F]fluorodopamine from plasma FDOPA, and to retain that product within vesicles, mainly located in dopaminergic nerve terminals. FDOPA K ${}_{\text{in}}^{\text{app}}$ is thus widely accepted as a surrogate marker for the activity of DOPA‐decarboxylase and is interpreted as an index of dopamine synthesis capacity [Kumakura and Cumming, 2009]. Studies in Parkinson' disease patients with a loss of nigrostriatal nerve terminals found reduced FDOPA K ${}_{\text{in}}^{\text{app}}$ [Kumakura and Cumming, 2009] and therefore this macroparameter may reflect the density of dopaminergic innervations [Pate et al., 1993]. Therefore, one may have expected to find a positive correlation between VS PE signaling and dopamine synthesis capacity.

Indeed, dopamine depletion via blockade of synthesis reduces electrically evoked dopamine release [Venton et al., 2006]; however, such a rather profound stimulation of dopamine release rapidly depletes presynaptic dopamine storage pools and may impair continuous tonic as well as stimulus‐driven phasic [Goto et al., 2007] dopamine release. In our study, we observed a negative correlation in right VS between dopamine synthesis capacity and the functional PE signal. One explanation for this comes from animal and human studies with DA agonists and antagonists, which suggest that dopamine synthesis capacity is regulated by presynaptic autoreceptors [Cumming et al., 1997; Vernaleken et al., 2006]. In FDOPA PET studies of cats and healthy human volunteers Hassoun et al. calculated the magnitude of FDOPA influx in striatum relative to a reference brain region during the interval of 60 – 90 minutes after tracer injection. Comparison of the results obtained at baseline and during sensory stimulation (i.e. radial nerve stimulation) revealed in both species a significant decline in the apparent rate constant for FDOPA utilization in striatum [Hassoun et al., 2005]. In earlier [¹¹C]raclopride studies from the same research group, the same stimulus had evoked declines in dopamine D_2/3 receptor availability, suggestive of increased dopamine release [Thobois et al., 2004]. Although we note that their reference tissue FDOPA utilization estimates, which assumed irreversible trapping, could have been under‐estimated in a condition of increased dopamine turnover, their results were interpreted to reveal an inverse relationship between dopamine synthesis capacity and stimulus‐evoked dopamine release. Interestingly, a recent dual tracer PET study with [¹¹C]raclopride and the alternate DOPA decarboxylase tracer L‐[β‐¹¹C]DOPA likewise reported a negative correlation between baseline D_2/3 receptor availability and dopamine synthesis capacity in striatum of healthy human subjects [Ito et al., 2011]. In the context of our present findings, we suggest that presynaptic D_2/3 autoreceptors may normally mediate a compensatory balance between pre‐ and postsynaptic aspects of dopamine transmission, characterized by an inverse relationship between synthesis capacity and dopamine release.

Therefore, our finding of an inverse relationship between synthesis capacity and functional PE signal may indicate a homeostatic regulation of dopamine synthesis, under the control of autoreceptors. Indeed, dopaminergic autoreceptor availability in human mesencephalon was inversely correlated with striatal dopamine release evoked by amphetamine [Buckholtz et al., 2010]. Animal research has also suggested that a rather continuous, tonic mode of dopamine release is negatively correlated with a phasic dopamine signal [Goto et al., 2007; Grace, 1991; Grace, 2000]. The FDOPA K ${}_{\text{in}}^{\text{app}}$ is a steady‐state parameter measured during an hour, presumably reflecting the long‐lasting rather than phasic aspects of dopaminergic neurotransmission, although the exact mechanism of synthesis capacity and dopamine release remains to be elucidated. Therefore, one possible interpretation of the observed inverse correlation between the VS PE signal and local dopamine synthesis capacity is that these two measures reflect a phasic versus tonic component of dopamine‐related neurotransmission.

Higher dopamine synthesis capacity in some subjects does not necessarily reflect hereditary differences in DA neurotransmission; instead, elevated DA levels have, for instance, been found in socially stressed primates [Morgan et al., 2002; Nader et al., 2006]. In subjects with rather high levels of DA synthesis capacity and putatively higher stress levels (such as patients with psychosis [van Os et al., 2010], both impaired cognitive performance and a reduced VS PE signal has been reported [Heinz and Schlagenhauf, 2010; Murray et al., 2008]. Further studies are needed to assess acute and chronic stress effects on DA neurotransmission and their respective effects on cognitive performance.

Limitations of our study include the cross‐sectional nature of our data, the limitation to male subjects, and the relatively limited sample size of n = 17 for the multimodal imaging group, which nevertheless yielded significant correlations between VS PE signaling and dopamine synthesis capacity in VS. We only observed a significant correlation between FDOPA K ${}_{\text{in}}^{\text{app}}$ and the functional PE signal in the right but not left VS, and further studies may be required to clarify whether there are indeed lateralization differences in the mediation of PE signaling by striatal dopamine.

Altogether, our data reveal a potential mechanism directly linking individual differences in functional brain activation associated with reward PEs to flexible cognitive performance. Indeed, phasic alterations in striatal PE signaling appear to preferentially facilitate complex reasoning and staying on task [Aalto et al., 2005; Goto and Grace, 2005]. It has been suggested that cognitive performance can be targeted pharmacologically by dopaminergic agents such as modafinil [Grady et al., 2010]; however, the inverse correlation observed in this study between right VS dopamine synthesis and PE signaling in VS suggests that pharmacological intervention seeking to optimize cognitive abilities may face unexpected side‐effects due to the apparently complex nature of striatal dopaminergic signaling.

Supporting information

Additional Supporting Information may be found in the online version of this article.

Supporting Information