Dissociable dopaminergic control of saccadic target selection and its implications for reward modulation

Abstract

To investigate mechanisms by which reward modulates target selection, we studied the behavioral effects of perturbing dopaminergic activity within the frontal eye field (FEF) of monkeys performing a saccadic choice task and simulated the effects using a plausible cortical network. We found that manipulation of FEF activity either by blocking D1 receptors (D1Rs) or by stimulating D2 receptors (D2Rs) increased the tendency to choose targets in the response field of the affected site. However, the D1R manipulation decreased the tendency to repeat choices on subsequent trials, whereas the D2R manipulation increased that tendency. Moreover, the amount of shift in target selection resulting from the two manipulations correlated in opposite ways with the baseline stochasticity of choice behavior. Our network simulation results suggest that D1Rs influence target selection mainly through their effects on the strength of inputs to the FEF and on recurrent connectivity, whereas D2Rs influence the excitability of FEF output neurons. Altogether, these results reveal dissociable dopaminergic mechanisms influencing target selection and suggest how reward can influence adaptive choice behavior via prefrontal dopamine.

As the primary means of exploring the visual environment, we shift our gaze several times each second via saccadic eye movements. Where we look depends not only on the physical salience of visual stimuli, but also on their reward value (1–3). For example, while shopping, your attention may be captured differently by colored sales tags upon realizing that they indicate different discounts (e.g., yellow tags for 20% vs. red tags for 40% off). Studies of value-based behavior have established that midbrain dopamine (DA) neurons signal different aspects of reward (4) and that neurons in many cortical structures receiving dopaminergic projections represent the reward value of visual stimuli (5). The frontal eye field (FEF), the area of prefrontal cortex (PFC) most directly involved in triggering saccadic eye movements, receives inputs from structures encoding reward value (6), and these inputs might be sufficient to control value-based, saccadic choice (target selection). It is also possible that this control operates via FEF projections to caudate in the basal ganglia, where DA-mediated plasticity modulates reward-dependent target selection (7). In addition, the direct dopaminergic inputs to the FEF (8) could provide a mechanism that independently modulates target selection based on reward signals. Determining how dopamine within the FEF influences saccades is thus important for understanding the mechanism by which reward modulates the selection of visual targets for eye movements.

To address these questions, we manipulated dopaminergic activity within the FEF of monkeys performing a saccadic choice task. In the task, monkeys freely selected between two identical visual targets appearing at varying temporal asynchronies. By systematically altering the target onset asynchrony (TOA), we could measure the bias in selecting either target, the change in selection probability due to previous choice, and the sensitivity of choice to the TOA. The selection of either target was rewarded after the saccade, so the bias in target selection corresponded to the delay in reward that the animal endured for selecting its preferred target. Dopaminergic FEF activity was manipulated either by blocking D1 receptors (D1Rs) with a selective antagonist or by stimulating D2 receptors (D2Rs) with a selective agonist. We recently reported that these two manipulations produce equivalent effects on saccadic target selection, both increasing the propensity of monkeys to make saccades to stimuli within the response fields (RFs) of affected FEF neurons (9). Both receptor subtypes are known to exert complex, modulatory effects on neural activity and behavior (10–13). For example, D1Rs exhibit dose-dependent, inverted U-shape effects on the persistent activity of PFC neurons (14, 15) and on working memory (16). To understand the complex modulatory effects of FEF dopamine on saccadic choice behavior we looked for differential influences of D1Rs and D2Rs on that behavior and simulated those influences using a plausible cortical network. We observed dissociable influences of D1R- and D2R-mediated FEF activity on saccadic choice that could be explained by dopaminergic modulation of synaptic plasticity and neural activity within different cortical layers.

Results

Experimental Findings.

To quantify target selection, we used a task in which monkeys were trained to choose one of two stimuli as the targets of saccadic eye movements (Fig. 1A). During each experiment, we positioned one of the two targets (T_in) within the RFs of FEF neurons at the site of a drug infusion and the other target (T_out) at a diametrically opposite location. Targets were presented at varying TOAs over a wide range of values (∼ ±200 ms) and monkeys were rewarded regardless of which target they chose. Positive (negative) values of the TOA denote that T_in (T_out) appeared first, followed by the appearance of the second target TOA milliseconds later. To select the second target, the monkey thus had to wait (an amount of time equal to the TOA) until it appeared. All TOAs were presented with equal probability and were pseudorandomly interleaved such that on any given trial the monkey could not predict the TOA.

Fig. 1.

Similar effects of manipulating D1R- and D2R-mediated FEF activity on saccadic choice behavior. (A) The saccadic choice task. In the task, two targets appeared on the display asynchronously (Δt), and the monkey was rewarded for making a saccadic eye movement to either one of them. One of the targets (T_in) appeared within the FEF RF. (B) Psychometric functions from two example experiments measuring the proportion of T_in choices across TOAs before (black) and after manipulation of D1R- (red) or D2R-mediated (blue) FEF activity. Solid curves show the logistic regression fit. The vertical dashed lines denote the TOAs yielding a 0.5 proportion of T_in choices (i.e., PES). (C–E) Distribution of PES values (C), overall choice probability [p(T_in)] (D), and stochasticity of choice (σ) (E) before and after the dopaminergic manipulations. Triangles show the median of each distribution and asterisks denote significant differences (P < 0.05) between distributions.

When targets appeared simultaneously, monkeys tended to choose one target more often than the other, exhibiting a bias. By varying the TOA, we could measure that bias in terms of the delay in onset at which the monkey began to choose the alternative target (17). We refer to that bias as the point of equal selection (PES), namely the TOA at which the monkey chooses the two targets with equal probability. The PES was determined from a logistic fit of the probability of choosing T_in as a function of the TOA (Materials and Methods). Positive values of the PES denote biases in favor of T_out, whereas negative values denote biases in favor of T_in.

Saccadic choice was measured during control trials and after the manipulation of either D1R- or D2R-mediated FEF activity via the local infusion of the selective D1R antagonist SCH23390 or the selective D2R agonist quinpirole. In total, we performed 34 experiments on two monkeys: 21 D1R antagonist infusions (14 in monkey A and 7 in monkey B) and 13 D2R agonist infusions (8 in monkey A and 5 in monkey B). The infusions in monkeys A and B were done in the left and right hemispheres, respectively. We analyzed data from blocks of trials before (control) and after the drug infusion (drug). These data consisted of an average of 166 (control, SD = 48) and 173 (drug, SD = 42) trials in each of the D1R experiments and 201 (control, SD = 43) and 205 (drug, SD = 44) trials in each of the D2R experiments. As the results were consistent between the two monkeys and there were no effects of learning (Fig. S1 and SI Text, Consistency of Experimental Results Between Two Monkeys and Learning Effects), we performed our analyses using the combined data.

Fig. 1B shows the psychometric functions obtained in two example experiments in monkey A before and after a D1R antagonist or a D2R agonist infusion into the FEF. In both cases, the PES during control was slightly positive, indicating a small bias toward T_out. Following both the D1R and the D2R manipulations the PES was shifted leftward, indicating an increase in T_in selection. This pattern of results was consistent across all experiments in the two monkeys. Although there were no significant differences between the PESs measured during the separate D1R and D2R experiments, either before (control_D1R vs. control_D2R, z = −0.99, P = 0.3) or after the drug infusion (D1R vs. D2R, z = −0.64, P = 0.5), within each experiment the PES was significantly reduced by both drug manipulations (ΔPES = −29.3 ± 4.0 ms for D1R, z = −4.0, P < 0.0001; ΔPES = −20.5 ± 8.1 ms for D2R, z = −2.1, P < 0.04) (Fig. 1C). Furthermore, the magnitude of the shift toward T_in choices was independent of the control PES for both drug manipulations (r = 0.03, P = 0.9 for D1R; r = −0.09, P = 0.8 for D2R), indicating a fixed increment in T_in preference. We observed no effects of either drug manipulation on saccadic latency or amplitude (SI Text, Drug Effects on Saccade Metrics). In addition to measuring the effect of the drug manipulations on the choice bias (PES), we also examined its effect on the overall probability of selecting T_in, or p(T_in). Consistent with the PES effects, we found that both blocking D1Rs [Δp(T_in) = 0.071 ± 0.010, z = −4.0, P < 0.0001] and stimulating D2Rs [Δp(T_in) = 0.046 ± 0.016, z = −2.3, P < 0.02] increased p(T_in) above that of control trials (Fig. 1D). Thus, saccadic target choice was shifted in favor of T_in following both dopaminergic manipulations.

In addition to measuring bias in the selection of targets, we also measured the stochasticity of the monkeys’ choices, determined from σ of the fitted psychometric function (Materials and Methods). Larger values of σ correspond to greater stochasticity, i.e., a larger range of TOAs at which the choice is not determined solely by the TOA, whereas lower values of σ correspond to more deterministic choice behavior. We found that there were significant differences between the σ-values measured during the separate D1R and D2R manipulations both before (control_D1R vs. control_D2R, z = 2.4, P < 0.01) and after the drug infusion (D1R vs. D2R, z = 3.8, P < 0.0001) (Fig. 1E). The differences might be due to the different experiments being carried out on different days. However, the key comparison is between control and drug values within each manipulation, carried out on the same day. For this comparison, we found that neither the D1R (z = −0.43, P = 0.7) nor the D2R (z = −1.4, P = 0.2) altered σ. Thus, the stochasticity of choice behavior was unaltered by either dopaminergic manipulation.

In contrast to the nearly identical effects of blocking D1Rs and stimulating D2Rs on target preference [p(T_in) and PES] and on the stochasticity of choice (σ), we found that the two manipulations altered choice behavior in very different ways. First, we found that the D1R and D2R manipulations exerted different effects on the tendency of monkeys to repeat choices on subsequent trials. We quantified the tendency to repeat choices with a repetition index (RI), positive values indicating a probability of repetition that exceeds the tendency due solely to the choice bias (SI Text, Repetition Index as a Measure of Repetition in Choice). Fig. 2A shows the distribution of RIs before and after the D1R and D2R manipulations. Although neither of the control RIs significantly differed from zero (control_D1R, z = −1.4, P = 0.2; control_D2R, z = −0.73, P = 0.5), both drug manipulations yielded nonzero RIs, yet in opposite ways. The D1R antagonist reduced RIs below zero (RI = −0.032 ± 0.009, z = −2.8, P < 0.005), to values significantly less than control (ΔRI = −0.028 ± 0.010, z = −2.4, P < 0.02), indicating that monkeys became less likely to repeat target choices on subsequent trials. In contrast, the D2R agonist increased RIs above zero (RI = 0.050 ± 0.012, z = −2.8, P < 0.005), to values significantly greater than control (ΔRI = 0.040 ± 0.012, z = −2.6, P < 0.01), indicating that monkeys became more likely to repeat target choices on subsequent trials. Consequently, although the control RIs did not differ before the drug infusion (control_D1R vs. control_D2R, z = 0.78, P = 0.4), they did so after the drug infusion (D1R vs. D2R, z = 4.0, P < 0.00005). Thus, the two dopaminergic manipulations exerted opposite effects on the tendency of monkeys to repeat saccadic choices.

Fig. 2.

Dissociable effects of blocking D1Rs and stimulating D2Rs on saccadic choice behavior. (A) Distribution of the repetition index values before and after the dopaminergic manipulations. Other conventions are as in Fig. 1. (B) Correlations between changes in p(T_in) due to drug manipulations and the stochasticity of choice (σ) measured during control trials. Correlation coefficients, significance, and linear fits (solid line) are shown for the two datasets.

Second, we found that although the two dopaminergic manipulations increased T_in choices equally, the magnitude of their effects correlated with the choice stochasticity in opposite ways. Fig. 2B shows how the increases in T_in choices varied as a function of choice stochasticity measured during control trials. Following the D1R antagonist infusion, the increase in p(T_in) was positively correlated with the σ measured during control trials (r = 0.61, P = 0.003). This result indicates that during experiments in which monkeys exhibited greater stochasticity in their choices (larger σ-values, i.e., choices were less determined by the TOA), blocking D1Rs led to greater increases in T_in choices. In contrast, the increase in p(T_in) following stimulation of D2Rs was negatively correlated with the σ measured during control trials (r = −0.64, P = 0.02). Thus, during experiments in which monkeys exhibited greater stochasticity in their choices, D2R stimulation led to smaller increases in T_in choices. In addition, an analysis of covariance (ANCOVA) confirmed the contrasting relationships between stochasticity and increases in T_in selection produced by the D1R and D2R manipulations [F(1, 30) = 15.7, P = 0.0004]. Finally, we tested whether changes in choice bias and repetition were generated through the same mechanisms by computing the correlation between the two. The PES and RI were not correlated either before (control_D1R, r = 0.069, P = 0.8; control_D2R, r = 0.13, P = 0.7) or after the drug infusion (D1R, r = 0.22, P = 0.3; D2R, r = 0.44, P = 0.1). In addition, we found no significant correlation between changes in the PES and RI due to blocking D1Rs (r = 0.28, P = 0.2) or stimulating D2Rs (r = −0.36, P = 0.2).

Modeling Results.

In an attempt to account for the observed dopaminergic effects on saccadic choice behavior, we constructed a biophysically plausible cortical network model of target selection and examined how changing model parameters altered choice behavior in the task. The model was composed of two FEF columns, consisting of pools of excitatory neurons within superficial (layers II and III) and deep (layers V and VI) cortical laminae (Fig. 3A). Each column contained two pools of excitatory pyramidal neurons, one in superficial layers and one in deep layers. In addition, a pool of inhibitory interneurons mediated mutual inhibition between the excitatory pools of the superficial layers. The excitatory pools within superficial and deep layers had RFs corresponding to the location of the two saccadic targets. Both superficial layer pools were driven by three types of input: background, visual, and value based (SI Text, Computational Model). These pools competed in a winner-take-all fashion to drive deep layer (output) pools, which in turn rendered a choice. The winner-take-all property was due to connectivity of excitatory and inhibitory pools in the superficial layers. In the deep layers, however, there was only weak recurrent excitation between neurons with similar selectivity. The excitatory pools in the deep layers sent outputs to the brainstem or the superior colliculus, driving target selection. Therefore, the activity of neural pools in the deep layers determined the network’s choice on each trial. Specifically, we assumed that the network’s choice on a given trial was the target that was represented by the deep layer pool whose activity reached 15 Hz first. Consequently, changes in the excitability or input efficacy of deep layer pools can also affect decision. The model also implemented dopaminergic modulation of synaptic plasticity and neural activity, providing a means by which to simulate the D1R and D2R manipulations (Materials and Methods).

Fig. 3.

Network architecture of the model used to simulate saccadic choice and the effects of manipulating different network elements on choice behavior. (A) The model comprised two FEF columns (T_out and T_in), consisting of excitatory neurons within superficial and deep layers. Inhibitory (Inh) interneurons mediated mutual inhibition between the excitatory pools within the superficial layers. Superficial layer pools were driven by value-based, visual, and background inputs and projected to deep layer pools. Deep layer pools projected to brainstem oculomotor structures to render a saccadic choice. Connections and neural activity in the two columns were modulated by DA. To reproduce the effects of the drug infusion, only elements within the T_in column were altered in the simulations (orange). Inset shows example activity trajectories of the T_out and T_in pools [r(T_in) vs. r(T_out)] during trials on which the target appearing first (dark green or dark blue) or second (pale green or pale blue) was selected. The gray ellipse highlights the activity trajectories measured after the appearance of the first target and before the appearance of the second target (TOA epoch) on trials in which the first target (T_in) was chosen. Note that activity within the two pools diverges quickly during this epoch. The gray circle highlights the activity trajectory during the TOA epoch on trials in which the second target was chosen. Note that activity within the two pools remains equal during this interval. (B) Summary of choice behavior changes resulting from alterations to different network sites. The direction of alteration at each site was chosen such that it increased T_in selection (except for IE). Zero represents no change. For comparison, a summary of the experimental findings is shown at the bottom.

The model selected between the two saccade targets with a probability that depended on the TOA. The probability of choosing T_in tended to be 1 or 0 when the absolute value of TOA was large (i.e., when the T_in appears long before or after T_out), due to a lack of visual input to one of the columns during the interval between target onsets and activity within that column being suppressed by visually driven activity within the other. On the other hand, when the TOA was close to zero, choice behavior was less determined by the TOA. This occurs because the time interval in which visual input differs is near zero and thus the outcome of the competition between T_in and T_out pools depends on other inputs. Fig. 3A (Inset) depicts the simulated response trajectories of pools within the superficial layers of T_in and T_out columns during trials in which either the first- or the second-appearing target was chosen (Fig. S2). In the latter case, the responses of the two pools tended to remain equal in the interval between target onsets (indicated by the gray circle), but could diverge later solely due to random fluctuations in the inputs. Notably, we found that the model’s choice behavior could be fit as a sigmoid function of the TOA, similar to the experimental data (Fig. S3A). Moreover, by modulating the background inputs and overall visual inputs we could account for the experimentally observed variability in PES and σ-values (Fig. S3B and SI Text, Computational Model).

After establishing that the model can qualitatively replicate saccadic choice behavior during the control experiments, we next studied how independently altering different elements of the network changes that behavior (SI Text, Effects of Drug-Induced Alterations on the Model’s Choice Behavior). The model considered two classes of alterations, one static and one dynamic; the former corresponded to alterations to history-independent synaptic efficacy, whereas the latter corresponded to alterations to history-dependent processes (e.g., short-term plasticity, STP). First, we found that static alterations to all types of input, recurrent connections, and the excitability of the output pool (via alterations of the efficacy of the deep layer background input) could alter the probability of T_in choices, p(T_in) or equivalently the PES (Fig. 3B). Increasing the efficacy increased p(T_in) for all sites except the inhibitory–excitatory (IE) connections, where increased efficacy decreased p(T_in). These results are expected as increases in the efficacy of the aforementioned sites (except IE connections) increase the activity in the T_in column and therefore increase the selection of the T_in target. Note that alterations in the efficacy of excitatory–inhibitory connections and of superficial to deep layer pools resulted in qualitatively similar effects to those of alterations in IE and the excitability of output pools, respectively. Second, static alteration to all sites produced changes in p(T_in) that depended on the value of σ (stochasticity in choice) (Fig. S4), resembling our experimental observations. Namely, changes in p(T_in) were larger when σ was larger for static alterations to all sites except the deep layers where these changes were smaller for larger σ-values (see SI Text, Comparison of the Effects of Alterations to the Superficial Layers vs. the Deep Layers for an intuitive explanation). Third, we found that altering value-based inputs was the only static alteration that could produce changes in the RI (Fig. S4). This is because only the efficacy of value-based input carried information about the choice on the previous trial. Fourth, dynamic alteration to all sites except value-based input [via changing the rates of long-term depression (LTD) and potentiation (LTP)] produced changes in p(T_in). Moreover, as with the static alterations, changes in p(T_in) depended on the value of σ. Fifth, dynamic alteration to all sites except visual and background inputs altered the RI (Fig. S5). Specifically, reduction in STP at recurrent connections resulted in changes in the RI that were positive at excitatory–excitatory (EE) and negative at IE connections. Note that STP reduces the strength of EE connections within the recently active column and results in alternation; therefore, reduction in STP increases repetition. In addition, increases in the rates of LTD and LTP and increases in afterdepolarization (AD, an increase in membrane potential that is dependent upon a preceding action potential for its initiation) both increased the RI (Fig. S5).

After exploring the effects of manipulating individual elements of the network, we used a combination of those alterations to reproduce our experimental findings (Fig. 4). Specifically, we sought to recapitulate the increase in p(T_in) observed with both the D1R and the D2R manipulations, opposite effects of the two manipulations on the RI, and the contrasting correlations between the p(T_in) increases and baseline σ. Based on modeling results summarized in Fig. 3B, one can notice that negative correlations between the p(T_in) increases and σ can be obtained only via alterations to activity in the deep layer pool, whereas any alterations to superficial layers resulted in positive correlation between the p(T_in) increases and σ. We reproduced the D1R effects with the network simulations by (i) increasing the efficacy of all inputs and both types of recurrent connections, (ii) decreasing the rates of LTD and LTP, and (iii) decreasing STP at all inputs and recurrent connections (EE and IE), but less strongly at EE connections (Fig. S6). The D2R effects were reproduced by (i) increasing the excitability of the output pool and (ii) increasing the AD within the output pool (Fig. S6). Both sets of network alterations resulted in increases in p(T_in) that were comparable to those observed experimentally (∼0.08) (Fig. 4A). In contrast, the two sets of alterations produced opposite changes to the RI. The first set of alterations produced D1R-like decreases in the RI (ΔRI = −0.04), whereas the second set produced D2R-like increases in the RI (ΔRI = 0.04) (Fig. 4B). The D1R-like decrease in the RI was achieved via imposing weaker decreases in STP at EE connections compared with other sites. The D2R-like increase in the RI was achieved via increases in the AD within the output pool. Finally, similar to the experimental results, the first set of alterations yielded an increase in p(T_in) that was positively correlated with σ during control trials, whereas the second set of alterations produced an increase in p(T_in) that was negatively correlated with σ (Fig. 4C).

Fig. 4.

Replication of experimental results with two sets of alterations to network elements. (A–C) Distributions of p(T_in) (A) and RIs (B) before and after the two sets of network alterations and correlations (C) between alteration-induced changes in p(T_in) and stochasticity of choice (σ). Red and blue symbols indicate D1R-like and D2R-like effects, respectively. Other conventions are as in Fig. 2.

Discussion

We observed dissociable influences of DA-mediated FEF activity on saccadic target selection. Increases in target selection were achieved either by blocking D1Rs or by stimulating D2Rs. However, the former manipulation decreased the tendency to repeat choices on subsequent trials, whereas the latter increased that tendency. Moreover, the amount of shift in choice resulting from blocking D1Rs was positively correlated with baseline stochasticity of choice, whereas the amount of shift due to stimulating D2Rs was negatively correlated with stochasticity. A simple prediction of these results is that decreases in target selection would result from stimulating D1Rs or blocking D2Rs, yet the dissociable effects on repetition and choice stochasticity would remain. Our simulations using a plausible cortical model of target selection reproduced the experimental effects and pointed to the biophysical mechanisms underlying DA’s influence on target selection. Specifically, our model predicts that D1Rs influence target selection mainly through their effects on the strength of inputs to the FEF and on recurrent connectivity within superficial layers, whereas D2Rs influence the excitability of FEF output neurons.

It is significant that our cortical network model was able to reproduce the experimental results via a unique (in terms of loci and relative strength) set of alterations to multiple network elements (Fig. S6). Nonetheless, it is also important to consider whether the alterations needed to achieve the modeling results are consistent with the known prefrontal distributions of D1Rs and D2Rs and their effects on neural activity (11). Notably, it is known that D1Rs modulate the efficacy of inputs (18, 19) and both excitatory and inhibitory recurrent connections (20, 21), modulate STP at inputs (18) and recurrent connections (22–24), and influence both LTD and LTP within the PFC (25–28). These properties of D1Rs are consistent with the network alterations necessary to reproduce the experimental results, namely increases in the efficacy of inputs and recurrent connections, decreases in STP, and decreases in the rates of LTD and LTP. The requirement of stronger modulation of STP in inhibitory interneurons is also compatible with the observation that GABA-ergic activity within the FEF contributes to target selection (17). Moreover, the decrease in rates of LTD and LTP is consistent with a recent finding that D1R-mediated activity within the lateral prefrontal cortex contributes to learning of novel visuomotor associations (29).

In addition, D2Rs are expressed largely within deep layers of cortex (30, 31), where in the FEF, layer V pyramidal neurons provide the primary output to the superior colliculus and brainstem oculomotor nuclei (32, 33). In contrast, D1Rs are expressed throughout cortical layers (30). Furthermore, prefrontal D2Rs are known to enhance the excitability of layer V pyramidal neurons (34) and it was recently reported that AD is also enhanced by D2R agonists in a subtype of layer V pyramidals (35), thereby prolonging activity of these neurons for hundreds of milliseconds. Consistent with these properties, an increase in the excitability and AD of the deep layer pool was precisely the alterations to the model network required to reproduce our experimental D2R effects. Thus, not only was the model able to reproduce the dissociable effects of the D1R and D2R manipulations on target selection, but also the alterations to the model required to reproduce those effects were consistent with the known properties of D1Rs and D2Rs and their distributions across cortical layers.

Role of Dopamine in Saccadic Target Selection.

Saccadic target selection is determined in part by the reward value of potential targets (1–3). Given the clear role of the FEF in target selection, it is surprising that few studies have explored the contribution of FEF dopamine to this behavior. It has been shown that reward-dependent modulation of saccadic target selection relies on dissociable effects of D1Rs and D2Rs on neural activity in the striatum, presumably through modulation of long-term synaptic plasticity in the caudate (7, 36). Both caudate and FEF neurons have been shown to exhibit modulation of activity by the reward value of saccadic targets (37). Thus, it is important to determine the relative contributions of these two areas to value-based target selection. Similar to the caudate studies, our study also demonstrates dissociable roles of D1R- and D2R-mediated FEF activity on target selection. However, it is possible that dopaminergic modulations within these two areas could contribute to reward control of target selection under different conditions: modulation within the FEF for when the reward values are changing or are unpredictable (e.g., timing of the second target in our experiment) and modulation within the caudate for when the rewards for different targets are fixed and predictable (e.g., small and big rewards) and a bias is desirable (7, 36).

The dissociable dopaminergic effects we observed suggest potential mechanisms that may underlie different aspects of adaptive choice behavior. First, we found opposing effects of the D1R and D2R manipulations on repetition (RI), namely decreases and increases in repetition for the two manipulations, respectively. The observed decrease in repetition following the infusion of a D1R antagonist is consistent with previous results showing increases in repetitive behavior (perseveration) following the infusion of a D1R agonist into prefrontal cortex (16). One might suggest that variation in the degree of repetition may be related to the exploration–exploitation trade-off observed in adaptive choice behavior. This trade-off reflects the balance between maximizing reward based on current knowledge and testing alternative actions to acquire new knowledge (38). A recent study found that DA levels in mice influence the degree of exploitation (39). Another study in humans found evidence of a correlation between exploration and prefrontal DA (40). In the context of this trade-off, the D1R effects we observed might be viewed as an increase in exploration, whereas the D2R effects might be viewed as an increase in exploitation.

Second, although both manipulations equally increased the probability of T_in selection, those increases depended on the baseline stochasticity in choice in opposite ways. For the D1R antagonist, larger increases in T_in selection were associated with greater stochasticity in choice with respect to the TOA. In contrast, for the D2R agonist, larger increases in T_in selection were associated with smaller stochasticity. This dissociation suggests a possible mechanism for adjusting behavior according to the sensitivity to relevant information. Given that greater stochasticity reflects lower sensitivity to visual information, specifically the TOA, DA could adjust choice bias according to that sensitivity. Differential adjustments of choice bias according to sensitivity could be beneficial when multiple cues exist, but only one carries reward information. When behavior relies on the relevant cue and cue sensitivity is high, the degree of bias adjustment should be small (i.e., inversely proportional to sensitivity). On the other hand, when behavior relies on an irrelevant cue, and sensitivity to that cue is high, then the degree of bias adjustment should be large (i.e., proportional to sensitivity). The latter case may be particularly important for escaping local maxima in reward maximization (38).

Previous modeling work has helped elucidate how DA contributes to working memory via D1R-mediated differential changes to NMDA and AMPA currents (41) and differential dopaminergic modulation of NMDA currents in excitatory and inhibitory neurons (42). Here, we were able to pinpoint possible neural mechanisms through which D1Rs and D2Rs differentially alter saccadic target selection by virtue of their effects in different cortical layers. Our model suggests how dopaminergic modulation of the afferents to the FEF could alter reward-dependent choice. The predictions of this model could be tested in experiments in which reward delivery is probabilistic and therefore the animal’s choice is determined by the integration of reward history (43, 44). One might predict, for example, that after blocking D1Rs within the FEF, the form and time constant of reward integration would be altered such that the impact of previous rewards on current choices could be increased or decreased. Such an outcome could further clarify the contribution of FEF dopamine to reward-dependent choice behavior.

Materials and Methods

Experimental Procedures.

Two monkeys (Macaca mulatta) were trained on a saccadic choice task. All experimental procedures were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals, the Society for Neuroscience Guidelines and Policies, and the Stanford University Animal Care and Use Committee. For detailed general and surgical procedures see SI Text, Experimental Procedures.

Dopaminergic Manipulation of FEF Activity.

We used a microinjectrode system for simultaneous microstimulation and microinfusion of drugs (45). The center of the RF of the FEF site under study was defined by the endpoint of the saccades evoked by its electrical microstimulation. We positioned one of the targets (T_in) in the RF of the FEF site and the other one in the opposite hemifield (T_out). Small volumes (0.5–1 µL) of the selective D1R antagonist SCH23390 or the D2R agonist quinpirole were delivered into the FEF with infusion rates less than 100 nL/min. Volumes of this size diffuse ∼1–3 mm within the cortex (46) and thus affect neurons within only a few columns of the FEF. Both drugs were obtained from Sigma-Aldrich. The acidic PH of the D1R antagonist solution was adjusted to 5.5–6.0 before the infusion, whereas the D2R agonist required no PH adjustment. Given the limitations to the number of possible repeated infusions in a single cortical region, due to the risk of damage (45), we chose the two (of four possible) dopaminergic manipulations most likely to yield interpretable results.

Data Analysis.

To quantify target selection, for each experiment we measured the psychometric function by computing the probability of selecting T_in as a function of the TOA, for trials before and after drug infusion. The psychometric function was then fitted by a sigmoid (logistic function), which yielded two parameters: a bias parameter (PES) that determined the TOA for which the two targets were selected with equal frequency and a measure of stochasticity in choice (σ, often referred to as the temperature) with respect to the TOA. For one of the D2R manipulation experiments in monkey B, the choice probability plateaued at about 0.2 and 0.8 for the minimum and maximum values of the TOA, respectively. To get a better fit for this experiment, we bounded our logistic function between 0.2 and 0.8; however, our results were unaffected by this choice of fitting.

Unless otherwise mentioned, we used the Wilcoxon signed-rank test for comparison between control and drug experiments and the Wilcoxon rank sum test for comparisons between the two drug conditions (for which the z and P values are reported). For correlation measures we reported the Pearson correlation coefficient and its significance value. Unless otherwise mentioned, data are expressed as a mean plus or minus the SEM.

Model Implementation.

To simulate the superficial layers we implemented a mean-field reduction of a detailed spiking network model (47). The details of this implementation have been described previously (47, 48). The deep layer pools were simulated using a firing-rate model with a realistic response function. In addition, we incorporated short-term (STP) and long-term synaptic plasticity in the inputs, STP in connections between pools in the superficial layers, and the effect of afterdepolarization on neural activity in the deep layer pools (SI Text, Computational Model).

Simulation of Dopaminergic Manipulations.

We assumed that reward harvest is signaled globally by the phasic activity of midbrain DA and this signal results in an elevation of prefrontal DA for a few hundred milliseconds, during which short-term and long-term plasticity are modulated by DA (see SI Text, Computational Model for more details). In addition, the drug manipulation could alter connections, synaptic plasticity, or neural excitability of the column that was infused with drug, in two different ways. First, the drug manipulation could change the synaptic efficacy of a given pathway or the neural excitability of a pool (static alterations). Second, the drug manipulation could alter STP of a given pathway, DA-dependent rates of LTD and LTP of value-based inputs, or the activity-dependent change in neural excitability between trials (dynamic alterations). These alterations could affect the network through the following pathways: background, visual, and value-based inputs to the superficial layers; the strength of connections between excitatory pools, between inhibitory and excitatory pools, and from superficial to deep layer pools; and the excitability of the deep layer pools (see SI Text, Computational Model and Effects of Drug-Induced Alterations on the Model’s Choice Behavior for more details).