Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2007 May 2;27(18):4810–4818. doi: 10.1523/JNEUROSCI.0577-07.2007

The Influence of Expected Value on Saccadic Preparation

David M Milstein 1, Michael C Dorris 1,✉
PMCID: PMC6672101  PMID: 17475788

Abstract

Basing higher-order decisions on expected value (reward probability × reward magnitude) maximizes an agent's accruement of reward over time. The goal of this study was to determine whether the advanced preparation of simple actions reflected the expected value of the potential outcomes. Human subjects were required to direct a saccadic eye movement to a visual target that was presented either to the left or right of a central fixation point on each trial. Expected value was manipulated by adjusting the probability of presenting each target and their associated magnitude of monetary reward across 15 blocks of trials. We found that saccadic reaction times (SRTs) were negatively correlated to the relative expected value of the targets. Occasionally, an irrelevant visual distractor was presented before the target to probe the spatial allocation of saccadic preparation. Distractor-directed errors (oculomotor captures) varied as a function of the relative expected value of, and the distance of distractors from, the potential valued targets. SRTs and oculomotor captures were better correlated to the relative expected value of actions than to reward probability, reward magnitude, or overall motivation. Together, our results suggest that the level and spatial distribution of competitive dynamic neural fields representing saccadic preparation reflect the relative expected value of the potential actions.

Keywords: motor planning, reaction time, reward, probability, motivation, dynamic field theory

Introduction

The generation of saccadic eye movements is influenced by bottom-up factors such as the contrast and luminance of visual stimuli as well as top-down factors such as goals, expectations, and context (Thompson and Bichot, 2005; Fecteau and Munoz, 2006). Of these, top-down factors are more difficult to define, quantify, and manipulate in the laboratory, and consequently, their influence is less understood (Sparks, 1999; Maunsell, 2004; Schall, 2004). Here, we manipulated a set of variables that were easily defined and quantified to better understand how top-down factors influence the advanced allocation of saccadic preparation.

Expected value (defined as the product of the probability of reward for an action and the magnitude of that reward) is a critical factor when making any decision, because choosing the option with the highest expected value maximizes the intake of reward over time. Behavioral and neurophysiological studies linking each of these individual components of expected value to changes in saccadic generation have recently been described. For example, experiments describing probability effects involved probability manipulations while reward magnitude remained constant (Basso and Wurtz, 1998; Dorris and Munoz, 1998; Platt and Glimcher, 1999). Similarly, experiments describing reward magnitude effects involved magnitude manipulations while probability remained constant (Leon and Shadlen, 1999; Platt and Glimcher, 1999; Lauwereyns et al., 2002; Takikawa et al., 2002; Ding and Hikosaka, 2006). Therefore, attributing changes in behavior and/or neural activity to manipulations of probability or reward magnitude was confounded by the fact that expected value was concomitantly manipulated in these studies. Here, we hypothesize that the advanced preparation of saccades is better accounted for by the expected value of potential actions than either probability or magnitude of reward alone.

The effect of expected value on saccadic preparation was measured with two behavioral indices. First, saccadic preparation has traditionally been measured by its influence on saccadic reaction time (SRT), defined as the time required to initiate a saccade toward a target after its presentation (Dorris et al., 1997; Munoz et al., 2000). Second, we used a novel probe of saccadic preparation in the form of an irrelevant visual distractor that was occasionally flashed at specific locations in the visual field before target presentation. We reasoned that the preexisting level of saccadic preparation would determine whether the addition of a distractor-related visual transient was sufficient to overcome the threshold level needed to trigger an erroneous saccade, also known as an oculomotor capture (Theeuwes et al., 1998, 1999). Thus, the distractor acts as a noninvasive probe of the otherwise covert process of saccadic preparation in a manner analogous to how deviations in saccadic endpoints resulting from microstimulation of the oculomotor circuitry have been used to study the preparation of saccades during motion discrimination tasks (Gold and Shadlen, 2000, 2003).

Overall, these behavioral indices suggest that relative expected value of potential actions accounts for the level and spatial allocation of saccadic preparation better than the probability or magnitude of reward associated with those actions or motivation resulting from differences in the accumulation of reward across blocks of trials. We discuss how our results can be accounted for by valuation preshaping competitive dynamic neural fields representing the spatial allocation of saccadic preparation.

Materials and Methods

Subjects performed a saccadic eye movement task for monetary reward. All experimental procedures were reviewed and approved by the Queen's University Human Research Ethics Board.

General methods.

Subjects sat in a chair with their head placed on a chin rest to centralize and stabilize their gaze with respect to a computer monitor (refresh rate of 100 Hz) situated 59 cm in front of the subject and spanning 32° of their central visual field. Subjects were fitted with an Eyelink II infrared eye tracker (SR Research, Osgoode, Ontario, Canada) headset that sampled the position of the left eye at 250 Hz and 0.1° spatial resolution. Behavioral paradigms, visual displays, and storage of eye-position data were under the control of a Pentium 4 personal computer running a real-time data acquisition software package (Gramalkn; Ryklin Software, New York, NY). All data analyses were performed off-line on a Pentium 4 personal computer running Matlab version 7.0.4 (MathWorks, Natick, MA).

Experimental subjects.

A total of 11 human subjects (five female, six male; 20–34 years of age; including the two authors) participated in these experiments. Ten subjects participated in experiment 1 (see below) (Fig. 1A–C), and six subjects participated in experiment 2 (see below) (Fig. 1A,C,D). Five subjects participated in both experiments. Subjects were paid based on performance, with payoffs ranging from $63.04 to $71.09 and from $78.01 to $91.41 for these two experimental conditions, respectively.

Figure 1.

Figure 1.

Open in a new tab

Schematic of behavioral paradigms. Each panel denotes a successive display on a computer monitor. Subjects received a reward on trials in which they made a target-directed saccade (red arrows), and reward was withheld for distractor-directed saccades (green arrows). Open circles are not actually presented on the computer monitor but denote possible locations of targets and distractors. expt. 2, Experiment 2.

Behavioral paradigms.

Subjects received monetary reward for successfully completing a simple oculomotor task that involved directing a saccade toward a single visual target that could be presented either to the left or right of a central fixation point on each trial (Fig. 1). The relative value of these potential responses was manipulated across blocks by varying the probability of left versus right target presentation as well as the magnitude of reward associated with each. The influence of valuation on the level of saccadic preparation was indexed, for the most part, by the time required to initiate target-directed saccades. The influence of valuation on the spatial allocation of saccadic preparation was probed occasionally by flashing a visual distractor that could trigger a saccadic error to its location. Finally, the manner in which saccadic preparation was temporally updated was examined during a minority of trials in which reward could be harvested by selecting either of the two targets that were presented. The details of these three randomly interleaved trial types are outlined below.

Control trials (Fig. 1A) made up the core of each block (60%) and used SRTs to index the level of saccadic preparation allocated to each target. To begin each trial, subjects were required to foveate a circular red fixation point [luminance, 5.2 cd; visual radius, 0.25°; red (R), 100%; green (G), 0%; blue (B), 0%] presented in the center of the computer monitor. After 600 ms of fixation, the fixation point was extinguished, followed 400 ms later by the presentation of a peripheral saccade target (luminance, 5.2 cd; visual radius, 0.25°; R, 100%; G, 0%; B, 0%) that appeared either 8° to the left or right of the fixation point. This 400 ms warning period between fixation point offset and target onset was critical because it facilitated the preparation of saccadic responses in advance of the target presentation (Saslow, 1967; Dorris et al., 1997). Subjects had 300 ms to initiate a saccade to the new target location and were required to fixate it for 600 ms. After this time, the target disappeared, and the monetary payoff was displayed for 1000 ms. At the onset of the experiment, subjects were informed that 4 cents was divided between the left and right targets on each trial. When subjects completed a trial successfully, the reward amount was displayed as a percentage of 4 cents (e.g., $70% corresponded to 2.8 cents). Finally, the intertrial interval was set at 1000 ms.

If at any time, the subject failed to meet the time constraints or keep his or her gaze within an invisible window surrounding the visual stimuli (±3°), a red “X” was displayed in the center of the screen, and the participant received no monetary reward for that trial. Anticipatory saccades initiated between 200 ms before to 70 ms after target appearance were followed by the presentation of a white “$0%,” and monetary reward was withheld.

Distractor trials were a minority within a block (30%) (Fig. 1B,D). The level of saccadic preparation at a specific time and location in the visual field was indexed by the proportion of oculomotor captures triggered by an abrupt-onset visual distractor. Distractor trials were identical to control trials, except that an irrelevant circular green distractor (luminance, 5.2 cd; visual radius, 0.25°; R, 0%; G, 100%; B: 0%) was flashed for 70 ms beginning halfway through the warning period. If subjects looked to the distractor (i.e., oculomotor capture), the trial was immediately aborted, followed by the display of a white 0% in the center of the screen for 1000 ms, indicating that monetary reward was withheld.

Dual-target trials (Fig. 1C) were the least frequent trial type (10%). These trials were identical to control trials, except that, instead of the presentation of one target, both left and right targets were presented simultaneously. Subjects were free to harvest the reward from either of the two targets by acquiring it with a saccade. Because reward was guaranteed for both targets, probability of reward was no longer a factor in determining the valuation of the targets. Therefore, to maximize intake of reward, the target with the highest reward magnitude should be selected during these trials.

Dual-target trials served two purposes. First, we were concerned that a monetary reward paid as a cumulative amount at the end of an experimental session was too abstract to be an effective incentive for influencing saccadic preparation. Saccades directed toward the high-reward-magnitude target during these trials would suggest that it was a salient variable. Second, dual-target trials could provide us with preliminary insight into how saccadic preparation is temporally updated when relevant top-down factors suddenly change.

Experiment 1: influence of expected value on saccadic preparation.

The main goal of this experiment was to quantify the effects of expected value on saccadic preparation using the behavioral indexes of SRTs and oculomotor captures. A secondary goal of this experiment was to examine how saccadic preparation was temporally updated as reward probability was removed as a factor during dual-target trials. The expected value of the saccadic targets was manipulated across 15 blocks of trials by varying the probability of the left/right target presentation over three levels and the magnitude of left/right monetary reward over five levels (Table 1). Distractors were presented at one of four locations (Fig. 1B): 8° left or right of the fixation point (i.e., at possible target locations) or 8° up or down relative to the fixation point (i.e., orthogonal to where targets were presented).

Table 1.

A list of top-down factors used during blocks of trials for experiments 1 and 2

Relative expected value Relative reward probability Relative reward magnitude
0.01 0.1 0.1
0.05 0.1 0.3
0.1 0.1 0.5
0.1 0.5 0.1
0.21 0.1 0.7
0.3 0.5 0.3
0.5 0.1 0.9
0.5 0.5 0.5
0.5 0.9 0.1
0.7 0.5 0.7
0.79 0.9 0.3
0.9 0.5 0.9
0.9 0.9 0.5
0.95 0.9 0.7
0.99 0.9 0.9
Open in a new tab

Experiment 1: all rows. Experiment 2: bold rows.

Each block consisted of 220 trials (60% control trials, 30% distractor trials, and 10% dual-target trials) and lasted ∼15 min in length. Subjects were informed of their cumulative payoffs at the end of each block of trials. Subjects were paid at the end of each experimental session the total payoff for all performed blocks. For each day of testing, subjects completed two to three blocks of trials, resulting in a 45–60 min (including setup and breaks) experimental period. Additional days of testing were required until all 15 blocks were completed. Block order was randomized across subjects.

Experiment 2: influence of expected value on the spatial allocation of saccadic preparation.

The goal of this experiment was to explore how expected value influences the spatial allocation of saccadic preparation. This condition was identical to that of experiment 1 with two exceptions. First, the distractor could appear at one of 50 locations (Fig. 1D) rather than only four. These distractors were arranged into two grids of 25 appearing 0°, ±3°, and ±6° horizontally and vertically around each target location. Second, to obtain sufficient data for each distractor location, it was not feasible to perform all 15 expected value conditions. Therefore, subjects performed three repetitions of five specific 240 trial blocks that spanned the relative expected value range (Table 1, bold rows). In addition, distractor trials accounted for 40% of trials during experiment 2 compared with 30% in experiment 1.

Subjects performed 720 trials for each expected value condition. This resulted in five to six presentations of the distractor at each distractor location for each of the five expected value conditions. Data were pooled for all six subjects, resulting in ∼33 presentations of each distractor location for each expected value condition (see Fig. 8A).

Figure 8.

Figure 8.

Open in a new tab

Spatial allocation of saccadic preparation based on relative expected value of options. A, Three-dimensional contour maps showing the proportion of oculomotor captures made at each distractor location. Each white dot represents a possible distractor location, and the red dots indicate possible target locations for reference. Note that distractors can be presented at the location of targets as well. B, Proportion of oculomotor captures directed toward distractors as a function of their distance from the target in the same hemifield. Each color represents one of the five different relative expected value conditions in experiment 2. Red, 0.99; green, 0.79; black, 0.50; yellow, 0.21; blue, 0.01.

Data analysis.

SRTs were defined as the time between target presentation and the initiation of the first correct saccade. Saccadic initiation was the time at which peak velocity reached 30°/s. A correct saccade was defined as the first saccade initiated between 70 and 370 ms after target onset that landed within 3° of the target. Oculomotor captures were defined as saccades that landed within ±5° of the distractor during the 200 ms period extending from 70 ms after distractor onset to 70 ms after target presentation. Spatial constraints were relaxed (i.e., from ±3° to ±5°) for distractor-directed errors, because these saccades tended to be hypometric relative to target-directed saccades (Theeuwes et al., 1998).

The relative expected value associated with a target was defined as the proportion of expected value of one target divided by the sum of expected values for both targets: Relative Expected Value = [p(T1) × r(T1)]/([p(T1) × r(T1)] + [p(T2) × r(T2)]), where p(T1) and p(T2) denote the percentage of times target 1 (T1) and target 2 (T2) appear, respectively, during a given block of trials, and r(T1) and r(T2) denote the percentage of 4 cents allocated to each of the two targets, respectively.

Relative reward probability was defined as the proportion of times a target appeared at one target location divided by the sum of reward probabilities for both targets regardless of reward magnitudes associated with those targets: Relative Reward Probability = p(T1)/[p(T1) + p(T2)]. Relative reward magnitude was defined as the proportion of 4 cents allocated to one target divided by the sum of the proportions of 4 cents allocated to both targets regardless of reward probability associated with those targets: Relative Reward Magnitude = r(T1)/[r(T1) + r(T2)]. Throughout, left and right data sets that had the same reward probabilities and magnitudes were collapsed together.

Motivation was defined as the mean amount of monetary reward earned per trial of a block (adapted from Roesch and Olson, 2004). Some blocks of trials resulted in larger total earnings for the subjects (e.g., 0.9 reward probability and 0.9 reward magnitude to the T1 compared with 0.9 reward probability and 0.1 reward magnitude to T1). A higher amount of money per trial could result in a higher motivation level and conceivably lead to an increase in overall saccadic preparation (Stellar and Stellar, 1985). Unlike the other top-down factors described above, motivation was associated with a block of trials and not spatially associated with each target. Because motivation should influence all saccadic behaviors globally, all locations for each saccadic behavior (i.e., T1 and T2 for SRT; up, down, left, and right distractor locations for oculomotor captures) were collapsed together. Motivation was calculated by obtaining the total amount of reward accrued in a given block of trials and then dividing that by the number of correct trials performed in that block: Motivation = [r1(T1) + r1(T2)]/Tc, where rt(T1) and rt(T2) denote the total monetary reward accrued during one block of trials for correct saccades made toward T1 and T2, respectively, and Tc denotes the total number of correct trials for one block of trials.

Statistical analysis.

The relationship between top-down factors (expected value, probability, magnitude, and motivation) and the various behavioral measures (SRT, oculomotor captures, and dual-target selection) were measured with the Pearson correlation coefficient. Kruskal–Wallis ANOVA on ranks for non-normal samples tests followed by an all-pairwise multiple-comparison procedure (Student–Neuman–Keuls) determined whether cumulative correlation distributions varied significantly between top-down factors (see Fig. 5). All statistics were performed with the statistical software package SigmaStat version 3.1 (Systat Software, San Jose, CA) with significance levels set at p < 0.05 unless otherwise stated.

Figure 5.

Figure 5.

Open in a new tab

Cumulative distributions of correlation coefficients for top-down factors and behavioral measures. Each data point represents the correlation coefficient from one subject calculated in the same manner as in Figures 3 and 4. Blue, Relative reward probability; green, relative reward magnitude; red, relative expected value; black, motivation; filled circles, significant correlation (p < 0.05).

Results

Experiment 1: influence of expected value on saccadic preparation

The influence of expected value on the level of saccadic preparation was first indexed by measuring SRTs associated with each target during control trials. Figure 2A shows a representative block of trials in which 0.79 of relative expected value was associated with the right target and 0.21 with the left target. Saccades were initiated more quickly after the presentation of the high-valued right target than the low-valued left target, as evidenced by the earlier upward deflections of the eye traces (Fig. 2A). For all subjects, across all 15 blocks of trials, there was a negative correlation between SRT and relative expected value of the targets (Fig. 3A) (R2 = 0.87; p < 0.05).

Figure 2.

Figure 2.

Open in a new tab

Eye traces for an individual subject during one block of trials. The left (downward deflections) target had a relative expected value of 0.21 (reward magnitude, 0.7; probability, 0.1), and the right (upward deflections) target had a relative expected value of 0.79 (reward magnitude, 0.3; probability, 0.9). The times of target and distractor presentation are denoted by red and green vertical lines, respectively. Target-directed correct saccades and distractor-directed oculomotor captures are denoted by red and green eye traces, respectively. The shaded red and green windows represent the time during which saccade initiation must have occurred to be classified as a correct saccade or an oculomotor capture, respectively.

Figure 3.

Figure 3.

Open in a new tab

Influence of top-down factors on SRT. A–D, Mean SRT data for 10 subjects plotted as a function of relative expected value (A), relative reward probability (B), relative reward magnitude (C), and motivation across the 15 blocks of trials (D). Each data point represents a combination of a certain reward probability (square, 0.1; circle, 0.5; triangle, 0.9) and reward magnitude (blue, 0.1; black, 0.3; red, 0.5; green, 0.7; cyan, 0.9). The black lines represent the least-squares linear regression. The asterisks denote a statistically significant correlation (p < 0.05).

The level of saccadic preparation was also probed by the abrupt presentation of distractors (Fig. 1B). The proportion of oculomotor captures assessed saccadic preparation at nontarget locations (i.e., up/down) in advance of target presentation (i.e., halfway through the warning period). If saccadic preparation is allocated based on the relative expected value of potential actions, the proportion of oculomotor captures to left and right distractors should vary with the relative expected value of these targets, whereas oculomotor captures should rarely be directed toward the distractors presented at the valueless up and down locations.

During the same representative block of trials, distractors presented at the location of the higher-valued target induced more oculomotor captures than distractors presented at the location of the lower-valued target (Fig. 2B). For all subjects, across all 15 blocks of trials, there was a significant positive correlation between relative expected value of the collapsed left and right targets and the proportion of oculomotor captures directed to distractors at those locations (Fig. 4A, filled symbols) (R2 = 0.82; p < 0.05). Subjects rarely, if ever, looked toward the up and down distractors (Fig. 4A, open symbols).

Figure 4.

Figure 4.

Open in a new tab

Influence of top-down factors on oculomotor captures. A–D, Mean proportion of oculomotor captures for 10 subjects plotted as a function of relative expected value (A), relative reward probability (B), relative reward magnitude (C), and motivation across the 15 blocks of trials (D). The format used is the same as in Figure 3. A–C, Oculomotor captures are segregated into those directed to either the left/right (filled symbols) or up/down (open symbols) distractors. The least-squares regressions (black lines) are fit to the left/right distractor data and not the up/down data. D, Oculomotor captures to all four distractor locations are collapsed together.

We performed a control experiment with four subjects in which the saccadic targets were presented vertically instead of horizontally to rule out the possibility that there was a tendency for oculomotor captures to be preferentially directed toward distractors presented along the horizontal plane. Only a subset of blocks was used for this control experiment (Table 1, bold rows). We found that not only were more oculomotor captures directed toward the up and down distractors than those left and right, but the proportion of oculomotor captures varied with the relative expected value of the up and down targets (data not shown) (Student–Newman–Keuls method; p < 0.05).

Influence of other top-down factors on saccadic preparation

Other top-down factors in addition to relative expected value could conceivably affect saccadic behaviors during the present paradigm. If relative expected value is correlated to saccadic preparation, then the subcomponents of expected value (relative reward probability and relative reward magnitude) should also be correlated to saccadic preparation, albeit to a lesser degree. Moreover, changing levels of motivation resulting from differences in overall payoffs across blocks of trials may play a role in saccadic preparation (Roesch and Olson, 2003, 2004; Ravel and Richmond, 2006). Therefore, we examined the extent to which these three other top-down factors affected behavioral measures of saccadic preparation.

For all subjects, across all 15 blocks of trials, SRT was negatively correlated with each of relative reward probability (Fig. 3B) (R2 = 0.51; p < 0.05), relative reward magnitude (Fig. 3C) (R2 = 0.40; p < 0.05), and motivation (Fig. 3D) (R2 = 0.71; p < 0.05). Oculomotor captures to the horizontal distractors were positively correlated to relative reward probability (Fig. 4B, filled symbols) (R2 = 0.65; p < 0.05). However, the correlations between oculomotor captures and each of relative reward magnitude (Fig. 4C) (R2 = 0.19; p > 0.05) and motivation failed to reach significance (Fig. 4D) (R2 = 0.24; p > 0.05). No top-down factor was correlated to the proportion of distractors presented at locations orthogonal to the targets (Fig. 4B,C, open symbols) (p > 0.05).

To quantify whether relative expected value was better correlated to saccadic preparation than these other top-down factors, we plotted the correlation coefficients generated from each subject during these analyses (Table 2) in cumulative form (Fig. 5). The number of individual subjects that displayed significant correlations (Fig. 5, filled symbols; Table 2, bold type) was greater for relative expected value than all other top-down factors. In fact, every subject, except subject 6, had a statistically significant correlation between relative expected value and SRT. Overall, relative expected value was better correlated to SRT than reward probability, reward magnitude, and motivation (Fig. 5A) (Student–Newman–Keuls method; p < 0.05).

Table 2.

Correlation coefficients between top-down factors and behavior for individual subjects in experiment 1

Subject Relative reward probability Relative reward magnitude Relative expected value Motivation
Control trials: SRT
    1 −0.71 −0.46 −0.81 −0.58
    2 −0.50 −0.79 −0.85 −0.50
    3 −0.77 −0.52 −0.86 −0.43
    4 −0.78 −0.33 −0.81 −0.62
    5 −0.49 −0.65 −0.76 −0.64
    6 −0.01 −0.49 −0.28 −0.04
    7 −0.67 −0.58 −0.90 −0.52
    8 −0.68 −0.33 −0.74 −0.45
    9 −0.82 −0.49 −0.95 −0.62
    10 −0.43 −0.54 −0.62 0.36
Distractor trials: proportion of oculomotor captures
    1 0.76 0.31 0.75 0.22
    2 0.53 0.53 0.75 0.05
    3 0.78 0.35 0.79 −0.17
    4 0.70 0.21 0.73 0.28
    5 0.49 0.31 0.61 0.25
    6 0.30 0.11 0.28 0.20
    7 0.66 0.65 0.90 0.40
    8 0.72 0.13 0.65 0.21
    9 0.81 0.50 0.91 0.71
    10 0.59 0.36 0.68 −0.18
Dual-target trials: percentage of selections
    1 0.23 0.90 0.73
    2 0.18 0.92 0.71
    3 0.19 0.90 0.68
    4 0.74 0.59 0.97
    5 0.11 0.94 0.65
    6 0.43 0.78 0.78
    7 0.44 0.81 0.89
    8 0.48 0.78 0.79
    9 0.48 0.81 0.91
    10 0.31 0.80 0.72
Open in a new tab

Bold type indicates significance at p < 0.05 (Pearson correlation).

A similar pattern was observed for oculomotor captures (Fig. 5, filled symbols; Table 2, bold type). Again, every subject showed a statistically significant correlation between relative expected value and oculomotor captures except for subject 6. Overall, relative expected value was significantly better correlated to oculomotor captures than magnitude of reward and motivation (Fig. 5B) (p < 0.05), and the difference seen between relative expected value and relative reward probability approached significance (Fig. 5B) (p = 0.054).

Validity of oculomotor captures as an index of saccadic preparation

To test the validity of oculomotor captures as a probe of saccadic preparation, it was compared with an established measure of saccadic preparation, SRT, on a subject-by-subject basis (Fig. 6). The correlation that a subject had between a top-down factor and SRT was also predictive of the correlation that a subject had for that top-down factor and oculomotor captures. This relationship was statistically significant for relative expected value (Fig. 6A) (R2 = 0.90; p < 0.05), reward probability (Fig. 6B) (R2 = 0.90; p < 0.05), and reward magnitude (Fig. 6C) (R2 = 0.39; p < 0.05), but not motivation (Fig. 6D) (R2 = 0.33; p > 0.05). That these two behavioral measures were correlated for each subject across manipulations of the three spatially localized top-down factors bolsters the use of oculomotor captures for probing saccadic preparation at the time and location of distractor presentation.

Figure 6.

Figure 6.

Open in a new tab

Comparison of oculomotor captures and SRT correlations on a subject-by-subject basis. Each data point represents the correlation coefficient from one subject (Table 2) for oculomotor captures compared with the correlation coefficient for SRT plotted for each top-down factor. A, Relative expected value. B, Relative reward probability. C, Relative reward magnitude. D, Motivation.

Temporal updating of saccadic preparation: dual-target trials

Whereas only one saccadic target was presented during distractor and control trials, two targets were simultaneously presented during dual-target trials. Probability was suddenly removed as a top-down factor, because selecting either target was guaranteed to yield its associated reward. Therefore, selection of the target with the highest reward magnitude maximized reward during dual-target trials. Indeed, target selection was significantly correlated to relative reward magnitude (Fig. 7C) (R2 = 0.77; p < 0.05) but not to relative reward probability (Fig. 7B) (R2 = 0.13; p > 0.05). Target selection was still significantly correlated with the relative expected value of the targets (Fig. 7A) (R2 = 0.67; p < 0.05).

Figure 7.

Figure 7.

Open in a new tab

Influence of top-down factors on saccadic selections during dual-target trials. A–C, Mean proportion of saccadic selections for 10 subjects plotted as a function of relative expected value (A), relative reward probability (B), and relative reward magnitude (C). D, Comparison of SRTs for dual-target trials (open symbols) and control trials (filled symbols). The format used is the same as in Figures 3 and 4.

Responses were on average 37 ms slower during dual-target trials (Fig. 7D, open symbols) than during control trials (Fig. 7D, filled symbols) (paired t test; p < 0.05). There was no significant difference in the correlations between target selection and relative reward magnitude or relative expected value (Fig. 5C) (p > 0.05), suggesting that 37 ms was insufficient time to completely alter ongoing saccadic preparation processes.

Experiment 2: the influence of relative expected value on the spatial allocation of saccadic preparation

The aim of experiment 2 was to examine how expected value influenced the spatial allocation of saccadic preparation. Results from distractor trials in experiment 1 provided preliminary evidence for such a spatial allocation; oculomotor captures were distributed in proportion to the relative expected value of the left and right targets and almost never directed toward distractors presented at the valueless up and down locations. To achieve higher resolution of spatial encoding, distractors were presented at 50 possible locations (Fig. 1D) in experiment 2. Thus, the pattern of oculomotor captures provided an index of how saccadic preparation was spatially allocated just before the presentation of valued targets.

In total, 18,438 trials were collected from six subjects, 8209 of these being distractor trials. A distractor appeared at each of the 50 locations (Fig. 1D) on average 33 times for each of the five expected value conditions. The spatial distribution of saccadic preparation can be observed qualitatively in the contour plots of the proportion of oculomotor captures directed to distractor locations (Fig. 8A). As the expected value of each target varied, so did the proportion of oculomotor captures surrounding those targets.

The relationship between oculomotor captures and the distance between distractors and valued targets was further quantified in Figure 8B. The proportion of oculomotor captures was calculated as a function of the absolute distance of the distractors from the targets (Fig. 8B). For example, the absolute distance of the distractor presented at the target location was 0°, and the absolute distance of the distractors presented at the nearest locations above, below, left, and right of the target was 3°. Distractors of the same absolute distance were averaged together. Only distractors presented in the same hemifield were included in this analysis. The proportion of oculomotor captures decreased in a linear manner as the distance between distractors and targets increased across all five relative expected value conditions (p < 0.05 in all cases). Together, the results from experiment 2 suggest that saccadic preparation is allocated based on the absolute expected value of the two targets and decreases with distance from those valued targets.

Discussion

Our findings suggest that the relative expected value of potential outcomes is a strong predictor of saccadic preparation. We quantified saccadic preparation using two behavioral measures, SRTs and oculomotor captures. In many instances, reward probability, reward magnitude and motivation were correlated to these behavioral measures (Figs. 3, 4). Overall, however, saccadic behaviors were better correlated to relative expected value than these other top-down factors (Table 2, Fig. 5A,B). Reward probability was abruptly removed as an informative top-down factor during the occasional dual-target trial (Fig. 7B), and target selection was correlated only to reward magnitude and expected value (Fig. 7A,C). The second experiment examined how saccadic preparation was spatially allocated during manipulations of expected value. We found that the level of saccadic preparation decreased linearly with distance from valued targets (Fig. 8).

Oculomotor captures as a temporal and spatial probe of saccadic preparation

SRTs are commonly used as an index of saccadic preparation (Carpenter and Williams, 1995; Munoz et al., 2000). Here, oculomotor captures provided supplementary information about the level of saccadic preparation at specific times and spatial locations not provided by target-directed SRTs. Although abrupt-onset visual distractors are known to trigger saccades (Theeuwes et al., 1998, 1999), influence SRTs (Walker et al., 1997), and cause deviations in saccade trajectories (McSorley et al., 2004), they have not been used as a directed probe of underlying saccadic preparation. Abrupt-onset distractors served a function here similar to the function that abrupt-onset electrical microstimulation of the visuosaccadic circuitry served for the examination of saccadic preparation during a perceptual discrimination task (Gold and Shadlen, 2000). These authors argued that deviations in the endpoints of stimulation-induced saccades reflected motor bias for producing the choice response associated with the evolving perceptual decision. Importantly, these deviations in saccadic endpoints occurred only when there was direct and predictable mapping between stimulus and response (Gold and Shadlen, 2003), suggesting that oculomotor captures more accurately reflect advanced saccadic preparation rather than cognitive process related to the decision-making process itself.

Oculomotor captures were not directed to all distractor locations equally, but appeared to reflect underlying biases in saccadic preparation associated with the relative expected value of targets. The probability that a distractor would trigger an oculomotor capture was correlated to the relative expected value of the targets presented at the same location (Fig. 3, filled symbols) and uncorrelated to distractors presented at the valueless up/down locations (Fig. 3, open symbols). A control experiment ruled out the possibility that a tendency to make oculomotor captures preferentially toward horizontal rather than vertical stimuli influenced our results. When the valued targets were placed at the up/down locations, oculomotor captures were directed to these locations and rarely to horizontal distractors. Moreover, the influence of top-down factors on oculomotor captures was correlated to the influence of those top-down factors on an established measure of saccadic preparation, namely SRTs, on a subject-by-subject basis (Fig. 6). This suggests that the neural mechanisms responsible for variability in SRTs and triggering oculomotor captures are linked. Together, these results validated oculomotor captures as a behavioral probe of the otherwise covert process of saccadic preparation at both the time and location of distractor presentation.

The effects of relative expected value on saccadic preparation

An important finding is that saccadic preparation is better correlated to the relative expected value of actions than the components of relative expected value, reward probability and reward magnitude (Fig. 5). This finding suggests that previous work claiming behavioral and neural correlates of reward probability (Carpenter and Williams, 1995; Basso and Wurtz, 1998; Dorris and Munoz, 1998; Platt and Glimcher, 1999) and reward magnitude (Leon and Shadlen, 1999; Platt and Glimcher, 1999; Lauwereyns et al., 2002; Takikawa et al., 2002; Ding and Hikosaka, 2006) on motor behaviors may be more accurately interpreted as correlates of expected value.

Saccadic preparation was also better correlated to relative expected value than motivation, defined as the overall rate of reward intake within a block of trials. In other experiments, in which the magnitude of rewards or punishments were manipulated under conditions in which the expected value of the available options remained constant, motivational effects have been clearly observed (Roesch and Olson, 2004; Ravel and Richmond, 2006). Therefore, additional experiments are required to discern how expected value and motivation are combined across different contexts.

Although we have interpreted our results as relative expected value influencing saccadic preparation processes, another possibility is that our results were mediated by the influence of expected value on attentional resources. Because the target stimulus and saccadic response were both confined to the same region of visual space, our task was not designed to tease apart saccadic preparation and visuospatial attention mechanisms (Sparks, 1999; Maunsell, 2004). To do so would require tasks in which the motor response and attentional resources were dissociated from each other (Juan et al., 2004). However, if expected value only influenced attentional mechanisms, then to achieve the observed saccadic effects (i.e., SRTs and oculomotor captures) would require a strong link between attentional resources and saccadic preparation (Rizzolatti et al., 1987).

Dynamic field theory

Dynamic field theory provides a useful framework for interpreting the results of this study [for review of dynamic field theory, see Wilimzig et al. (2006); for comparable models, see Usher and McClelland (2001) and Ratcliff (2006)]. Briefly, a saccade is generated when activation across a topographically organized map of potential saccadic vectors either surpasses a threshold level or overcomes fixation-related activation. Spatially adjacent regions excite each other and distant regions inhibit each other (Koch and Ullman, 1985; Munoz and Istvan, 1998), resulting in competitive integration of signals across this map.

The results from experiment 1 illustrate how the level of dynamic neural fields may be influenced by the relative expected value of potential saccades. First, removal of the fixation point reduces fixation-related activation, thus disinhibiting the saccade-related regions of the map during the warning period (Dorris et al., 1997). The relative expected value of the potential saccades then shapes this baseline level in a more spatially localized manner, as evidenced by its influence on target-directed SRTs (Fig. 3) and distractor-directed oculomotor captures (Fig. 4). Whether an oculomotor capture is triggered presumably depends on whether the distractor-related visual transient combines with the existing baseline activation to overcome the threshold or fixation-related activation. This interpretation is consistent with the lack of oculomotor captures directed toward distractors presented at valueless locations orthogonal to the targets. Presenting the distractor at any one of 50 different locations in experiment 2 (Fig. 1D) revealed the influence of relative expected value on the spatial distribution of dynamic neural fields in more detail. We interpret the contour plots of oculomotor captures (Fig. 8A) as reflecting snapshots of how dynamic fields of saccadic preparation are preshaped by relative expected value at the time of distractor presentation.

Dynamic field theory also provides a possible mechanism for the dual-target results. When two targets appear at distant locations in visual space, the activity associated with these two regions compete through inhibitory interactions, so that this sensory activation takes longer to reach threshold, thus slowing SRTs (Walker et al., 1997; Munoz and Istvan, 1998), which, in our case, averaged 37 ms. This extra time may have allowed the contribution of reward probability toward saccade preparation to be attenuated and the contribution of reward magnitude to be enhanced. However, 37 ms seems a very short time to allow such a strategic transition of top-down signals to take place.

An alternative dynamic neural field mechanism that could account for our dual-target results involves probability and magnitude of reward being represented as separate signals within the field. For example, some neurophysiological evidence suggests that probability may exert its effects most heavily on pretarget activity (Dorris and Munoz, 1998), whereas reward magnitude may exert its effects by adjusting the gain of incoming visual responses across the dynamic field (Ikeda and Hikosaka, 2003). Therefore, it is plausible that probability would have its greatest effect when there was little competition during single-target trials. Under these conditions, high pretarget activity could summate with the visual transient to surpass the threshold level required to trigger short SRTs and oculomotor captures. During dual-target trials, competition would prevent either of the visual signals from immediately surpassing threshold. Therefore, the pretarget shaping of dynamic neural fields by probability would have little effect. Modification of visual inputs by reward could then exert a more pronounced and long-lasting effect under these conditions of delayed SRTs.

Conclusion

We conclude that expected value is not only an important factor for deliberative decision making but also for the advanced preparation of simple motor actions, such as saccadic eye movements. Our results suggest that dynamic neural fields representing potential saccadic vectors are shaped, in large part, by the potential value of the available options. Future physiological recordings are required to determine how different top-down factors such as the probability and magnitude of reward and motivation are combined at the neuronal level to achieve the observed saccadic behaviors.

Footnotes

This work was supported by the Canadian Institutes of Health Research. D.M.M. was supported by a Queen's University graduate fellowship. M.C.D. was supported by the Canadian Research Chairs program. We thank J. Green, F. Paquin, S. Hickman, R. Pengelley, and J. Turner for technical assistance and E. Ryklin for the customization of the data acquisition program.

References

  1. Basso MA, Wurtz RH. Modulation of neuronal activity in superior colliculus by changes in target probability. J Neurosci. 1998;15:7519–7534. doi: 10.1523/JNEUROSCI.18-18-07519.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Carpenter RH, Williams ML. Neural computation of log likelihood in control of saccadic eye movements. Nature. 1995;377:59–62. doi: 10.1038/377059a0. [DOI] [PubMed] [Google Scholar]
  3. Ding L, Hikosaka O. Temporal development of asymmetric reward-induced bias in macaques. J Neurophysiol. 2006;97:57–61. doi: 10.1152/jn.00902.2006. [DOI] [PubMed] [Google Scholar]
  4. Dorris MC, Munoz DP. Saccadic probability influences motor preparation signals and time to saccadic initiation. J Neurosci. 1998;18:7015–7026. doi: 10.1523/JNEUROSCI.18-17-07015.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Dorris MC, Pare M, Munoz DP. Neuronal activity in monkey superior colliculus related to the initiation of saccadic eye movements. J Neurosci. 1997;17:8566–8579. doi: 10.1523/JNEUROSCI.17-21-08566.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Fecteau JH, Munoz DP. Salience, relevance, and firing: a priority map for target selection. Trends Cogn Sci. 2006;10:382–390. doi: 10.1016/j.tics.2006.06.011. [DOI] [PubMed] [Google Scholar]
  7. Gold JI, Shadlen MN. Representation of a perceptual decision in developing oculomotor commands. Nature. 2000;404:390–394. doi: 10.1038/35006062. [DOI] [PubMed] [Google Scholar]
  8. Gold JI, Shadlen MN. The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commands. J Neurosci. 2003;23:632–651. doi: 10.1523/JNEUROSCI.23-02-00632.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Ikeda T, Hikosaka O. Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron. 2003;39:693–700. doi: 10.1016/s0896-6273(03)00464-1. [DOI] [PubMed] [Google Scholar]
  10. Juan CH, Shorter-Jacobi SM, Schall JD. Dissociation of spatial attention and saccade preparation. Proc Natl Acad Sci USA. 2004;101:15541–15544. doi: 10.1073/pnas.0403507101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Koch C, Ullman S. Shifts in selective visual attention: towards the underlying neural circuitry. Hum Neurobiol. 1985;4:219–227. [PubMed] [Google Scholar]
  12. Lauwereyns J, Watanabe K, Coe B, Hikosaka O. A neural correlate of response bias in monkey caudate nucleus. Nature. 2002;418:413–417. doi: 10.1038/nature00892. [DOI] [PubMed] [Google Scholar]
  13. Leon MI, Shadlen MN. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron. 1999;24:290–291. doi: 10.1016/s0896-6273(00)80854-5. [DOI] [PubMed] [Google Scholar]
  14. Maunsell JH. Neuronal representations of cognitive state: reward or attention? Trends Cogn Sci. 2004;8:261–265. doi: 10.1016/j.tics.2004.04.003. [DOI] [PubMed] [Google Scholar]
  15. McSorley E, Haggard P, Walker R. Distractor modulation of saccade trajectories: spatial separation and symmetry effects. Exp Brain Res. 2004;155:320–333. doi: 10.1007/s00221-003-1729-5. [DOI] [PubMed] [Google Scholar]
  16. Munoz DP, Istvan PJ. Lateral inhibitory interactions in the intermediate layers of the monkey superior colliculus. J Neurophysiol. 1998;79:1193–1209. doi: 10.1152/jn.1998.79.3.1193. [DOI] [PubMed] [Google Scholar]
  17. Munoz DP, Dorris MC, Pare M, Everling S. On your mark, get set: brainstem circuitry underlying saccadic initiation. Can J Physiol Pharmacol. 2000;78:34–44. [PubMed] [Google Scholar]
  18. Platt ML, Glimcher PW. Neural correlates of decision variables in parietal cortex. Nature. 1999;400:233–238. doi: 10.1038/22268. [DOI] [PubMed] [Google Scholar]
  19. Ratcliff R. Modeling response signal and response time data. Cognit Psychol. 2006;53:195–237. doi: 10.1016/j.cogpsych.2005.10.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Ravel S, Richmond BJ. Dopamine neuronal responses in monkeys performing visually cued reward schedules. Eur J Neurosci. 2006;22:277–290. doi: 10.1111/j.1460-9568.2006.04905.x. [DOI] [PubMed] [Google Scholar]
  21. Rizzolatti G, Riggio L, Dascola I, Umilta C. Reorienting attention across the horizontal and vertical meridians: evidence in favor of a premotor theory of attention. Neuropsychologia. 1987;25:31–40. doi: 10.1016/0028-3932(87)90041-8. [DOI] [PubMed] [Google Scholar]
  22. Roesch MR, Olson CR. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J Neurophysiol. 2003;90:1766–1789. doi: 10.1152/jn.00019.2003. [DOI] [PubMed] [Google Scholar]
  23. Roesch MR, Olson CR. Neuronal activity related to reward value and motivation in primate frontal cortex. Science. 2004;304:307–310. doi: 10.1126/science.1093223. [DOI] [PubMed] [Google Scholar]
  24. Saslow MG. Effects of components of displacement step stimuli upon latency for saccadic eye movements. J Opt Soc Am. 1967;57:1024–1029. doi: 10.1364/josa.57.001024. [DOI] [PubMed] [Google Scholar]
  25. Schall JD. On building a bridge between brain and behavior. Annu Rev Psychol. 2004;55:23–50. doi: 10.1146/annurev.psych.55.090902.141907. [DOI] [PubMed] [Google Scholar]
  26. Sparks DL. Conceptual issues related to the role of the superior colliculus in the control of gaze. Curr Opin Neurobiol. 1999;9:698–707. doi: 10.1016/s0959-4388(99)00039-2. [DOI] [PubMed] [Google Scholar]
  27. Stellar JR, Stellar E. The neurobiology of motivation and reward. New York: Springer; 1985. [Google Scholar]
  28. Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O. Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res. 2002;142:284–291. doi: 10.1007/s00221-001-0928-1. [DOI] [PubMed] [Google Scholar]
  29. Theeuwes J, Kramer AF, Hahn S, Irwin DE. Our eyes do not always go where we want them to go: capture of the eyes by new objects. Psychol Sci. 1998;9:379–385. [Google Scholar]
  30. Theeuwes J, Kramer AF, Hahn S, Irwin DE, Zelinsky GJ. Influence of attentional capture on oculomotor control. J Exp Psychol Hum Percept Perform. 1999;25:1595–1608. doi: 10.1037//0096-1523.25.6.1595. [DOI] [PubMed] [Google Scholar]
  31. Thompson KG, Bichot NP. A visual salience map in the primate frontal eye field. Prog Brain Res. 2005;147:251–262. doi: 10.1016/S0079-6123(04)47019-8. [DOI] [PubMed] [Google Scholar]
  32. Usher M, McClelland JL. The time course of perceptual choice: the leaky, competing accumulator model. Psychol Rev. 2001;108:550–592. doi: 10.1037/0033-295x.108.3.550. [DOI] [PubMed] [Google Scholar]
  33. Walker R, Deubel H, Schneider WX, Findlay JM. Effects of remote distractors on saccade programming: evidence for an extended fixation zone. J Neurophysiol. 1997;78:1108–1119. doi: 10.1152/jn.1997.78.2.1108. [DOI] [PubMed] [Google Scholar]
  34. Wilimzig C, Schneider S, Schoner G. The time course of saccadic decision making: dynamic field theory. Neural Netw. 2006;19:1059–1074. doi: 10.1016/j.neunet.2006.03.003. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES