Significance
Studies using limb movements to probe cognitive states have suggested an interactive model of cognition and motor action. Despite ample evidence showing that the neural activity in oculomotor brain circuits is correlated with the formation of decisions about where to move the eyes, it is not known whether there are interactions between such decision-related activity and decision-irrelevant oculomotor processing. By probing the oculomotor system with a decision-irrelevant saccadic eye movement during decision formation, we found that saccade metrics are affected by the strength and direction of motion evidence. Our results suggest that decision signals continuously flow into the oculomotor system during decision making, and further extend the interactive model of cognition and action into the oculomotor system.
Keywords: visual motion, eye movement, decision making
Abstract
It is well established that ongoing cognitive functions affect the trajectories of limb movements mediated by corticospinal circuits, suggesting an interaction between cognition and motor action. Although there are also many demonstrations that decision formation is reflected in the ongoing neural activity in oculomotor brain circuits, it is not known whether the decision-related activity in those oculomotor structures interacts with eye movements that are decision irrelevant. Here we tested for an interaction between decisions and instructed saccades unrelated to the perceptual decision. Observers performed a direction-discrimination decision-making task, but made decision-irrelevant saccades before registering their motion decision with a button press. Probing the oculomotor circuits with these decision-irrelevant saccades during decision making revealed that saccade reaction times and peak velocities were influenced in proportion to motion strength, and depended on the directional congruence between decisions about visual motion and decision-irrelevant saccades. These interactions disappeared when observers passively viewed the motion stimulus but still made the same instructed saccades, and when manual reaction times were measured instead of saccade reaction times, confirming that these interactions result from decision formation as opposed to visual stimulation, and are specific to the oculomotor system. Our results demonstrate that oculomotor function can be affected by decision formation, even when decisions are communicated without eye movements, and that this interaction has a directionally specific component. These results not only imply a continuous and interactive mixture of motor and decision signals in oculomotor structures, but also suggest nonmotor recruitment of oculomotor machinery in decision making.
Perception, cognition, and action are often modeled as discrete, serial stages of processing (1, 2). In the context of a simple perceptual decision-making task, it is conventional to assume that motor output is generated only after a decision regarding the sensory information is complete. However, a growing body of literature suggests a more interactive model, in which actions are continuously affected by ongoing cognitive processing (3, 4). In a variety of reaching tasks, limb trajectories have been shown to be modulated by attention (5), language processing (6), and numeric representation (7). And during a well-studied motion-direction discrimination task (8), the reflex gain in limb muscles during decision making was modulated by the accumulated evidence of motion direction (9).
It is not known, however, whether cognitive signals similarly affect oculomotor functions. There are many reasons why such interactions might not be present or may not have been detected. In principle, limb movements are relatively slow and more continuous, and so online correction may be more appropriate than for fast and ballistic saccadic eye movements. In practice, it might have been difficult to assess how decision formation affects oculomotor output, because in previous perceptual decision-making studies, the oculomotor output is typically consistent with the sensory stimuli and the corresponding decisions (e.g., making a rightward saccade after viewing rightward motion).
Given the ample electrophysiological evidence in nonhuman primates showing decision-making–related neural responses in oculomotor brain areas, such as the lateral intraparietal area (10–13), the superior colliculus (14), and the frontal eye fields (15–17), and human neuroimaging evidence suggesting decision-related neural activity in counterpart brain areas, such as the posterior parietal cortex (18–20), we hypothesized that such decision-related activity in oculomotor structures might affect eye movements themselves. To test this hypothesis, we devised a dual-task paradigm in which instructed saccadic eye movements had to be made, but were independent of a concurrent perceptual decision-making task. Our dual-task paradigm is different from previous paradigms that assessed how saccadic eye movements affect concurrent processes in which observers perform a task during saccades (21–23). We reversed the logical order, measuring saccade metrics during an ongoing decision-making task (Fig. 1). Observers were required to judge the net direction of a random-dot motion stimulus, and later to press a button indicating the net direction of motion. However, in between the motion stimulus and the decision response, we presented a saccade target in particular locations following the offset of the motion stimulus. Observers had to make a saccade to this decision-irrelevant target. After the saccadic eye movement was made, the decision was reported. We emphasized speeded responses for the saccade to the target location, whereas accuracy was emphasized for the motion-direction judgment.
Fig. 1.
Procedure. On a given trial, random-dot motion stimuli were displayed after observers fixated stably for 1 s. At motion offset, a saccade target appeared to either the left or right side of the visual field. Observers made a saccadic eye movement to the target as quickly as possible. After the saccade, they were instructed to report the motion direction with a button press. The contrast polarity of dots and the background in the figure is reversed for illustrative purpose only. The dashed lines depicting the aperture were not shown in the experiments. An example trial from Exp. 1 is shown here.
Results
Interactions Between Decision-Making and Decision-Irrelevant Saccades.
In Exp. 1 we tested for direction-specific interactions between motion strength and saccade generation in the context of a motion-direction discrimination task (8–17). Observers performed decision and saccade tasks in which the axes of visual motion and saccade direction were aligned. On a given trial, after fixation was established, a random-dot motion stimulus of variable coherence (3%, 6%, 12%, 24%, or 48%) and variable duration (100, 200, or 400 ms) was displayed around fixation. After the offset of the motion stimulus, a saccade target was displayed on either the left or right side of the display, displaced 20° horizontally from the fixation point. Observers were instructed to make a saccadic eye movement to the target as quickly as possible. After the saccade, they reported their motion direction judgment by pressing designated keyboard buttons with fingers on their right hand. This arrangement is schematized in Fig. 1. Motion direction could be either toward the left or right, and the saccade target could appear in either the left or right visual field. In this geometric arrangement, motion direction and saccade direction could be either congruent (i.e., rightward visual motion and a rightward instructed saccade) or incongruent (i.e., rightward visual motion and a leftward instructed saccade). Note that although the axes of visual motion and saccade direction were aligned, making a saccadic eye movement was irrelevant to motion-direction discrimination.
Motion-direction discrimination accuracy varied systematically with motion coherence. This conventional dependency confirms that observers were engaged in the decision task (Fig. 2A; individual psychometric functions are shown in Fig. S1). The psychometric functions were nearly identical between the congruent and incongruent conditions, indicating that congruency between saccade and motion direction did not affect psychophysical performance, even though the saccade took place in between motion viewing and the subsequent decision response. This independence supports the notion that observers treated the saccade task as irrelevant to performance of the direction discrimination task; the intervening saccades did not affect choices in a direction-selective manner. This process allowed us to test whether motion strength or direction affected the reaction time or other oculomotor metrics of the decision-irrelevant saccade.
Fig. 2.
Psychometric functions and saccade metrics in Exps. 1 and 2. (A) The psychometric functions for congruent vs. incongruent conditions, depicted by red and blue lines, respectively. The error bars on the data points reflect SEM across observers, and the horizontal error bars on the 75% accuracy threshold indicate bootstrapped 68% confidence intervals (CIs). (B and C) Saccade reaction times (B) and saccade peak velocities (C) as a function of motion coherence in Exp. 1. (B) Color-scaled data represent viewing duration in the congruent and the incongruent condition (red vs. blue: 100 ms, orange vs. light blue: 200 ms, and yellow vs. cyan: 400 ms). (C) Red and blue indicate saccade peak velocities in the congruent and incongruent conditions, respectively. Solid and dashed lines are the best-fitting lines for the congruent and incongruent conditions, respectively. Error bars are bootstrapped 95% CIs. (D and E) Saccade reaction times (D) and saccade peak velocities (E) as a function of motion coherence in Exp. 2.
Fig. S1.
Individual psychometric functions for the motion-direction discrimination task in Exp. 1. Red and blue lines indicate psychometric function for the congruent and incongruent condition, respectively. Horizontal error bars represent bootstrapped 68% CIs on the 75% accuracy threshold. This suggests that the main finding of Exp. 1 did not result from the different task performance between the congruent and incongruent conditions.
Indeed, saccade reaction times were affected by the directional congruency of the preceding visual motion. Reaction times were faster in the congruent condition than in the incongruent condition (Fig. 2B) [main effect of congruency on saccade reaction times, F(1, 4) = 10.11, P = 0.03] (see Table S1 for individual data). Interestingly, the effect of motion coherence on saccade reaction times was different between congruent and incongruent conditions [interaction between congruency and coherence, F(4, 16) = 6.28, P = 0.003]. In the congruent condition, reaction times decreased as coherence increased [F(4, 16) = 7.24, P = 0.002], whereas in the incongruent condition this was not the case [F(4, 16) = 0.05, P = 0.99]. Thus, decision-irrelevant saccade reaction times were modulated by motion coherence, suggesting an interaction between decision making and the initiation of saccades, and this interaction occurred only when motion was directionally congruent with the saccade. Motion viewing duration also affected saccade reaction times [F(2, 8) = 114.53, P < 0.0001], but the range of viewing durations we used did not strongly affect psychophysical performance in the motion-direction discrimination task (Fig. S2). Thus, it is likely that the effect of viewing duration on saccade reaction times is a distinct consequence of simple response readiness (24, 25) and not of decision formation, in which longer viewing durations increased subject readiness, resulting in shorter reaction times.
Table S1.
Individual saccade reaction times in Exp. 1
| Subject | Coherence | |||||
| 3% | 6% | 12% | 24% | 48% | ||
| Subject 1 | ||||||
| Duration | 100 ms | 333.9 (328.8) | 339.2 (340.8) | 340.7 (340.7) | 322.8 (340.0) | 324.4 (338.6) |
| 200 ms | 295.6 (305.9) | 305.4 (299.0) | 303.5 (295.3) | 294.9 (299.0) | 275.7 (293.6) | |
| 400 ms | 265.7 (269.8) | 267.6 (271.3) | 263.1 (269.4) | 255.6 (265.9) | 250.0 (273.6) | |
| Subject 2 | ||||||
| Duration | 100 ms | 357.3 (369.5) | 361.6 (369.7) | 367.9 (358.9) | 353.9 (360.4) | 340.4 (343.8) |
| 200 ms | 339.0 (338.8) | 323.1 (333.2) | 316.1 (334.0) | 309.9 (327.1) | 297.7 (302.0) | |
| 400 ms | 307.3 (307.9) | 304.0 (325.3) | 293.7 (298.7) | 290.0 (285.7) | 278.1 (283.6) | |
| Subject 3 | ||||||
| Duration | 100 ms | 301.6 (302.5) | 294.3 (297.6) | 295.1 (296.1) | 293.5 (314.1) | 303.9 (324.9) |
| 200 ms | 253.8 (264.4) | 253.1 (271.1) | 256.6 (262.5) | 263.3 (272.7) | 255.1 (275.4) | |
| 400 ms | 244.3 (248.5) | 240.3 (253.1) | 242.8 (246.7) | 235.0 (241.7) | 245.2 (245.8) | |
| Subject 4 | ||||||
| Duration | 100 ms | 344.2 (369.8) | 365.1 (365.1) | 353.9 (356.4) | 353.9 (370.6) | 340.2 (381.0) |
| 200 ms | 340.4 (323.7) | 326.4 (338.6) | 326.6 (345.0) | 307.0 (363.5) | 288.1 (333.4) | |
| 400 ms | 304.8 (316.6) | 314.1 (322.9) | 313.9 (314.6) | 314.4 (330.0) | 317.1 (343.9) | |
| Subject 5 | ||||||
| Duration | 100 ms | 298.9 (303.8) | 315.4 (296.5) | 305.6 (310.6) | 292.6 (306.9) | 286.3 (313.5) |
| 200 ms | 276.5 (283.4) | 277.4 (278.5) | 278.4 (285.7) | 262.7 (282.8) | 262.7 (294.4) | |
| 400 ms | 266.9 (257.2) | 263.8 (258.5) | 255.7 (268.4) | 250.7 (261.7) | 248.1 (263.9) | |
Units are milliseconds. Saccade reaction times for the congruent and incongruent (in parentheses) are shown.
Fig. S2.
Psychometric functions in Exp. 1 for the congruent (Left) and incongruent (Right) conditions, respectively. Color-scaled data represent viewing duration in the congruent and incongruent conditions (red vs. blue: 100 ms; orange vs. light blue: 200 ms; and yellow vs. cyan: 400 ms).
Saccade peak velocities were also influenced differentially by the congruency of the visual motion. Fig. 2C depicts peak velocities for the congruent and incongruent conditions as a function of motion coherence (after collapsing across motion viewing duration). Congruent trials led to saccades with higher velocities than incongruent trials [F(1, 4) = 7.24, P = 0.05]. Motion coherence also had an effect, with higher coherences leading to faster velocities [F(4, 16) = 8.04, P = 0.001]. Similar to saccade reaction times, motion coherence affected the saccade peak velocity in the congruent condition [F(4, 16) = 8.78, P = 0.001] but not in the incongruent condition [F(4, 16) = 1.92, P = 0.16]; although the interaction between congruency and coherence was not statistically significant [F(4, 16) = 1.88, P = 0.16].
We tested whether this linear dependency between motion coherence and saccade peak velocity was simply predictable by the saccadic “main sequence” (26). For example, higher motion coherence might have resulted in larger saccade amplitude and thus higher saccade peak velocity. However, we found that saccade amplitude did not vary with coherence (Fig. S3A). This finding suggests that our reported modulation of saccade peak velocity cannot be explained as a simple consequence of the main sequence.
Fig. S3.
Saccade amplitude as a function of motion coherence. (A) Saccade amplitude in Exp. 1. Color-scaled data represent viewing duration in the congruent and incongruent conditions (red vs. blue: 100 ms; orange vs. light blue: 200 ms; and yellow vs. cyan: 400 ms). Saccade amplitude did not increase as motion coherence increased in both the congruent [F(4, 16) = 1.749, P = 0.19] and incongruent conditions [F(4, 16) = 2.217, P = 0.11]. (B) Saccade amplitude in Exp. 3. Red, orange, and yellow lines represent saccade amplitude for 100, 200, and 400 ms viewing duration, respectively. All of the lines are best-fitting lines to the data.
Overall, our results suggest that saccade reaction times and saccade peak velocity were modulated by motion strength, and that saccade generation can interact with decision-related signals in direction-specific parts of the oculomotor circuitry.
Saccades Are Affected by Decision Making but Not Motion Viewing.
In Exp. 2 we conducted a single-task experiment in which observers only performed the saccade task. This allowed us to test whether the oculomotor consequences we observed in Exp. 1 were simply a result of viewing visual motion, as opposed to the formation of decisions per se. In this experiment, the stimuli and procedure were identical to the preceding experiment in which the motion direction and the axis of saccadic eye movements were aligned, but now observers were instructed to ignore the motion stimulus that preceded the saccade target appearance, and simply perform the instructed saccades.
Although the visual stimuli in Exp. 2 were identical to those in Exp. 1, motion coherence no longer modulated saccade reaction times, in either the directionally congruent or incongruent conditions (Fig. 2D) [congruent condition, F(4, 16) = 1.06, P = 0.40; incongruent condition, F(4, 16) = 0.89, P = 0.49, respectively] (see Table S2 for individual data). Congruency itself had only subtle effects on saccade reaction times that were not statistically significant [F(1, 4) = 5.80, P = 0.07]. Only motion viewing duration significantly affected reaction time [F(2, 8) = 54.67, P < 0.0001], supporting our interpretation that the effects of motion viewing duration seen in the first experiment are more likely to indicate general response readiness compared with motion evidence (as there was no decision component in this experiment). Saccade peak velocity was not affected by congruency either (Fig. 2D) [F(1, 4) = 0.02, P = 0.89]. Finally, there was no longer a clear relationship between saccade peak velocity and coherence [congruent condition, F(4, 16) = 3.74, P = 0.07; incongruent condition, F(4, 16) = 2.94, P = 0.05].
Table S2.
Individual saccade reaction times in Exp. 2
| Subject | Coherence | |||||
| 3% | 6% | 12% | 24% | 48% | ||
| Subject 1 | ||||||
| Duration | 100 ms | 267.7 (271.8) | 269.4 (264.2) | 264.2 (263.9) | 277.4 (274.5) | 268.9 (272.0) |
| 200 ms | 241.5 (240.6) | 243.9 (239.4) | 246.9 (234.6) | 244.1 (247.5) | 240.3 (244.8) | |
| 400 ms | 226.2 (226.9) | 222.8 (225.8) | 223.0 (222.5) | 229.8 (226.9) | 226.0 (226.8) | |
| Subject 2 | ||||||
| Duration | 100 ms | 315.6 (315.1) | 302.5 (310.2) | 320.1 (307.0) | 308.8 (306.2) | 316.0 (323.6) |
| 200 ms | 281.2 (285.9) | 281.0 (279.6) | 278.5 (292.7) | 288.8 (287.7) | 274.3 (292.0) | |
| 400 ms | 270.4 (271.8) | 268.7 (273.4) | 263.9 (277.3) | 259.5 (273.6) | 252.0 (267.4) | |
| Subject 3 | ||||||
| Duration | 100 ms | 238.0 (250.2) | 236.8 (238.5) | 236.7 (240.4) | 239.2 (323.9) | 237.4 (249.4) |
| 200 ms | 219.5 (220.6) | 226.9 (231.5) | 224.3 (220.5) | 221.0 (224.9) | 231.4 (239.7) | |
| 400 ms | 213.0 (215.5) | 218.1 (217.3) | 217.7 (221.5) | 214.1 (214.6) | 216.9 (218.1) | |
| Subject 4 | ||||||
| Duration | 100 ms | 280.9 (282.3) | 284.3 (275.1) | 273.1 (277.5) | 285.3 (282.5) | 271.2 (286.2) |
| 200 ms | 259.8 (254.7) | 259.8 (254.6) | 252.7 (258.2) | 251.8 (263.0) | 240.0 (265.6) | |
| 400 ms | 252.1 (249.3) | 246.0 (244.3) | 242.6 (245.7) | 242.1 (243.8) | 242.6 (234.8) | |
| Subject 5 | ||||||
| Duration | 100 ms | 309.8 (310.9) | 308.8 (314.0) | 303.4 (305.4) | 303.5 (318.9) | 306.3 (302.7) |
| 200 ms | 278.2 (274.6) | 291.9 (284.4) | 272.9 (281.8) | 275.0 (280.4) | 280.2 (279.9) | |
| 400 ms | 278.5 (285.0) | 279.3 (282.3) | 277.7 (285.4) | 274.0 (280.3) | 277.5 (279.1) | |
Units are milliseconds. Saccade reaction times for the congruent and incongruent (in parentheses) are shown.
To compare the results from this passive-viewing experiment (Exp. 2) to the results of the active decision-making experiment (Exp. 1), we collapsed the data over duration for each condition and plotted the difference between the congruent and the incongruent conditions as a function of motion coherence (Fig. 3). The difference in saccade reaction time between these conditions increased systematically with coherence for the decision-making task, but remained close to zero for the passive viewing task. At the very highest motion coherence, there was a slight increase when passively viewing the motion stimulus (mean ± SEM; 7.19 ± 2 ms), but this increase is smaller than that of the active decision-making experiment (21.59 ± 7 ms). We suspect that this small effect might be because some of our observers failed to completely ignore the motion stimuli, given that four of five observers conducted the active decision-making experiment before conducting the passive-viewing control experiment. Taken together, these results suggest that the relationship between decision difficulty and saccade metrics—linear decreases in saccade reaction times, and linear increases in saccade peak velocity, as a function of coherence—cannot be explained by sensory signals alone.
Fig. 3.
Comparison of saccade reaction time benefits between Exps. 1 and 2. Saccade reaction time benefit was calculated by subtracting saccade reaction times for the congruent condition from saccade reaction times for the incongruent condition. Red and blue lines represent saccade reaction time benefit in Exps. 1 and 2, respectively. Error bars are bootstrapped 95% CIs.
Decision Making but Not Spatial Attention Modulates Decision-Irrelevant Saccades.
Although the preceding experiments lend support for the notion of decision making modulating the oculomotor system, we sought to isolate purely decisional factors from other processes that likely co-occur with decision making. It is possible that the observed modulation in saccade metrics is a function of attentional allocation in concert with the direction of motion. For example, strong rightward motion could serve as an attentional cue to the right, resulting in a facilitation of rightward saccade reaction time and velocity (27).
In Exp. 3 we tested whether the modulation of saccade metrics in prior experiments could be caused by stimulus-driven spatial attention: specifically, a shift of attention to a target location congruent with the direction of discriminated motion. In this experiment, the axis of visual motion for direction discrimination was now perpendicular to the axis of potential saccade target locations (i.e., up versus down visual motion, and left or right saccade targets). In this geometric configuration, a direction-specific shift in spatial attention would be either above or below the motion aperture, both spatially unrelated to the left/right saccade. If an attentional shift accounts for the results of Exp. 1, then we would not expect to observe a modulation of saccade metrics here.
As in Exp. 1, direction-discrimination accuracy varied systematically with motion coherence (individual psychometric functions are shown in Fig. S4). Because there was no explicit mapping between motion direction (up/down) and the saccade target location (left/right), the saccadic eye movements should not be affected by the motion direction, but could still be affected by the nondirectional magnitude of the motion coherence. Consistent with this, motion direction did not exert a reliable effect on saccade reaction times [F(1,4) = 2.27, P = 0.21]. Thus, we collapsed the data over motion direction for further analyses.
Fig. S4.
Individual psychometric functions for the motion-direction discrimination task in Exp. 3. Each observer reported motion direction after making a saccadic eye movement by pressing a designated keyboard button. All observers’ psychometric functions indicate that the difficulty of the task was modulated by the motion coherence. Horizontal error bars represent bootstrapped 68% CIs on the 75% accuracy threshold.
Saccade reaction times to the decision-irrelevant saccade target were systematically affected by motion coherence (Fig. 4A; see Table S3 for individual data). Saccade reaction times were slower while making a decision based on weaker motion evidence, and quickened progressively with increased motion coherence [F(4, 16) = 7.12, P = 0.002]. A roughly linear inverse dependency between motion coherence and saccade reaction times was evident. The reaction time modulation by motion coherence was largest for the longer viewing durations (for 200 ms, slope = −0.61, r = 0.97, P = 0.001; for 400 ms, slope = −0.63, r = 0.99, P = 0.0001), and was weak if present at all in the shortest duration (for 100 ms, slope = −0.10, r = 0.73, P = 0.14), confirmed by an interaction between motion viewing duration and coherence [F(8, 32) = 3.78, P = 0.003].
Fig. 4.
Saccade metrics in Exps. 3 and 4. (A) Saccade reaction times in Exp. 3 are plotted as a function of motion coherence for each viewing duration (red: 100 ms; orange: 200 ms; and yellow: 400 ms), averaged across motion direction (up and down). (B) Saccade peak velocities in Exp. 3 are plotted as a function of motion coherence for each viewing duration (red: 100 ms; orange: 200 ms; and yellow: 400 ms), averaged across motion direction (up and down). (C) Manual reaction times in Exp. 4 are plotted as a function of motion coherence for each viewing duration (orange: 200 ms; yellow: 400 ms), averaged across motion direction (up and down). Best-fitting lines are shown. Error bars are bootstrapped 95% CIs.
Table S3.
Individual saccade reaction times in Exp. 3
| Subject | Coherence | |||||
| 3% | 6% | 12% | 24% | 48% | ||
| Subject 1 | ||||||
| Duration | 100 ms | 304.5 | 303.4 | 306.8 | 301.8 | 301.3 |
| 200 ms | 266.5 | 274.4 | 274.6 | 266.4 | 256.6 | |
| 400 ms | 242.7 | 239.6 | 240.0 | 237.2 | 228.6 | |
| Subject 2 | ||||||
| Duration | 100 ms | 359.9 | 357.9 | 360.9 | 354.4 | 346.5 |
| 200 ms | 351.6 | 352.2 | 344.2 | 318.5 | 307.1 | |
| 400 ms | 317.5 | 307.1 | 313.0 | 279.0 | 280.5 | |
| Subject 3 | ||||||
| Duration | 100 ms | 312.5 | 309.3 | 321.1 | 322.3 | 333.6 |
| 200 ms | 270.8 | 269.4 | 268.5 | 283.3 | 266.7 | |
| 400 ms | 248.1 | 242.8 | 238.7 | 239.4 | 227.3 | |
| Subject 4 | ||||||
| Duration | 100 ms | 345.0 | 348.8 | 342.9 | 337.7 | 329.7 |
| 200 ms | 304.3 | 315.5 | 315.2 | 297.5 | 281.3 | |
| 400 ms | 274.1 | 274.0 | 274.6 | 264.7 | 250.4 | |
| Subject 5 | ||||||
| Duration | 100 ms | 374.7 | 370.6 | 370.5 | 374.1 | 371.1 |
| 200 ms | 349.9 | 352.7 | 353.9 | 337.7 | 340.1 | |
| 400 ms | 350.8 | 350.2 | 336.7 | 332.8 | 316.5 | |
Units are milliseconds.
Motion viewing duration also affected saccade reaction times (Fig. 4A) [F(2, 8) = 61.32, P < 0.0001], likely because general response readiness increased as motion viewing duration increased (24, 25). Saccade peak velocity was also linearly dependent on motion coherence. Peak velocity increased progressively as the coherence increased (Fig. 4B) [F(4,16) = 9.85, P = 0.0003]. Motion viewing duration also affected saccade peak velocity [F(2, 8) = 9.89, P < 0.01]. Consistent with the results in Exp. 1, there was no linear relationship between saccade amplitude and motion coherence (Fig. S3B) [F(4, 16) = 2.431, P = 0.09], whereas motion viewing duration affected saccade amplitude [F(2, 8) = 11.645, P < 0.005].
To summarize Exp. 3, saccade reaction times and peak velocities were modulated by motion coherence even in the absence of a geometrical mapping between saccade and motion direction. This finding argues against an attentional shift in congruence with motion direction, and instead supports the idea that the difficulty of the decision governs interactions between decisions and saccades.
Dual-Task Interference Because of Decision Making Is Oculomotor-Specific.
It is also logically possible that the results of Exp. 1 depend on task difficulty, but not a specific interaction between decision-making signals and oculomotor circuits, as a result of general dual-task interference (28). To test whether our results are more precisely specific to decisions interacting with oculomotor functions, we conducted Exp. 4, in which we changed the mappings between decisions and motor responses, under otherwise identical experimental settings to Exp. 3.
If our results were caused by general dual-task interference, the modality of the response to the decision-irrelevant target (i.e., an eye movement or a button press) should not matter, because general dual-task interference, by definition, should affect all effectors equally. Thus, the procedure of Exp. 4 was identical to Exp. 3, but instead of a saccade to a decision-irrelevant target, subjects were instructed to report the location of the decision-irrelevant target with a speeded button press. To maximize statistical power, we used only the two longer motion durations (200 and 400 ms) for which the modulation of saccade reaction times was most pronounced.
Inconsistent with the predictions of general dual-task interference, we found no modulation of manual reaction times as a function of motion coherence (Fig. 4C) [F(4, 16) = 1.14, P = 0.37] (see Table S4 for individual data). Motion-viewing duration affected manual reaction times [F(1, 4) = 18.164, P = 0.01], consistent with our previous experiments that suggested duration separately affected general task readiness. Performance on the motion-direction discrimination task was nearly identical to the one in Exp. 3 (Fig. S5), confirming that the different results in the manual reaction-time task were not a result of different psychophysical performance levels between experiments. To summarize Exp. 4, manual reaction times were not affected by motion coherence when the location of the decision-irrelevant target was reported with a button press, arguing against general dual-task interference, and in addition, add further support to the notion that our main finding is specific to the oculomotor system.
Table S4.
Individual manual reaction times in Exp. 4
| Subject | Coherence | |||||
| 3% | 6% | 12% | 24% | 48% | ||
| Subject 1 | ||||||
| Duration | 200 ms | 363.8 | 352.3 | 357.3 | 359.5 | 363.1 |
| 400 ms | 325.2 | 327.4 | 324.7 | 312.7 | 320.0 | |
| Subject 2 | ||||||
| Duration | 200 ms | 437.3 | 434.5 | 440.6 | 446.0 | 440.4 |
| 400 ms | 432.9 | 432.3 | 429.4 | 432.3 | 437.6 | |
| Subject 3 | ||||||
| Duration | 200 ms | 443.9 | 433.8 | 441.2 | 434.5 | 429.4 |
| 400 ms | 412.0 | 406.7 | 404.4 | 417.8 | 427.3 | |
| Subject 4 | ||||||
| Duration | 200 ms | 354.2 | 464.6 | 353.2 | 356.7 | 365.9 |
| 400 ms | 352.4 | 350.5 | 337.6 | 338.2 | 351.8 | |
| Subject 5 | ||||||
| Duration | 200 ms | 399.3 | 403.8 | 409.5 | 401.8 | 415.9 |
| 400 ms | 394.9 | 389.8 | 394.8 | 398.3 | 390.7 | |
Units are milliseconds.
Fig. S5.
Psychometric functions in Exp. 3 (blue) and Exp. 4 (red). The horizontal error bars represent 68% CIs around 75% accuracy threshold from 1,000 bootstrapped fits. In Exp. 3, threshold is 15.27% and CI = (14.48, 15.52), and in Exp. 4, threshold is 16.25% and CI = (15, 16.89). These results suggest that the different results in Exps. 3 and 4 were not caused by different behavioral performance on motion-direction discrimination tasks in each experiment.
Overall, our results demonstrate that performing a challenging perceptual decision task results in significant modulations of oculomotor performance. The observed changes in saccade reaction times and peak velocities point to interactions within the oculomotor circuit, in which activity related to accumulating evidence during a perceptual decision making and activity related to generation of decision-irrelevant saccades are not kept completely separate.
Discussion
We tested whether visually guided, instructed saccades to a single target were influenced by the formation of perceptual decisions known to elicit decision-related activity in the oculomotor brain structures of primates. We found that saccadic eye movements to a decision-irrelevant target were systematically modulated by the motion strength that informed perceptual decisions about visual motion direction. This pattern of results suggests that decision-making processes interact with saccadic eye movements: specifically, saccade reaction times and peak velocities were slower given difficult motion judgments (i.e., decisions driven by weaker sensory evidence). We also found that the spatial congruency between motion direction and saccade direction mattered: motion strength did not affect saccadic eye movements when the decision-irrelevant saccades were opposite the direction of the motion decision.
The results reported here are most consistent with an explanation based on an active decision-making process. Passively viewing the motion stimulus did not result in the same effects on saccades (Exp. 2). We also observed changes in saccade reaction times and velocities when motion and saccade directions were uncoupled (Exp. 3), ruling out a directional form of spatial attention as a possible explanation of our results. Additionally, when manual reaction times were measured instead of saccade reaction times (Exp. 4), reaction times were no longer modulated as a function of motion coherence, even though all other aspects of experiment remained the same as in Exp. 3. This constellation of results suggests an oculomotor-specific dual-task interference effect, and supports the prospect of oculomotor circuitry being involved in decision making, even when decisions are not communicated with eye movements.
An interesting remaining question is whether attending to the stimuli without making decisions can result in the coherence-dependent effect in our paradigm. Attention and decision processes are tightly related, and it is difficult to distinguish whether an effect is caused by one process or the other. For example, even when observers perform an attention-demanding task on moving stimuli, it is difficult to know whether they covertly make decisions on motion direction. In the same vein, we speculate that making covert decisions on motion direction may have caused a very small effect in the highest coherence condition in Exp. 2, where there was no explicit decision component.
Our results show that only when there is a conflict between saccade and motion direction (the incongruent condition in Exp. 1), saccade metrics were not modulated by motion strength. This might be because of the fact that saccades in the incongruent condition involve an unnatural stimulus-response mapping. In fact, during visual inspection of raw eye-position traces, we found that in a small portion of trials in the incongruent condition (which were removed from the main analyses), observers initiated saccades toward the location congruent with motion direction (opposite to the target location) and then corrected their gaze toward to the target location. Interestingly, the number of these error trials increased as the motion coherence increased (3, 6, 7, 15, and 19 trials across observers for 3%, 6%, 12%, 24%, and 48% coherence, respectively). This observation suggests that the higher the motion coherence, the more interference between motion evidence and saccades in the incongruent condition, and thus less of a speed-up as a function of motion strength.
It is known that saccade peak velocities can be influenced not only by the intrinsic value of the saccade target (29, 30) but also by the subjective preference or value of the saccade target (31). It is possible, therefore, that the saccade target in our experiments might have been associated with a value based on the task difficulty. For example, the saccade target in difficult (low coherence) conditions (where observers made more incorrect responses) might have been associated with low values, resulting in slower saccade reaction times. However, in Exp. 1 there was not only an effect of motion coherence but also a dependency on congruency. In congruent versus incongruent trials, the motion judgment was equally hard (for matched coherences), but the coherence effect was only pronounced in the directionally congruent condition. This argues against interpretations of our results based on the value of targets and simple effects of task difficulty.
Although the majority of behavioral studies addressing cognitive influences on motor output have focused on tasks using arm reaches as the motor response, there has been some evidence suggesting an interaction between decision making and oculomotor output. Microstimulation in the frontal eye field during perceptual decision-making tasks evokes saccadic eye movements that deviate in proportion to motion evidence (e.g., coherence) along the motion direction, suggesting that accumulated evidence is represented in this oculomotor planning structure, or in structures connected to it (16, 17). Our findings are consistent with this type of decision and premotor multiplexing, and demonstrate that such interactions are tacitly present even without artificial perturbation of brain circuits. Furthermore, saccadic trajectories exhibit systematic deviations in their endpoints when saccadic eye movements are used to report decisions in a motion-direction discrimination task (32). Our results are not only consistent with such observations of oculomotor output being affected by decisions, but also reveal that eye movements can be affected even when the decision-making task does not involve an oculomotor response. This finding suggests that oculomotor circuits may be recruited during decision making, even when they are not tightly tied to premotor or motor function.
A growing body of literature has revealed multiplexing of signals in oculomotor areas. Recent single-cell recordings have documented that neural responses in oculomotor brain areas show heterogeneous selectivity for different sources, such as visual events, decision formation, and saccade execution, often within the same neurons and the same single-trial spike trains. These include the lateral intraperiatal area (11, 33–35), frontal eye field (36), and superior colliculus (14). Our findings suggest that multiple signals in oculomotor areas can interact with each other, specifically, signals related to decision formation and oculomotor execution. Ultimately, it appears that the read out of oculomotor structures cannot completely distinguish activity driven by decision formation from activity related to the planning of even decision-irrelevant eye movements (37).
Materials and Methods
Participants.
A total of seven observers, including the authors, with normal or corrected-to-normal vision, participated in the experiment voluntarily. Distinct subsets of five observers from this pool participated in each experiment. All experimental sessions were conducted with the written consent of each observer and in accordance with the University of Texas at Austin Institutional Review Board.
Apparatus, Stimuli, and Procedure.
Eye position was recorded monocularly with an Eyelink1000 (SR Research) in “remote mode” with a sampling rate of 500 Hz. Stimuli were created using MATLAB (The Mathworks) in conjunction with the Psychophysics Toolbox (38, 39) on a Mac Pro. Stimuli were displayed on a Samsung liquid crystal display TV (1,920 × 1,080 resolution, 60-Hz refresh rate, subtending 80° horizontally). The viewing distance was 60 cm.
We used random-dot motion stimuli (300 dots) that were displayed in a circular aperture (20° in diameter) centered around the fixation mark. Dots were white (78.43 cd/m2), subtended 0.17°, and moved at the speed of 5°/s on a black background (0.23 cd/m2). Each dot was assigned a random lifetime from a uniform distribution between 0 and 150 ms (nine video frames). When a dot’s lifetime expired, it was randomly placed within the aperture and assigned the maximum lifetime (150 ms). The central 3° around fixation was blank. Motion coherence was defined as the percentage of dots moving together in the same direction among dots moving in random directions. We used five coherence levels: 3%, 6%, 12%, 24%, and 48%. We also manipulated the viewing duration of motion stimuli (100, 200, or 400 ms).
Each trial started with a fixation mark (red, 15.08 cd/m2) at the center of the display. After establishing stable fixation for 1 s, random-dot motion stimuli were displayed for variable duration. At the offset of the motion stimulus, the fixation mark disappeared and a saccade target (red, 15.08 cd/m2) appeared at a location that was displaced by 20° horizontally (either left or right) from the center of display. Observers made a saccadic eye movement to the saccade target as quickly as possible. After re-establishing stable fixation after the saccade landing for 500 ms, the color of the saccade target changed from red to blue (5.44 cd/m2), prompting observers to report the net direction of the random dot stimulus. Observers used computer keyboard arrow keys to report the motion direction with a finger button-press. The experiment did not proceed until observers reported the motion direction. The intertrial interval was 1 s, and after this interval the fixation mark appeared at the center of the display. Auditory feedback was presented for the motion discrimination task (correct vs. incorrect). A 3° wide square window was used to determine whether stable fixation was established. When observers broke fixation (i.e., their eye position moved outside of this square window), the trial was aborted and discarded.
In Exp. 1, the motion direction (left or right) was aligned with the axis of saccadic eye movements (left or right). Because the motion direction and the saccadic eye movements shared the same axis, there were two more conditions in this experiment: congruent (the saccadic direction and motion direction were the same) and incongruent (the saccadic direction was the opposite of the motion direction). Observers used left and right arrow keys to report their decision about motion direction. In Exp. 2, all aspects of the experiment remained the same as in Exp. 1 except that there was no motion direction discrimination component: observers were not prompted to report the motion direction after making the instructed saccade. In Exp. 3, the motion direction (up or down) was perpendicular to the axis of saccadic eye movements (left or right). Observers used up and down arrow keys to report the motion direction. In Exp. 4, all aspects of the experiment remained the same as in Exp. 3, except that manual reaction times were measured instead of saccade reaction times. Observers reported the target location using left or right arrow keys as quickly as possible after target onset using their right hand, and they used designated buttons to report the motion direction, up (“a”) or down (“z”) using their left hand. The fixation mark was always displayed to force observers to maintain fixation. We excluded from analyses trials in which observers made an eye movement to the target (<1%).
Each experimental session consisted of 31 experimental blocks. Within each block, 15 trials (3 durations × 5 coherences) were presented in pseudorandomized order (Exps. 1, 2, and 3) and 10 trials (2 durations × 5 coherences) were similarly pseudorandomized (Exp. 4). The motion direction and saccadic direction were randomized across trials. The first block served as a practice block and was not included in the data analysis. Each observer participated in three to four experimental sessions for Exps. 1 and 3, and two experimental sessions for Exps. 2 and 4.
Data Analysis.
We detected saccades offline by applying a velocity threshold. We slid a 10-ms moving window and defined saccade onset as the first time point of the window during which the velocity exceeded 20°/s. Saccade reaction times were defined as the elapsed time from saccade target presentation to the detected onset of the saccade. Only saccade reaction times between 150 ms and 500 ms were included in data analysis, excluding <1% of trials in Exps. 1, 2, and 3. We included all of the data regardless of whether the reported motion direction was correct or not for analyses. Only including correct trials did not change the pattern of the results. For statistical evaluation, we used repeated-measures ANOVA. The assumptions for ANOVA were met, unless otherwise stated.
Motion-discrimination performance was evaluated by defining a psychometric function. We used a maximum-likelihood procedure to estimate the best-fitting Weibull function to the data. Then, we used a parametric bootstrapping procedure to calculate the confidence intervals of the estimated psychophysical thresholds.
Acknowledgments
This research is supported by National Institutes of Health National Eye Institute Grant EY020592 (to A.C.H.) and by a Howard Hughes Medical Institute International Student Research Fellowship (to L.N.K.).
Footnotes
The authors declare no conflict of interest.
This article is a PNAS Direct Submission.
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1520309113/-/DCSupplemental.
References
- 1.Donders FC. On the speed of mental processes. Acta Psychol (Amst) 1969;30:412–431. doi: 10.1016/0001-6918(69)90065-1. [DOI] [PubMed] [Google Scholar]
- 2.Luce RD. Response Times: Their Role in Inferring Elementary Mental Organization. Oxford Univ Press; Belfast, Ireland: 1986. [Google Scholar]
- 3.Song J-H, Nakayama K. Hidden cognitive states revealed in choice reaching tasks. Trends Cogn Sci. 2009;13(8):360–366. doi: 10.1016/j.tics.2009.04.009. [DOI] [PubMed] [Google Scholar]
- 4.Spivey M. The Continuity of Mind. Oxford Univ Press; New York: 2007. [Google Scholar]
- 5.Song J-H, Nakayama K. Target selection in visual search as revealed by movement trajectories. Vision Res. 2008;48(7):853–861. doi: 10.1016/j.visres.2007.12.015. [DOI] [PubMed] [Google Scholar]
- 6.Spivey MJ, Grosjean M, Knoblich G. Continuous attraction toward phonological competitors. Proc Natl Acad Sci USA. 2005;102(29):10393–10398. doi: 10.1073/pnas.0503903102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Song J-H, Nakayama K. Numeric comparison in a visually-guided manual reaching task. Cognition. 2008;106(2):994–1003. doi: 10.1016/j.cognition.2007.03.014. [DOI] [PubMed] [Google Scholar]
- 8.Newsome WT, Britten KH, Movshon JA. Neuronal correlates of a perceptual decision. Nature. 1989;341(6237):52–54. doi: 10.1038/341052a0. [DOI] [PubMed] [Google Scholar]
- 9.Selen LPJ, Shadlen MN, Wolpert DM. Deliberation in the motor system: Reflex gains track evolving evidence leading to a decision. J Neurosci. 2012;32(7):2276–2286. doi: 10.1523/JNEUROSCI.5273-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Shadlen MN, Newsome WT. Motion perception: Seeing and deciding. Proc Natl Acad Sci USA. 1996;93(2):628–633. doi: 10.1073/pnas.93.2.628. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bennur S, Gold JI. Distinct representations of a perceptual decision and the associated oculomotor plan in the monkey lateral intraparietal area. J Neurosci. 2011;31(3):913–921. doi: 10.1523/JNEUROSCI.4417-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Shadlen MN, Newsome WT. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol. 2001;86(4):1916–1936. doi: 10.1152/jn.2001.86.4.1916. [DOI] [PubMed] [Google Scholar]
- 13.de Lafuente V, Jazayeri M, Shadlen MN. Representation of accumulating evidence for a decision in two parietal areas. J Neurosci. 2015;35(10):4306–4318. doi: 10.1523/JNEUROSCI.2451-14.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Horwitz GD, Newsome WT. Separate signals for target selection and movement specification in the superior colliculus. Science. 1999;284(5417):1158–1161. doi: 10.1126/science.284.5417.1158. [DOI] [PubMed] [Google Scholar]
- 15.Ding L, Gold JI. Neural correlates of perceptual decision making before, during, and after decision commitment in monkey frontal eye field. Cereb Cortex. 2012;22(5):1052–1067. doi: 10.1093/cercor/bhr178. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Gold JI, Shadlen MN. Representation of a perceptual decision in developing oculomotor commands. Nature. 2000;404(6776):390–394. doi: 10.1038/35006062. [DOI] [PubMed] [Google Scholar]
- 17.Gold JI, Shadlen MN. The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commands. J Neurosci. 2003;23(2):632–651. doi: 10.1523/JNEUROSCI.23-02-00632.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hebart MN, Donner TH, Haynes J-D. Human visual and parietal cortex encode visual choices independent of motor plans. Neuroimage. 2012;63(3):1393–1403. doi: 10.1016/j.neuroimage.2012.08.027. [DOI] [PubMed] [Google Scholar]
- 19.Liu T, Pleskac TJ. Neural correlates of evidence accumulation in a perceptual decision task. J Neurophysiol. 2011;106(5):2383–2398. doi: 10.1152/jn.00413.2011. [DOI] [PubMed] [Google Scholar]
- 20.Kayser AS, Buchsbaum BR, Erickson DT, D’Esposito M. The functional anatomy of a perceptual decision in the human brain. J Neurophysiol. 2010;103(3):1179–1194. doi: 10.1152/jn.00364.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Irwin DE, Brockmole JR. Suppressing where but not what: The effect of saccades on dorsal- and ventral-stream visual processing. Psychol Sci. 2004;15(7):467–473. doi: 10.1111/j.0956-7976.2004.00703.x. [DOI] [PubMed] [Google Scholar]
- 22.Shioiri S, Cavanagh P. Saccadic suppression of low-level motion. Vision Res. 1989;29(8):915–928. doi: 10.1016/0042-6989(89)90106-5. [DOI] [PubMed] [Google Scholar]
- 23.Burr DC, Morrone MC, Ross J. Selective suppression of the magnocellular visual pathway during saccadic eye movements. Nature. 1994;371(6497):511–513. doi: 10.1038/371511a0. [DOI] [PubMed] [Google Scholar]
- 24.Ross LE, Ross SM. Saccade latency and warning signals: Stimulus onset, offset, and change as warning events. Percept Psychophys. 1980;27(3):251–257. doi: 10.3758/bf03204262. [DOI] [PubMed] [Google Scholar]
- 25.Kingstone A, Klein RM. What are human express saccades? Percept Psychophys. 1993;54(2):260–273. doi: 10.3758/bf03211762. [DOI] [PubMed] [Google Scholar]
- 26.Bahill AT, Clark MR, Stark L. The main sequence, a tool for studying human eye movements. Math Biosci. 1975;24(3-4):191–204. [Google Scholar]
- 27.Bosbach S, Prinz W, Kerzel D. A Simon effect with stationary moving stimuli. J Exp Psychol Hum Percept Perform. 2004;30(1):39–55. doi: 10.1037/0096-1523.30.1.39. [DOI] [PubMed] [Google Scholar]
- 28.Pashler H, Carrier M, Hoffman J. Saccadic eye movements and dual-task interference. Q J Exp Psychol A. 1993;46(1):51–82. doi: 10.1080/14640749308401067. [DOI] [PubMed] [Google Scholar]
- 29.Xu-Wilson M, Zee DS, Shadmehr R. The intrinsic value of visual information affects saccade velocities. Exp Brain Res. 2009;196(4):475–481. doi: 10.1007/s00221-009-1879-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O. Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res. 2002;142(2):284–291. doi: 10.1007/s00221-001-0928-1. [DOI] [PubMed] [Google Scholar]
- 31.Reppert TR, Lempert KM, Glimcher PW, Shadmehr R. Modulation of saccade vigor during value-based decision making. J Neurosci. 2015;35(46):15369–15378. doi: 10.1523/JNEUROSCI.2621-15.2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.McSorley E, McCloy R. Saccadic eye movements as an index of perceptual decision-making. Exp Brain Res. 2009;198(4):513–520. doi: 10.1007/s00221-009-1952-9. [DOI] [PubMed] [Google Scholar]
- 33.Meister MLR, Hennig JA, Huk AC. Signal multiplexing and single-neuron computations in lateral intraparietal area during decision-making. J Neurosci. 2013;33(6):2254–2267. doi: 10.1523/JNEUROSCI.2984-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Park IM, Meister MLR, Huk AC, Pillow JW. Encoding and decoding in parietal cortex during sensorimotor decision-making. Nat Neurosci. 2014;17(10):1395–1403. doi: 10.1038/nn.3800. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Rishel CA, Huang G, Freedman DJ. Independent category and spatial encoding in parietal cortex. Neuron. 2013;77(5):969–979. doi: 10.1016/j.neuron.2013.01.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Mante V, Sussillo D, Shenoy KV, Newsome WT. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature. 2013;503(7474):78–84. doi: 10.1038/nature12742. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Raposo D, Kaufman MT, Churchland AK. A category-free neural population supports evolving demands during decision-making. Nat Neurosci. 2014;17(12):1784–1792. doi: 10.1038/nn.3865. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Brainard DH. The Psychophysics Toolbox. Spat Vis. 1997;10(4):433–436. [PubMed] [Google Scholar]
- 39.Pelli DG. The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spat Vis. 1997;10(4):437–442. [PubMed] [Google Scholar]