Attention during active visual tasks: counting, pointing, or simply looking

Abstract

Visual attention and saccades are typically studied in artificial situations, with stimuli presented to the steadily fixating eye, or saccades made along specified paths. By contrast, in the real world saccadic patterns are constrained only by the demands of the motivating task. We studied attention during pauses between saccades made to perform 3 free-viewing tasks: counting dots, pointing to the same dots with a visible cursor, or simply looking at the dots using a freely-chosen path. Attention was assessed by the ability to identify the orientation of a briefly-presented Gabor probe. All primary tasks produced losses in identification performance, with counting producing the largest losses, followed by pointing and then looking-only. Looking-only resulted in a 37% increase in contrast thresholds in the orientation task. Counting produced more severe losses that were not overcome by increasing Gabor contrast. Detection or localization of the Gabor, unlike identification, were largely unaffected by any of the primary tasks. Taken together, these results show that attention is required to control saccades, even with freely-chosen paths, but the attentional demands of saccades are less than those attached to tasks such as counting, which have a significant cognitive load. Counting proved to be a highly demanding task that either exhausted momentary processing capacity (e.g., working memory or executive functions), or, alternatively, encouraged a strategy of filtering out all signals irrelevant to counting itself. The fact that the attentional demands of saccades (as well as those of detection/localization) are relatively modest makes it possible to continually adjust both the spatial and temporal pattern of saccades so as to re-allocate attentional resources as needed to handle the complex and multifaceted demands of real-world environments.

Introduction

People cannot attend to multiple objects or events without some loss of perceptibility. These so-called attentional bottlenecks, key to the understanding of immediate visual experience and awareness, have attracted the interest of researchers for decades (James, 1890; Broadbent, 1958; Treisman, 1969; Sperling & Dosher, 1986; Sperling & Melchner, 1978; Shaw, 1982; Neisser & Becklin, 1975; Mack & Rock,1998; Bonneh, Cooperman & Sagi, 2001; Lavie, Hirst, de Fockert & Viding, 2004; Kahnemann, Beatty & Pollack, 1967; Carrasco, Ling & Reed, 2004; Kastner, DeWeerd, Desimone & Ungerleider, 1998).

Much of what we know about the characteristics of attentional bottlenecks in vision has come from studies in which sequences of brief stimuli are presented to the steadily fixating eye. For example, several studies have shown that attending to a central visual task (visual search, typically) can impair the ability to discriminate the contrast, orientation or spatial frequency of eccentric gratings (Morrone, Denti & Spinelli, 2004; Huang & Dobkins, 2005; Lee, Itti, Koch & Braun, 1999; Dosher & Lu, 2005; Joseph, Chun, & Nakayama, 1997; Schwartz, Vuilleumier, Hutton, Maravita, Dolan & Driver, 2005). In general, the more difficult the central task, the greater the impairment of perceptual discrimination of the eccentric stimuli.

Studying attention during maintained fixation, as the prior work has done, has real virtues. It allows precise control over the spatial and temporal properties of stimuli, making it possible to study the fine grain properties of attention during brief and manageable intervals of time. But this admittedly artificial situation, in which attention is held at a single central region for prolonged periods, does not represent how attention normally functions. Under normal circumstances attention is not rigidly held in place by a fixation target, but is in continual motion, as saccadic eye movements take the line of sight from one region to another.

Studies of attention during steady fixation have another drawback, namely, they cannot consider the role played by attention in the control of the saccades themselves. Perceptual attention must be allocated to the target of a saccadic eye movement during the interval preceding each saccade to avoid saccadic error or delays (Hoffman & Subramaniam, 1995; Kowler, Anderson, Dosher & Blaser, 1995; Deubel & Schneider, 1996; Godijn & Theeuwes, 2003; McPeek, Maljkovic & Nakayama, 1999; Cohen, Schnitzer, Gersch, Singh & Kowler, 2007; Moore & Armstrong, 2003). Under some conditions the resulting enhancement of visual performance at the saccadic goal can spread to different locations along the planned saccadic path (Gersch, Kowler & Dosher, 2004; Gersch, Kowler, Schnitzer & Dosher, 2008; Baldauf & Deubel, 2008; Godijn & Theeuwes, 2003). These prior results came from studies of saccades made to designated targets, another convenient, but artificial, situation. Dictating the choice of saccadic targets is a useful experimental tactic for removing ambiguity about where subjects intend to look, thus facilitating the comparison of visual performance on and off a planned saccadic path. Nevertheless, such restrictions do not allow us to address basic questions about the links between attention and saccades when the targets are chosen freely, rather than being specified by the experimenters.

The goal of the present paper was to reduce the artificial restrictions on attention and eye movements that have characterized the past work – both the psychophysical and oculomotor research – and study attention during intersaccadic pauses while saccades were made to freely-chosen targets. The way we approached this problem was influenced by prior work. In particular:

1. We included two types of saccadic conditions, one in which saccades were made with no purpose other than to look around the displays, and another in which saccades were made to accomplish a specific visual task. Two visual tasks were studied: counting a display of randomly-positioned dots (Experiment 1), or pointing to the same dots with a visible cursor (Experiment 2). The inclusion of motivating tasks to generate the saccadic sequences (rather than limiting testing to saccades made for no purpose other than to look at targets) was prompted by prior results of Epelboim and colleagues (1995, 1997,1998). They found that saccadic velocities were faster, and intersaccadic pauses shorter, when saccades were made while tapping a sequence of rods than when simply looking at the rods. The apparent facilitation of saccades during tapping was surprising because Epelboim et al.’s looking-only task required the same spatial sequence of saccades as tapping, and seemed, on the face of things, to be the easier (less demanding) of the two. Epelboim et al. proposed that some of the effects of the motivating task on saccades could be due to differential contributions of attention, but did not test this suggestion with psychophysical measures.

2. We used a forced-choice psychophysical discrimination task (identifying the orientation of a briefly-presented grating) to evaluate attention during the pauses between saccades. Use of a perceptual discrimination task parallels the approach taken in the classical studies of attention, in which displays are presented during periods of steady fixation (see above). Many recent studies have attempted to infer properties of attention during active (saccadic) visual tasks by using measures of change detection (e.g., Mack & Rock, 1998; Rensink, 2000; Simons, 2000; Henderson & Hollingworth, 2003; Droll, Hayhoe, Triesch & Sullivan, 2005). Most change detection tasks, however, rely on experimental paradigms in which the changes are infrequent or unexpected. Thus, performance is influenced not only by limits of perception and memory, but also by strategies that determine how people notice or interpret unlikely or implausible events (Melcher, 2006; Fernandez-Duque & Thornton, 2002). Forced-choice psychophysical measures of discrimination, on the other hand, provide a less ambiguous way to assess the limits on attention during active saccadic tasks because they do not rely on the detection of improbable events. In our study (described in detail below), there was some uncertainty about the precise time and location of the critical stimulus, but it was established that a stimulus would appear, and a response be required, on each trial.

During the free-scanning tasks we tested, the planned pathways of sequences of saccades could not be known in advance. As a result, we did not set out to compare perceptual attention at locations on and off the planned saccadic path, as prior work on saccades and attention has done (e.g., Gersch et al., 2008). Instead we investigated the extent to which the performance of the different primary tasks (counting, pointing, and looking-only) impaired performance of the secondary, orientation identification task.

Experiment 1: Counting

The main primary task in Experiment 1 was to count the number of elements (10 to 19 randomly-positioned dots) shown in the display and report the result at the end of each trial. Counting was chosen because it readily encourages, and benefits from, the use of saccades (Landolt, 1891; Kowler & Steinman, 1977, 1979), and because it can be done with simple, unstructured visual displays. Counting tasks can be performed during steady fixation, although the accuracy of the reports suffers (Kowler & Steinman, 1977, 1979). Note that while we asked subjects to count the dots, we did not attempt to control or to monitor the various estimation or grouping strategies that subjects might have adopted (Liss & Reeves, 1983; Dehaene, Piazza, Pinel & Cohen, 2003; Gelman & Gallistel, 1978; Hurewitz, Gelman & Schnitzer, 2006). Our goal was to keep the subjects engaged in a task that generated saccades, and not to test a particular model of numerical judgments.

Counting was deemed to be the primary task (task instructions will be discussed in more detail below). The secondary visual task – identifying the orientation of a briefly-presented tilted Gabor patch – was used to characterize attention during randomly-chosen intersaccadic pauses. Gabor orientations were separated widely (+/− 22.5 deg), well above discrimination thresholds under conditions of full attention with high contrast targets (Regan & Beverly, 1985). These procedures allowed us to determine how attention to the primary task affected contrast thresholds for orientation identification (Dosher & Lu, 2000a, b; Gersch et al., 2004), and how effects due to attention depended on stimulus contrast.

Orientation identification during counting was compared to that obtained during a steady-fixation baseline (no counting and no saccades). We also tested two other versions of the primary task, namely, (a) counting during steady fixation, with shifts of attention substituting for the saccades, and (b) making comparable patterns of saccades without counting. This allowed us to evaluate the demands on attention made by saccades with respect to those made by the cognitive or perceptual requirements of counting itself. Finding strong suppression of secondary task performance when saccades are made (either during counting, or simply when looking around the display) relative to performance during steady fixation (with or without counting) would indicate that saccades contributed substantially to the overall attentional demand. On the other hand, finding strong suppression of secondary task performance during counting (with or without saccades), and less suppression while making saccades without counting, would support a weaker contribution of saccades to the overall demand on attention relative to the demands imposed by other perceptual or cognitive aspects of the motivating task.

Methods

Stimulus

Stimuli were displayed on a Dell P793 CRT monitor (13 deg × 12 deg; viewing distance 115 cm, 1.46 pixels/min arcs; refresh rate 75 Hz, non-interlaced).

The stimulus for the counting task was a set of 10–19 dark grey dots (8′ radius, 15 cd/m²) presented on a medium gray (54 cd/m²) background. Dots were randomly distributed about the screen (minimum edge separation 3′) within the central 8 deg of the display. The Gabor probe appeared in one of four locations (north, south, east, or west), centered on a location 3 deg from the center of the display. The Gabor probe was generated according to the following:

$$\text{l}(\text{x},\text{y})={\text{l}}_0(1.0+\text{a}sin(2^{\ast }{\text{pi}}^{\ast }\text{f}(\text{x}cos(\theta )+/-\text{y}sin(\theta ))exp(-({\text{x}}^2+{\text{y}}^2)/(2^{\ast }{\sigma }^2)))$$

where f is the spatial frequency (2.24 cycles/deg), l₀ is the mean luminance (54 cd/m²), θ is the orientation (+/− 22.5 deg from vertical), σ the standard deviation of the Gaussian window (0.89 deg), (x, y) the spatial coordinates in the display, and a the amplitude. Maximum luminance was 108 cd/m^2. Amplitude was determined from the contrast (the difference between maximum and minimum luminance divided by twice the mean luminance), and contrast was selected at random from 6 or 7 values ranging from 4 to 96%. Three frames of the Gabor were interleaved with single frames of background luminance. The background luminance frames were replaced with external noise frames in a control condition, to be described below. All frames of the Gabor appeared within a 66 ms time period.

Sequence of display frames

Before each trial the display contained a white fixation cross located either in the center of the display (for conditions requiring saccades) or at a randomly-chosen location that would later be occupied by one of the dots (for conditions requiring steady fixation). The subject started each trial when ready by pressing a button. Two hundred ms later the array of dots appeared and remained on throughout the 3 sec trial.

For conditions requiring saccades, the Gabor probe appeared during a randomly-chosen fixation pause. The fixation pause was selected by detecting on-line the first saccade to occur after a randomly-selected delay (600 ms – 2 s) following the start of the trial. The Gabor then appeared after an additional random delay of 25–100 ms so that the Gabor would appear at various times after saccadic offset but before the next saccade to be made (this was verified by off-line data analyses). For conditions requiring steady fixation, the timing of the Gabor was the same except that the Gabor appeared immediately after the first randomly-selected delay (600 ms–2s). Note that due to the random choice of fixation position at the start of the steady fixation trials, the retinal eccentricities of the Gabors (which takes into account both their position on the display and the position of fixation) were the same across conditions (see Table 2 for verification). Trials in which the Gabor appeared during any part of a saccade, regardless of the experimental condition, were eliminated. Figure 1 shows the display (dots and Gabor probe) along with representative eye movement traces from trials requiring saccades and from trials requiring steady fixation.

Table 2.

Eccentricity of the Gabor in Experiment 1 (Counting) and Experiment 2 (Pointing).

	Gabor Eccentricity (min arc)
	Mean (SD)	N
Experiment 1: Counting
Counting with saccades
JT	222 (95)	734
SDK	209 (74)	995
ES	233 (108)	567
GT	221 (96)	796
Mean	221
Saccades only
JT	241 (104)	656
SDK	212 (82)	930
ES	252 (117)	554
GT	241 (109)	649
Mean	237
Fixate
JT	226 (80)	868
SDK	215 (79)	1021
ES	214 (86)	883
GT	229 (86)	670
Mean	221
Fixate and count
JT	196 (77)	879
SDK	216 (85)	992
ES	215 (85)	619
GT	222 (89)	774
Mean	212
Experiment 2: Pointing
Pointing with saccades
AS	182 (28)	383
AW	184 (30)	662
LM	183 (28)	669
SDK	198 (52)	845
JW	196 (62)	482
GT	182 (23)	391
Mean	188
Saccades only
AS	183 (26)	367
AW	182 (26)	629
LM	184 (26)	657
SDK	192 (52)	791
JW	199 (67)	423
GT	182 (18)	356
Mean	187
Fixate
AS	182 (33)	274
AW	182 (21)	483
LM	182 (19)	607
SDK	181 (20)	852
JW	204 (78)	479
GT	181 (17)	359
Mean	185

Figure 1.

Representative eye movements during trials requiring saccades (left) and steady fixation (right) shown both superimposed on the displays (top) and also as a function of time (bottom). The Gabor is included in the display to show its appearance relative to a representative aggregate of dot stimuli. In the experiment the Gabor was presented briefly (66 ms) at a randomly chosen time.

Immediately after the trial was over, an arrow appeared that disclosed the location of the Gabor. This ‘post-cue’ was included in all conditions to remove uncertainty about the location on which to base the response (Dosher & Lu, 2000a, b). The subject then reported by means of button presses: (1) the number of dots (in conditions requiring counting), and (2) the tilt (left or right) of the Gabor. Feedback was given after each trial by showing the display frame containing the Gabor for 500 ms and by announcing the number of dots displayed.

Experimental conditions

There were 4 experimental conditions tested in separate experimental sessions (75–100 trials each), and each condition was typically tested once (75–100 trials each) per day. The 4 conditions used the same displays but different instructions: (1) Steady fixation (baseline): subjects maintained a steady line of sight on the initial fixation cross, and continued fixating the same location when the trial started and the fixation cross was replaced by the random dots. (2) Counting with saccades: Subjects used any pattern of saccades they chose while counting the dots. At the end of the trial, the number of dots was reported followed by the report of the orientation of the Gabor. (3) Counting during steady fixation: Steady fixation was maintained, as in the baseline condition, and subjects counted as best they could by shifting attention instead of shifting the line of sight (Kowler & Steinman, 1977, 1979). (4) Saccades only: Subjects inspected the dots with saccades, attempting to reproduce the approximate pattern of saccades they used during counting, without actually counting or reporting the count at the end of the trial. (Note: As will be shown below, subjects had no difficulty producing saccadic patterns that resembled the patterns used during counting.) A total of 700 – 1100 trials/subject were run for each of the 4 experimental conditions listed above.

Task prioritization

For conditions in which counting was required (either with or without saccades), counting was defined as the primary task. Subjects were told to try to continue counting without interruption either in anticipation of, or in reaction to, the appearance of the Gabor.

Variations

A few variations of the basic experiment were included: (1) An easy perceptual discrimination (500 Hz vs. 1000 Hz tone, 200 ms) was tested in place of the report of the Gabor. This control was included to verify that any errors in reports about the Gabor were not due to the need to deliver two responses at the end of the trial (the dot number and the Gabor report); (2) Effective Gabor contrast was increased by 67% by presenting 5 consecutive frames of the Gabor (rather than 3 frames interleaved with background luminance); (3) Gabors were oriented vertically with a spatial frequency either 1.7 or 2.7 c/d. The task was to report which frequency (the higher or lower of the pair) was presented; (4) Single frames containing high-contrast visual noise (diam 80′) were interleaved with the frames of the Gabor, and shown at all 4 of the possible Gabor locations; (5) The post-trial arrow disclosing Gabor location was eliminated and instead observers reported Gabor location (N, S, E, W) rather than orientation. Subjects ran in 1500–2000 trials for each of these 5 additional experimental variations.

Subjects

Four volunteers were tested (JT, SDK, GT, ES), all with uncorrected normal vision, and all naïve as to the purpose of the experiment. Two additional naïve observers (AW and LM) were tested in the version of the experiment with interleaved noise frames.

Eye movement recording and analysis

Horizontal and vertical movements of the right eye were recorded using a Generation IV Double Purkinje Image Tracker (Crane & Steele, 1978). The left eye was covered and the head was stabilized with a dental biteboard. The tracker’s voltage output was fed on-line through a low pass 100 Hz filter to a 12-bit analog to digital converter (ADC). The ADC, controlled by a PC, sampled the eye’s position every 2 ms. The digitized voltages were stored for analysis. The time of appearance of the Gabor was monitored by a photocell, which received a signal from a small white square on the display, out of the subject’s view, whenever the Gabor appeared. The output of the photocell was fed to a channel of the ADC and recorded along with the eye position to ensure accurate temporal synchronization between the stimulus display and the eye movement recording.

Tracker noise level was measured with an artificial eye after the tracker had been adjusted so as to have the same first and fourth image reflections as the average subject’s eye. Filtering and sampling rate were the same as those used in the experiment. Noise level, expressed as a standard deviation of position samples, was 0.4′ for horizontal and 0.7′ for vertical positions. Recordings were made with the tracker’s automatically movable optical stage (auto-stage) and focus servo disabled.

The beginning and ending positions of saccades were detected off-line by means of a computer algorithm employing an acceleration criterion (Gersch et al., 2004). The value of the criterion was determined empirically for individual observers by examining a large sample of analog recordings of eye positions. Saccades as small as the microsaccades that may be observed during maintained fixation (Steinman, Haddad, Skavenski, & Wyman, 1973) could reliably be detected by the algorithm.

Trials were eliminated if tracker lock was lost during the trial (1.3%) or if the Gabor appeared during a saccade (10.2%). The eliminated trials were distributed approximately equally across all experimental conditions.

Performance on the orientation identification task was analyzed by fitting Weibull functions to the psychometric data using the ‘psignifit’ algorithm (Wichman & Hill, 2001). Contrast thresholds were taken at the 75% correct level. Given that performance in many cases never reached this level even at maximum contrast, we will also report the peak performance at the 100% contrast level.

Results

Effects of attention on orientation identification

Figure 2 contains the main result of this experiment: the proportion of correct identifications of Gabor orientation as a function of contrast under the 4 conditions tested.

Figure 2.

Perceptual judgments of Gabor orientation as a function of Gabor contrast during counting, looking and steady fixation. Proportion correct orientation identifications (tilt left vs. tilt right) are shown for each subject and for the 4 conditions tested: FIX: steady fixation baseline. FIX/COUNT: Counting during steady fixation. SAC: Saccades only with no counting. SAC/COUNT: Counting with saccades. Data are fit with Weibull functions using the Psignifit algorithm (Wichman & Hill, 2001). Psychometric functions are based on 50–200 observations/point.

Counting resulted in substantially poorer identification performance than found for the steady fixation baseline, even at the highest Gabor contrast tested. Orientation identification during counting with saccades did not exceed about 75% correct, which means (given that chance performance was 50% correct) subjects failed to identify the orientation of the highest contrast Gabor on about half of the trials. The poor performance at the highest Gabor contrast, and the fact that there was little improvement with increases in contrast above 32%, suggests that the suppression of the Gabor during counting was not due exclusively to a reduction in the effective Gabor contrast due to inattention (sometimes referred to as a reduction in contrast gain; e.g., Morrone et al., 2004; Huang & Dobkins, 2005). The pattern of results suggest that failures to identify the high contrast Gabors was due in part to an overall response suppression that cannot be overcome completely by increases in stimulus contrast.

Losses in orientation identification while counting during maintained fixation followed the same pattern, but were smaller than losses while counting with saccades. Performance reached about 85% correct at the highest contrast tested, better than obtained for counting with saccades, but still considerably poorer than the baseline, steady fixation condition.

Making saccades without counting had modest effects on orientation identification and revealed a more complex pattern of losses. Two subjects (ES, GT) showed a loss only at the lower Gabor contrasts, one (SDK) only at high contrasts, and one (JT) at both.

The pattern of results in Figure 2 shows that, using orientation identification as a benchmark of attention, counting was more demanding of attention than the act of making saccades across the display. Counting could, on about half the presentations, render the orientation of even high contrast Gabors nearly impossible to perceive.

Various checks and controls

Why was orientation identification so poor at high stimulus contrasts during counting? We made additional observations that allowed us to reject a few possibilities unrelated to interference from attention to the primary counting or saccadic tasks:

1. Remembering two reports: Poor orientation identification during counting was not due to the need to remember two reports until the end of each trial. Subjects performed well (>90% correct) when, instead of Gabor orientation, they had to report which of two highly discriminable tones (500 vs. 1000 Hz) was presented during the counting trials. (Additional evidence showing that the number of required reports was not critical will be described in the section Detection and Localization, below).

2. Insufficient Gabor contrast: Orientation identification remained poor when effective Gabor contrast was increased by 67% by presenting 5 consecutive frames of the Gabor (Fig. 3a), instead of 3 frames interleaved with background luminance.

3. Characteristics specific to encoding of orientation: The pattern of results was similar when instead of orientation, subjects reported the spatial frequency (1.7 vs 2.7 c/d) of a vertically-oriented Gabor (Fig. 3b).

4. Differences in saccadic patterns: The characteristics of the saccadic patterns (sizes and pause durations) were quite similar with and without counting (Table 1). The main difference, consistent across subjects, was that saccade size was smaller during counting (average 82′ across the 4 subjects) than when saccades were made without counting (average 111′ across the 4 subjects). As a result, the average retinal eccentricity of the Gabor was smaller during counting (221′) than when only making saccades (average=237′) (see Table 2), a small difference that, if anything, would favor performance in the counting condition. Thus, the poorer performance during counting cannot be attributed to overt differences in the patterns of saccades. Note that Table 2 also shows that average retinal eccentricity of the Gabor was comparable across all conditions (including steady fixation).

5. The experiment was repeated with single frames of high-contrast, spatially-limited visual noise interleaved with frames of the Gabor. Noise frames were presented at all 4 possible Gabor locations. This condition was included for compatibility with prior work on saccades and attention (Gersch et al., 2004), and prior work where attention during fixation was manipulated by means of location cues (Dosher & Lu, 2000a, b). Results from the two subjects tested shows that adding visual noise frames during the intersaccadic pauses did not abolish the suppression due to counting (Fig. 3c).

Figure 3.

Additional experiments testing perceptual judgments about the Gabor during counting, looking or steady fixation. In each graph proportion correct reports are shown as a function of Gabor contrast. A Orientation identification (tilt left vs. tilt right) when effective Gabor contrast was increased by 67% relative to the contrast in the basic experiment. Effective contrast was increased by showing additional frames of the Gabor during the display interval (see Methods). B. Frequency discrimination (1.7 vs 2.7 c/d) of a vertical Gabor. C. Orientation identification (tilt left vs. tilt right) when frames of the Gabor were interleaved with frames of visual noise (see Methods). For each experiment performance is shown for the two subjects and the 4 conditions tested: FIX: steady fixation baseline. FIX/COUNT: Counting during steady fixation. SAC: Saccades only with no counting. SAC/COUNT: Counting with saccades. Data are fit with Weibull functions using the Psignifit algorithm (Wichman & Hill, 2001). Psychometric functions are based on about 30–80 observations/point.

Table 1.

Characteristics of saccades in Experiment 1 (Counting) and Experiment 2 (Pointing).

	ALL SACCADES AND INTERSACCADIC PAUSES
	Saccade Size (min arc)		Pause Duration (ms)
	Mean (SD)	N	Mean (SD)	N
Experiment 1: Counting
Counting with saccades
JT	87 (63)	6615	256 (151)	5881
SDK	85 (51)	6485	359 (197)	5490
ES	75 (47)	4743	279 (170)	4176
GT	82 (55)	7912	241 (107)	7116
Mean	82		284
Saccades only
JT	111 (85)	6208	234 (137)	5552
SDK	91 (56)	5945	353 (215)	5015
ES	118 (65)	3922	325 (176)	3368
GT	125 (75)	6117	236 (110)	5468
Mean	111		287
Experiment 2: Pointing
Pointing with saccades
AS	60 (42)	2308	411 (183)	1925
AW	58 (41)	4275	290 (178)	3613
LM	70 (32)	3639	459 (196)	2970
SDK	57 (37)	4205	505 (257)	3360
JW	61 (41)	3030	405 (171)	2548
GT	66 (41)	2978	317 (126)	2587
Mean	62		407
Saccades only
AS	82 (54)	2872	296 (156)	2505
AW	70 (51)	4873	305 (133)	4244
LM	80 (43)	4956	307 (123)	4298
SDK	67 (46)	4072	375 (189)	4072
JW	78 (53)	4021	236 (106)	4021
GT	68 (46)	3480	229 (87)	3124
Mean	74		291

Effects of attention on detection and localization

The poor performance in the orientation identification task during counting could have been due to a failure to detect the Gabor, or, alternatively, to difficulty creating or maintaining an accurate representation of its features. The distinction between detection and identification comes up often in the attention literature, with some studies showing that a primary task interferes with the ability to notice (i.e., detect) unexpected stimuli (Mack & Rock, 1998; Bonneh et al., 2001), and others showing that detection (and localization) are far less demanding of attention than identification (Sagi & Julesz, 1985; Huang et al., 2007). In agreement with these latter observations, one of our subjects (JT) confidently asserted after the experimental data were collected that he usually knew where the Gabor was even though he could not always determine its properties.

To distinguish detection from identification, the secondary task was changed to one of localizing the Gabor. All other aspects of the experiment remained the same except for the post-trial display, where the arrow disclosing Gabor location while waiting for the subject’s report was removed.

Figure 4 confirms JT’s impressions and shows that reports of the location of the Gabor (N,S,E, or W) were much more accurate than reports of its orientation. Two subjects (JT and SDK) showed only a small loss relative to baseline in any of the experimental conditions. ES and GT showed losses during counting relative to the baseline conditions, but not as severe as found for the orientation identification task, particularly at the high Gabor contrasts where localization performance in the 4AFC task reached 85% correct.

Figure 4.

Perceptual judgments of Gabor location (N, S, E or W) as a function of Gabor contrast during counting, looking or steady fixation. Proportion correct localizations are shown for each subject and for the 4 conditions tested: FIX: steady fixation baseline. FIX/COUNT: Counting during steady fixation. SAC: Saccades only with no counting. SAC/COUNT: Counting with saccades. Data are fit with Weibull functions using the Psignifit algorithm (Wichman & Hill, 2001). Psychometric functions are based on about 50–70 observations/point.

The high level of performance in the localization task shows that poor orientation identification during counting cannot be explained solely by failure to notice the Gabor stimulus, or by confusions about its location during the trial. The main effect of counting is an interference with the representation of the features of the Gabor.

Intersaccadic pause durations during counting

Another indication that the presence of the Gabor was registered by the visual system came from an analysis of the duration of pauses between saccades. Figure 5a, b, based on data from the identification task, shows that the duration of the intersaccadic pause containing the Gabor increased with Gabor contrast. This was true even for trials in which the Gabor orientation was reported incorrectly (Fig. 5a). The slopes of the functions in Fig. 5a, b were significantly greater than 0 (average slope=.67, t=6.51, p=.000043) with pause durations increasing by 25–100 ms over the full range of contrasts tested.

Figure 5.

Mean duration of the critical intersaccadic pause containing the Gabor as a function of Gabor contrast for 4 conditions: A. Identification of Gabor orientation while counting with saccades. B. Identification of Gabor orientation while making saccades only. C. Localization of the Gabor while counting with saccades. D. Localization of the Gabor while making saccades only. Within each graph data are shown separately for trials with correct (solid symbols) and incorrect (open symbols) judgments about the Gabor. Best fit straight lines are shown. Error bars are +/− 1 SE.

If the longer intersaccadic pause durations were due solely to the transient flash of the Gabor, then the pause durations also should have increased during the localization task. Figure 5c, d shows that in the localization condition the slopes of the functions relating Gabor contrast and intersaccadic pause duration were shallower (mean=.06) and not significantly different from 0 (t=.42, p=.68). This leads to an interesting speculation, namely, that the increased pause durations during the orientation task was not an automatic reaction to the appearance of the Gabor, but rather a strategic response tailored to the orientation task, an attempt to briefly interrupt counting, or looking, in order to devote more attention to the time-consuming task of identifying the orientation of the Gabor.

Counting and task tradeoffs

We also examined the reports of the number of dots in the display. Performance was represented by the correlation between the actual number of dots displayed and the reports of dot number. The correlation provides a reasonable measure of how much useful information was obtained from the dot displays across the different conditions.

Figure 6a shows that the correlations were substantially higher when saccades were used (triangles) than during steady fixation (circles), as was expected. Correlations did not vary systematically with Gabor contrast, suggesting that detection of the Gabor (which was more likely at higher contrasts; Fig. 4) did not interfere with obtaining information from the dot displays. The same pattern of results was obtained regardless of whether the Gabor was reported correctly or incorrectly. Figure 6a also shows that for 3 subjects, correlations for the counting task did not differ as a function of the type of Gabor report (orientation or location) that was required.

Figure 6.

Performance in the counting task. Data are shown separately for the 4 subjects tested (A): Correlations (R²) between the number of dots displayed and the report of dot number as a function of the contrast of the Gabor probe. Performance is shown separately for the two types of oculomotor instructions (saccades or steady fixation) and when counting was paired with either type of Gabor report (orientation or location). The arrows along the righthand ordinate show performance during trials when counting alone during steady fixation (blue) or with saccades (black) without any concurrent reports about the Gabor.

(B): Same results in the form of Attentional Operating Characteristics (AOC’s), with counting performance (R²) on the abscissa and peak Gabor performance (percent correct at 100% contrast) on the ordinate. Horizontal lines represent baseline Gabor performance (identification, solid; localization, dashed) when the Gabor tasks were done alone during steady fixation. Vertical lines represent counting performance when the task was done alone either during steady fixation (blue) or with saccades (black). The intersections of the pairs of vertical and horizontal lines are the independence points, showing expected performance when the tasks are done concurrently if there were no mutual interference. To make clear which task pairs corresponded to each of the 4 independence points, small black symbols are superimposed on the independence points in subject SDK’s AOC. The task pairs denoted by the 4 symbols are as follows: filled circle: counting during steady fixation, Gabor orientation judgments; open circle: counting during steady fixation, Gabor location judgments; filled triangle: counting with saccades, Gabor orientation judgments; open triangle: counting with saccades, Gabor location judgments.

These results show that the higher levels of performance obtained in the Gabor localization task did not come as a result of a tradeoff with counting. There was, however, evidence for other tradeoffs. The arrows along the righthand ordinates of Figure 6a show counting performance for experimental sessions in which counting alone was tested, without any judgments about the Gabor. In cases where counting was performed using saccades (black arrows), correlations for counting alone were either the same as (subjects JT and GT) or slightly higher than (ES and SDK) those obtained when both the counting and Gabor tasks were done in the same trials. When counting was carried out during steady fixation (blue arrows), correlations were reduced relative to the counting-along baseline by the addition of the Gabor task for all subjects except GT.

The losses in both Gabor judgments and counting performance when both tasks were done together, relative to performance when either task was done alone, are shown in the form of Attentional Operating Characteristics (Sperling & Dosher, 1986) in Fig. 6b. These results confirm the mutual interference of the counting and the Gabor judgments. The fact that small losses were observed even in the primary counting task suggests that there were few resources to spare, and that any attempts to improve Gabor judgments would have come at the expense of counting.

Experiment 2: Pointing

The perceptual losses found for orientation identification during counting were substantial. Would such losses be limited to a task such as counting, with a demonstrable cognitive load (Gelman & Gallistel, 1978; Hurewitz, Gelman & Schnitzer, 2006; Dehaene et al., 2003), or would they obtain more generally with any purposeful task involving saccades? To address this question we performed a second experiment that used a different and less cognitively-demanding primary task to motivate the saccades. The task we used was pointing to sets of dots using a visible cursor controlled by moving an unseen pen across a digitizing tablet. Pointing, like counting, benefits from the use of saccades (Biguer, Prablanc, & Jeannerod, 1985).

Method

Stimulus

The stimulus for the pointing task was the same as for the counting task except that fewer dots (7–16) were displayed per trial.

Experimental conditions

Steady fixation (baseline) was the same as in Experiment 1. Pointing-with-saccades: Subjects used the pen of a digitizing tablet to move a visible cursor from one dot to the next. The tablet and pen were out of view, located on a shelf mounted below a tabletop in front of the subject. No constraints were imposed on the cursor movements, other than to make a reasonable attempt to hit each dot. For technical reasons (incompatibilities in software) we did not record the output of the digitizing pen, but did verify by inspection of the movements of the visible cursor on the display screen that subjects were performing the task. Saccades only: Subjects were asked to look from one dot to the next, as if they were pointing to the dots with the cursor. A total of 1000–2700 trials were run/subject, of which 9.1% were eliminated due either to loss of tracker lock or to a Gabor occurring during a saccade.

Subjects

Six subjects were tested, including 4 who participated in the counting experiment (SDK, GT, AW, LM) and 2 new subjects (AS, JW). Other than author JW, all the subjects were naïve as to the purpose of the experiments.

Results

Characteristics of saccades during pointing

Table 1 summarizes characteristics of the saccades. During the pointing task, saccades were smaller and intersaccadic pause durations longer than during counting. As was the case for counting, saccades were larger than when solely looking around the display. Five of the 6 subjects showed longer intersaccadic pauses during pointing than when solely looking. Intersaccadic pause durations once again increased with Gabor contrast when looking only (Fig. 7b). Increases in pause duration as a function of contrast were not found during pointing (Fig. 7a), perhaps because the intersaccadic pause durations were already prolonged due to the requirements of the pointing task.

Figure 7.

Mean duration of the critical intersaccadic pause containing the Gabor as a function of Gabor contrast for 2 conditions: A. Identification of Gabor orientation while pointing. B. Identification of Gabor orientation while making saccades only. Within each graph data are shown separately for trials with correct (solid symbols) and incorrect (open symbols) judgments about the Gabor. Best fit straight lines are shown. Error bars are +/− 1 SE.

Effects of the tasks on orientation identification

The effect of pointing on orientation identification is shown in Fig. 8. Four subjects (JW, SDK, GT and LM) showed losses during pointing, and while looking-only without pointing. The magnitude of the losses during pointing was less than during counting (Figs. 2 and 3), particularly for LM. Subjects AW and AS showed little or no effect of either pointing, or looking-only, on orientation identification.

Figure 8.

Perceptual judgments of Gabor orientation as a function of Gabor contrast during pointing, looking or steady fixation. Proportion correct orientation identifications (tilt left vs. tilt right) are shown for each subject and for the 3 conditions tested: FIX: steady fixation baseline. SAC: Saccades only. POINT: Pointing to the dots using a visible cursor. Data are fit with Weibull functions using the Psignifit algorithm (Wichman & Hill, 2001). Psychometric functions are based on about 50–100 observations/point.

Note that two of the subjects with good orientation identification during the pointing task (LM and AW) had shown substantial losses in orientation identification during counting in Experiment 1 (Fig. 3c). Both also showed losses in the looking-only condition, with LM showing losses in Experiment 2 (Fig. 8) and AW in Experiment 1 (Fig. 3c). Thus, their good performance during the pointing task can be attributed to differences in the distribution of attention in the visuomotor tasks tested (pointing or looking-only), rather than to a global immunity from interference from any of the primary tasks.

Summary of the attentional demands of the different tasks

The results presented so far showed substantial losses in orientation identification during counting, smaller losses during pointing, and smaller losses still while making saccades by themselves.

Figure 9 provides a summary of the effects of counting, pointing and saccades on orientation performance. Fig. 9a shows mean contrast thresholds in two conditions, baseline steady-fixation, and saccades-only. Thresholds were averaged over all 14 instances where orientation identification was measured under these two conditions (Figs. 2, 3a, 3c and 8). Analyses of thresholds are limited to these two conditions because orientation performance during either counting or pointing usually did not reach levels high enough in a sufficient number of cases to allow meaningful threshold comparisons.

Figure 9.

A. Mean contrast threshold for orientation identification for the baseline steady fixation condition and for the conditions requiring saccades only (no counting or pointing). Thresholds (75% correct) were obtained from 14 psychometric functions (see Figs. 2, 3a, 3c and 8). B. Mean performance at 100% contrast for 5 conditions: baseline steady fixation, saccades only (no counting or pointing), counting during steady fixation, counting with saccades, and pointing.

Fig. 9a shows that average contrast threshold increased from 16% in the steady fixation baseline to 22% while making saccades. The magnitude of this increase in average threshold (37%) is in the range of the effects of saccades on attention found during tasks where saccades were made along specified paths (Gersch et al., 2004), as well as the effects of attentional cues measured during steady fixation (Dosher & Lu, 2000a, b; Carrasco, Penpeci-Talgar, & Eckstein, 2000).

Figure 9b summarizes the effects of the different tasks on peak performance (performance at 100% Gabor contrast). Of the different primary tasks, counting led to the poorest orientation identification, and looking-only to the best. Performance when counting during steady fixation was better than when counting with saccades. Performance during pointing was better than either counting task, as well as more variable due to the differences among subjects (see Fig. 8). Note that both measures, threshold and peak performance, show that the attentional demands of purposeless saccades were significant, but smaller in magnitude than when the saccades were motivated by a purposeful task.

Saccadic patterns and perceptual performance

Can any of the variation in perceptual performance during the primary tasks be traced to overt differences in characteristics of the saccades? Figure 10 contains scatterplots showing the relationship between the perceptual measures in the orientation task (thresholds and peak performance) and two measures of saccades: average saccade size and average intersaccadic pause durations. Each datum point in the scatterplot represents the results of one subject tested with one of the primary tasks studied (counting, pointing, or saccades-only).

Figure 10.

Scatterplots showing the relationship between psychophysical and oculomotor performance. Left: contrast thresholds (75% correct) vs. either the average duration of the intersaccadic pause (top) or the average size of the saccade (bottom). Right: Peak performance (percent correct at 100% contrast) vs. either the average duration of the intersaccadic pause (top) or the average size of the saccade (bottom). Performance is shown for 3 conditions tested in Experiments 1 and 2: Counting (red diamonds), pointing (green asterisks), or looking-only (blue circles). The individual data points within each scatterplot correspond to performance for a given subject and condition. Correlation coefficients (R) are also shown.

Figure 10 shows weak relationships between the saccadic measures and perceptual performance. There was a small tendency for lower levels of perceptual performance to be associated with either longer intersaccadic pauses (top) or shorter saccades (bottom). Neither of these saccadic characteristics should have hurt identification of the Gabor (if anything, longer pauses should have helped). We suspect that the longer pauses, and perhaps the shorter saccades, did not impair Gabor performance, but rather are indicators of increased attention to the primary task. Thus, the correlations are consistent with the view that the more difficult primary tasks were associated with less attention to the Gabor targets.

General Discussion

Much of what we know about attention, and the relationship between attention and saccades, is based on studies in which the patterns of eye movements are restricted in the interests of achieving precise experimental control. In the case of psychophysical investigations of attention, one or more stimuli are typically presented briefly to the steadily fixating eye. In the case of investigations of the link between attention and saccades, a single saccade, or a sequence of saccades, are made to specified targets. As useful as such experimental approaches have been, and will continue to be, for testing hypotheses about attention, if we are to accept the assumption that the processes discovered are relevant to attention and saccades in natural viewing, then it becomes necessary to attempt comparable psychophysical investigations when the constraints on eye movements are removed.

The present work removed some of the conventional constraints by studying attention during the pauses between saccades while subjects were engaged in different active visual tasks, namely, counting, pointing, or simply looking around the display. These active tasks were performed with patterns of saccades that were not dictated by the experimenter, but were chosen subject only to the needs of the motivating task. Attention was evaluated during intersaccadic pauses by means of a psychophysical task (identifying the orientation of a briefly-presented Gabor probe).

We found that all of the active tasks we studied, including looking-only, led to losses in performance of the secondary, orientation identification task. Finding such performance losses shows that the attentional requirements attached to the control of saccades is not limited to tasks where saccadic targets were specified by the experimenters, and can apply as well to saccades made along freely-chosen paths. (There were differences across tasks and individuals in the losses associated with saccades; this issue will be taken up later in the discussion.)

Saccades made less of a demand on resources than other perceptual or cognitive aspects of the tasks. Orientation identification was poorer during counting, either with or without saccades, than during either looking or pointing. The pattern of losses during counting was particularly striking: Orientation identification reached a level of only 75–80% correct at a moderate contrast (32%), and did not improve with further contrast increases. These losses at high stimulus contrast are similar to those found in experiments where discrimination thresholds were measured during performance of a competing task (e.g., Morrone et al., 2004; Huang & Dobkins, 2005). In cases like these, where the loss of performance appears to persist despite increasing contrast, losses can be due to any number of processes, including failure to accurately encode the features of the stimulus, or failure to maintain a representation of the stimulus long enough for the perceptual decision to be made.

Detection and localization

In contrast to the poor identification of the orientation of the Gabor during counting, the ability to detect and locate the Gabor was largely spared. Detection and localization require less attention, and less processing time, than identification of stimulus features, a result described by Sagi & Julesz (1985), and confirmed recently under different conditions by Huang, Treisman & Pashler (2007).

The relatively good detection/localization performance was consistent with the increased intersaccadic pause duration found for the high contrast Gabors (see also Henderson & Hollingworth, 2001). The transient appearance of the Gabor could have contributed to increasing pause duration, but was not the only factor responsible because pause duration was not inflated by the appearance of the Gabors during the perceptual localization task. Thus, the increase in intersaccadic pause duration during the identification task was associated with task difficulty, and was perhaps a sign of an attempt to shift some processing resources to the Gabor in order to better identify its features. These increases in pause duration did not improve orientation performance, presumably because the Gabor was gone, along with any persisting memory of its features, before orientation could be determined.

Attention and saccades

The pointing and the looking-only tasks produced clear losses in orientation performance, although less than those produced by counting. These effects are consistent with the view that attention is engaged by the need to plan and program freely-chosen patterns of saccades. ¹

The magnitude of the effects of looking-only, or pointing, differed across experiments and observers, and there were instances where either pointing or looking did not impair orientation performance (Fig. 8, AW and AS; Fig. 3, ES and LM). Individual differences, even large ones, are certainly not without precedent in studies of attention (Shaw & Shaw, 1977), even studies of attention in monkey (Boudreau, Williford & Maunsell, 2006). The individual differences we observed were not correlated with overt characteristics of saccades, such as the average sizes or intersaccadic pause durations. Thus, there was no evidence that good identification performance while looking or pointing was achieved by sacrificing the control of the saccades. And, given that all observers showed substantial losses in orientation performance during counting, and comparable identification performance in the steady fixation baseline, we do not believe that the individual differences stemmed from phenomena related to the orientation identification task itself. This leaves us with the issue of attentional strategies, and how attention was distributed while making saccades.

Both the pointing and the looking-only tasks may have allowed flexibility in the choice of attentional strategies. In a prior study of attention preceding single saccades, Kowler et al. (1995) found that it was possible to voluntarily allocate some attention to non-goal locations without much cost either to saccadic accuracy or saccadic latency (see Attentional-Operating Characteristics in their Fig. 11). These findings (and comparable ones applying to smooth pursuit; Khurana & Kowler, 1987) show that a given pattern of eye movements can be produced either by allocating full attention to the target, or by sparing some attention in order to analyze non-target stimuli.

Flexibility is interesting because of what it implies about the underlying relationship between perceptual attention and saccades. All that eye movements may ever require in order to maintain an acceptable level of spatial and temporal accuracy is a modest alteration in the distribution of attention, large enough so that the peak level coincides with the saccadic target during either the entire latency interval of the saccade, or some critical portion of the latency interval. With such a distribution of attention, the spatial goal of the saccade can be determined by a “winner take all” network that finds the locus of peak attentional strength (Koch & Ullman, 1985; Tsotsos, 1990). Such networks would make it possible to allocate considerable attention to locations other than the saccadic goal, with little or no cost to saccades. Recently, Gersch et al. (2008) found evidence supporting this view in a study of attention during saccadic sequences. They found that when saccades were made along color cued paths, perceptual attention could be distributed along the path to locations both ahead and behind the immediate saccadic goal without disrupting the saccadic pattern. (Peak attentional strength remained at the saccadic goal.) It is possible that the subjects in our study who showed good performance on the orientation task during pointing or looking may have found a way to distribute attention efficiently, so that the peak strength remained at the immediate saccadic goal, but the distribution of attention across the display was sufficient to support the orientation task.

Counting

Counting was another matter. Performance on the orientation task in all observers suffered during counting, even at high stimulus contrasts. The flexible strategies of distributing attention, which might have been adopted by some subjects during pointing or looking, were evidently no longer available. This was a surprising outcome in that counting a display of dots makes only modest demands on pattern recognition systems, and thus would appear to present little basis for competition with identification of the features of the Gabor.

What aspects of the counting task were responsible for the grab of resources? Selection of a dot or a group of dots as a saccadic target, or shifts of spatial attention to the targets, could have contributed to the losses. But given that target selection was also required during looking or during pointing, it is not likely that target selection accounted for all of the losses during counting. We can also rule out processes related to generating and remembering a report of the number of dots since we did not find comparable losses in other tasks (detection and localization; or auditory discrimination) that also required a second response.

This leaves us with the operations performed at stages in between perceptual selection of a local group of dots and the generation of the counting report. These operations include estimating the number of dots in a group, keeping track of the running sum, and retrieving and generating the count-words (see, for example, Dehaene et al., 2003; Gelman & Gallistel, 1978; Hurewitz et al., 2006; Liss & Reeves, 1983). In addition, demands are made on both memory and executive processes to keep track of which dots have already been counted, and ensure the process continues in an effective way over time.

These operations may have been demanding enough to use up all the perceptual or cognitive resources available during a given fixation pause (Lavie et al., 2004; Epelboim & Suppes, 2001), and thus prevent the maintenance of a representation of the Gabor in a relatively durable form (see the following section for a related point). But an overload of fixed-capacity processors is not the only explanation for the difficulties in identifying the Gabor. An alternative explanation involves attentional strategies. Specifically, as part of a strategic adjustment to the difficulty of counting, attentional filters may have been invoked to shield visual or cognitive analyzers from inputs unrelated to the primary counting task. Difficult or demanding primary tasks, such as counting, may encourage such filters to spring into action even when there is capacity to spare. Other circumstances might as well. For example, studies have found that even a relatively low-demand primary task could encourage a conservative strategy of ignoring the secondary task under circumstances where the probability of a secondary target appearing is low (Droll et al., 2005; Boudreau et al., 2006). The relative roles of attentional strategies and processing capacity in accounting for dual task performance has been a fundamental issue in evaluating dual-task performance (e.g., Navon & Gopher, 1979; Sperling & Dosher, 1986), and needs to be re-examined in the context of attempts to understand the role and uses of attention in active and complex tasks. Although we cannot definitively distinguish the roles of capacity and strategies in determining our results, the fact that there were some losses in the primary counting task (Fig. 6), accompanying the losses in the Gabor tasks, is an indication that performance limits had been approached, and that any inducements to devote more time or attention to the Gabor would have led to even more errors in the counting task.

The role of processing time

Our results can be related to a finding from a very different domain. Supèr, Spekreijse & Lamme (2001a, b) found a correlation between the late component of activity in V1 in monkey and the performance of perceptual tasks. In these studies monkeys had to make a saccade to a motion-defined or a texture-defined target that appeared in a randomly-chosen location in the display. The probability of making a saccade to the correct location (i.e., the probability of correctly finding the target) was positively correlated with the strength of the late V1 response to the target figures. The late V1 response began about 100 ms following stimulus onset, and could last as long as 1–2 seconds, even when the target was no longer present on the screen. The late activity in V1 was attributed to feedback signals from higher-level cortical areas.

The relevance of late and persisting neural signals to perception indicates the importance of preserving representations of selected stimuli for periods of time at least as long as the duration of a typical fixation pause. One consequence, however, of relying on extended processing time is that it enhances the opportunity for competition among concurrent tasks. Persisting neural signals may remain in a fragile state, subject to disruption by other targets or tasks present at the same time (Supèr et al., 2001b; also, Sigman & Dehaene, 2005). Such disruption of the representation of the Gabor may have occurred in our experiments due to the attention to the dot stimuli and to the operations connected to performing the different primary tasks. This disruption would account for failures to identify the Gabor, even at high stimulus contrasts. It would be of considerable interest to know whether, in a dual-task behavioral paradigm comparable to ours, maintained attention to a primary task is able to abolish the late-V1 response to newly-presented secondary targets, and with it, the ability of the animal to identify critical features of these targets.

Attentional strategies during active tasks

Studying attentional bottlenecks during free-viewing, where the spatial and temporal patterns of saccades are not constrained by instruction, is interesting for obtaining some insight about strategies of compensation for the bottlenecks. Free-viewing permits a re-allocation of resources in at least two ways, namely, by allowing intersaccadic pause duration to increase long enough to accommodate multiple task demands, or by commanding shifts of attention or shifts of gaze to a new place.

We found that tasks connected to such reallocation of resources made modest demands. These included the saccadic shifts of gaze to look around the display, and the perceptual detection or perceptual localization of the Gabor probe. These tasks could be done in combination without substantial loss. By comparison, the perceptual identification of features (orientation) was costly, as was a cognitive task – counting – that involved perception, memory and thinking. These tasks could not be done well in combination.

These results show that the operations needed to detect or locate targets, and to control saccades, can continually run in the background, enabling rapid re-distribution of the limited analytical and memory resources as needed. This is useful. Given that demanding tasks are required so often in real-world settings, the ability to rapidly re-orient and re-allocate processing resources by appropriate modifications of saccadic patterns is both a valuable skill and a necessary option in natural viewing. Our results show that the limits on the ability to perform multiple operations during single fixation pause are so severe that vision reduces to a serial process in which the continual management of resources by saccades becomes a crucial component of successful task performance.

Summary

1. We studied attention during different saccadic tasks, namely, counting, pointing, or only looking. Saccadic patterns were not constrained by instruction or by designating specific saccadic targets. Attention was assessed by the ability to identify the orientation of a Gabor stimulus presented during a randomly selected pause between saccades.

2. All the primary tasks resulted in poorer identification performance, even at high stimulus contrasts, with the greatest losses imposed by counting, then pointing, and finally looking.

3. The primary tasks did not interfere appreciably with the ability to detect or locate the Gabor.

4. In response to the appearance of the Gabor during the orientation task, intersaccadic pause durations increased, perhaps as part of an attempt to interrupt the primary task and switch attention to the Gabor. The brief duration of the Gabor, and the inability to maintain a representation in a short-term visual store, rendered such strategies useless.

5. Counting is a difficult task with many components, all of which people can manage to handle with minimal error during brief intervals of time. The fact that the Gabor could not be identified consistently during counting suggests that the poor identification resulted either from exhaustion of perceptual or cognitive capacity (which are evidently impressive, given the ability to count accurately), or from the operation of attentional filters which can be set to block irrelevant inputs from reaching selected processing centers.

6. The tasks we studied that were based on the ability to orient to a stimulus (that is: the saccadic tasks, as well as perceptual detection and localization) showed the least mutual interference. By contrast, the tasks that required identification of the features of a stimulus, or that required high-level cognitive operations, showed the greatest mutual interference. This evident sparing of the abilities to re-orient attention or the eye to relevant locations, and to extend the duration of a fixation pause, facilitates the continual re-allocation of processing resources. Given how difficult it proved to be to make independent perceptual or cognitive decisions during a fixation pause – for example, maintaining an accurate count and identifying a feature of a grating – the effective use of saccades in concert with the control of attention is crucial to manage allocation of resources over time and ensure success in natural visual tasks.