THE DEPENDENCE OF VISUAL SCANNING PERFORMANCE ON SEARCH DIRECTION AND DIFFICULTY

Abstract

Phillips & Edelman (2008) presented evidence that performance variability in a visual scanning task depended on oculomotor variables related to saccade amplitude rather than fixation duration, and that saccade-related metrics reflected perceptual span. Here, we extend these results by showing that even for extremely difficult searches trial-to-trial performance variability still depends on saccade-related metrics and not fixation duration. We also show that scanning speed is faster for horizontal than for vertical searches, and that these differences derive again from differences in saccade-based metrics and not from differences in fixation duration. We find perceptual span to be larger for horizontal than vertical searches, and approximately symmetric about the line of gaze.

INTRODUCTION

Foveate animals can perceive only a small portion of the visual world with high acuity at any given time, so they use saccades, rapid flicks of the eye, to acquire visual information rapidly. When looking from place to place in a visual scene, the eyes’ behavior is governed both by the spatial extent of the portion of the visual field that can be perceived in a single fixation, as well as how long it takes for visual information to be obtained during a single glance (Jacobs, 1986, Motter & Belky, 1998a, Motter & Belky, 1998b). These visual processing abilities are reflected in the size and direction of each saccadic eye movement as well as the duration of each eye fixation. Therefore, an investigation of the metrics of saccades and fixations during visual analysis of a scene can provide insight into how motor and perceptual systems interact as visual information is being gathered. In addition, the use of gaze-contingent displays, in which the visual scene can be altered rapidly based on the measurement of eye movement, affords the direct measurement of what can be visually discriminated during a single eye fixation.

One difficulty in using metrics of oculomotor performance and assessment of perceptual span from fixation to fixation is the complexity of two (and sometimes three) dimensional oculomotor behavior as well, of course, as the complexity of the visual scene. In a recent study from this lab (Phillips & Edelman, in press) we took a novel approach of examining saccade and fixation behavior by having subjects scan with saccades along a one-dimensional dense, regular column of visual stimuli., Use of a one-dimensional search task facilitated the examination of performance metrics in several ways. First, it allowed the measurement of search speed, and not merely mean reaction time. Search speed is well-defined for a single trial, whereas mean reaction time in a two-dimensional search task is defined only across a set of trials on account of the enormous intertribal variability in scanpath length. Second, using a regular, linear array reduces the amount of variability in the search path by eliminating a degree of freedom in the search path trajectory. Third, use of a homogeneous array reduces the variability of saccade amplitude both within and across trials, making the effects of perceptual learning easier to observe. And fourth, using a dense linear array simplifies the measurement of perceptual span—the area of space that can be perceived in a single fixation—since the configuration of the array affords the probing of target discriminability at different distances from the point of fixation. Using this task, we determined whether saccade- and fixation-related metrics change as a function of performance, both within and across sessions. Thus, we determined whether the visual system learns by ‘seeing more’ in a single fixation, or by ‘seeing more quickly’—reducing the amount of time spent processing a set amount of information.

The principal finding of the previous study was that both within and across sessions, saccade metrics accounted for much more of the variability and improvement in performance than did fixation duration. Increases in search speed were due primarily to the processing information from a greater area of the visual field, rather than processing information from a fixed area more quickly, though there was a small but consistent decrease in fixation duration across sessions. In that study we distinguished between fixational perceptual span, which is measured in a task with peripherally presented stimuli and no eye movements (Geisler & Chou 1995), and saccadic perceptual span, which is measured during a task which involves eye movements, such as our own and (Motter & Belky, 1998a, Motter & Belky, 1998b, Rayner, 1998). The increase in performance we found derived from an increase in saccadic perceptual span and not merely an increase in subjects’ efficiency in ‘tiling’ the search array with regions of visibility.

These results raise two questions. First, are these findings robust to changes in the spatial arrangement of the display and its density, such that intertrial variability of search speed depends more upon saccadic perceptual span than upon fixation duration regardless of search orientation and difficulty? Second, will overall differences in performance on tasks differing in array orientation and difficulty reflect differences in perceptual span, fixation duration, or both?

In a task involving search for a particular letter, Jacobs (1986) and Jacobs & O’Regan (1987) showed that scanning strategy depends on the difficulty of finding the target, with fixation duration increasing as search difficulty increased. But, it is not clear which elements of saccade and fixation metrics underlie intertrial variability when search is difficult. While we have shown (Phillips & Edelman, in press) that with a task of fixed difficulty subjects performing a scanning task increase performance by increasing the number of items scanned per fixation and forward saccade amplitude much more so than they do by decreasing fixation duration, it is unknown is whether this stronger intertrial dependence on the saccadic metrics of scanning itself depends on task difficulty. On one hand, as search difficulty increases, perceptual span may shrink such that its entire extent can be perceived with high acuity. At this stage, visual crowding, not acuity, may limit performance, and the execution of a saccade would not provide enhanced resolution of what is being scanned, but merely disrupt visual processing while the eye is in flight. Thus, one might expect that the amount of time required to analyze what is perceived in a single duration, reflected by the fixation duration, may determine search performance.

The prior study examined vertical scanning from the top of a vertical array to the bottom, both because that is the way printed lists generally are scanned, as well as because reading is generally performed horizontally (at least for our native Anglophone subjects) and scanning tactics evident on a horizontal scan may be heavily influenced by reading processes, which would thus interfere in the more basic scanning processes that we were interested in. But, the conclusions of that study raised the question of whether the variables determining vertical scanning performance would also determine horizontal scanning performance, whether it be in the same or opposite direction of that of normal reading. Reading, of course, requires visual scanning, and numerous studies have investigated how visual information is processed fixation by fixation (O’Regan, 1990, Rayner, 1998). During a single fixation more information is obtained from the side where gaze is heading (i.e. to the right for English) than from the side where gaze has just visited. This raises the question of whether such asymmetries in perceptual span would occur during a horizontal visual eye movement task that didn’t involve reading, as well as whether horizontal scanning is faster when it is performed in the same direction as that of reading.

But, the motivation to study horizontal scanning using our tasks also arises from a more general question of whether the overall speed of horizontal and vertical scanning differ, and, if such differences exist, whether differences in saccadic metrics, fixation metrics, or both, explain these scanning speed differences. While there is evidence that horizontal reading is faster than vertical reading (Ojanpää, Näsänen & Kojo, 2002), even in languages such as Japanese and Koreans where reading can occur in both orientations (Kajii & Osaka, 2000, Seo & Lee, 2002), and that these differences may reflect differences in saccade metrics rather than fixation metrics (Ojanpää et al., 2002, Seo & Lee, 2002), it is not clear whether such differences hold in a general scanning task, or whether they are specific to reading.

In the present study we use the linear saccadic scan approach described above, assessing the metrics of saccades and fixations, to investigate how oculomotor search behavior depends on the difficulty of finding the target during visual search and on the orientation of the search array (and the direction of search). As in the previous study (Phillips and Edelman, 2008), we also examined how search speed depends upon saccade and fixation metrics on a trial-by-trial basis. We also compared saccadic perceptual span for horizontal scanning with that of vertical scanning to determine whether any differences we observed in saccade metrics reflected differences in span. Finally, we examined whether the same visual asymmetries that are present during reading are also present during scanning.

METHODS

General

Eye movements of 4 subjects were recorded at 500Hz using video oculography (EyeLink II, Sensorimotor Instruments). Stimuli were presented on a 22″ Compaq P1220 CRT monitor, set to 1024×768 resolution with a refresh rate of 85Hz. The computer ran an in-house program written in C using the Vision Shell libraries (Comtois, 2003). Subjects sat 62 cm from the monitor and had their heads stabilized with a full impression bite bar. The research was performed under a protocol approved by the Institutional Review Board of The City College of New York and the CUNY Medical School.

Subjects

Five subjects participated in these experiments, three in experiment one (Easy vs. hard search) and five in experiment two. All subjects who participated in experiment one also participated in experiment two. One subject (hh) was an author; the other four were naïve to the purposes of the experiment. All subjects except (ll) were experienced psychophysical observers.

Stimuli and procedure, general

At the beginning of each trial, subjects fixated a small square at the top, bottom, left, or right side of the screen (depending on experiment and condition). After a 700–1000ms variable delay it disappeared and a 3 × 30 (experiment 1) or 3 × 24 (experiment 2) array of boxlike distractors appeared, oriented such that stimuli along the ‘long’ dimension aligned with the initial fixation square (fig. 1c,d). All stimuli (target and distractors) were 0.45deg long per segment, and segments had a thickness of 0.074deg. The distance from the center of one stimulus item to the center of its horizontal and vertical neighbors was 0.62deg.

Figure 1.

Experimental stimuli. Targets used in experiments 1 (a) and 2 (b) are shown, as are example trials from experiments 1 (c) and 2 (d). In (c) and (d), the subject’s scanpath is shown; the target is marked red for expository purposes. Light gray orthogonal bars indicate the total duration of the fixation at the corresponding location; a duration of 200ms is indicated by the dashed reference line in each plot. In (c), trials from subject pp in the Easy (top), Medium (middle) and Hard (bottom) conditions are shown. In (d), trials from downward (left) and upward (right) searches are shown. The upward search trial in (d) is a transient target trial; the location of the eye at the time of target onset is indicated by a red strikethrough disk in the scanpath.

Subjects were instructed to find the target and report its orientation as quickly as possible while minimizing the number of errors (fig. 1). They were instructed to scan in the direction of the end of the array only, and received an error message displayed on the screen and heard a ‘beep’ from the computer speaker if they made a regressive (backwards) saccade of more than 5 deg. Subjects were instructed to guess the target orientation if they reached the end of the list without having seen the target. Target placement was restricted to the center row/column of boxes, and subjects were apprised of this. The flanking rows/columns were added in order to help equalize the discriminability of the 4 target orientations, as well as to increase task difficulty. Subjects did not know in advance which of the four targets would be presented on a given trial. Responses (4AFC) were recorded by key press, after which a second fixation square appeared for 750ms. Subjects received a feedback screen after every trial. On correctly performed trials, this screen displayed their speed for the trial just completed and their cumulative error rate for that block. On error trials, the screen displayed an appropriate error message and subjects heard a ‘beep’ from the computer. Subjects controlled when the program advanced to the next trial by key press.

Stimuli and procedure, experiment one

Subjects ran two sessions, with a least a day and no more than a week separating the two, after successfully completing a shorter practice session. Each experimental session consisted of at least nine blocks of fifteen trials each; the practice session consisted of three blocks of fifteen trials each. In this experiment, the initial fixation point was positioned on the left side of the screen and the direction of search was left-to-right. The target was one of four “Landolt C”-like boxes with a notch on one of the four sides (fig. 1a). Distractors consisted of either boxes (Easy condition, fig. 1c top), boxes with one side removed (Medium condition, fig. 1c middle), or boxes with an off-center notch in one of the four sides (Hard condition, fig. 1c bottom). Distractor type varied between blocks in a counterbalanced way, but within a block distractors were of the same type. To ensure that subjects were completing the task, subjects were required to redo a block if they made more than three errors (> 20% incorrect). Subject hh was required to redo one block (Easy condition) and subject pp five blocks (one Easy condition, four Hard condition).

Stimuli and procedure, experiment two

Subjects ran five sessions excluding an initial practice session over the course of two weeks. Each session consisted of eight blocks of 25 trials each; search direction differed across blocks in a counterbalanced way, but within blocks search direction was the same for all trials. In all trials, the target consisted of a box with a notch in one of the 4 sides, and the distractors consisted of boxes with an open side (fig. 1b,d). These stimuli were similar to the stimuli in the Medium condition of experiment one (fig. 1c). In this experiment there were two trial types, trials with stationary (80%) and with transient (20%) targets, a breakdown identical to that used in Phillips and Edelman (2008). In the stationary trials the target was in the display in a fixed location for the entire trial. In the transient trials, the target was displayed only for the duration of a single fixation; otherwise the location contained a distractor. The change was made in mid-saccade. Transient targets were distributed ahead and behind the point of fixation in equal proportions; after offline calibration they were found to be, by subject, median locations ahead/behind fixation were dd: [2.4, −3.1], hh: [3.0, −3.4], ll: [1.9, −2.7], pp: [3.2, −3.1], zz: [3.1, −3.7]. Otherwise, the two trial types were identical, although they were analyzed differently (see Methods—experiment 2 and Results--Saccadic perceptual span below).

Calibration

Subjects were calibrated at the beginning of each session using the 9-point calibration routines built into the EyeLink II and Vision Shell software packages in succession. In addition, a drift correction was performed between each block of trials. In offline analysis, a MATLAB program was used to identify the fixation made to the second fixation square (See ‘Stimuli and procedure’), and the scanpath from each trial was translated so that this fixation overlaid the square.

Data Analysis, general

Our offline analysis program identified a saccade as occurring when the eye velocity (lowpass filtered, 200Hz Butterworth) exceeded 40deg/sec until it fell below 10deg/sec, with a total duration of at least 8 ms. We did not use the entire “scanpath,” the trajectory of eye position during search, in our analysis. Rather, we analyzed the scanpath beginning from the first fixation within 2 deg of the beginning of the list, until the end of the fixation prior to the fixation during which the key press was made. Hence, the fixation during which the key press was made, as well as the saccade immediately prior to it, were not considered in the calculation of any of the oculomotor data. We did this because for both experiments, as well as in Phillips & Edelman (2008), the duration of the fixation during which the key press was made tended to be much longer than the average fixation duration, and its duration varied widely, from <100ms to >1000ms. This variation is also partially a result of variability in the timing of the key press at the end of the trial. The last saccade prior to the key press was generally much shorter than average and so it too was excluded. Possibly as a result of these measures we found very few corrective saccades (latency < 60ms) in either experiment (0–2% across subjects and experiments, mean 0.6%).

Search time, then, was defined as the amount of time used to traverse the scanpath. Of course, search time will be faster for targets located near the top of the array. Therefore, the metric of performance we used was search speed, defined as search time divided by the number of items from the beginning of the list to the target, inclusive. This quantity, items per second, describes the rate at which the subject traverses the list during search and is thus distinct from the traditional measure of search time or reaction time used to assay performance in visual search tasks. In both experiments we correlated this performance measure with the following three oculomotor variables: items scanned per fixation, defined as the number of fixations in the scanpath divided by the number of items from the beginning of the list to the target, inclusive; forward saccade amplitude, defined as the mean amplitude of the saccades made in the top-to-bottom direction, and fixation duration. We considered items scanned per fixation and forward saccade amplitude to be saccade-related metrics and fixation duration to be a fixation-related metric. Only trials with a stationary target (all trials in experiment 1, 80% of all trials in experiment 2) were used in the calculation of these quantities. Both forward saccade amplitude and items scanned per fixation have been used as indirect measures of span in the literature (Jacobs 1986; Ojanpää et al. 2002). They are however different metrics; items scanned per fixation but not forward saccade amplitude will reflect regressive saccades that were made, placement of the initial fixation within the search array, and the distance of the final fixation in the scanpath to the target. Rather than decide a priori which measure is a better indicator of oculomotor strategy and saccadic perceptual span, we used both in our analysis. They are, as we will see, empirically distinct. Our direct measurement of span in experiment two using transient target trials will be described in Results—Saccadic perceptual span below.

All analyses of the data were performed using routines written in the MATLAB (MathWorks) software, and Sigma Plot (SPSS) programming environments.

Data analysis, experiment one

A significant fraction of trials (hh: 29%, pp: 32%, zz: 32%) were excluded from analysis. Excluded trials included: trials in which the subject made a single saccade directly to the target, or in which the scanpath contained only one fixation (hh: 22%, pp: 18%, zz: 25%), in which case forward saccade amplitude was undefined; trials in which an incorrect key press was made (hh: 3%, pp: 4%, zz: 1%); trials in which the key press occurred when the eye was fixated more than 5 degrees below the target (hh: 1%, pp: 3%, zz: 2%); and trials in which the cumulative amplitude of regressive saccades was over 5deg (hh: 0%, pp: 2%, zz: %0). The remaining excluded trials consisted of trials in which there was either a blink or an initial fixation error. There were no significant differences between conditions in exclusion rates, except that, not surprisingly, subjects were more likely to make one or zero fixations in the less difficult conditions (hh: 13%, 7%, 2%, in order of increasing difficulty; pp: 9%, 4%, 4%; zz: 15%, 8%, 2%). Remaining excluded trials consisted of trials in which there was either a blink or an initial fixation error.

Data analysis, experiment two

We performed a similar analysis of error trials in experiment 2 as in experiment 1, applied to stationary target trials. Again, a significant fraction (dd: 23%, hh: 19%, ll: 19%, pp: 17%, zz: 19%) were excluded from analysis.

Excluded trials comprised trials in which the scanpath contained only one fixation, or subject made a single saccade directly to the target (dd: 11%, hh: 9%, ll: 11%, pp: 12%, zz: 14%); trials in which an incorrect keypress was made (dd: 8%, hh: 5%, ll: 6%, pp: 4%, zz: 3%); trials in which the keypress occurred when the eye was fixated more than five degrees below the target (dd: 3%, hh: 4%, ll: 2%, pp: 1%, zz: 2%); and trials in which the cumulative amplitude of regressive saccades was over 5 deg (<1% all subjects). Remaining error trials consisted of blinks or initial fixation errors.

Analysis of perceptual span

Transient trial data were fit with a two-component Weibull function, largely as in (Phillips & Edelman, 2008), which described proportion correct response as a function of the retinal eccentricity of the target. We measured forward span—the region of visibility below fixation, i.e. in the forward direction of search—and backward span, the region of visibility above fixation, simultaneously by fitting a function composed of two Weibull cumulative distribution functions (cdfs) which modeled the respective forward and backward halves of the data set:

$$y=\gamma +(1-\gamma -\lambda )(e^{-{(\mid x\mid /{\alpha }_1)}^{{\beta }_1}}I{(x)}_{(-\infty ,0]}+e^{-{(\mid x\mid /{\alpha }_2)}^{{\beta }_2}}I{(x)}_{(0,\infty )})$$

This procedure ensured that the model function was continuous at x=0, while allowing the slope (β₁, β₂) and inflection (α₁, α₂) parameters of the two component cdfs to vary independently. Parameter γ(floor) was fixed at chance (0.25), and λ(the lapse rate, equal to the difference between 1 and the maximum value) was restricted to the interval [0, 0.75]. I(x)_(a,b) is defined as equal to 1 over the interval (a,b) and 0 everywhere else. The spatial region comprising the subject’s saccadic perceptual span for that session was then defined as the portion of the visual field within which target identification was 62.5%, i.e. halfway between ceiling (100%) and chance on the 4-AFC task (25%), or better.

RESULTS

Experiment One -- Easy vs. Medium vs. Hard Search

Decreasing the similarity of targets and distractors dramatically increased scanning performance, increased the size of saccades made during scanning, and to a lesser extent decreased the duration of fixations in between saccades (fig. 2). Mean performance (search speed) for the three subjects was 18.5 items/sec when targets were most dissimilar to distractors (square distractors – Easy condition), 8.2 items/sec with more similar distractors (open squares – Medium condition), and 4.3 items/sec for gapped distractors, which were the distractors most similar to the targets (Hard condition). Thus, overall performance was slowed by a factor of four from the easiest to the most difficult condition.

Figure 2.

Task performance (items scanned per second) by subject and distractor type in experiment 1. Larger values indicate faster searches. Error bars represent 1 standard error (SE).

As we would predict from numerous previous studies that have examined task effects on oculomotor variables (Jacobs, 1986, Jacobs & O’Regan, 1987, Phillips & Edelman, in press), saccade-related oculomotor metrics varied far more strongly with performance than did fixation duration. Whereas items per fixation and forward saccade amplitude generally decreased by at least a factor of three from the easiest to the most difficult condition (mean 5.0 to 1.4 items/fixation and mean 4.9 to 1.3 degrees/saccade, respectively), fixation duration increased only by a factor of 1.4 (mean 200ms to 285ms). These patterns were consistent across all subjects.

We found that the intertrial correlations between performance and the three oculomotor variables of interest remained approximately the same regardless of difficulty level (fig. 3). In particular, with one exception, fixation duration never correlated more strongly with performance than either of the saccade-related oculomotor metrics for any subject, despite the fact that 2/3 subjects were only able to process approximately one item per fixation in the most difficult condition. The only exception to this amongst the nine combinations of subject and condition concerned the performance of subject hh on the Easy condition, for which performance correlated more strongly with fixation duration than it did with forward saccade amplitude. However, hh was the only subject to produce errors (incorrect target choice, overshoot, and regression error, combined) in this condition (10%), even though all subjects produced errors in the Medium and Hard conditions (fig. 4). Moreover, hh produced more errors in the Easy condition than he did for the other two conditions. This suggests that hh may have violated the general relationship between forward saccade amplitude, fixation duration, and performance only by adopting a particularly error-prone strategy, e.g. one in which saccade amplitude was greater than the amount of the array perceived. For this reason we do not think that hh’s data in the Easy condition threaten our general conclusion from this experiment.

Figure 3.

Correlations between performance and oculomotor variables for experiment 1, plotted as a function of task difficulty level. Grayscale levels distinguish the subjects hh (black), pp (dark gray) and zz (light gray).

Figure 4.

Error rates by subject and condition, experiment 1. There are no bars denoting errors in the Easy search condition for subjects pp and zz because they did not make any errors in this condition.

Experiment two -- Horizontal vs. Vertical Search

Subjects generally did not exhibit substantial differences between left and right search nor between upward and downward search. Two of the five subjects were significantly faster for rightward rather than leftward searches: ll, 9% faster, p < 0.01; zz, 12% faster, p < 10⁻⁴. One subject was significantly slower for upward rather than downward searches: zz, −8%, p < 0.005. At the population level, there was a slight but significant general advantage (0.38 items/sec) for rightward searches (11.5 items/sec vs. 11.12 items/sec) as determined by a 2-factor (Subject × Direction) ANOVA (F = 4.77, p < 0.05); no such difference existed for upward vs. downward searches (F = 0.44, p > 0.5). Given the low magnitude of these differences (max 1.3 items/sec) we collapsed leftward/rightward and upward/downward searches in subsequent analysis.

The orientation of the search array significantly affected search performance (fig. 5); horizontal searches were between 13% and 41% (mean 31%) faster than vertical searches, with differences significant for every subject. There was no difference in the error rate between horizontal and vertical for any subject (p > 0.1 in all cases). The observed effect on performance was manifested almost exclusively in changes in saccade-related metrics; horizontal scanning was higher in terms of items scanned per fixation (range 3–35%, mean 26%) and forward saccade amplitude (range 0–58%, mean 32%); in each case all comparisons were highly significant by t-test (p < 10⁻⁵) except those of pp which were not significant. The change in fixation duration was much smaller (−1% to −13%, mean −6%), and significant for 3/5 subjects. One subject, pp, showed greater horizontal/vertical variation in fixation duration than in saccade-related metrics, a 13% decrease in fixation duration vs. a 3% increase in items scanned per second and a 0% increase in forward saccade amplitude. However, pp also exhibited a substantially smaller performance difference than the other subjects (13% compared to a mean of 36% from the other subjects), and the subject’s change in fixation duration, though highly significant, was modest in comparison to the proportional changes in saccade-related metrics made by other subjects.

Figure 5.

Each bar represents the proportional change from vertical to horizontal search for the corresponding variable and subject, for experiment 2. Change was calculated using the equation (X_horizontal − X_vertical)/X_vertical, where X_horizontal is the value of the variable in question for horizontal searches, and X_vertical is the value for vertical searches. For example, a value of 0.32 for items scanned per second for subject dd indicates that dd’s horizontal searches were 32% faster than dd’s vertical searches. ‘*’ indicates the horizontal difference was significant by t-test at the level of p < 10⁻⁵; unmarked comparisons n.s.

In terms of intertrial variability, the hierarchy of correlation strengths we observed before (Phillips & Edelman, 2008) and in experiment one held here as well: invariably, across all subjects and all directions of scanning, items scanned per fixation correlated most strongly with performance, followed by forward saccade amplitude, followed by fixation duration (fig. 6). There was no mean change in these correlation strengths from horizontal to vertical; for items scanned per fixation, the mean r values were 0.76 and 0.77, respectively; for forward saccade amplitude they were 0.50 and 0.54; and for fixation duration they were 0.15 and 0.14. Overall, these results suggest that intertrial variability in horizontal search performance, like that of vertical search performance, depends on saccade metrics, not fixation duration.

Figure 6.

Correlations between performance and oculomotor variables for experiment 2, plotted by search direction and subject. Each of the three panels displays correlations between performance and the corresponding variable for horizontal (left points) and vertical (right points) searches.

Saccadic perceptual span

We obtained a direct measure of perceptual span by presenting targets for the duration of a single fixation on 20% of trials. The proportion of correct responses as a function of the target’s distance from the point of fixation determined the linear extent of perceptual span along the axis defined by the search array.

Performance at identifying the transiently appearing targets was generally higher for horizontal scanning trials than for vertical scanning trials, particularly for intermediate distances of the target from fixation, suggesting that horizontal saccadic perceptual span was greater than that of vertical perceptual span. We investigated the effect of orientation of scanning on saccadic perceptual span both by directly comparing the proportion of correct responses as well as by fitting curves to the relation between performance and target distance from fixation as in Phillips & Edelman (2008).

It would be expected that performance for very small distances between the target and fixation was close to ceiling and for very larger differences at chance. Thus, we speculated that differences in performance on horizontal and vertical trials would emerge at intermediate target/fixation differences. To examine this, we first grouped horizontal and vertical scanning trials and binned target locations for each scanning orientation into one-degree bins by rounding the distance in degrees of each target from fixation to the nearest whole number. (This bin width is arbitrary, but the results described henceforth depended only slightly on this particular value; smaller (to 0.5deg) and larger (1.5deg) bins produced very similar results at the group land individual levels.) Next we calculated, across all subjects, and for each combination of scanning orientation (horizontal or vertical), and one deg. eccentricity bin, the proportion of trials with correctly identified targets, and finally compared horizontal and vertical trial performance using a z-test for proportions.

Performance was above 90% for the 0.0–1.0deg bin for both orientations, but then declined rapidly as eccentricity increased. As had been speculated, significant differences were found for bins intermediate eccentricity, namely the 2.0 – 3.0deg bin (horiz.: 54%, vert.: 36%, p = 0.03) and the 3.0–4.0 deg. Bin (horiz.: 49%, vert: 30%), but were similar at larger and smaller eccentricities. To analyze each subject’s data, we pooled together data from these two eccentricity bins Analyzing each subject’s data individually revealed that for all five subjects correct responses were numerically higher for horizontal trials than vertical trials. The proportion correct of horizontal trials minus that of vertical trials was (horiz. - vert percent change,: dd, 6%; hh, 39%; ll, 21%; pp, 10%; zz, 5%), though this reached statistical significance in only in subject, hh. A similar analysis for individual scanning directions (up, down, left, right) found only small and statistically insignificant differences between upward and downward scanning and between leftward and rightward scanning for all eccentricity bins.

Saccadic perceptual span measured by fitting the performance data with a two-part Weibull function and determining the eccentricity at which performance reached 62.5% (see Methods), was also in general correspondingly larger for horizontal than for vertical scanning (fig. 7). In terms of percent change, increases in span size were as follows: dd, 92%; hh, 95%; ll, −23%; pp, 14%; zz, 0%. There was large variability between subjects, and little apparent correspondence between the magnitude of the change and the percentage change in span-related oculomotor variables, by subject. However, span measured by this metric and by proportion correct, as calculated above, corresponded reasonably well by subject. One subject (ll) exhibited greater vertical than horizontal span; however as the curve indicates this subject was substantially below ceiling (but above threshold) even for foveal targets, and hence may have achieved greater peripheral span in this case only by sacrificing foveal performance. Indeed, as calculated above, ll’s performance as measured by percentage correct was higher for horizontal than vertical trials. This interpretation is supported by ll’s post-session report that he found vertical searches to be particularly difficult. Overall, the quality of the raw (non-bootstrapped) data curve fits was moderate (R² value range 0.21–0.48, median 0.28), so we cannot say with certainty at this point whether the observed lack of correspondence is due to partial uncoupling of oculomotor strategy from perceptual span, or is due rather to uncontrolled variability in the data. In any case, the trend across subjects for saccade perceptual span is consistent with that observed in the oculomotor data and with the direct comparison of proportion of correct trials.

Figure 7.

Saccadic perceptual span for each subject in the two major search directions. The central shaded area indicates the portion of the graph for which the two-part Weibull has a value greater than the threshold of 0.625. The solid vertical lines in the right half of the graph correspond to mean forward saccade amplitude for searches in the indicated direction. The dashed reference line in the center of each graph intersects the origin.

Lastly, we replicated our earlier result (Phillips & Edelman, 2008) that there is little difference between forwards and backwards span. For each subject and absolute eccentricity, we separated trials with targets placed ahead of fixation with those placed behind fixation. We then repeated our analysis on proportion of trials with correct performance as described above for bins of one deg. Percentage was highly similar for all eccentricity bins (all p > 0.4, z-test on proportions). Similarly, subjects’ spans as measured using the Weibull curve fitting were generally symmetric, with the absolute difference between forwards and backwards span <1deg in all cases except for the vertical span of subject ll (range: −0.67deg to 1.94deg, median 0.34deg); backward span was larger in 4/10 cases and forward span was larger in 6/10 cases. As our subjects were all native English speakers, we also examined symmetry in left-to-right searches specifically; here we again found no systematic forward bias, whether by analysis of percentage correct or by comparing subjects’ spans as determined by curve fits. In this latter case, forward/backward discrepancies were as follows: dd: −0.52deg, hh: −0.24deg, ll: 1.92deg, pp: −0.87deg, zz: 0.26deg.

DISCUSSION

Confirming the results of Phillips and Edelman (2008), the findings described here provide additional support for the idea that the speed of visual scanning depends upon how much is perceived during a single fixation, rather than how long it takes to process what is seen during that fixation. In terms of both intertrial variability and task dependence, subjects modulate their scanning strategy by altering the size of their saccades to adjust for how much they can perceive during a single fixation, not for how long that single fixation takes. Even when search is extremely challenging, such as in the Difficult condition of experiment 1, and fixation durations are relatively long, variability of scanning performance across trials of a single task depends little upon fixation duration. Thus, the current study suggests that the conclusions of the study of Phillips and Edelman (2008) were not unique to downward vertical scanning with searches of moderate difficulty, but pertain to visual scanning in general.

Experiment 1 showed that as we increased the difficulty of a task by manipulating target/distractor similarity, our saccade-based metrics of items scanned per fixation and saccade duration decreased by a far greater percentage than fixation duration increased. Furthermore, except for one subject in the Easy condition, intertrial variability better reflected variability in saccade-based metrics than did fixation duration. Our second experiment demonstrated that horizontal scanning was faster than vertical scanning, and that, again, these changes resulted from a larger number of items scanned per fixation and an increase in saccade size rather than a decrease in fixation duration. As for experiment 1 and the results of Phillips & Edelman (in press), intertrial variability in horizontal scanning reflected changes in saccade-based metrics rather than fixation duration. In addition, similar to what we found in the previous study, the area perceived about the line of gaze during search was approximately symmetric. This is unlike the case for reading, where evidence suggests that more can be perceived in the directions towards which the eye is moving than in the direction from which gaze angle just shifted (O’Regan, 1990, Rayner, 1998).

Relation to previous results on difficulty and horizontal/vertical scanning

Using a scanning task similar to ours, in that subjects were required to span a horizontal array of letters and find a target letter amongst a homogeneous set of distractor letters, Jacobs (1986) and Jacobs & O’Regan (1987) showed that both saccadic perceptual span (“visual span,” in their parlance) and saccade amplitude decreased as targets were made more similar to distractors. Fixation duration also depended on target/distractor similarity,, though to a lesser degree. Here, we demonstrate that even when target and distractors are quite similar, intertrial variability in search performance is dependent more on the variability of saccade size and items scanned per fixation than on fixation duration. Like Jacobs (1986) and Jacobs & O’Regan (1987), we did find that fixation duration increased substantially with increased target/distractor similarity, suggesting that subjects adopt a different scanning strategy for more difficult searches that involves longer fixation durations. This perhaps reflects the ability to see with high acuity more than what one can, due to visual crowding, select attentionally (He, Cavanagh & Intriligator, 1996, Intriligator & Cavanagh, 2001). The subject would fixate longer, since a small saccade would simply disrupt vision for several tens of milliseconds without appreciably aiding scanning, and perhaps adopt a more serial scanning strategy to process different portions of what is visible during a single fixation. Our findings indicate that even if subjects undergo this change in strategy for difficult searches, performance on any given trial still depends more on the amount of the search array visible at a given fixation, which would in turn affect saccade amplitude, than on how rapidly such a small portion of visual space can be processed.

The results of Experiment two are in accord with various studies suggesting that perceptual span has a greater extent horizontally than vertically. Comparisons in reading showed that more letters can be perceived during a single fixation during horizontal reading than in vertical reading of the Korean Hangul character set. (Seo & Lee, 2002) and of Japanese words (Kajii & Osaka, 2000). A study of the scanning of word lists has also indicated a greater perceptual span along the horizontal axis than along the vertical axis (Ojanpää et al., 2002). These studies of reading and scanning are consistent with the finding that target detection, discrimination, and location, in both cued and uncued conditions in an eyes-fixed paradigm, are superior along the horizontal as opposed to the vertical axis (Carrasco, Talgar & Cameron, 2001). As these authors hypothesize, these findings appear to reflect the fact that acuity drops off faster with vertical distance from the fovea than with horizontal distance (Weymouth, Hines, Acres, Raaf & Wheeler, 1928) which in turn reflects anatomical studies of cone spacing in the fovea and perifovea (Curcio & Allen, 1990). In corroborating and extending these prior studies, our results suggest that in response to smaller perceptual span and greater fall-off in acuity along the vertical axis subjects adopt a scanning strategy in which the items scanned per fixation and saccade amplitude are smaller while fixation duration increases much less. Interestingly, the one of our five subjects who did not show this pattern also did not show greater scanning speed in the horizontal directions. Furthermore, intertrial variability in scanning speed reflected saccade-based metrics rather than fixation duration regardless of the orientation of scanning.

Thus, our results suggest that for all types of scanning, regardless of direction or difficulty, acuity and visual crowding determine perceptual span and perceptual span determines the items scanned per fixation and the size of the saccades. Modulation of the duration of fixation is generally not adopted as a strategy for dealing with differences in the ability to locate a target.

Relation to visual search

A traditional visual search task generally differs from our task in several respects; there is no pre-set path, scanpaths are extended in 2 or 3 dimensions, not just 1; and distracter elements are not homogenous. As discussed in (Phillips & Edelman 2008), we chose a dense linear array for our experiments to simplify the saccadic scanpath as well as to simplify the measurement of perceptual span. The question arises as to whether our results concerning the dependence of performance upon saccade-based metrics extend to the general case of visual search. In our task, both the direction of the impending saccade as well as the coarse configurational properties of the array are known to the subject in advance. More conventional visual search in two or three dimensions generally involves more complex and less predictable search arrays, with a greater variability in the amount and structure of visual information available at a given fixation. Thus, peripheral visual analysis should become more important to optimize search. Indeed, Motter & Belky (1988) found that in a two-dimensional search task as stimulus density varied, saccade amplitude varied more than fixation duration. Similarly, we expect that even with equating the visual world from saccade to saccade, intertrial variability would result more from variability in saccade metrics than from fixation duration.

It is less obvious whether perceptual span will be symmetric during ordinary two- or three-dimensional visual search, but acuity considerations make this the default assumption. Without a clear direction or magnitude for the next saccade, peripheral visual analysis will be much more intensive, and will generally itself determine the endpoint of the saccade. Thus, there is not a clear reason to expect a greater attentional advantage at the saccade endpoint in standard visual search than in one-dimensional visual scanning, even if peripheral processing plays a greater role. In particular, we expect that stimulus identification processes stay focused on what is clearly seen (i.e. the visual world near their fovea), rather than on a future saccade target.

As we mentioned in our previous report (Phillips and Edelman, 2008), our finding that span is symmetric about the point of fixation may be seen as somewhat surprising given that 1) numerous studies have shown that stimuli at the saccade goal undergo some selective presaccadic processing (Deubel & Schneider, 1996, Gersch, Kowler & Dosher, 2004, Hoffman & Subramaniam, 1995, Kowler, Anderson, Dosher & Blaser, 1995), and 2) electrophysiological studies have shown that weak microstimulation of oculomotor areas (such as the FEF or SC) produces perceptual advantages at the location to where stronger stimulation would elicit a saccade (Cavanaugh & Wurtz, 2004, Moore & Fallah, 2001, Moore & Fallah, 2004). As we speculated previously (Phillips and Edelman, in press), the discrepancy between behavioral results may result from specific saccades being instructed in the studies involving saccades and attention, whereas specific saccades are not explicitly instructed in our task. The neurophysiological studies involved stimulation while the monkey was maintaining steady fixation amidst a sparse visual array, and its consequences could be quite different if the stimulation during active visual exploration of a more realistically crowded visual scene.

Relation to reading

As in our previous study, perceptual span was approximately symmetric about the line of gaze, regardless of the orientation of the stimulus array and the direction of search. This is in contrast to studies of reading in which more is perceived in front at locations that are about to be fixated than at locations that have just been fixated (O’Regan, 1990, Rayner, 1998). This “look ahead” may both aid in the programming of subsequent saccades and in aiding the lexical processing of words currently being fixated. O’Regan (1990) has distinguished “visual span,” the area of space that can be perceived, bounded by limitations in fundamental aspects of vision, such as acuity and crowding, and “perceptual span,” in which context obtained by lexical processing may in effect increase visibility. As our study did not explicitly rely on lexical processes, our experiments could be said to address the extent of “visual span.” Previous studies assessing span during scanning have not measured (or at least have not reported) whether span is symmetric (e.g. Jacobs, 1986). Our results indicating that perceptual span is as symmetric for left-to-right scanning as it is for scanning in other directions suggests that the asymmetry found in perceptual span during reading (reviewed in O’Regan, 1990) is inherent to the reading process, and that it is not in effect for other visual scanning behaviors. It is thus reasonable to think that “perceptual span” as defined by O’Regan (1990) is asymmetric, whereas “visual span,” as he defines it, is not.

The sharp discrepancy between the shape of perceptual span in the case of reading and in scanning as we report it here and in (Phillips & Edelman 2008) raises the question of whether we would have obtained the same results if we had had subjects scan through an array of letters rather than boxes. We think so, since the forward span skew exhibited during reading is probably due to particular benefits that parafoveal preview incurs to the reader. Fixation duration, and whether a word is fixated or skipped, have been shown to depend upon lexical information—word familiarity and length—obtained parafoveally (Blanchard, Pollatsek & Rayner, 1989, Inhoff & Rayner, 1986). Since neither length nor familiarity can vary meaningfully among single-letter stimuli, we doubt that such an alteration would induce a change in allocation of attention. We of course do not rule out the possibility that search arrays could exist for which subjects would exhibit skewed perceptual span.

Relation to studies and theory on fixation duration

The results of the experiments described here as well as the previous report from our lab (Phillips and Edelman, 2008) suggests that while fixation duration does vary from trial to trial, its mean value is relatively insensitive to changes in task demands whose consequence is a change in the size of perceptual span. The saccade-based metrics of items scanned per fixation and saccade size seem to respond to changing task demands and anisotropies in acuity much more strongly. There is some evidence implying that the exposure time necessary to perceive information during a given fixation, assuming that the stimuli are at a suprathreshold luminance level, is less than that of a typical saccade duration (Sperling, 1990). A saccade-contingent study controlling the duration of visual exposure, complementing our saccade-contingent approach of controlling the spatial extent of exposure, could shed light on this issue.