Role of peripheral vision in saccade planning: Learning from people with tunnel vision

Abstract

Both visually salient and top-down information are important in eye movement control, but their relative roles in the planning of daily saccades are unclear. We investigated the effect of peripheral vision loss on saccadic behaviors in patients with tunnel vision (visual field diameters 7°–16°) in visual search and real-world walking experiments. The patients made up to two saccades per second to their pre-saccadic blind areas, about half of which had no overlap between the post- and pre-saccadic views. In the visual search experiment, visual field size and the background (blank or picture) did not affect the saccade sizes and direction of patients (n=9). In the walking experiment, the patients (n=5) and normal controls (n=3) had similar distributions of saccade sizes and directions. These findings might provide a clue about the extent of the top-down mechanism influence on eye movement control.

Introduction

Salient visual stimuli in peripheral visual fields (VF) are regarded as important triggers and targets for saccades, which are usually referred as bottom-up information. Models have been proposed to predict eye movements in visual search based on the salient features (Foulsham & Underwood, 2008; Itti, 2005; Itti & Koch, 2000; Najemnik & Geisler, 2005; Pomplun, Reingold, & Shen, 2003). However, studies of eye movement in natural situations have suggested that many eye movements are proactive (not induced by salient stimuli) rather than reactive (induced by salient stimuli) (Hayhoe & Ballard, 2005; Land & Furneaux, 1997). It has been argued that top-down information (e.g. prior knowledge, intention) provides important guidance to eye movements (Land, Mennie, & Rusted, 1999; Wolfe, 1994; Wolfe, Butcher, Lee, & Hyle, 2003). Readers can refer to papers by Henderson and Rothkopf et al for comprehensive reviews of bottom-up and top-down models regarding eye movement control (Henderson, 2003; Rothkopf, Ballard, & Hayhoe, 2007).

It appears that bottom-up and top-down information may both play important roles in saccade planning (Foulsham & Underwood, 2008; Henderson, 2003), but it is still unknown how each of them is weighted and how they interact in eye movement control for daily activities in the real world. The respective influences of visual salience and top-down information on the hundreds of thousands of daily saccades are usually difficult to differentiate, as both often exist simultaneously. Gaze-contingent paradigms have been used to control visual salience shown on displays by simulating peripheral and central vision loss (Cornelissen, Bruin, & Kooijman, 2005; Geisler, Perry, & Najemnik, 2006; Pomplun, Reingold, & Shen, 2001). It was found that, when stimuli were only visible within a small window (e.g. 5° in diameter) contingent to gaze, subjects sometimes made saccades that were beyond the restricted windows (Cornelissen et al., 2005; Pomplun et al., 2001). It is unknown whether these beyond window saccades to featureless areas were generated purely through a top-down mechanism, because in reality even the black area outside the visible window is usually distinguishable from the surrounding. When having an instantaneous image of a blank gap between current gaze point and the search area boundary, it is possible that subjects make beyond window saccades to reveal un-visited spots. The un-visited yet visible area can also be considered as a piece of bottom-up information guiding eye movement. So, it remains ambiguous what mechanisms may be involved in such beyond window saccades by normally sighted people in the gaze-contingent studies. To completely block particular visual information (central or peripheral) and investigate its effect on eye movement, studying patients with VF loss due to eye diseases may offer an effective approach, especially in outdoor environments, where a gaze contingent system is difficult to implement.

We analyzed eye movement data from patients with severely restricted peripheral fields (tunnel vision) which we collected in a visual search experiment and a real-world walking experiment for the purpose of visual aid development. Surprisingly, we found that these patients frequently made one or two saccades per second beyond their residual VFs. Some questions that can be raised from this finding are: What is the visual experience like when they saccade to pre-saccadic blind locations? Why are so many beyond-VF saccades generated? How is the world represented in patients’ brains when they sample the world as if viewing through discrete “holes”?

In this paper, we do not intend to answer all these questions, but only report the unexpected eye movement data, and briefly discuss the implication on eye movement control and visual perception across saccades.

Methods

The research followed the tenets of the Declaration of Helsinki and was approved by the institutional review board at Schepens Eye Research Institute.

The visual search and real-world walking experiments were originally conducted separately, although both were related to the development of low vision aids for people with tunnel vision. Nine patients with tunnel vision participated in the visual search experiment, and five patients plus three normally sighted control subjects participated in the walking experiment. Patients’ VF defects were all due to retinitis pigmentosa, a retinal disease. The boundary between functional and blind areas in this type of patients is usually sharp. Some patients with tunnel vision may have separate functional areas in the periphery called residual islands. These islands may be useful for the detection of targets, thereby confounding our tunnel vision criterion. To eliminate this possibility, we verified that no patient in our study had peripheral islands (Goldmann II4e).

The methodology of the visual search experiment is described in more detail by Luo and Peli (Luo & Peli, 2006). VF was measured with an 18mm white target from 1 meter. The horizontal binocular VFs of the nine patients ranged from 7° to 16° wide. They performed the visual search within an area of 66°(H)×54°(V) for “pop-out” targets (size 3° and 5°). The targets were presented on a projection screen in a random sequence at eccentricities of 15°, 22°, and 29°, all of which were outside their VFs. In a pseudo-randomized order, subjects carried out the visual search tasks under a combination of two cue conditions (with or without auditory cues) and two types of background (blank and real-world images). The auditory cues were chirping sounds lasting 5 seconds from one of eight piezoelectric buzzers placed around the projection screen. These buzzers indicated the approximate direction to a target but not its eccentricity. Patients were allowed free head movement during the search. An eye tracking system (ISCAN, Burlington, MA) was used to record subjects’ eye movements at 60 Hz.

The methodology of the walking experiment is described in more details by Vargas-Martin and Peli (Vargas-Martín & Peli, 2006). Five patients with retinitis pigmentosa (VF diameter: 7°–15°) and three normally sighted subjects participated in the experiment. The patients’ VFs were measured with a 6-mm white target from 1 meter. The subjects walked for more than 30 minutes through unfamiliar indoor environments and along city streets in downtown Boston. They received no instructions other than to walk normally to various points along the route. The environments were not controlled. Free head movements were allowed. Eye movements were recorded using a portable recording system, and processed off line using the same ISCAN eye tracking system. Eye tracking calibration was performed before walking and several times between walking segments.

The criteria for saccade detection were: speed >15°/s; peak speed < 700°/s; saccade duration < 200ms. To verify the validity of our saccade detection, we compared the main sequence of our data with others obtained at higher sampling rates (Figure 1), and found good consistency. We found that a slight change in the speed threshold of saccades (e.g. changing the speed criterion from the commonly used 15°/s to 12°/s) would result in a large change in the number of small saccades (generally those < 2°). In this study, these small saccades were not of interest, but their count affects the total number of saccades, and therefore the proportion of large saccades. Instead of presenting the proportion of saccades as in Bahill’s paper (Bahill, Adler, & Stark, 1975), we present data as the frequency of saccades for a given saccade size, which is not affected by the total saccade count. For the same reason, we excluded saccades smaller than 2° in curve fitting, even though some small saccades were detected using the 15°/s criterion.

Figure 1.

Comparison of main sequences between our data and published data by Boghen, et al (Boghen, Troost, Daroff, Dellosso, & Birkett, 1974, infrared reflection technique, 100Hz), Garbutt, et al (Garbutt, Harwood, & Harris, 2001, IR limbus and EOG, 1090Hz), Epelboim, et al (Epelboim et al., 1997, magnetic coil, 976Hz), Wilson, et al (Wilson, Glue, Ball, & Nutt, 1993, EOG, 256Hz), and Baloh, et al (Baloh, Sills, Kumley, & Honrubia, 1975, EOG, 200Hz).

Results

1. Saccade size

In total, 16,582 saccades in the visual search and 26,344 saccades in the walking experiment were detected. Histograms of saccade occurrence as a function of saccade amplitude are plotted in Figure 2. The gray zone in each figure indicates the range of VF sizes of the patients in each experiment.

Figure 2.

Frequency histogram of saccades per minute as a function of saccade size. Gray zones indicate the VF range of the patients. Error bars are standard deviation across patients. (a) Saccades in a visual search under different conditions, indicated by the legends. These conditions made no significant difference to the distribution of saccade size. (b) Saccades when walking indoors and outdoors. The saccade size of normally sighted people and patients were similar. There were fewer large saccades than in visual search.

The curves shown in Figure 2a are averages across the nine patients in the visual search experiment for different search conditions. It can be seen that the frequency of saccades was almost the same for different cues and backgrounds. The black dots in Figure 2a represent averages across conditions and patients. The error bars show standard deviations across patients. The small differences between search conditions and the large standard deviation demonstrate that individual patients had different saccadic habits, in terms of saccade frequency, that were not affected by cue and background conditions. We did not compare the patients with normally sighted controls in this experiment because their saccade sizes were strongly controlled by the three eccentricities (15°, 22°, and 29°) where “pop-out” targets were presented. Many of their saccades were 15° and 22°, which is an artificial effect. If the targets were presented at other eccentricities, their saccade sizes would have been different accordingly.

The two curves in Figure 2b are averages across five patients and three normally sighted subjects, respectively, in the walking experiment. Results for the two groups were very similar. Compared with the visual search data, there were fewer large saccades in the walking experiment.

Like Bahill et al, who studied the saccades of normally-sighted subjects walking through a university campus (Bahill et al., 1975), we fit exponential curves to our data (nonlinear regression with SPSS 11.5). The fitted exponential curves (R² ≥ 0.97) were:

$$\begin{array}{l}\begin{array}{ll}\text{Patients}\text{in}\text{visual}\text{search}-\hfill & S=37.5\times e^{-\frac{A}{10.64}}\hfill \\ \text{Patients}\text{while}\text{walking}-\hfill & S=95.3\times e^{-\frac{A}{3.75}}\hfill \\ \text{Normaly}\text{sighted}\text{subjects}\text{while}\text{walking}-\hfill & S=125.1\times e^{-\frac{A}{2.89}}\hfill \end{array}\\ \end{array}$$

where S is the saccade frequency in occurrences per minute, and A is the saccade size. The decay coefficients (10.64, 3.75 and 2.89) inversely indicate how fast the curve decreases. Larger decay coefficients represent more large saccades. The decay coefficients were also calculated for each patient, and they did not correlate significantly with patients’ VF (Pearson r=0.43, p=0.24 for search, r=−0.55, p=0.34 for walking). Note the correlation was opposite for search and walking experiments, which suggests there was indeed no correlation between decay coefficients and VF size.

Bahill et al reported that “Most naturally occurring human saccades have magnitudes of 15 degrees or less” (Bahill et al., 1975). Compared with the decay coefficient they found (7.6), both our patients (3.75) and controls (2.89) in the walking experiment had a smaller decay coefficient, while the patients in our visual search experiment had a larger decay coefficient (10.64). These decay coefficients indicate that, overall, saccades in visual search were larger, and saccades when walking were smaller than in Bahill’s study.

2. Saccades beyond the VF

Table 1 lists the frequencies of the total saccades and the saccades beyond the VF in patients with tunnel vision. Saccades beyond the VF include those larger than half of the VF, and those larger than the full VF. In the former case, post-saccadic fixations were invisible at saccade onset. In the latter cases, the pre- and post-saccadic views did not overlap at all. This interpretation is a conservative approximation, since the patients’ VFs were not perfectly round, but generally oval-shaped. VF sizes reported here were measured along the long horizontal axis. If a saccade larger than half VF is along the short axis, the post-saccadic fixation is actually farther out than just invisible at the VF boundary.

Table 1.

Frequencies ±SD (in saccades per minute) of total saccades and those beyond the VFs of patients with tunnel vision in the visual search and walking experiments.

	Searching	Walking*
Total saccades	171±34	160±28
Saccades > half VF (%)	107±17 (63± 9%)	56±23 (35±14%)
Saccades > full VF (%)	54±17 (33±14%)	24±16 (15±10%)

^*normal controls in walking: 145/min

The frequencies of saccades of patients in the visual search and walking experiments were similar – nearly 3 per second. The patients made a large number of saccades beyond their VFs in both experiments. Based on the saccades we detected, about two-thirds of the saccades (or 107/min) in the visual search and one-third of the saccades (or 56/min) in the walking experiment were beyond the VFs. Among these beyond-VF saccades, saccades larger than full VFs averaged almost one per second (54/min) in the visual search, and averaged one every 2.5 seconds (24/min) while walking.

3. Saccade direction

Figure 3 and Figure 4 illustrate the distribution of saccade directions in the visual search and real-world walking experiments, respectively. Although targets or salient features in both experiments were evenly distributed in all directions, the dominant directions of saccades tended to lie in the horizontal and vertical directions. In the visual search (Figure 3), saccade direction distribution was not affected by whether or not picture backgrounds were presented, while auditory cues seemed to cause more saccades along vertical and less along horizontal direction. In the walking experiment, the distribution of saccade directions was similar for patients and normally sighted controls – the frequencies of horizontal and vertical saccades were about the same in both groups.

Figure 3.

Saccade direction polar histograms (average across subjects) in a visual search experiment by combination of auditory cues and picture background. Unit along radii: saccade/minute. Bin size: 15°. Background did not have an obvious effect on saccade direction. Auditory cues seemed to elicit more vertical saccades and less horizontal saccades per minute.

Figure 4.

Saccade direction polar histograms (average across subjects) in a real-world walking experiment. Unit along radii: saccade/minute. Bin size: 15°. Left: saccade direction of five patients with tunnel vision. Right: saccade direction of three normal controls. No obvious difference is noted between the two groups.

Discussion

We have found that people with tunnel vision make beyond-VF saccades very frequently (one to two times per second) not only in the laboratory visual search experiment, but also in one of their daily activities – walking. These beyond-VF saccades could not be triggered by instantaneous visual salience. The smooth transition of the distribution curves of saccade size from within-VF to beyond-VF range might imply that the sizes or goals of the saccades are influenced by a non-visual mechanism. This is also supported by our findings that the decay coefficient of saccade size distribution did not correlate with VF size. Furthermore, we found that normally sighted people appeared to have almost the same distributions of saccade size and direction as the patients in the real world walking experiment, although they had much more visual salient features in their full VFs. This finding may further support an important role of the non-visual mechanism of eye movement control in both normally sighted and patient groups.

Habit is unlikely to be the non-visual mechanism as the saccades of the two patient groups in the searching and walking experiments were quite different. In other words, saccade size appears to be task related: the patients and normally sighted controls had similar saccade size and direction in the same walking task in spite of their very different VF sizes, the two patient groups had different saccades in the searching and walking conditions despite their similar VF sizes, and our normally sighted subjects walking on Boston streets had different saccade sizes from the subjects in Bahill’s study who walked through the UC Berkeley campus (Bahill et al., 1975). We think this non-visual mechanism is probably a top-down process, which has been widely shown to be involved in eye movement control (see Hayhoe & Ballard, 2005 for reviews). Our findings provide a clue about the extent of the top-down mechanism influence on eye movement control. It seems to be large in the patients, given the fact that one-third of their post-saccadic gazes in real world walking and two-thirds in searching were outside pre-saccadic views (Table 1). Normally sighted people may not need to rely on top-down information as much as the patients, but the strong similarities in saccade size and direction they shared with the patients might suggest a similarly important role for top-down mechanism also in people with full VFs. This is also supported by the findings of Rothkopf et al in their virtual environment study: gaze points are directed toward task related regions, and non-task related bottom-up saliency is not a good predictor of gaze allocation (Rothkopf et al., 2007).

However, we also believe there should be some differences in eye movements between normally sighted people and patients with tunnel vision, since it is well known that bottom-up mechanisms based on visual saliency do play an important role in eye movement control. It has been shown that an ideal searcher model based on a “visibility map” matches human observers’ eye movement in search for sin-wave targets (Najemnik & Geisler, 2005). We have previously reported that the patients had horizontally narrower distribution of eye position than normally sighted while same vertically (Vargas-Martín & Peli, 2006). We think this could be because the patients had less salient features to attract their eyes to left or right (bottom-up mechanism), and also because they needed to look down at the ground for potential obstacles more frequently (top-down mechanism). It would be more insightful to investigate what the patients try to look at when they make beyond-VF saccades. Due to technical difficulties of recording head movement outdoors and analyzing the data about targets they look at, we were not able to do that in this study. We believe that large saccades may not necessarily bring gazes right on targets for the patients, but making large saccades is probably an effective approach to bring targets they want to see into their fields of view. Gazes can then be guided to targets based on bottom-up saliency.

After we found that the patients make so many beyond-VF saccades, we called back four of the patients and debriefed them about their visual experience when they made beyond-VF saccades. Their responses did not suggest that beyond-VF saccades cause any unusual visual experience, such as sudden image change or object jump. Subjects said that objects in their post-saccadic views are perceived as if they have been there all along and just became visible, rather than perceived as suddenly emerging. Along the same line, objects in their pre-saccadic views become simply out of sight, rather than abruptly disappear.

To confirm what they reported, we conducted a simple test with a stripe pattern and 4-dot patterns as illustrated in Figure 5. The patients were instructed to make non-overlapping saccades between two fixation crosses. The two fixations were placed apart far enough (~24°) to ensure they did not appear simultaneously in VFs at the tested distance (14 inches). The patients never perceived shifting stripes, although the stripes (0.5 cycle/deg) were unlikely to fall on the same retinal position from pre- to post-saccadic views. In the right 4-dot pattern, the right dot was 2° farther away from its fixation mark than that in the left 4-dot pattern, and the other three dots were the same in both patterns. The patients never perceived a jumping right dot.

Figure 5.

When patients with tunnel vision make saccades larger than their full VFs over a stripe pattern, they never perceive jumping stripes, although the stripes are unlikely to fall on the same retinal position. When patients made a saccade from the left 4-dot pattern to the similar pattern on the right side in which one dot is 2° further away from the fixation mark, they never perceived a jumping dot. Two fixations are placed apart far enough (~24°) to ensure saccades larger than full VFs at a distance of 14 inches.

Patients’ reports and our simple tests show that when they make beyond-VF saccades, the pre- and post-saccadic views of patients do not fuse together. Instead, they are pinned at different (and probably correct) locations on the world representation map and thus allow patients to maintain reasonably good orientation. This can be accounted for by the phenomenon of saccadic remapping based on efference copy of oculomotor signals (Duhamel, Colby, & Goldberg, 1992; Sommer & Wurtz, 2006).

In cases of beyond-VF saccades, the post-saccadic view becomes visible from a blindness status, not from a visible background. Some forms of trans-saccadic integration of visual memory that may occur in normally-sighted people (Currie, McConkie, Carlson-Radvansky, & Irwin, 2000; Hayhoe, Lachter, & Feldman, 1991; McConkie & Currie, 1996; Melcher, 2005, 2007; Prime, Niemeier, & Crawford, 2006) are not possible in the patients. Thus, there is no basis for them to perceive any image change across saccades. Along the same line, we speculate that a short visual blank resulting from saccadic suppression might probably have a similar effect to prevent the detection of intrasaccadic image change (Bridgeman, Hendry, & Stark, 1975; Currie et al., 2000; McConkie & Currie, 1996), even though some high level memory probably can be integrated across saccades. The effect of such a visual blank might be similar to that of a blank flicker between two images (Rensink, Oregan, & Clark, 1997) or eye blinks (O’Regan, Deubel, Clark, & Rensink, 2000) that can cause change blindness. However, in some specific situations where stimuli are presented intermittently rather than in a continuous illumination manner, intrasaccadic visual perception is not effectively suppressed and motion perception may occur (Castet, Jeanjean, & Masson, 2002; Castet & Masson, 2000; Deubel, Schneider, & Bridgeman, 1996; Garcia-Perez & Peli, 2001; Hershberger, 1987; Schlag & Schlagrey, 1995).