Looking at the center of the targets helps multiple object tracking

Abstract

The ability to move our gaze to locations of interest facilitates interactions in everyday life. Where do participants direct gaze when multiple locations are of interest simultaneously? We previously demonstrated that, when tracking several moving targets amidst distractors in a multiple object tracking (MOT) task, participants primarily looked at a central point in between the targets (H. M. Fehd & A. E. Seiffert, 2008). This strategy of center-looking is in contrast to a target-looking strategy where participants would saccade from target to target. Here we investigated what factors influence the use of center-looking as well as its effectiveness. By decreasing object speed, we determined that center-looking is not a result of avoiding costly eye movements during tracking. Decreasing object size showed that peripheral visibility is necessary for tracking, but that center-looking continues up to the limits of peripheral visibility. Further analysis revealed that participants often engaged in both target-looking and center-looking by switching gaze from the center to targets and back again. Directly comparing participants’ performance when they either did or did not include center-looking along with target-looking revealed that center-looking facilitates tracking performance. These results suggest that there is value in looking at the center that relates directly to the process of tracking multiple objects.

Introduction

Eye movements often reflect the intentions of an observer, such as fixating a kettle when making a cup of tea (Land, Mennie, & Rusted, 1999). More complex goals often engender more deliberate eye movements. For instance, participants given the task of reading a passage for content typically engage in either a linear strategy of reading every line without looking back to prior sentences or a look-back strategy where they return to topic phrases repeatedly throughout the passage (Hyönä, Lorch, & Kaakinen, 2002; Hyönä & Nurminen, 2006; Rayner, 1998). Notably, readers using the look-back strategy do better than linear readers on tests of retention (Hyönä & Nurminen, 2006). In a task where they had to move along a path, participants habitually fixated the center of objects they approached and the edges of objects they avoided (Rothkopf, Ballard, & Hayhoe, 2007). Participants’ object fixation patterns had such regularity that looking at the data alone could predict the task condition. Strategic use of eye movements can increase the efficiency with which a task is performed. During a simple task of arranging colored blocks to copy a pattern, participants frequently looked back at the pattern to be copied rather than only relying on their memory of it (Ballard, Hayhoe, Li, & Whitehead, 1992). While these eye movements were not essential to task completion, preventing participants from moving their eyes caused them to take about three times as long. This literature suggests that routine gaze shifts can reveal strategic eye movement patterns that can ultimately improve performance. The main questions of the present study are what factors determine where participants look during a task demanding attention to multiple objects and if there is value in utilizing a given looking strategy.

Because people have one focal point, there can be a dilemma about where to look during a task when they must attend to more than one thing. We previously examined the patterns of eye movements participants made during a multiple object tracking (MOT) task (Fehd & Seiffert, 2008) in which attention is required (Tombu & Seiffert, 2008). This task, introduced by Pylyshyn and Storm (1988), requires participants to keep track of a subset of several identical moving objects (see Cavanagh & Alvarez, 2005 for review). A naive strategy for this task might be to maintain fixation on a target and switch gaze between targets periodically. Landry, Sheridan, and Yufik (2001) demonstrated that participants do this when monitoring the movements of multiple objects for the purpose of detecting potential collisions. Such a strategy, however, yields high resolution for only the fixated target and much lower resolution for the other targets.

We found that people tend to look toward the center of the shape formed by the objects they are tracking in addition to looking at the targets themselves. For example, when tracking 3 targets, participants often looked at the center of mass, or centroid, of the triangle formed by the targets. We refer to this tendency to place gaze at the center of a target formation as center-looking. All participants tested spent time both center-looking and target-looking during multiple object tracking. The center-looking tendency presents an opportunity to study the cognitive components of eye movements made during tracking. What is it that draws peoples’ gaze toward the center? Do people simply look at the center out of laziness? Or is there some value in keeping gaze there? How do people employ both center-looking and target-looking during object tracking?

Several studies show that people look at the center of objects with which they are planning to interact. When participants make saccades to peripheral objects, their gaze tends to land in the center of the object (He & Kowler, 1991; Kowler, 1995; Vishwanath & Kowler, 2003). Scanning from one object to the next, participants fixate the center of objects and are not influenced by parts or object configuration (Vishwanath & Kowler, 2003). When about to grasp an object, participants also initially look at the center of the object, before saccading to areas of the object that require visual feedback for a precise grasp (Brouwer, Franz, & Gegenfurtner, 2009). These results and others suggest that people may look at the center of target objects because of a localization process that roots gaze to the center of the object that they are attending to and/or acting upon (Aivar, Hayhoe, Chizk, & Mruczek, 2005; Hayhoe, Shrivastava, Mruczek, & Pelz, 2003; Mennie, Hayhoe, & Sullivan, 2007).

While these studies examined the manner in which the eyes move to localize specific targets in a display, the current study extends this work by using displays in which multiple targets are intermixed with distractors and distributed across the display. Previous work found that when participants made saccades to a group of peripheral targets, the saccadic endpoint landed near the center of the target group, both when the targets were alone and amidst differently colored distractors (Cohen, Schnitzer, Gersch, Singh, & Kowler, 2007; McGowan, Kowler, Sharma, & Chubb, 1998). However, unlike the stationary target groups employed previously, we have investigated eye movements during MOT in which targets move and can only be distinguished from distractors via accurate attentive tracking. Looking toward the center of an object may be essential for interacting with it. In the case of MOT, continually maintaining the distinction between targets and distractors may also involve a similar orienting response to the dynamic target group. Thus, one might look to the center of the target group in MOT as a way to spatially localize the target group and help differentiate them from distractors.

We investigated the possible benefits of a center-looking strategy in MOT and explore two potential explanations for center-looking. First, we determined if center-looking is an artifact of a general strategy of avoiding making many eye movements during tracking by making the objects move more slowly, allowing time for more eye movements. Second, we investigated whether center-looking occurs only because peripheral vision is sufficient to keep track of targets by making the objects so small that participants needed to fixate targets more often. Finally, we determined the usefulness of center-looking by directly comparing tracking performance when participants engaged in center-looking along with target-looking compared to when participants only looked at targets. We have also analyzed the patterns of eye movements in more detail to better describe gaze behavior during object tracking. To anticipate, we discovered that looking at the center was a pervasive gaze strategy that was used often in alternation with saccades to targets, a new strategy we have labeled “center–target switching”.

Experiment 1

Experiment 1 examined whether avoiding saccading to targets is a factor governing the use of center-looking in MOT. It is possible that center-looking was the predominant strategy participants used in Fehd and Seiffert (2008) because the targets moved too fast to make saccading to targets worthwhile. Although keeping gaze at the center likely involved both saccadic and smooth pursuit eye movements, switching gaze between targets would usually necessitate larger saccades, on the order of several degrees. Visual information is somewhat suppressed during saccades and this suppression is longer for larger saccades (Bridgeman, Hendry, & Stark, 1975; Burr, Morrone, & Ross, 1994; Campbell & Wurtz, 1978; Riggs, Merton, & Morton, 1974; Volkmann, 1986; Wurtz, 2008; Zuber & Stark, 1966). Visual information must also be updated retinotopically after a saccade occurs (Colby, Duhamel, & Goldberg, 1995; Duhamel, Colby, & Goldberg, 1992; Klier & Angelaki, 2008). Because the dots continue to move during a saccade, perception of dots after the saccade must then be compared to the memory of dot locations from before the saccade in order to recover the displaced targets. Previous work has demonstrated impairments in the recovery of targets after they disappear momentarily and keep moving while absent (Fencsik, Klieger, & Horowitz, 2007; Horowitz, Birnkrant, Fencsik, Tran, & Wolfe, 2006; Keane & Pylyshyn, 2006). The degree of impairment increased with the duration of the disappearance, and thus distance of target displacement.

The probability of losing a target during a saccade can be modulated by dot speed. In our previous work demonstrating center-looking, dots moved at about 15°/s, so the distance that a target moved during a 200-ms saccade was approximately 3°. This was larger than the diameter of the dots and may have been too large for the target’s location to be accurately recovered after the saccade. Participants may have used center-looking as the default strategy not because it is more effective than target-looking, but because of the probability of losing targets during saccades. Here, we investigated the effect of reducing the speed of the dots in the display to reduce participants’ risk of losing targets while making an eye movement. At the slowest speed tested, 3°/s, targets moved only about 0.5° during a 200-ms saccade that was only about one fourth of the size of the dots. If participants only employ center-looking to avoid costly saccades to targets, then center-looking should be absent when the targets move slowly because they will move only trivial distances during the time it takes to make a saccade. However, if participants look at the center for reasons other than avoiding costly eye movements, participants should continue to look at the center even when the targets move slowly.

Methods

Participants

Thirteen people (5 females; aged 18–22) from Vanderbilt University participated in this experiment following the procedures for the protection of human participants provided by the Vanderbilt University Institutional Review Board and defined in the APA Code of Ethics (2002). Participants’ data were subject to exclusion from the final analysis if sufficient eye movement data were not acquired. Individual trials were included if less than 10% of the eye movement data were lost due to errors with the equipment, calibration, or participants’ motion (such as blinks or head motion). Data from one participant were removed because the number of excluded trials for this participant exceeded 30%. The average number of excluded trials for the rest of the participants was 6.3%.

Apparatus and stimuli

We monitored eye movements using an Applied Systems Laboratory EYE-TRAC 6000 (ASL, Bedford, MA, USA) running at 120 Hz. Participants used a chinrest and headrest to sit 38.5 cm from the computer monitor. Stimuli were created with Matlab for OS X and the Psychophysics toolbox (Brainard, 1997; Pelli, 1997). The visual display was generated by a Macintosh eMac driving a Sony Trinitron Multiscan E540 monitor at 120 Hz.

Stimuli were 10 red dots, 2.1° of visual angle in diameter, presented within a white rectangular frame (39.8° × 39.8°) on a black background. Green rings, 2.9° in diameter, were used to designate four dots as targets. Dots appeared in randomized starting positions that prevented dots from overlapping with each other or the frame. Each dot moved in a random Brownian-like motion constrained so that each one moved a set number of pixels per frame. We tested five speeds, with the dots moving 0.5 (3°/s), 1 (6°/s), 2 (12°/s), 3 (18°/s), or 4 (24°/s) pixels per frame. When the speed was set to 0.5 pixel per frame, dots moved 1 pixel every two frames. The range of speeds used here was similar to that of previous studies investigating speed changes in MOT (Alvarez & Franconeri, 2007; Liu et al., 2005) and was intended to cover a wide range of performance accuracies. The center was defined as the centroid, or center of mass of a polygon with the targets as its vertices. Because the center’s movement depended on the configuration of the targets, its speed was reduced to roughly half that of the dots for each of the 5 speeds. The median number of pixels the center moved per frame was 0.27, 0.54, 1.08, 1.63, and 2.21 for each speed, respectively.

Procedure

Participants completed one 60-min session containing 120 experimental and 5 practice trials. At the beginning of the session and after every block of 15 trials, the ASL system was calibrated using a 17-point calibration. We tested five speed conditions, mixed within each block. Each trial began with a display of stationary dots and green rings designating 4 of the dots as targets for 3 s. Target cues disappeared and all dots started to move. Dot motions were chosen randomly and each item continued along its vector unless it was in danger of overlapping with the bordering frame or fellow stimuli. To prevent overlap, a repulsion mechanism redirected a target or distractor to move away from any other target or distractor or the frame if it was less than one dot diameter away. Because the period of motion was 5 s and the eye tracker collected data at a rate of 120 Hz, 600 frames of eye movement data were recorded on each trial. At the end of each trial, participants selected each of the tracking targets with the computer mouse. A high or low tone provided feedback for each correct and incorrect selection, respectively.

Eye movement analysis methods

We assessed the location of gaze with a region of interest method. Regions that were 5° or twice the diameter of each stimulus were defined over each of the 10 stimuli and the center of the target group. The number of movie frames that the eye position overlapped with each of the regions was summed for each trial. For frames in which gaze overlapped with more than one region, the frame counted for both of them. We divided the total number of frames of overlap by the total number of frames in the trial to create the percentage of gaze overlap for each region. For the statistical analysis, these percentages were then averaged across each of the targets to create the average percentage of gaze overlap on targets per condition per participant. The same was done for distractors. In the figures, we have shown the data for each region of interest separately to clearly represent how gaze was distributed in time across regions. Only trials in which all targets were correctly identified were selected for eye movement analysis. We did not analyze trials in which participants correctly tracked only a subset of targets because participants may not have known the location of the center of the target group on those trials.

Results

Accuracy was defined as the percentage of trials in which all targets were correctly identified, a measure with a chance level of 0.48% (1 correct out of 210 possibilities of choosing 4 of the 10 dots). The average percent correct across all speed conditions was 58%. An ANOVA on the accuracy data showed a significant main effect of speed (F(4,44) = 119.7, p < 0.01), with accuracies ranging from 13% for the fastest speed to 95% for the slowest speed (Figure 1A).

Figure 1.

Accuracy and gaze overlap percentages across dot speeds. (A) Percentage of trials for each speed condition in which all targets were correctly identified. (B) Average percentage of gaze overlap with the regions surrounding each of the four targets (blue), each of the six distractors (black), and the center of the targets (red) shown for only correct trials across speed conditions. Error bars depict standard errors.

We analyzed the percentage of gaze overlap to see if viewing time varied as a factor of the region with which it overlapped (target average, distractor average, or the center of the targets) and the speed of the dots (Figure 1B). An ANOVA showed a significant main effect of region (F(2, 16) = 86.6, p < 0.01), but the effect of speed was not significant (F(4, 32) < 1). The interaction between region and speed was also not significant (F(8, 64) < 1). Simple effects analysis showed that the percentage of overlap with the center did not increase as speed increased (F(4, 32 < 1) and the decrease in overlap time with the targets as speed increased was only marginally significant (F(4, 32) = 2.5, p = 0.06).

To further understand the use of eye movements, we calculated the duration of the dwell times on each region. Dwell time was operationalized as the number of consecutive frames a particular region was the closest to gaze. The average dwell time decreased significantly as the speed of dots increased (F(3, 30) = 9.7, p < 0.01). In addition, the overall number of switches, or changes from one region to the next, showed a slight, but not significant, increase with speed (F(3, 30) = 2.1, ns). There were more switches between the center and a target than between two targets (7.4 vs. 4.5; F(1, 10) = 28.9, p < 0.01). This difference did not vary across speeds (F(4, 32) < 1).

Further inspection of these switches revealed evidence of systematic streaks of alternations between two regions. The data suggested that people often looked from the center to a target and back again, a strategy we refer to as “center–target switching”. A streak was defined as any repeated switches between two regions. For example, a switch from the center to a target, back to center, and back to a target was defined as a center-to-target streak. A switch from one target to another that was repeated was defined as a target-to-target streak. There were 19.1 streaks defined on average per trial, and center-to-target streaks were more common (5.9) than target-to-target streaks (1.9). For each of the streaks, the duration of the streak was calculated by summing the durations of each dwell time within the streak. Analysis of the average streak durations found that there was a significant effect of streak type (F(1, 8) = 92.5, p < 0.01) and no effect of speed (F(4, 32) = 1.2, ns). Center-to-target streaks lasted much longer (1.0 s) than target-to-target streaks (0.2 s) on average. Given that the duration of the trial was 5 s, center–target switching took about 20% of a trial on average. About half of the time spent on center-to-target streaks was center-looking (0.53 s). The average amount of center-looking that was not within a center-to-target streak was about the same (0.52 s). Thus, it seems that center–target switching explains some of the eye movement behavior during object tracking, but there is more to the behavior than we have accounted for thus far.

To summarize, these data indicate that as the speed of the dots decreased, the amount of center-looking remained constant. Additionally, a common pattern of eye movements observed was a repeated switch from looking at the center to looking at a target and back.

Discussion

The objective of this experiment was to test whether people choose to look at the center when tracking multiple objects because of the difficulties that arise from the distance targets move during saccades. We found that there was a strong tendency for observers to engage in center-looking, even at the slowest speeds tested that would allow participants more time to make saccades without missing substantial target movement. The total amount of time that gaze was at the center of the target group per trial did not change as speed decreased, but the dwell time of continuous fixations at the center increased. At the slower speeds, participants viewed the center for prolonged periods of time, while at higher speeds they made quick glances to the center. Notably, we also observed that participants often used a center–target switching strategy in that the majority of the gaze shifts between regions were from the center to targets and back. These results also suggest that participants never completely abandoned the center in favor of the targets. In fact, at the slowest dot speeds—when the price of eye movements was lowest—there was still more center–target switching than switching between targets. These results suggest that looking at the center of the targets might be useful to the tracking process for reasons other than avoiding costly eye movements.

Experiment 2

During center-looking, the information about the targets is accessible in the visual periphery. If participants are looking at the center, they usually will not be fixating any particular target. Center-looking may be so prevalent because peripheral information about each target is sufficient to maintain representations of target locations. If center-looking works as a strategy because participants can use peripheral information to track targets, then limiting the ease of access to this information should decrease the use of the center-looking strategy. In this experiment, the size of the targets in the display was reduced so that peripheral visibility of the targets was drastically limited. As visibility declines, the number of target fixations is likely to increase because foveation would ensure perception of even the smallest targets. If center-looking relies on the peripheral resolution of the targets, participants should abandon the center-looking strategy when the targets are too small to see in the periphery. However, if participants employ the center-looking strategy even when access to peripheral information is limited, it would suggest that center-looking is helpful to participants for reasons beyond the use of peripheral vision.

Methods

The methods for Experiment 2 were the same as in Experiment 1, except for the following alterations. Twenty-seven people (5 females; aged 20–33) from Vanderbilt University participated in this experiment. Using the same eye data criterion as in Experiment 1, data from six participants were removed because their number of excluded trials exceeded 30%. The average number of excluded trials for the rest of the participants was 11.8%. Participants completed one 60-min session containing 125 experimental and 5 practice trials. At the beginning of the session and after every block of 25 trials, the ASL system was calibrated using a 17-point calibration. The 5 size conditions tested were randomly mixed within blocks.

Unlike Experiment 1, where the stimuli were circular, targets and distractors in Experiment 2 were squares. Squares ranged from 1 to 5 pixels on a side (0.06° to 0.3° of visual angle). The largest size targets were 7 times smaller than those used in Experiment 1. Squares were light gray (CIE x = 0.284, y = 0.294, Y = 46.9) on a background that was dark gray (CIE x = 0.286, y = 0.288, Y = 20.8) and moved at 12°/s. We chose this speed because in Experiment 1, the tracking accuracy was above chance and below ceiling (~65%) at this speed and the total amount of the center was viewed was roughly equivalent to the sum of the average amount each target was viewed (39% vs. 44%).

Results

Accuracy was defined as the percentage of trials in which all targets were correctly identified (chance = 0.48%). The average percent correct across all size conditions was 37%, ranging from 0.01% to 53.9% from the smallest to largest sized targets (Figure 2A). An ANOVA on the accuracy data showed a significant main effect of size (F(4, 80) = 86.4, p < 0.01), indicating that accuracy diminished significantly with the size of the targets. The accuracy was so low at the smallest size (0.06°) that only 4 of the participants managed to successfully track all 4 targets on one trial each, providing an inadequate sample size of correct trials to be analyzed for eye movements. Data for 0.06° have not been plotted in Figure 2B for this reason. This result suggests that the stimuli at this size were past the limits of peripheral acuity or attentional resolution (Intriligator & Cavanagh, 2001).

Figure 2.

Accuracy and gaze overlap percentages across dot sizes. (A) Percentage of trials for each size condition in which all targets were correctly identified. (B) Average percentage of gaze overlap with the regions surrounding each of the four targets (blue), each of the six distractors (black), and the center of the targets (red) shown for only correct trials across all but the smallest dot size conditions, where performance was at chance. Error bars depict standard errors.

We assessed percentage of gaze overlap with the same region of interest method as Experiment 1, with the following modifications. First, we used a region for the analysis that was reduced from 5° to 0.6°, so that it was twice the size of the largest dots. An ANOVA of the correct trials showed that the main effects of region (F(2, 36) = 79.8, p < 0.01) and size (F(3, 54) = 4.3, p < 0.01) were significant, as well as their interaction (F(6, 108) = 4.6, p < 0.01). Gaze overlapped with the center (4.0%) much more of the time than the targets (0.9%) or distractors (0.3%). This replicates the pattern of our previous results; however, the percentages of overlap are much lower than previously observed due to the smaller regions used in this analysis. Simple effects analysis showed that the percentage of overlap with the center lowered from 5.0% to 3.2% as size decreased (F(3, 54) = 4.6, p < 0.01) while the target overlap increased from 0.9% to 1.3% (F(3, 54) = 4.4, p < 0.01). These results are plotted in Figure 2B. As a secondary analysis, we increased the region size to 5 degrees, as used in Experiment 1, and found slightly different results. The enlarged region analysis found a significant main effect of region (F(2, 36) = 241.1, p < 0.01) but no effect of size (F(3, 54) = 1.2, ns) and no interaction (F(6, 108) = 1.7, ns). Gaze overlapped with the center much more of the time (34.3%) than the targets (7.8%) or distractors (3.1%). Taken together, the results indicate that, as target size decreased, gaze was directed less to the precise location of the center but maintained within the region near it.

As in Experiment 1, participants spent a considerable amount of time engaged in center–target switching. Again, there were more center-to-target streaks on average (7.2) than there were target-to-target streaks (1.5). Analysis of the average streak durations found that there were significant effects of streak type (F(1, 18) = 232.5, p < 0.01) and size (F(3, 54) = 3.6, p < 0.05) but no significant interaction (F(3, 54) < 1). Center-to-target streaks lasted much longer (1.4 s) than target-to-target streaks (0.1 s). Given that the duration of the trial was 5 s, center–target switching in this experiment took about 25% of a trial on average. Again, about half of the time spent on center-to-target streaks was center-looking (0.79 s). The average amount of center-looking that was not within a center-to-target streak was about the same or less (0.65 s). Both types of streaks had longer durations at the 0.12 dot size (0.2 for target-to-target and 1.5 for center-to-target) than any of the larger dot sizes (0.1 for target-to-target and 1.4 for center-to-target). These data indicate that participants systematically viewed and revisited the center during tracking despite the reduced size of the targets.

Discussion

The purpose of Experiment 2 was to determine if the ease of access to peripheral information was a contributing factor in the choice for observers to engage in center-looking. We limited access to peripheral information by decreasing the size of the targets in the display. This manipulation had a strong effect, as also found in previous work (Bettencourt & Somers, 2009). Accuracy fell off steeply at the smallest target sizes with chance performance at the smallest size tested. This drop in performance indicates that these stimuli did push observers to their limits of tracking small objects. The critical finding was that center-looking persisted even though target visibility in the periphery was greatly reduced. Participants spent time looking at the center of the target group even when the targets were so small that foveating these targets would have provided much better information about their locations than peripheral viewing did. These results suggest that participants employ center-looking because it provides some value to tracking objects beyond the use of peripheral vision. We also found additional evidence that participants engage in center-looking as part of a center–target switching strategy, which we investigated further in Experiment 3.

Experiment 3

The previous experiments found that participants commonly look at the center of the target array during tracking, making repetitive glances to targets throughout the tracking period. Although we have shown that this viewing strategy is persistent, it is not clear if this strategy of eye movements is actually helpful for maintaining each of the targets. Experiment 3 examined whether the center-looking strategy is beneficial to tracking performance. In this experiment, we measured tracking accuracy while participants followed two different strategies, center–target switching and target-looking. Target-looking refers to the eye movement strategy of looking from target to target during tracking. Center–target switching refers to the eye movement strategy of keeping gaze rooted at the center of the target array with occasional glances to targets. We chose center–target switching rather than only center-looking because it reflects the strategy that we observed most participants engaged in for substantial proportions of the time (20% in Experiment 1 and 25% in Experiment 2). In addition, center–target switching differs from target-looking in that participants are asked to also fixate the center of the targets. If looking at the center is useful to tracking, then performance should be improved when participants use center–target switching compared to when they use the target-looking strategy.

Methods

The methods for Experiment 3 were the same as in Experiment 1, except for the following alterations.

Participants, apparatus, and stimuli

Twenty-five people (13 females; aged 18–25) from Vanderbilt University participated in this experiment. As in previous experiments, trials were analyzed for eye movements if less than 10% of the eye movement data were lost due to errors with the equipment, calibration, or participants’ motion (such as blinks or head motion). The average number of excluded trials across participants was 5.6% and none of the data from a particular participant were excluded. The apparatus was the same as described in the previous experiments.

Stimuli were 12 red circles (1.8°) presented within a white rectangular frame (36.8° × 36.8°) on a black background. The 3 targets were designated by changing their color to green during the 3-s cue period. After the cues were removed, dots moved 2 pixels per frame (12°/s) for 3 s in the same pseudo-random motion described previously.

Procedure

Participants completed two 60-min sessions containing a total of 240 experimental and 11 practice trials. At the beginning of each block of 20 trials, the ASL system was calibrated using a 17-point calibration. In the first session, participants completed 6 blocks of 20 trials each for a total of 120 trials. They were not informed of either strategy and were allowed to freely view the display during tracking. We asked participants to indicate their tracking errors by pressing the space bar when they thought they might have lost a target.

Participants returned within 30 h, but no sooner than 12 h, to complete the second session. In the second session, each participant completed another 6 blocks of 20 trials each, two blocks for each of the three strategies: center–target switching, target-looking, and free-looking. The first block for every participant was a free-looking block to get a pure measure of the participants’ natural strategy. After they completed the first free-looking block, we instructed participants about target-looking and center–target switching. The order of the following blocks was controlled by assigning each subject to one of five independent orderings created using a reduced Latin square. There were five participants in each possible block ordering.

A visual aid was used to describe the strategies to the participants (Figure 3). To emphasize the importance of looking at the center, the center–target switching was referred to as center-looking in all the instructions to participants. Center–target switching instructions emphasized that participants should keep their gaze near the center point of the target group or near a target. Specifically, the participants were told when they looked at one of the targets, to look back at the center before looking at another target. Target-looking instructions emphasized that participants should keep their gaze near a target. Specifically, the participants were told when they looked away from one target, to be sure to look at another target. For both strategies, participants were not required to view targets in a certain sequence, look at a certain number of targets or at any location for any specified duration. During practice trials, the experimenter visually inspected eye movement patterns by watching the monitor with eye position overlaid on a depiction of the display screen. If participants’ eye movements did not seem to reflect the strategy that was instructed, the experimenter would further clarify the instructions. As a reminder, a shortened version of the written instructions for each strategy was shown at the beginning of each block. At the end of the second session, participants were asked which of the two strategies they preferred and why they preferred it.

Figure 3.

Visual aids for gaze strategies. (A) In the center–target switching condition, participants were told to look at the center or at one of the targets. If they looked at a target, they were told to look back at the center before looking at another one. (B) In the target-looking condition, participants were told to always look at one of the targets but not at the center. Participants were not told to look at the targets in a certain sequence, to look at any specific targets, or for any specific amount of time.

Results

Tracking accuracy

This experiment focused on the effect of the eye movement strategy manipulation on tracking accuracy. Accuracy was measured as the percentage of trials in which all the targets were correctly selected at the end of the trial, a result with a chance level of 0.45% (1 correct out of 220 possibilities of choosing 3 of 12 dots). Average performance for the first session, in which participants were free to use any eye movement strategy they chose, was 80.4%. The accuracy of free-looking trials in the second session was 82.5%, which did not differ significantly from the first session data (F(1, 24) = 2.0, ns). Performance was lower when participants were instructed to pursue a specific eye movement strategy, regardless of whether this was target-looking (57.0%) or center–target switching (76.7%; Figure 4A). An ANOVA on the second session accuracy data confirmed that there was a significant main effect of instructions (F(2, 48) = 52.6, p < 0.01). Adding the constraint of a specific eye movement strategy did reduce accuracy for both center–target switching (t(24) = 2.1, p < 0.05) and target-looking (t(24) = 10.0, p < 0.01). However, there was a benefit from engaging in center–target switching over the target-looking strategy (t(24) = 7.6, p < 0.01). This result clearly supports the notion that center-looking is valuable to the tracking process.

Figure 4.

Accuracy and gaze overlap percentages across instruction types. (A) Percentage of trials in which all targets were correctly identified for the free-looking conditions from session 1 (gray) and session 2 (black), as well as target-looking (blue) and center–target switching (red) instruction conditions. (B) Average percentage of gaze overlap with the regions surrounding targets, distractors, and the center shown for each instruction condition. Colors are the same as in (A). Error bars depict standard errors.

As in the previous experiments, we conducted the region of interest analysis to calculate the percentages of gaze overlap for each region. Similar to those results, the data from session 1 showed a significant effect of region (F(2, 48) = 181.9, p < 0.01), with a higher percentage of time that gaze overlapped the center (40.2%) than the targets (12.7%) and distractors (3.8%) on average. Data from the second session showed a significant effect of region (F(2, 48) = 377.7, p < 0.01) and instructions (F(2, 48) = 21.9, p < 0.01), as well as a significant interaction between the two (F(4, 96) = 95.6, p < 0.01). Simple effects analysis for each region showed a significant effect of instructions for each one (Center: F(2, 48) = 74.1, p < 0.01; Targets: F(2, 48) = 108.5, p < 0.01; Distractors: F(2, 48) = 12.3, p < 0.01). These differences across instructions can be seen in Figure 4B. Consistent with instructions, participants looked at the center less when given target-looking instructions than when given center–target switching or no instructions. Complementarily, participants looked at targets more when given target-looking instructions than the other conditions.

It is possible that participants’ compliance with the instructed strategies varied across trials, potentially interacting with tracking accuracy. To quantify compliance with instructions, we calculated the number of trials in which participants modified their eye movement behavior consistent with instructions. For a trial to be counted as a compliant target-looking trial, a participant had to have looked at the targets more than he or she did, on average, in session 1, and looked at the center less than in session 1. For a trial to be considered compliant with center–target switching, a participant had to have increased the time of center-looking and decreased the time of target-looking from session 1. Participants were significantly more compliant with target-looking instructions (71.6%) than center–target switching instructions (36.8%; F(1, 24) = 45.7, p < 0.01). This is most likely due to the fact that participants were spending a lot more time center-looking when given free-looking instructions, so the criterion for compliance was harder to reach. However, both compliant and incompliant trials show higher accuracy for center–target switching relative to target-looking instructions (76.0% vs. 53.3% for compliance and 80.3% vs. 61.0% for incompliance). An ANOVA of these data shows that the main effect of instructions was significant (F(1, 24) = 34.7, p < 0.01), and the effect of compliance was marginal (F(1, 24) = 3.1, p = 0.09), but the interaction was not significant (F(1, 24) < 1). Thus, even when the data were reduced to only those trials that exhibit signs of following the strategy indicated, there was still a clear benefit seen in tracking performance due to center-looking.

Discussion

The goal of Experiment 3 was to determine if asking participants to engage in center-looking would improve tracking performance relative to target-looking alone. Results indicated that performance was higher when participants engaged in center–target switching, which allowed center-looking, compared to target-looking, which did not. This result indicates that center-looking is beneficial to tracking performance. However, performance in both strategy conditions was impaired relative to the free-looking condition where participants’ eye movements were unconstrained. It appears that while center–target switching did result in better performance than target-looking, center–target switching did not entirely reflect participants’ natural behavior during tracking. In sum, instructed center-looking is beneficial to tracking but not as helpful as allowing participants to move their eyes naturally.

Conclusions

People can use eye movements to strategically aid attentional selection when they need to pay attention to more than one location. The goal of this study was to investigate a specific eye movement strategy, center-looking, that we previously found participants engaged in when simultaneously attending to multiple objects (Fehd & Seiffert, 2008). We tested two potential theoretical explanations for why people employ center-looking by investigating the factors involved in the choice to use it. First, center-looking does not seem to be a compromise made because people are attempting to avoid making costly eye movements to targets. Participants continued to gaze at the center even when the targets moved slowly, lowering the risk of losing a target due to a saccade. Instead, they increased the dwell time of the fixations on the center as the target speed decreased. Second, we found that the ease of maintaining access to peripheral information about each target could not fully explain the tendency to use center-looking. Participants viewed the center even when the small size of the targets pushed their limits of tracking in the periphery. Finally, when we directly compared tracking accuracy when participants were instructed to either look at the center periodically or not at all, we found participants tracked more effectively when they looked in the center. The results of these experiments strongly support the notion that looking at the center of the target group facilitates the process of tracking multiple moving targets. In this discussion, we will consider the implications of both target-looking and center-looking on tracking as well as the cognitive mechanisms involved.

When should people look at targets?

Directing fixation to an object is an obvious way to update one’s mental representation of the object. When involved in a food preparation task, gaze is directed to the object being manipulated with the hands accompanied by repeated glances to the next object that will be handled (Land & Hayhoe, 2001; Land et al., 1999). Looking at the objects with which one is interacting can be helpful to locate the object in addition to controlling its manipulation (Land & Hayhoe, 2001). During multiple object tracking (MOT) the primary focus of the participant is to continually locate each target. Though we have found that participants view the center the most, participants also often direct gaze to targets and may need to do so to some extent for accurate tracking. All participants spent some amount of time looking directly at targets. Target-looking and center-looking usually were interspersed throughout tracking, a pattern of eye movements that we described as center–target switching. On average, participants spent about 20% of the time of a trial looking from the center to a target and back to the center, with about even time spent looking at targets and the center during that time.

The current work suggests that participants fixate targets more of the time if they are too small to see peripherally. These results are based on overall averages, however, which blur the particular temporal patterns of when shifts are made in relation to certain parameters. The key to the value of target-looking may be in the timing. For instance, participants may shift gaze from the center to a target when distractors crowd that target. As crowding has been shown to be a strong limiting factor to tracking performance (Alvarez & Franconeri, 2007; Franconeri, Pylyshyn, Fisher, & Enns, 2008; Shim, Alvarez, & Jiang, 2008; Tombu & Seiffert, 2008), it may be a good starting place for investigating the intricacies of when gaze is shifted to targets. We found evidence in support of this idea because, on average, distractors were closer to targets immediately before a center-to-target gaze shift than prior to a target-to-center gaze shift (Experiment 1: 10.3 vs. 12.2, respectively, t(11) = 23.3, p < 0.01; Experiment 2: 9.6 vs. 9.9, respectively, t(20) = 5.2, p < 0.01).

Looking at targets is clearly important to tracking, yet when we required participants to always look at the targets, they had more difficulty in accurately tracking them. All but three of the twenty-five participants in Experiment 3 reported verbally that they preferred using the center–target switching because it was easier to keep track of targets that they were not fixating. It may be that shifting gaze to the targets is helpful only to the target fixated and harmful to the other targets. Certainly, the eccentricity of the non-fixated targets increases with every saccade from the center to a target. This could degrade the representation of non-fixated targets and make them more susceptible to confusion with nearby distractors. Determining when the best moment is to look at targets might be a crucial part of center–target switching. It is possible that participants focused on the center to determine when they should shift gaze to targets.

Implications of center-looking

As reported in Experiments 1 and 2, a closer look at the pattern of eye movements across time revealed that participants frequently alternated between looking at the center and looking at targets. This finding provided the framework to define center–target switching, in which participants alternate between looking at the center, looking at a target, then looking back at the center. In this strategy, gaze is anchored at the center, but glances to targets are performed intermittently. The unusual aspect of this strategy is, of course, that participants are not looking at the targets all of the time. In fact, they choose to look at a location of empty space that has no information about the targets and indeed is often closer to a distractor.

One aspect of center-looking to consider is whether participants attend to the center in addition to the targets. If we assume that attentional resources are limited, then devoting attention to the center location would actually diminish the strength of attention at each of the peripheral targets. One possibility is that attention is automatically directed to the locus of fixation. There is some evidence to suggest that it is difficult to ignore information presented at fixation. For instance, in a peripheral covert attention task, distractors presented at fixation are harder to ignore than those presented in the periphery (Beck & Lavie, 2005; Goolkasian, 1981). However, it is possible that the ability to disengage attention from fixation depends on task relevance, with irrelevant distractors being more easily ignored (Lichtenstein-Vidne, Henik, & Safadi, 2007). Currently, it is not clear if the point of fixation at the center is attended in addition to the targets when tracking multiple objects.

An additional facet of the center-looking strategy to consider is that it may be a more difficult strategy to employ because it requires the online computation of the center of mass of the targets. Perceptual estimations of the center of stationary dots have been shown to be influenced by symmetry and elongation (Friedenberg & Liby, 2008), thus it stands to reason that some configurations the targets form might produce more errors in determining the center. It may be that the differences between the free-looking and center–target switching in Experiment 3 come from participants performing more taxing mental computations to fixate the actual center of the targets, while their natural tendency could be less constrained.

Cognitive mechanisms for tracking objects

Researchers are currently debating the cognitive mechanisms underlying the ability to track multiple moving objects. What can our eye movement data tell us about the theories of these mechanisms? Visual index theory describes a limited number of pre-attentive visual indexes that are tagged to objects in a scene and dynamically update as they move (Pylyshyn, 2001; Pylyshyn & Storm, 1988; Pylyshyn, 2006). Attention can then be allocated to one of the objects at a time by using the index as a pointer. While this theory does not stipulate any relationship between attention and eye movements, it is possible that a target-looking strategy is more consistent with the theory if gaze is moved from target to target similarly to attention. However, the observation that participants look at the center of the target group, and that looking at the center has value to the tracking task, does not follow from visual index theory.

According to an alternative view, MOT is accomplished via independent attentional selection of all of the targets at once. This view, called multi-focal attention, suggests that attention is split up into multiple foci that are allocated simultaneously to the locations occupied by each tracked object (Allen, McGeorge, Pearson, & Milne, 2004, 2006; Cavanagh & Alvarez, 2005). This theory also does not make specific predictions about eye movements during tracking. However, if tracking is achieved by directing separate attentional foci to each target’s location simultaneously, gaze may be equally pulled in all the attended directions and end up directed to the balance point between these foci, which would be the center of the target group. Because attention is also directed to targets in this theory, eye movements to these targets may also be predicted. The timing of when participants direct gaze to targets and when they direct gaze to the center may depend on the dynamic distribution of attention across the target locations. Results from Experiments 1 and 2 found that participants sustain the center-looking strategy when targets are slow or very small. This supports the notion that participants evenly divide attention among the targets, resulting in center-looking despite changes in the display characteristics. Results from Experiment 3 showing better tracking performance with center-looking could be taken to suggest that divided attentional focus is better managed when equal attention is paid to all targets, rather than unevenly distributed to one target over the others, as one might expect during fixation of one target. Although there is a close relationship between spatial attention and fixation, it is not perfect (Lichtenstein-Vidne et al., 2007) so it is possible that these observations of eye movements do not reflect attentional allocation. A better test of this theory would be to tax attentional demand differently at each of the target locations to see if it changes the eye movement behavior. In summary, the results seem potentially consistent with multi-focal attention theory, although further testing is warranted.

A third theory posited to account for multiple object tracking is an object-based theory. Previous investigations of multiple object tracking have shown that tracking performance is improved when participants employed the strategy of mentally grouping the multiple targets into a single polygon and tracking the contorting “virtual” object as a whole (Yantis, 1992). Perhaps if people conceive of the targets as forming an object, then they may look at the center of the object formed by the targets. This idea is consistent with the findings that people look at the centers of objects they grasp (Brouwer et al., 2009), approach (Rothkopf et al., 2007), and saccade to (He & Kowler, 1991; Kowler, 1995; Vishwanath & Kowler, 2003). According to this object-based theory, gazing at the center of the object formed by the targets may help to somehow reinforce the mental representation of the object and its vertices, the tracking targets. This would predict the performance boost from center-looking observed in Experiment 3. The data from Experiment 1 could also be consistent with the object-based theory, as one might expect that representation of the virtual object would be unaffected by target speed. The object-based theory could account for the results of Experiment 2 in the same way, with the addition that the grouping process that forms the object representation breaks down when targets became so small that tracking was no longer possible. The object-based theory, therefore, could account for the results of these experiments.

Ultimately, the results of these experiments suggest that both the balance point of multi-focal attention and the object-based grouping theories can explain eye movement behavior during multiple object tracking. As such they also provide insight into why people tend to look at the center of the objects they are tracking. Looking at the center may be valuable to tracking because it allows for an equal balance of attention across multiple targets or because it directs attention to an object representation formed by imagining a connection between the targets. In order to further investigate the nature of center-looking, it will be necessary to clarify the role that targets play in the strategy. All of the present experiments found that recurring shifts between the center and targets were prevalent. The key to determining when it is important to look at the center will be to discern when directly looking at the targets is critical. Overall, these results suggest that looking at the center of the target group is a valuable eye movement strategy when the intention of the observer is to keep track of multiple moving objects.