Object-centered representations support flexible exogenous visual attention across translation and reflection

Abstract

Visual attention can be deployed to stimuli based on our willful, top-down goal (endogenous attention) or on their intrinsic saliency against the background (exogenous attention). Flexibility is thought to be a hallmark of endogenous attention, whereas decades of research show that exogenous attention is attracted to the retinotopic locations of the salient stimuli. However, to the extent that salient stimuli in the natural environment usually form specific spatial relations with the surrounding context and are dynamic, exogenous attention, to be adaptive, should embrace these structural regularities. Here we test a non-retinotopic, object-centered mechanism in exogenous attention, in which exogenous attention is dynamically attracted to a relative, object-centered location. Using a moving frame configuration, we presented two frames in succession, forming either apparent translational motion or in mirror reflection, with a completely uninformative, transient cue presented at one of the item locations in the first frame. Despite that the cue is presented in a spatially separate frame, in both translation and mirror reflection, human performance in visual search is enhanced when the target in the second frame appears at the same relative location as the cue location than at other locations. These results provide unambiguous evidence for non-retinotopic exogenous attention and further reveal an object-centered mechanism supporting flexible exogenous attention. Moreover, attentional generalization across mirror reflection may constitute an attentional correlate of perceptual generalization across lateral mirror images, supporting an adaptive, functional account of mirror images confusion.

1. Introduction

Our senses are constantly flooded with streams of information, necessitating mechanisms of attentional gating and selection. Attention allocation is mainly controlled by two distinct mechanisms (for reviews, see Corbetta & Shulman, 2002; Egeth & Yantis, 1997; Nakayama & Mackeben, 1989; Theeuwes, 2010): a flexible, endogenous (i.e. top-down, goal-directed) mechanism whereby one voluntarily selects information of interest for further processing, and a rigid, exogenous (i.e. bottom-up, stimulus-driven) mechanism whereby one's attention is involuntarily drawn to the location of salient information.

Reflecting that endogenous attention is goal-directed whereas exogenous attention is stimulus-driven, endogenous attention and exogenous attention are manipulated by using cues that are informative and uninformative regarding the target location, respectively (Carrasco, 2011). The classic paradigm that implements these manipulations is the Posner cueing paradigm (Posner, 1980): in endogenous attention, an informative cue indicates the likely location where the target would appear with a higher than chance probability (e.g., 70%); in exogenous attention, a transient, uninformative cue is randomly flashed at one of the possible target locations. In both cases, attention is found to be drawn to the cued location, manifesting as better performance when the target appears at the cued location than at non-cued locations.

By definition, then, endogenous attention is spatially flexible: it is usually allocated not to the location of the informative cue but rather to the location indicated by the cue (e.g., a cue presented on the left side of the display directs attention to the right when the cue predicts the target to likely appear on the right). Exogenous attention, on the other hand, is spatially non-flexible: it is attracted to the retinotopic location of the uninformative cue (e.g., a cue presented on the left side of the display attracts attention to the location of the cue, despite that the cue carries no information regarding the target)¹.

1.1. The spatial flexibility of exogenous attention

Given that uninformative cues that can attract exogenous attention are highly salient in the environment—such as a red bar among green bars, or an onset in an otherwise static image—attraction of exogenous attention to the retinotopic location of the salient cue seems adaptive, promoting rapid awareness of and resource mobilization toward important objects and urgent events.

However, on the other hand, urgent and important information in the natural environment is usually dynamic—for example, a dashing arrow, an approaching snake, or a jumping lion—rendering an exogenous attention mechanism rigidly locked to the original cue location maladaptive. Moreover, because objects in the natural world usually form specific spatiotemporal relations with their context, an exogenous attention mechanism blind to these structural regularities also seems inefficient. Coping with salient and dynamic visual stimuli would benefit from a flexible exogenous attention mechanism that incorporates environmental regularities and updates them regularly. Here, we ask whether exogenous attention can be flexibly deployed according to the spatiotemporal regularities in the environment.

That exogenous attention may be non-retinotopic has been explored in a few studies. In a cueing study measuring eye movement latencies by Abrams and Dobkin (1994), two objects were presented, one above and one below the fixation, and one of them was flashed to induce exogenous cueing; the two objects then moved to the left and right sides of the fixation, with a central cue at the fixation indicating which object the subjects were supposed to look to. Subjects were significantly faster (by 2.5 ms) to look to the previously cued object than the uncued object, indicating non-retinotopic exogenous attention. A similar effect (of ∼10 ms) was also observed in detection and discrimination tasks in normal subjects (Ro & Rafal, 1999) and split-brain patients (Tipper et al., 1997). New evidence was further found in a study that used apparent motion displays (Boi, Vergeer, Ogmen, & Herzog, 2011).

However, although these studies were intended to manipulate exogenous attention, a validity confound suggests that these manipulations may, instead, tap into endogenous attention. For instance, in the two-objects-moving studies (Abrams & Dobkin, 1994; Ro & Rafal, 1999; Tipper et al., 1997), the cue, though uninformative regarding the exact target location, predicted which two locations the target would subsequently appear: when the cue was presented on the left or right of the fixation, the two target objects would always appear above and below the fixation, and vice versa. Similarly, in the apparent-motion study (Boi et al., 2011), the cue in the experiments also predicted the location of the target frame: the target frame would always be on the opposite side of the cue frame.

Because demonstration of exogenous attention requires that the cues be uninformative regarding the target (e.g., Folk, Remington, & Johnston, 1992), this validity confound renders it problematic to attribute the non-retinotopic effect to exogenous attention rather than endogenous attention. This validity confound is not a trivial concern, as cue validity is the only defining theoretical attribute that differentiates endogenous attention from exogenous attention: indeed, as mentioned above, cue validity is deliberately manipulated so that one can probe either endogenous or exogenous attention by using either informative or uninformative cues, respectively (for extensive reviews, see Carrasco, 2011; Posner, 1980).

Not surprisingly, given its voluntary nature, endogenous attention has been known to be spatially flexible and can be directed to a remote, object-centered location when the cue is predictive of the target location in the subsequent frame (Barrett, Bradshaw, & Rose, 2003; Umilta, Castiello, Fontana, & Vestri, 1995). Since the non-retinotopic effects in previous studies of “exogenous attention” can be attributed to endogenous attention, it is important to provide conclusive evidence for non-retinotopic effects from purely uninformative cues, thereby establishing non-retinotopic exogenous attention. A study that directly pitted retinotopic effect against non-retinotopic effect shows that exogenous attention is retinotopic: it is based on egocentric reference frames rather than allocentric reference frames (Barrett, Bradshaw, Rose, Everatt, & Simpson, 2001). Thus, up to now, there is no conclusive evidence for spatially flexible exogenous attention.

1.2. The current study

The current study is to fill this gap and undercover its potential mechanisms. Using completely uninformative cues, we ask how exogenous attention might be flexibly deployed according to the spatial relations between the cue and its contextual object (i.e., a reference frame). Specifically, we test a non-retinotopic, object-centered mechanism in exogenous attention, proposing that exogenous attention is dynamically attracted to a relative, object-centered location when the contextual object moves. This hypothesis is motivated by an object-centered coding and updating account in visual information processing, in which visual representations are tightly coupled to the contextual object and are continuously being updated through an object-centered, location-specific mechanism. This account has been supported by evidence from iconic memory, priming and backward masking (Lin & He, 2012) and is also consistent with studies showing motion-based grouping in visual processing (e.g., Boi, Ogmen, Krummenacher, Otto, & Herzog, 2009; Boi et al., 2011).

By exploiting the spatial relations within the contextual object, such an object-centered account is distinct from an object-based account that treats each constituent moving object as a unit, as is the case in previous studies (e.g., Abrams & Dobkin, 1994; Ro & Rafal, 1999; Tipper et al., 1997). Following Boi et al (2011), Section 1 (Experiment 1 and 2) tests object-centered non-retinotopic exogenous attention across translational apparent motion.

Section 2 (Experiment 3 and 4) further tests the object-centered account in a new apparent motion illusion. Specifically, we capitalize on a unique phenomenon in object recognition that has fascinated artists, philosophers, cosmologists, and scientists for centuries (Corballis & Beale, 1976): mirror image confusion, in which many species, monkeys and humans included, confuse images with their bilateral, left-right mirror-reflection counterparts (i.e. enantiomorphs). In our configuration, a cue frame and a target frame that are in lateral, left-right mirror reflection are presented sequentially. This configuration elicits a new apparent motion illusion: the cue frame is perceived to flip to the target frame as a book page flips to the next one, an illusion we term “flipping apparent motion”. Thus, using this new apparent motion, Section 2 tests object-centered non-retinotopic exogenous attention across mirror image reflection.

2. Section 1: Flexible exogenous attention across translation

2.1. Experiment 1: Cueing of exogenous attention across apparent translational motion

The configuration and task in Experiment 1 closely followed Experiment 4 in Boi et al (2011), which used a two-frame translational apparent motion, but critically, here the cue was completely uninformative, neither predicting the location of the target frame nor predicting the target location within the search display. Specifically, each trial involved a sequence of two frames: i) the first frame (“cue frame”, during which an exogenous attention cue was presented) appeared randomly on the left or right of the display (Figure 1A, up); ii) the second frame (“target frame”, during which a visual search configuration was presented) also appeared randomly on the left or right (Figure 1A, down). Consequently, when both frames were on the same side, it was the retinotopic condition; when they were on different sides, it was the non-retinotopic condition, which was perceived as a single frame moving from one side to the other because of apparent motion (subjectively confirmed by each observer). Note that, by “random”, we mean that all levels of a given factor had the same appearance rate in the whole experiment (e.g., half left, half right) but the order was randomly distributed across trials.

Figure 1.

Cueing of exogenous attention across apparent translational motion (Experiment 1). (A) Stimuli: Observers fixated on a central dot throughout the experiment. The cue frame was presented first, followed by the target frame; the two frames appeared either on the same side of the fixation (retinotopic display condition; both left or both right) or on different sides (translation display condition; one left and one right, forming translation). The cue was uninformative, neither predicting the location of the target frame nor predicting the target location within the search display. Observers were asked to indicate whether the non-vertical red bar was tilted clockwise or counterclockwise. (B) Results (n = 13): In both the retinotopic and translation display conditions, responses were faster for valid (cue and target at the same or same relative location) trials than invalid (cue and target at different or different relative locations) trials. Stars represent levels of significance from two-tailed t tests: *, **, and *** are statistically significant difference at the level of p < 0.05, 0.01, and 0.001, respectively.

Thus, based on the logic of the Posner cueing paradigm, if exogenous attention generalizes across translational apparent motion, then in the non-retinotopic condition, one would expect better performance when the target appeared at the same relative frame position as the cue than at other locations.

2.1.1. Methods

2.1.1.1. Observers and apparatus

Thirteen subjects from the University of Minnesota community participated in Experiment 1 in return for money or course credit. One observer was author Z. L.; the others were naive to the purpose of the study. All had normal or corrected-to-normal vision and signed a consent form approved by the local institutional review board.

The stimuli were presented on a black-framed, gamma-corrected 22-in. CRT monitor (Hewlett-Packard p1230; refresh rate = 100 Hz, resolution = 1024 768 pixels) using MATLAB (The MathWorks, Natick, MA) and the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). Subjects sat approximately 57 cm from the monitor with their heads positioned in a chin rest in an almost dark room while an experimenter was present.

2.1.1.2. Stimuli and procedure

Before the main experiment, observers took part in a fixation training session and an apparent motion session. During the fixation training session, subjects viewed a square patch of black and white noise that flickered in counterphase, with each pixel alternating between black and white across frames. Each eye movement during the viewing would lead to perception of a flash, and subjects were asked to minimize the perception of flashes, thereby training to maintain stable fixation (see Guzman-Martinez, Leung, Franconeri, Grabowecky, & Suzuki, 2009; Lin & He, 2012). During the apparent motion session, subjects were exposed to apparent motion displays as in the main experiment but were asked to describe their perception of the displays and were trained until they were able to see apparent motion before proceeding to the main experiment.

Figure 1A shows the configurations. Specifically, after 800 ms presentation of a central gray fixation dot (diameter = 0.39°; luminance = 30.1 cd/m²) against a black background, the first frame—consisting of a central square (size = 3.1°; luminance = 40.1 cd/m² contoured by a white rectangle; square center to fixation distance = 1.85°) and two flanking squares (size = 2°; luminance = 40.1 cd/m²; center-to-center distance between the central square and each flanker square = 3.16°)—was presented either on the left or right of the display for 200 ms, during the 80–120 ms of which a dot cue (size = 0.3°; color = randomly selected from blue, yellow, and cyan for each trial to avoid sensory adaptation) was presented randomly at one of six locations within the central square (center-to-center distance between cues and the square center = 1.1°). After an interstimulus interval (ISI) of 60 ms, the second frame was presented either on the same side of the display as the first frame or on the other side for 200 ms, during the first 80 ms of which six orientation target bars (length = 0.5°; width = 0.1°) were presented. Of the six bars, three were green and tilted—two tilted 30° either clockwise or counterclockwise relative to vertical and one titled 30° to the opposite side—and the remaining three were red and vertical except for one that was tilted 30° either clockwise or counterclockwise. The task was to find the non-vertical red bar and indicate whether it was tilted clockwise (right) or counterclockwise (left) as quickly and accurately as possible. Thus, the cue to target stimulus onset asynchrony (SOA) was 180 ms. Note that we used different color sets for the cue (blue, yellow, and cyan) and the target (red and green) to control for feature-based attention. There were 96 practice trials (in 2 blocks) and 576 experimental trials (in 12 blocks).

2.1.1.3. Design

There were two within-subject variables, cue validity (2 levels: valid vs. invalid) and reference frame (2 levels: retinotopic vs. translation).

2.1.2. Results and discussion

Figure 1A illustrates the basic structure of each trial in Experiment 1. Note that the cue provided no information regarding the position of the target frame or the location of the target within the frame, fulfilling a critical criterion of exogenous attention cueing. This design also allowed us to directly compare retinotopic and object-centered exogenous attention. Therefore, if target performance was facilitated for the same relative frame location as the cue compared with different relative frame locations, this would constitute evidence for flexible exogenous attention across translation.

A repeated-measures ANOVA revealed that, regardless of retinotopic or translation display conditions (insignificant interaction of display conditions and cue conditions, F(1, 12) = 1.06, P = 0.32, ${\eta }_p^2=0.002$), reaction times (RTs) were significantly faster when the cue and the target coincided at the same or same relative location (valid) than at different or different relative locations (invalid; F(1, 12) = 16.92, P = 0.001, ${\eta }_p^2=0.045$, Figure 1B). While the superior performance in the retinotopic-valid trials than the retinotopic-invalid trials (t(12) = -3.55, P = 0.004, Cohen's d = -0.98) is consistent with classic findings from the Posner cueing paradigm, the superior performance in the translation-valid trials than the translation-invalid trials (t(12) = -2.54, P = 0.026, d = -0.70) reveals a flexible exogenous attention effect, in which exogenous attention transfers across translation. No significant differences were found in the accuracy data.

2.2. Experiment 2: Cueing of exogenous attention across apparent translational motion without explicit frames and unique target apparent motion

Because only one item (the cue) was presented during the cue frame, the translation effect in Experiment 1 might be due to the unique apparent motion between the cue and the target in the valid trials, limiting the scope of flexible exogenous attention. To address this issue, in Experiment 2 we developed a cue-masking procedure for the cue frame (Figure 2A), in which a transient cue was presented ahead of other items and disappeared at the same time with other items (Mulckhuyse, Talsma, & Theeuwes, 2007). As such, when the test items were subsequently presented on the other side of the fixation as the target frame, all masking items on the cue frame were engaged in apparent motion, allowing us to rule out the confound of unique apparent motion in the valid condition. Since both the cue frame and the target frame were implicit frames made up of the masking and search items, this design also had the added benefit of allowing us to test whether explicit frames were necessary for flexible exogenous attention.

Figure 2.

Cueing of exogenous attention across apparent translational motion without explicit frames and unique target apparent motion (Experiment 2). (A) Stimuli: The cue frame (first an uninformative disk cue alone, then immediately masked by four disks on the same side of the fixation) was presented first, followed by the target frame either on the same side as the cue frame or on the opposite side. Observers were asked to indicate whether the tilted bar was tilted clockwise or counterclockwise. (B) Results (n = 10): In both the retinotopic and translation display conditions, responses were more accurate for valid trials than invalid trials. Stars represent levels of significance from two-tailed t tests: *, **, and *** are statistically significant difference at the level of p < 0.05, 0.01, and 0.001, respectively.

In Experiment 2, to test the robustness of flexible exogenous attention, we used different stimuli (black bars) and a more difficult task with emphasis on accuracy only (discriminating the tilted bar as tilted clockwise or counterclockwise). Again, as in Experiment 1, we expected better performance when the target appeared at the same relative frame position as the cue than at other locations.

2.2.1. Method

The method is the same as Experiment 1 except as noted here.

2.2.1.1. Observers and apparatus

Ten subjects participated in Experiment 2.

2.2.1.2. Stimuli and procedure

Experiment 2–4 used distinct displays and tasks than Experiment 1, and thus the timing schemes also differed. In Experiment 2, as shown in Figure 2A, after 1250 ms presentation of a fixation dot (diameter = 0.39°; luminance = 30.1 cd/m²), a white disk (diameter = 1.2°; luminance = 80.2 cd/m²)—serving as the attention cue—was presented randomly at one of four corners that form an imaginary isosceles trapezoid (center-to-center distance between the two left corners of the trapezoid = 4°; between the two right corners = 2°; trapezoid height = 2°; trapezoid centroid to fixation distance = 2.5°) either on the left or right of the display for 30 ms. The cue was then masked by four disks of the same size and luminance on the same side of the display for 10 ms. These masks were then followed by the same four disks but now they were either on the same side of the display as the masks or on the other side for 40 ms, during which four black orientation target bars (length = 0.8°; width = 0.125°) were superimposed on the four disks. Of the four bars, three were either horizontal or vertical—two horizontal, one vertical or the reverse—but the remaining one was tilted, either clockwise or counterclockwise relative to vertical by 15°. The task was to find the tilted bar and indicate whether it was tilted clockwise (right) or counterclockwise (left) as accurately as possible without emphasis on RTs. This produced a cue to target SOA of 40 ms. This procedure thus ensures that goal or target directed eye movements could not take place between the cue onset and the target offset, as saccades require about 250 ms to occur (Mayfrank, Kimmig, & Fischer, 1987). There were 64 practice trials (in 1 block) and 256 experimental trials (in 4 blocks).

2.2.1.3. Design

There were two within-subject variables, cue validity (2 levels: valid vs. invalid) and reference frame (2 levels: retinotopic vs. translation).

2.2.2. Results and discussion

A repeated-measures ANOVA revealed that, regardless of retinotopic or translation display conditions (insignificant interaction between display conditions and cue conditions, F(1, 9) = 0.28, P = 0.61, ${\eta }_p^2=0.002$), observers performed significantly more accurately when the cue and the target coincided at the same or same relative location (valid) than at different or different relative locations (invalid; F(1, 9) = 16.85, P = 0.003, ${\eta }_p^2=0.145$, Figure 2B). Again, consistent with classic retinotopic exogenous cueing, performance in the retinotopic-valid trials was better than the retinotopic-invalid trials (t(9) = 3.10, P = 0.01, d = 0.98). Importantly, as in Experiment 1, an unconfounded exogenous attention effect across translation was also observed, with superior performance in the translation-valid trials than the translation-invalid trials (t(9) = 3.34, P = 0.009, d = 1.05). These findings thus extend flexible exogenous attention across translation to implicit frames, and without being confounded by unique apparent motion between the cue and the target.

3. Section 2: Flexible exogenous attention across mirror reflection

3.1. Experiment 3: Cueing of exogenous attention across mirror reflection

Flexibility across translation in exogenous attention reveals a non-retinotopic, object-centered effect. The results can be explained by either a translational object-centered account in which the attentional spotlight is mounted on the moving contextual object or a refined object-centered account that emphasizes perceptual correspondence across apparent motion. Experiment 3 tests an object-centered exogenous attention effect that is not consistent with the simple translational account but is explained by the correspondence-based account.

Specifically, as shown in Figure 3A, we examine an intriguing case of flexibility: generalization of exogenous attention across lateral, left-right mirror images. In the object recognition literature, confusion of left-right mirror images is ubiquitous across many species (Corballis & Beale, 1976): in young children, for instance, letters and their mirror-reflection counterparts sometimes get confused (e.g. letter R and its mirror reflection, as in the Toys “R” Us logo) (Rudel & Teuber, 1963); in human adults, perceiving an object automatically primes its mirror-reflected counterpart (Lavie, Lin, Zokaei, & Thoma, 2009; Stankiewicz, Hummel, & Cooper, 1998). If a similar mirror-reflection mechanism also exists for exogenous attention, it could be adaptive, allowing the organism to quickly orient to dynamic salient visual information.

Figure 3.

Experiment 3 and 4: Flexible exogenous attention across mirror reflection. (A) Stimuli in Experiment 3: The cue was uninformative, and the cue frame and the target frame either appeared on the same side of the fixation (retinotopic display condition; both left or right, indicated by the black arrows) or on different sides (mirror reflection display condition; one left and one right, forming mirror reflection, indicated by the gray arrows). Observers were asked to indicate whether the non-vertical red bar was tilted clockwise or counterclockwise. (B) Results from Experiment 3 (n = 8): In both the retinotopic and mirror reflection display conditions, responses were faster for valid trials than invalid trials. (C) Stimuli in Experiment 4: The cue frame started with a frame, followed by an uninformative cue (precue and cue were on the same side of the fixation). After a brief interval, the mask and then the target frame appeared (mask and target were on the same side), either on the same side as the cue frame or on the opposite side. Observers were asked to indicate whether the vertical bar (but not the tilted bars) was above or below its abutting horizontal bar. (B) Results from Experiment 4 (n = 8): In both the retinotopic and mirror reflection display conditions, responses were faster for valid trials than invalid trials. Stars represent levels of significance from two-tailed t tests: *, **, and *** are statistically significant difference at the level of p < 0.05, 0.01, and 0.001, respectively.

In Experiment 3, we presented a cue frame and a target frame that were in lateral, left-right mirror reflection in sequence, which generated a flipping apparent motion. The crux was to test whether target performance would be facilitated when the target appeared at the mirror-reflected location of the cue than at other locations. If so, this would constitute evidence for flexible exogenous attention across mirror reflection, pointing to the role of perceptual correspondence across apparent motion in supporting object-centered exogenous attention.

3.1.1. Method

The method is the same as Experiment 1 except as noted here.

3.1.1.1. Subjects and apparatus

Eight subjects participated in Experiment 3.

3.1.1.2. Stimuli and procedure

Figure 3A shows the cue display and the target display. Specifically, after 750 ms presentation of a fixation dot (diameter = 0.39°; luminance = 30.1 cd/m²), the first frame—a shell-like shape that was contoured by light gray lines (width = 0.08°; luminance = 70.2 cd/m²) and was composed of an isosceles trapezoid (length of the longer base = 4.5°; length of the shorter base = 2.5°; trapezoid height = 2.2°; trapezoid centroid to fixation distance = 2.4°; luminance = 40.1 cd/m²) and an arc (contained within an imaginary rectangle of 1.25° × 4.5°; luminance = 40.1 cd/m²)—was presented for 200 ms, during the 140–180 ms of which a dot cue (size = 0.3°; color = randomly selected from blue, yellow, and cyan for each trial) was presented randomly at one of four locations within the shell. These four locations formed an imaginary isosceles trapezoid (length of the longer base = 3.24°; length of the shorter base = 1.54°; trapezoid height = 1.72°) whose longer base aligned with that of the isosceles trapezoid in the shell frame. After an ISI of 60 ms, the second frame appeared either on the same side as the first frame or flipped to the opposite side for 200 ms, during the first 80 ms of which four orientation target bars (length = 0.5°; width = 0.1°) were presented. Of the four bars, two were green and tilted—one tilted clockwise and one tilted counterclockwise relative to vertical by 30°—and the remaining two were red, one vertical and one tilted either clockwise or counterclockwise by 30°. The task was to find the non-vertical red bar and indicate whether it was tilted clockwise (right) or counterclockwise (left) as quickly and accurately as possible. This produced a cue to target SOA of 120 ms. There were 128 practice trials (in 2 blocks) and 384 experimental trials (in 6 blocks).

3.1.1.3. Design

There were two within-subject variables, cue validity (2 levels: valid vs. invalid) and reference frame (2 levels: retinotopic vs. reflection).

3.1.2. Results and discussion

In the mirror reflection condition, a trial was considered valid when the cue and the target were on the same relative frame location that formed mirror reflection (e.g. upper left of the left frame vs. upper right of the right frame); otherwise, it was an invalid cue trial. The critical finding is that, in the mirror reflection display condition, RTs were significantly faster for mirror-valid trials than mirror-invalid trial (t(7) = 4.73, P = 0.002, d = -1.67, Figure 3B), revealing a novel flexible exogenous attention effect across mirror reflection. A significant effect of cue validity was also observed in the retinotopic display condition (t(7) = 3.16, P = 0.016, d = -1.12,), but the effect was reduced (significant interaction of display conditions and cue conditions, F(1, 7) = 8.93, P = 0.02, ${\eta }_p^2=0.003$). Thus, flexible exogenous attention across mirror reflection points to a structural, object-centered mechanism in play.

To examine whether there is a translation effect, in the mirror reflection condition, in addition to the mirror-valid trials as above, we further classified the mirror-invalid trials into two categories: translation-valid trials (as if the cue and the target formed translation, e.g. upper left of the left frame vs. upper left of the right frame), and invalid trials (all remaining trials). Critically, we found that RTs were significantly faster in the mirror-valid trials (648 ms) than the translation-valid trials (668 ms, t(7) = -2.75, P = 0.03, d = -0.97) or the invalid trials (676 ms, P < 0.001, d = -1.93). RTs in the translation-valid trials (668 ms), though slightly faster, were statistically comparable to the invalid trials (676 ms, t(7) = -0.92, P = 0.39, d = -0.33). No significant differences were found in the accuracy data. These results therefore demonstrate that when mirror reflection was perceived, mirror generalization dominates translational generalization. These results also rule out any account based on attentional shifts driven by eye movements (overt attention).

3.2. Experiment 4: Cueing of exogenous attention across mirror reflection with reduced masking from the cue

The smaller cueing effect in the retinotopic condition than the reflection condition seems counterintuitive. The difference may be due to a stronger masking effect from the cue in the retinotopic condition than the mirror reflection condition, or it may be attributed to a mirror reflection tendency in the retinotopic condition as well, which would, by definition, reduce the retinotopic validity effect. To test these ideas, a different configuration and a different task were used in Experiment 4 (Figure 3C). To reduce masking, a horizontal bar was used as a cue on the cue frame, followed by non-overlapping vertical bars on the target frame.

3.2.1. Method

The method is the same as Experiment 1 except as noted here.

3.2.1.1. Subjects and apparatus

Eight subjects participated in Experiment 4.

3.2.1.2. Stimuli and procedure

The procedure was depicted in Figure 3C. After 1000 ms presentation of a fixation dot (diameter = 0.39°; luminance = 30.1 cd/m²), the first frame—a shell-like shape that was contoured by red lines (width = 0.08°; luminance = 26.7 cd/m²) and was composed of an isosceles trapezoid (length of the longer base = 4.8°; length of the shorter base = 2.2°; trapezoid height = 2.2°; trapezoid centroid to fixation distance = 2.4°; luminance = 40.1 cd/m²) and an arc (contained within an imaginary rectangle of 1.25°× 4.8°; luminance = 40.1 cd/m²)—was presented for 100 ms either on the left or right of the display, during the last 40 ms of which a horizontal white bar (size = 0.16° × 1.0°; luminance = 80.2 cd/m²), serving as an attention cue, was presented randomly at one of three locations within the shell. These three locations formed an imaginary isosceles triangle (length of the base = 3.0°; triangle height = 1.55°) whose base aligned with that of the isosceles trapezoid of the shell frame. After an ISI of 30 ms, the second frame appeared either on the same side as the first frame or flipped to the opposite side for 100 ms, during the first 40 ms of which three horizontal white bars were presented. The three horizontal bars were then immediately followed by the same three bars plus three additional bars for 60 ms, serving as the target search display: two were tilted (one was tilted clockwise and the other counterclockwise relative to vertical by 30°) and one was vertical. The task was to find the vertical bar and indicate whether it was above or below its abutting horizontal bar as quickly and accurately as possible (of the two nontarget bars, one was right above its abutting horizontal bar and the other below its abutting horizontal bar). These three additional bars were of different sizes: the bar close to the fixation was 0.08° × 0.5°, whereas the two farther away ones were 0.08° × 0.67°. This produced a cue to target SOA of 110 ms. There were 216 practice trials (in 6 blocks) and 216 experimental trials (in 6 blocks).

3.2.1.2. Design

There were two within-subject variables, cue validity (2 levels: valid vs. invalid) and reference frame (2 levels: retinotopic vs. reflection).

3.2.2. Results and discussion

We observed a similar pattern as in Experiment 3 (Figure 3D): RTs were significantly faster for mirror-valid trials than mirror-invalid trial (t(7) = 4.02, P = 0.005, d = -1.42). A significant effect of cue validity was also observed in the retinotopic display condition (t(7) = 3.58, P = 0.009, d = -1.26), although the effect was smaller than the mirror reflection display condition (significant interaction of display conditions and cue conditions, F(1, 7) = 6.31, P = 0.04, ${\eta }_p^2=0.005$), underscoring the notion that exogenous attention is highly sensitive to mirror reflection. No significant differences were found in the accuracy data.

Given that the retinotopic cueing effect was much larger than that in Experiment 3, these results suggest that masking from the cue might undermine the retinotopic cueing effect in Experiment 3. Moreover, since the non-retinotopic cueing effect was stronger than the retinotopic cueing effect, as in Experiment 3, these results also point to a mirror reflection tendency in the retinotopic condition as well, which counteracts the retinotopic cueing effect.

4. General discussion

Traditionally, exogenous attention is widely assumed to be spatially non-flexible—it is attracted to the retinotopic location of the uninformative cue (Carrasco, 2011; Corbetta & Shulman, 2002; Egeth & Yantis, 1997; Theeuwes, 2010). Recent studies have started to cast doubt on this idea (Abrams & Dobkin, 1994; Boi et al., 2011; Ro & Rafal, 1999; Tipper et al., 1997), but the non-retinotopic effects in these previous studies can also be attributed to endogenous attention. Here, by using completely uninformative cues—cues that provided neither information for the target frame position nor information for the target location within the frame—we provide the first conclusive evidence that exogenous attention can be non-retinotopic.

In addition, we show that this non-retinotopic exogenous attention is supported by an object-centered structural mechanism, in which exogenous attention is dynamically attracted to a relative, object-centered location when the contextual object moves. By emphasizing the spatial structure within objects, these findings are not consistent with object-based accounts that treat visual objects as abstract units. By showing generalization of exogenous cueing across mirror reflection, these findings are also not consistent with translational mobile accounts in which the attentional spotlight is mounted on the moving contextual object. But these findings are consistent with a recent study showing object-centered coding and updating in iconic memory, priming and backward masking (Lin & He, 2012) and studies showing motion-based grouping in visual processing (e.g., Boi et al., 2009; Boi et al., 2011).

4.1. Mechanisms of object-centered, non-retinotopic exogenous attention

Although eye movements were not monitored in the current study, our results cannot be explained as due to eye movements for the following reasons. First, all our subjects went through an eye fixation training that has been shown to be effective in controlling eye fixation at the center (Guzman-Martinez et al., 2009). Second, the durations between the onset of the cue and the offset of the target (i.e., cue to target SOA + target duration) in Experiment 1–4 were 260 ms, 80 ms, 200 ms, and 150 ms, respectively. As saccades require about 250 ms to occur (Mayfrank et al., 1987), goal or target directed eye movements could not take place between the cue onset and the target offset, particularly in Experiment 2–4. Finally, if eye movements were responsible for the effects we observed, one would expect to see faster RTs in the translation-valid trials than the mirror-valid trials in Experiment 3, which was the opposite of what we found.

What kinds of attentional processes are responsible for the object-centered non-retinotopic exogenous cueing effects? Theories of attention distinguish between two effects of attention: perceptual coding of the target and decision integration. In the latter case, given that a stimulus represents a source of noisy information for a chance of false alarm, attention may improve performance by restricting the sources of information for decision making without affecting the perceptual coding of the target. For example, Palmer (1994) has provided evidence showing that set size effects in visual search can be explained by decision integration alone, particularly for simple perceptual tasks—tasks that “depend on only one attribute, the attribute has plausibility as a relevant psychological variable, the attribute is of a single object rather than a relation among objects, and the task introduces no distractor heterogeneity”—but for complex perceptual task, set size effects may require an additional attentional effect on perception (see also Shiu & Pashler, 1994).

Although when multiple targets/locations are involved, decision integration usually comes into play, the attentional effects reported here are likely to reflect not just decision integration but also perceptual coding of the target. First, the tasks in Experiment 1, 3 and 4 all involved two features and thus were complex tasks according to Palmer (1994). For Experiment 2, the non-target horizontal and vertical bars afforded no chance of false alarm, as they were orthogonal to the target decision (left vs. right). Second, unlike the endogenous cues used by Palmer (1994) and Shiu & Pashler (1994), exogenous cues similar to those used in our experiments have been routinely found to affect perceptual coding and even appearance in displays with multiple items (Carrasco, 2011; Carrasco, Ling, & Read, 2004). Finally, in a dot detection experiment that uses only a single target, we have observed object-centered non-retinotopic exogenous effects even when the cue is invisible (Lin & Murray, under review-b). Such a finding cannot be explained by a reduction in decision noise, which would require knowing the cued location and would be intentional by definition.

That exogenous cueing generalizes across translation and mirror reflection suggests that cue representation within the cue frame is not lost when the cue and the cue frame disappear; rather, the cueing effect is apparent when the cue frame is perceived to move to the target frame because of apparent motion, suggesting an important role of perceptual continuity and correspondence in supporting flexible exogenous attention. Crucially, unlike object-based effects, the cueing effect relies on an object-centered, location-specific mechanism, which hitherto is limited only to voluntary attention (Barrett et al., 2003; Schreij & Olivers, 2009; Umilta et al., 1995). These characteristics are consistent with the object cabinet account (Lin & He, 2012) derived from the object file theory (Kahneman, Treisman, & Gibbs, 1992; Treisman, 1992). According to this account, perceiving the cue frame induces a temporary, episodic representation in an object cabinet in which the cue is spatially tagged in the cue frame in an object-centered, location specific manner. During apparent motion in which the cue frame is perceived to be moving to the target frame, the object cabinet from the cue frame is being retrieved and copied to the target frame, resulting in object-centered, location-specific generalization of exogenous attention across translation and mirror reflection.

In essence, these results point to the idea that perceptual correspondence is object-centered, which determines how attention treats two locations as “the same” and thus the generalization of exogenous attention across non-retinotopic locations. By emphasizing perceptual correspondence, this account could potentially accommodate other spatial transformations such as contraction and expansion. More generally, it is consistent with the finding that priming is determined by global motion correspondence, overriding local contexts: for example, a leftward target is affected more a leftward prime than by a rightward prime, even though the target's local context matches that of the rightward prime but mismatches that of the leftward prime (Lin & Murray, under review-a).

4.2. Implications for exogenous attention and object-based attention

These flexible exogenous attention effects represent a new addition to the classic retinotopic effect in exogenous attention, suggesting that exogenous attention is highly adaptive to the dynamic visual environments. For instance, exogenous attention can show rapid visuospatioal learning, in which performance is improved when targets are constantly presented at the same relative location (Kristjansson, Mackeben, & Nakayama, 2001) or with the same color or shape (Kristjansson & Nakayama, 2003) within the cue frame (e.g. made up by two parallel lines). In addition, exogenous attention may spread within a visual object (Mortier, Donk, & Theeuwes, 2003) and may be modulated by the attentional control setting induced by the task (Folk et al., 1992). Thus, together with these findings, our results suggest that exogenous attention is highly plastic and adaptive, able to accommodate different visual stimulations and tasks. As such, flexible exogenous attention prepares the organism to rapidly and efficiently respond to unexpected changes or sudden movements that could signal danger or opportunity.

By underscoring how relative locations within objects guide exogenous attention, the object-centered effects reported here have important implication for object-based attention as well. Traditionally, studies in object-based attention focus on same-object advantages, such as better divided attention between two attributes belonging to one object than to two objects (Duncan, 1984), better focused attention on the target when the distractors do not belong to the target object than when they do (Kramer & Jacobson, 1991), and prioritized attention to a location within a cued object than within an uncued object (Egly, Driver, & Rafal, 1994). However, the relative spatial locations within objects are usually overlooked. Indeed, it is sometimes argued that object-based attention selects a location-independent object representation (Awh, Dhaliwal, Christensen, & Matsukura, 2001; Matsukura & Vecera, 2009; Vecera & Farah, 1994). The object-centered effects here demonstrate that spatial information within objects are not lost; instead, this spatial information is coded and continuously being updated in an object-centered manner. These object-centered effects thus bring together spatial accounts and object-based accounts in visual attention.

4.3. Implications for object recognition and mirror image confusion

In particular, flexible exogenous attention across mirror reflection may be a functional consequence of perceptual generalization of lateral, left-right mirror images, providing an adaptive account of mirror images confusion. Developmental and comparative psychologists have long been puzzled by the extreme difficulty children and other animals such as octopuses, rats, pigeons, cats, and monkeys have in differentiating left-right mirror images (e.g. / and \) but not mirror images about oblique axes (e.g. — and |) (Corballis & Beale, 1976; Rudel & Teuber, 1963; Sutherland, 1957). It has been debated whether confusion of lateral mirror images represents a limitation of the visual system, or it may serve an adaptive function (Corballis & Beale, 1976). For instance, although confusion of letters b and d or symbols < and > can be costly through the course of human civilization (e.g., reading and mathematics), in the natural world, mirror images are usually perceptually equivalent to each other (e.g. symmetry in the real world abounds where mirror images are simply two halves of the same object—the two sides of a face and a vase, or in the modern world a car and a TV). Moreover, given the abundant bilateral symmetries in the natural world and the fact that even the bodies and nervous systems have evolved to be more and more bilaterally symmetrical (Weyl, 1952), mirror-image confusion or generalization may be an adaptive strategy of the visual system (Gross & bornstein, 1978). Indeed, although through learning, literate adults can easily differentiate b vs. d and > vs. <, each of which has acquired the status as a distinct object, objects in the natural world rarely change their identities through mirror reversal—looking at the mirror, your toothpaste and toothbrush in the bathroom appear just as familiar as they are. Therefore, in the majority of cases, a perceptual system that treats mirror images as equivalent would be adaptive, facilitating rapid object recognition. The effect of flexible exogenous attention across mirror reflection reported here suggests that mirror images are treated as equivalent not just in the perceptual system but also in the attention system, such that cueing within one image generalizes to its mirror image in a relative-location specific manner. The perceptual and attentional equivalence of mirror images might thus provide a flexible mechanism for adaptive processing of the ubiquitous mirror images in the natural world.

5. Conclusion and future directions

In conclusion, our studies reveal that exogenous visual attention is flexible, generalizing across translation and mirror reflection through a structural, object-centered mechanism. By exploiting spatial and temporal structural regularities in the dynamic visual environment, flexible exogenous attention facilitates rapid and efficient perception and action.

Further studies may investigate how object-centered exogenous attention interacts with other types of attention to control behavior. Toward this, it would be helpful to minimize the masking effect from the cue by cueing the placeholders on the screen (e.g., the display may consist of several circles with target items presented within these circles). In addition, non-retinotopic, object-centered attentional effects may serve as a useful tool for investigating visual awareness (for example, whether and how unconscious attention may travel to affect an uncued location, Lin & Murray, under review-b). More general, how object-centered processing might affect other domains of cognition (for example, spatial localization, Lin, under review-a) and how perceptual correspondence operates beyond apparent motion (for an example in voluntary attention, see Lin, under review-b) remain major challenges for future research.

• We test a non-retinotopic, object-centered mechanism in exogenous attention.

• Performance is enhanced at the non-retinotopic, same relative location as the cue

• Exogenous attention generalizes across both translation and mirror reflection

• These results reveal a object-centered structural mechanism in exogenous attention

• Provides an attentional correlate of perceptual generalization across mirror images