Object recognition: Visual crowding from a distance

Abstract

Immediately before a large eye movement, a target object is crowded by clutter placed near the target’s future location. This new finding, from a recent study, shows that the brain’s remapping for the anticipated eye movement unavoidably combines features from the target’s current and future retinal locations into one perceptual object.

Object recognition proceeds through the selection and combination of features, governed by rules of grouping and crowding [1,2,3,4,5]. This is variously called binding, grouping, region growing, or crowding, depending partly on whether the observer recognizes the object. In “crowding”, target identification fails because the feature combination extends unavoidably over an inappropriately large area, jumbling elements of the adjacent clutter with those of the target, which spoils identification [6,7]. The size of this minimum area over which features are combined is roughly the same for all simple objects, like letters, and grows proportionally with eccentricity (distance from fixation) [5,8,9]. Transformed by the cortical magnification factor, the minimum size of the combination region corresponds to a fixed area in mm² on the surface of the cortex [10,11]. This compact region of compulsory combination is roughly centered on the target. In their recent Current Biology paper, Harrison et al. [12] find that, immediately before a large eye movement (saccade), features from two widely separated regions, at the present and future locations of the target, are combined to produce one perceptual object.

Harrison et al. present target and clutter briefly, immediately before a large saccade (Figure 1). When clutter is placed next to the future retinal location of the target (far away, in the other hemifield), the clutter impairs recognition nearly as strongly as when it is placed next to the target’s current location. As in normal crowding, the strength of the effect depends on similarity between the target and the clutter [13]: An L crowds an H, but an X does not crowd an O. The sensitivity to similarity shows that the impairment is due to crowding, not to the indiscriminate impairment of saccadic suppression [14,15].

Figure 1.

(A) Initially, the observer fixates the central dot. (B) Disappearance of that central dot cues the observer to move his or her eyes rightward to the green square. (C) The random target (shown as H) is briefly presented to the right of fixation. At the same time, clutter is briefly presented to the left of fixation. The duration is 17 ms. (D) The brief target and clutter are gone by the time the eye moves and finally arrives at the green square. (Our simplified diagram omits the other placeholders and the changing content of the green square, which are not discussed here.) (E) Feature Remapping Theory: The features of the target are displaced from its original retinotopic location to its anticipated post-saccadic location. That places the target amid the clutter. Identification unavoidably combines information over a region that includes the (displaced) target and clutter. The combination region, the critical zone for crowding is represented by a gray oval. This combination degrades identification despite the large offset between the flankers and target in the display. (F) Attention Remapping Theory: The remapping causes identification to be based on a combination of features from two locations: the present and future locations of the target [19]. The combination regions are shown as gray ovals. Again, combining the features of the target and clutter degrades identification. Based on Harrison et al. [12]. Funnel image from Merriam Webster’s Learner’s Dictionary. http://www.learnersdictionary.com/search/funnel

The story of remapping began with physiological studies of cells that direct saccades to their targets, showing a correction in their target locations for an upcoming eye movement [16]. It was then extended to the allocation of attention, where a similar correction for upcoming saccades was found [17,18]. Now, Harrison et al. use crowding to show that remapping displaces not just location information, but also content, effectively combining the content of two distant regions.

The discovery of remapping by Duhamel, Colby, and Goldberg in 1992 launched intense scientific activity evaluating the mechanisms, brain areas, and pathways involved. They found that almost all the neurons in lateral intraparietal cortex (LIP) begin to fire in response to a stimulus that will be brought into their respective fields by an eye movement, even if the stimulus is extinguished before the eyes arrive. This remapped response is only found for attended targets, either flashed or task-relevant e.g. [19], and always moves in the direction opposite the saccade, predicting where the attended target will be after the saccade. The predictive response of these neurons can begin as early as 100 ms before the saccade and tends to peak at the onset of the saccade, much earlier than the cell would be able to respond if the stimulus simply appeared in the cell’s receptive field following the eye movement. This pre-saccadic stimulus activates the cells for the remapped location (which depends on the saccade vector) and, at the same time, it also activates cells that normally respond to the target’s location. As a result, activity can be seen just before the saccade in two widely separated sets of cells, at the target’s actual retinal location and its remapped location, both in response to the same brief stimulus.

These physiological studies were followed by behavioral studies that showed a similar pre-saccadic shift of attention by placing probes [17] at the target’s remapped location just before a saccade. Moreover, there is compelling evidence from time stamping and masking, and now crowding, showing that remapping does not just displace location information but generates a perceived target object that combines target information from the target’s pre- and (expected) post-saccadic retinal locations. When observers are asked to saccade to a clock with rapidly spinning hands and report the time that their eyes arrive, they report times 40–60 ms before their eyes arrived [20]. Masking and crowding are very different [7]. Masking requires overlap; crowding requires proximity. Without a saccade, a mask must overlap the target to impair its visibility, but immediately before a saccade a mask is effective at the target’s post-saccadic location [18]. These “bi-local” results are now buttressed by the Harrison et al. finding of remapped crowding. As already noted, without a saccade, clutter must be near the target to crowd it, but, immediately before a saccade, clutter is also effective at the target’s post-saccadic retinal location [12]. Since the range of masking and crowding is fairly small, and the size of saccades is quite variable, it would be interesting to measure the spatial profile of remapping to discover how accurately the pre-saccadic remapping predicts the saccade.

Thus, Harrison et al. now show that the brain’s remapping for the anticipated eye movement unavoidably combines features from the current and future retinal locations of the target into one perceptual object (see Figure 1 for two proposals on how this might happen).