The Role of the Lateral Intraparietal Area in Orienting Attention and its Implications for Visual Search

Abstract

Orienting visual attention is of fundamental importance when viewing a visual scene. One of the areas thought to play a role in the guidance of this process is posterior parietal cortex. In this review, we will describe the way the lateral intraparietal area (LIP) of posterior parietal cortex acts as a priority map to help guide the allocation of covert attention and eye movements (overt attention). We will explain the concept of a priority map and then show that LIP activity is biased by both bottom-up stimulus driven factors and top-down cognitive influences and that this activity can be used to predict the locus of covert attention and initial saccadic latencies in simple visual search tasks. We will then describe evidence for how this system acts during covert visual search and how its activity could be used to optimize overt visual search performance.

Introduction

“Everyone knows what attention is …” (James, 1890). This famous and oft-quoted phrase is as true today as it was over a century ago. In the same passage, James described the process of attention in terms that we still think of today, it is “the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought…It implies withdrawal from some things in order to deal effectively with others”. We now know that not only does attention help focus our resources, but that it appears necessary for extracting detailed information about a scene (Simons & Levin, 1997) and, thus, for our interactions with the visual world.

For decades, clinicians have been aware that damage to posterior parietal cortex can cause dramatic deficits in our awareness of regions of the visual world (Critchley, 1953). Such deficits can be explained by an inability to focus attention on the regions of space represented by the damaged areas in cortex (Bays et al., 2010) and functional imaging data strongly supports the idea that human parietal cortex is involved in such a process (Serences & Yantis, 2007; Corbetta et al., 2008). In this review, we will describe the current model of how a specific region within parietal cortex, the lateral intraparietal area (LIP), acts to guide attention based on single neuron responses recorded from awake behaving primates.

LIP was only anatomically identified as being a specialized region of area 7a based on its connections to both pre-frontal cortex and the superior colliculus in the mid-80s (Andersen et al., 1985), although a number of prior physiological recording studies had identified this section of 7a as being involved in motor control (Mountcastle et al., 1975) or visual attention (Lynch et al., 1977; Robinson et al., 1978; Bushnell et al., 1981). These original findings led to a debate about the role of LIP, with some suggesting that LIP activity represented motor intention (Andersen, 1995; Mazzoni et al., 1996; Snyder et al., 1998; 2000), while others suggesting that it represented attention (Colby et al., 1996; Gottlieb et al., 1998). This debate has mostly been resolved by the understanding that LIP acts as a priority map, which guides both covert attention and eye movements (Gottlieb et al., 2009; Ipata et al., 2009; Bisley & Goldberg, 2010). Thus, while the activity does not directly represent attention or motor intention, it is involved in guiding both processes.

There is a growing consensus that a number of cortical and sub-cortical areas work in concert to allocate attention (Petersen et al., 1987; Thompson et al., 2005; Buschman & Miller, 2009; Lovejoy & Krauzlis, 2010; Noudoost et al., 2010). In this review, we examine the contribution of LIP as a part of this network. As we will describe, LIP itself has many of the hallmarks of a priority map, but it is more than likely that it works with these other areas to create the priority map used to ultimately guide attention.

Priority Map

Our view of a priority map is based on the saliency map models of Koch, Itti and colleagues (Koch & Ullman, 1985; Itti & Koch, 2000; Walther & Koch, 2006). In these models, local salient features are highlighted in low level feature maps and combined to make a global saliency map. In this case, salience refers to the prominence of a particular feature as a function of other features in that feature space; a tall thin tree in a green pasture is more salient than a tall thin tree in a forest of tress. These models also include top-down cognitive inputs, so that the resulting map is not just a function of pure bottom-up ‘salient’ inputs. Twenty years ago, Yantis and Jones (1991) pointed out that this sort of model and many other models that had been proposed at the time (Duncan & Humphreys, 1989; Bundesen, 1990; Cave & Wolfe, 1990) utilize prioritizing systems in which the order or priority in which items, locations or search strategies should be used is based on both bottom-up stimulus driven features and top-down cognitive inputs. So, because the term salience often refers to only stimulus driven features, we prefer not to use the term saliency map, as it may incorrectly imply that only these features are used as inputs. Further, because these models use prioritizing algorithms, we and others feel that the term priority map is more appropriate as it better describes the general function of the model (Fecteau & Munoz, 2006; Serences & Yantis, 2006; Ipata et al., 2009; Bisley & Goldberg, 2010). It is important to remember that this is just a semantic difference; the concept underlying the priority map is the same as that underlying a saliency map.

The basic premise of a priority map is that it combines bottom-up stimulus driven inputs with top-down cognitive biases to create a map of visual space in which the activity in a particular location is a representation of the attentional priority in that location. The output of the map is then used to guide the allocation of attention and saccade goal selection. Specifically, it has been proposed that saccades are targeted towards the location with the highest attentional priority at the time of saccade goal selection (Ipata et al., 2006a; Thomas & Pare, 2007) and that covert attention is similarly allocated in a winner-take-all fashion (Bisley & Goldberg, 2003; 2006). Thus, the activity in such a map will be correlated with the allocation of attention and the intention to make a saccade, but in a way that is relative to the activity across the map; the absolute response in a single location does not represent attention or motor intention – just the attentional priority.

Given this role, a priority map must have both top-down and bottom-up inputs and must be in a position to influence the guidance of attention and saccades. In terms of the bottom-up inputs, a priority map should have access to the rapid transient responses thought to be involved in the capture of attention by sudden onsets. Such signals are thought to be processed through the dorsal visual pathway (Bisley et al., 2004). In addition, it should have access to the sort of feature information that allows objects to become salient based on their conspicuity. Such information is present in many early visual areas. This means that a priority map should receive projections directly or indirectly from both the dorsal and ventral visual pathways. Furthermore, a priority map should receive top-down information that can explain and guide complex performance. This means higher order information, such as rules, reward information, reward history, overall executive function and a history of previous actions. So in addition to visual inputs, a priority map should have inputs from areas involved in these functions, such as prefrontal areas, anterior cingulate cortex and the dopaminergic reward system.

In addition to the appropriate inputs, a priority map should have outputs that can be used to guide attention. This requires feedback pathways to visual areas that have neurons that are modulated by attention or to thalamic nuclei that can then modulate the activity of neurons in the visual areas. It also requires projections to oculomotor areas involved in the programming of saccades.

LIP as a priority map

Although it may not be the only area to fulfill these criteria, LIP has all the features outlined in the previous section. With connections from many visual areas in both the dorsal and ventral streams (Blatt et al., 1990; Baizer et al., 1991; Lewis & Van Essen, 2000), LIP is in an ideal anatomical location to gather bottom-up information. It also receives inputs from anterior cingulate cortex, insular cortex, the claustrum and a range of pre-frontal areas (Blatt et al., 1990; Baizer et al., 1991; Lewis & Van Essen, 2000), providing potential top-down information. For guiding attention it projects directly and indirectly, via the thalamus, back to many visual cortical areas and for guiding saccades it projects to important oculomotor areas, such as the superior colliculus and frontal eye field (Andersen et al., 1985; Andersen et al., 1990; Blatt et al., 1990; Baizer et al., 1993).

Interestingly, some of the top-down inputs appear to be driven by saccade generation (Barash et al., 1991; Wurtz, 2008), which itself is potentially guided by activity in LIP. In other words, there is likely to be a positive feedback system within the network involved in the guidance of saccades such that once a peak of activity in LIP is set as the saccade target, the motor response feeds back to heighten the activity even further. Thus, the peak of activity used to identify the saccade goal is amplified, assuring that covert attention is allocated to that same location (Shepherd et al., 1986; Hoffman & Subramaniam, 1995; Kowler et al., 1995; Deubel & Schneider, 1996).

One might expect a similar feedback system to overemphasize local peaks of activity in LIP from the allocation of covert attention, which in turn would affect responses in the earlier visual areas that then project forward to LIP. However, the effects of attention on earlier visual areas do not enhance neural activity nearly as much as the way saccade programming does in the frontal eye field and superior colliculus (Reynolds & Chelazzi, 2004). When only one stimulus is within a receptive field, neurons in earlier visual areas show limited modulation and if multiple stimuli are within a receptive field, attention biases the neuron’s response towards its normal response to that stimulus presented alone – but generally the resultant activity is no greater than the normal response to that stimulus presented alone (Lee & Maunsell, 2010). In addition, if the attentional modulation in earlier visual areas is aimed at obtaining more information about the attended items, it may actually reduce the activity on the priority map if the items are not of interest or importance to the subject. Thus, we do not expect to see strong positive feedback occurring due to the allocation of attention in earlier visual areas and nor has it been reported.

Bottom-up responses

In addition to having the appropriate connections to be a priority map, responses of LIP neurons display bottom-up features and are strongly modulated by top-down influences. As in the original saliency map models, bottom-up features that stand out (i.e. that are salient) must be represented by elevated activity in a priority map. Many studies have shown that sudden onsets or moving stimuli generate elevated responses in LIP (Barash et al., 1991; Gottlieb et al., 1998; Kusunoki et al., 2000; Balan & Gottlieb, 2006; Fanini & Assad, 2009), which could be considered correlates of salience. The most persuasive of these comes from a study in which the responses to static stable stimuli in an array were compared to the responses from the same stimuli, but after they were flashed on (Gottlieb et al., 1998). The authors found that stable stimuli that were left on the screen for long periods of time (> 10 min) generated very little in terms of responses when they entered the neurons’ receptive fields following a saccade. But if the same stimulus was flashed on, then the response was elevated, even if it was brought into the receptive field by a saccade 500–2000 ms after the stimulus flashed.

Salience defined by stimulus features alone is also represented in LIP. Moving and flashing stimuli are clearly salient and easily grab our attention in everyday life, as evident by the use of flashing lights on the top of emergency vehicles. But stimulus features alone can be sufficient to grab our attention; a bright green balloon among red balloons can easily draw our gaze (also see Fig. 1A). Behaviorally, such ‘popout’ stimuli often speed up or slow down response times depending on whether the popout is something we are looking for or a distractor, respectively (Egeth et al., 1972; Treisman & Gelade, 1980; Theeuwes, 1992). Recent work has shown that the majority of LIP neurons respond more to a task irrelevant stimulus when it is salient than when it is not salient (Arcizet et al., 2011), however the mean difference in response is quite small (Fig. 1B). Nevertheless, the response is characterized by a very consistent increase, which suggests that a gain mechanism may be involved in highlighting the salience signal. Further, this signal appears in LIP earlier than a similar signal in V4, 7a or FEF (Constantinidis & Steinmetz, 2005; Buschman & Miller, 2007; Burrows & Moore, 2009; Arcizet et al., 2011), suggesting that LIP is the first location to globally represent a salience response. Interestingly, if a similar stimulus configuration is used, but the salient stimulus becomes behaviorally relevant, then the response in LIP is greatly elevated (Buschman & Miller, 2007; Thomas & Pare, 2007; Fig. 1C). This elevated response may be due to two potential top-down biases: the linking of the stimulus with a reward; or a saccade related signal indicating that a saccade will be made to the stimulus. In either case, the difference between the pure salience response and the top-down biased response shown in Fig. 1 illustrates the overwhelming influence top-down signals have on LIP neurons.

Fig 1.

Salience and top-down biases to a popout stimulus. (A) A green stimulus is a popout stimulus when surrounded by many red distractors. (B) The response to a popout stimulus is slightly higher than the response to a distractor when neither are behaviorally important. The solid black line shows the line of best fit. (C) The response to a popout stimulus is much higher than the response to a distractor when the popout is the target of search. Adapted from Arcizet et al (2011) and Thomas and Pare (2007), Am Physiol Soc, modified with permission.

Interestingly, the fact that task irrelevant salient stimuli produce such weak responses and top-down inputs have such a large effects on LIP activity is consistent with the fact that singleton stimuli have not always been found to capture attention. Intuitively, we are all aware that popouts tend to grab our attention (Theeuwes, 1991). However, well controlled studies have found that under certain conditions, singleton stimuli do not capture attention (Jonides & Yantis, 1988; Hillstrom & Yantis, 1994). This apparent ambiguity was theoretically explained by Bacon and Egeth (1994), who suggested that the capture was dependent on the way the subjects performed the task. In a ‘singleton detection mode’, task irrelevant singletons captured attention, but when subjects were in a ‘feature search mode’ the irrelevant singletons did not. Although it is unclear what the neural mechanisms are that set such modes in the brain, it is clear that the top-down inputs illustrated above could easily override the salience effect seen by Arcizet et al. (2011). Indeed, suppression of a popout has been shown in LIP for well-trained monkeys performing a visual search task in which the popout was task irrelevant (Ipata et al., 2006b).

Top-down responses

As implied in the previous section, a large number of top-down biases can influence the activity of neurons in LIP. For example, LIP responses are elevated around the time of a saccade (Barash et al., 1991), which implies that saccade generation signals may be fed back into parietal cortex. Early studies in the area showed that when a stimulus became behaviorally relevant, responses increased (Lynch et al., 1977; Robinson et al., 1978; Bushnell et al., 1981), a result that has been replicated under more complex behavioral conditions (Gottlieb et al., 1998; Toth & Assad, 2002; Klein et al., 2008), including the case described above in which a salient object that needed to be actively ignored had its response suppressed (Ipata et al., 2006b). Activity related to decision making has also been identified in LIP under many different behavioral conditions (Platt & Glimcher, 1999; Shadlen & Newsome, 2001; Roitman & Shadlen, 2002; Leon & Shadlen, 2003; Janssen & Shadlen, 2005; Seo et al., 2009). Interestingly, a number of studies have shown that LIP activity is related to reward expectancy or relative subjective value (Dorris & Glimcher, 2004; Sugrue et al., 2004; Peck et al., 2009; Louie & Glimcher, 2010; Rorie et al., 2010), so one could interpret the activity seen in the decision making tasks or even the tasks relating activity to behavioral relevance as just being a function of reward or subjective value. In any case, these are all variables that impact attentional priority and are not inconsistent with the concept that LIP can act as a priority map.

Although spatially specific, several cognitive responses identified in LIP are more difficult to fit into the priority map model. Neurons in LIP have been found to respond to numerosity (Roitman et al., 2007) and many neurons respond differentially to stimuli in categorization tasks depending on which category the stimuli fit into (Freedman & Assad, 2006). Similarly, a subset of neurons in LIP have been shown to have effector specificity; neurons respond preferentially to responses made by one hand over the other (Oristaglio et al., 2006). On its face, results such as these appear to be difficult to reconcile with the concept of a priority map, but because some neurons are always responsive to a particular stimulus, numerosity, category or effector and because the strong responses are always spatially specific, the average response across such a population will still provide an accurate representation of attentional priority across space. We would note that this interpretation does not suggest that these signals are not used or important in their respective tasks, just that the signals are not incompatible with the priority map hypothesis.

Evidence that LIP activity guides covert and overt attention

Covert attention

To be a priority map, LIP not only needs to have the appropriate anatomical connections and responses that represent both bottom-up and top-down influences, but its responses must also correlate with the guidance of covert attention and saccade selection. Although many studies have claimed to relate the activity in LIP with attention (Lynch et al., 1977; Robinson et al., 1978; Bushnell et al., 1981; Colby et al., 1996; Snyder et al., 1998; Bisley & Goldberg, 2003; Bendiksby & Platt, 2006; Quian Quiroga et al., 2006; Herrington & Assad, 2009), only a few have explicitly attempted to quantify attention or shifts in attention to which activity can be correlated (Bisley & Goldberg, 2003; Herrington & Assad, 2009). We will describe the results of the former, which provided the first evidence that LIP activity did not directly correlate with attention, but was used in the guidance of attention.

To identify the locus of attention, Bisley and Goldberg (2003; 2006) trained animals on a dual task in which they had to plan a memory guided saccade and also had to discriminate the orientation of a probe (Fig. 2A). From trial to trial, the luminance of the probe varied so that psychometric functions could be recorded and the locus of attention was defined as a probe location where the contrast threshold was significantly lower than recorded under control conditions. For all conditions, the contrast thresholds were normalized daily, so an attentional benefit was measured as a normalized contrast significantly less than 1 (Fig. 2B, upper plot). The underlying concept of the task was that attention should be initially allocated to the goal of the saccade (Shepherd et al., 1986; Hoffman & Subramaniam, 1995; Kowler et al., 1995; Deubel & Schneider, 1996), but that it may shift if a distractor is flashed in a novel location (Fig. 2A). This is what was found; attention was first at the saccade goal, but shifted to the distractor location when tested 200 ms after the distractor flashed (first pair of triangle points, upper panel, Fig. 2B). Importantly, attention shifted back to the saccade goal location within 700 ms of the distractor flashing (second and third pairs of triangle points).

Fig 2.

The task and data collected to compare the locus of attention with activity in LIP. (A) The task was based on a memory guided saccade task, with a task irrelevant distractor. In this task, animals had to plan an eye movement to the remembered location of a flashed target. During the delay, a task irrelevant distractor could flash at or opposite the target location. 500 ms before the fixation point (FP) was extinguished, 4 rings appeared for 1 video frame. One of the rings had a gap either on the left or the right (the probe). The monkey had to identify the side of ring that the gap was on and indicate it either by either making the planned memory guided saccade when the fixation point was extinguished (GO) or by canceling the saccade and maintaining fixation until the end of the trial after the fixation point was extinguished (NOGO). (B) Behavioral and physiological data. The thin traces in the top panel show the animal’s behavioral performance plotted as normalized threshold. Points that are significantly beneath the black dashed line indicate an attentional advantage (*). The thick traces in the lower panel show the spike density function of a single neuron (the width of the trace shows the SEM). Blue traces show data from trials in which the probe was placed at the target site, and the distractor had flashed elsewhere. Red traces show data from trials in which the probe was placed at the distractor site and the target had flashed elsewhere. The thin black trace shows the result of a running statistical test showing when the thick red and blue traces were indistinguishable (gray block). From Bisley & Goldberg (2003) as modified in Bisley & Goldberg (2006), Am Physiol Soc, used with permission.

Using the same task, the authors found that an attentional benefit was always at the location represented by the greatest activity in LIP. Initially, following the presentation of the saccade target, elevated activity was seen at the target location in LIP (Barash et al., 1991; Bisley & Goldberg, 2003; 2006) and nowhere else. During this time, attention was at the saccade goal. Shortly after the distractor flashed, the burst of activity at the distractor location rose above the delay activity at the saccade goal (red trace is greater than blue trace early in Fig. 2B, bottom panel). During this time, attention had shifted away from the saccade goal and to the distractor location. As the response to the distractor waned to a level beneath the delay activity, attention shifted back to the saccade goal (around 600–700 ms in Fig. 2B). These data suggested the hypothesis that attention was allocated to the location represented by the greatest activity in LIP. To test this, the authors probed the animals’ attention at the time in which there was no clear winner, identified by the dip in the black trace, representing a rise in the p-value comparing the difference between the traces in Fig. 2B (first pair of circle points, upper panel). They found that there was no attentional benefit in either location when the activity was approximately equal. Further, they found that different animals had different times in which attention shifted, as predicted by the neural activity from the individual animals (Bisley & Goldberg, 2003; 2006). Together with a more recent study (Herrington & Assad, 2009), these data provide strong evidence that LIP activity is tightly correlated with the allocation of covert attention in a way that is consistent with the priority map hypothesis. The level of activity does not represent attention, but the attentional priority; when a location is represented by the greatest activity across the map, attention will be allocated to that location – even if it is relatively low activity.

Guidance of saccade goal selection

Activity in LIP also predicts the saccadic latency for a saccade made to a stimulus appearing suddenly in the periphery. For LIP to act as a priority map, it should also be involved in guiding saccadic eye movements, which are really just shifts in overt attention. Although many studies had suggested that LIP activity was related to motor intention (Mountcastle et al., 1975; Andersen, 1995; Mazzoni et al., 1996; Snyder et al., 1998; 2000), none had actually tested whether the activity predicted the goal and timing of saccades. The closest to this had been a study on decision making, which showed that once activity in LIP reached a critical threshold, a saccade was generated (Roitman & Shadlen, 2002). However, two more recent studies independently showed that LIP activity predicts the goal and latency of the first saccade (Ipata et al., 2006a; Thomas & Pare, 2007). The underlying theory of these experiments is outlined in Fig. 3A, using data from the first study as an example. Briefly, if LIP activity is used to guide saccades, then once a unique peak in activity across the priority map is identified, a saccade should be made a set time afterwards. This has been termed the saccade goal selection time or split time, when comparing mean target and distractor traces, as in Fig. 3. This means that the difference in latency between short and long latency saccades should be explained by extra time taken to identify the saccade goal in LIP. This is represented by the solid lines in Fig. 3A. On the other hand, if the activity in LIP does not guide saccade goal selection directly, then the time at which the peak in activity is identified in LIP should not correlate with saccadic latency (dashed lines in Fig. 3A). Under both conditions in which the animals were able to make saccades to any location (Ipata et al., 2006a) and conditions in which animals were instructed to make a saccade to a singleton (Thomas & Pare, 2007), the time it took for a peak to emerge in LIP correlated with the saccadic latencies (Fig. 3B) and not with the time between the saccade goal selection and saccadic onset (Fig. 3C). On its own, this result could be used as evidence that LIP is involved in saccade goal selection. But, it could also mean that the saccade goal selection occurs before the activity reaches LIP. However, similar studies in FEF, which projects to LIP, have generally found equal or longer saccade goal selection times (Thompson et al., 1997; Bichot & Schall, 1999; Sato et al., 2001; Sato & Schall, 2003) and earlier visual areas, such as V4, do not have saccade goal selection times which correlate with saccadic latencies (Gee et al., 2010). Thus, the data support the hypothesis that saccade goal selection is initially driven by the appearance of a novel peak in activity in LIP.

Fig 3.

Relationship between LIP activity and first saccadic latency. (A) An example short latency trial is compared to 2 possible long latency trials showing the 2 extreme possibilities in how the extra time is added to latency. In the upper long latency example (dashed line), the time from array onset until the split time is identical, so all the variability in latency time comes in to the process after LIP. This would suggest LIP is not involved in saccadic selection. In the lower long latency example (solid line), the extra latency time comes before the split time in LIP. This suggests that LIP or some area before it is first identifying the saccade goal. The 2 sets of hypothesized results (dashed and solid) are plotted in the small panels comparing the split time calculated by array onset and split time calculated by saccade onset. (B) The time from array onset to split time is plotted against the mean first saccadic latency for each group for each cell. The dotted line shows an example slope of 1. (C) The time from the split time to saccade onset is plotted against the mean first saccadic latency for each group for each cell. The flat lines suggest that a set time after a peak is identified in LIP, a saccade is generated. For (B) and (C), the black lines connect points from the same cell and the solid red lines connect the population means. Adapted from Ipata et al. (2006a) with permission.

The role of the LIP priority map in visual search

Covert visual search

Just using the information provided above, it is easy to see how LIP can help guide covert visual search. Covert visual search involves finding an object among distractors without moving the eyes (eg. Theeuwes, 1992). Thus a priority map, which guides attention, should identify potential targets by highlighting objects that are similar to the target or, in the terminology used above, objects that are behaviorally relevant. In some cases, when the target perceptually pops out from the remaining stimuli (as in Fig. 1A), the priority map will highlight the target immediately, as it did even when no search was necessary (Fig. 1B). This would lead to rapid reaction times and could be thought of as a neural correlate of ‘pre-attentive’ or ‘parallel’ visual search (Treisman & Sato, 1990; Wolfe, 1994), although it does not involve any additional parallel processing and still leads to the guidance of attention.

Even under conditions in which the target does not perceptually pop out, the target is represented by elevated activity in LIP. In a study by Oristaglio and colleagues (Oristaglio et al., 2006), monkeys were trained to perform covert visual search in which the orientation of the target was indicated by releasing one of two bars. The authors found elevated activity representing the target and this elevated activity emerged early enough for it to be viable in the role of guiding attention, which in turn guided the manual response. To show that such target related activity is actually used to guide covert attention, two studies have reversibly inactivated LIP using Muscimol, a GABA_freceptor agonist, after which monkeys performed a covert visual search task (Wardak et al., 2004; Balan & Gottlieb, 2009). Both studies found that LIP inactivation significantly slowed the time it took the animals to find and signal, using a hand movement, the presence or orientation of the target among distractors. However, the time it took to respond to a stimulus without any distractors present was not lengthened (Wardak et al., 2004). Thus, the inactivation did not slow the action or recognition of the stimulus, just the ability to covertly identify it among distractors. Conceptually, one can think of the inactivation reducing the activity of the stimuli represented in the inactivated region of space so that they have a lower attentional priority than they should have based on the top-down instruction. Thus, it takes longer to find the target because the distractors in the non-inactivated region of space have higher activity and are more likely to be attended to first during the covert search.

Overt visual search

In most behavioral settings we search a scene by moving our eyes, not by fixating one location and attempting to find a target using covert attention. Thus, a priority map should also be involved in prioritizing objects so that observers can find the target in a minimal number of steps. Part of this requires a prioritization of stimuli so that objects that share features with the target are given higher priorities and objects that are clearly distinguishable from the target are given lower priorities. But a second mechanism that makes sure that observers are not duped into looking at non-targets multiple times is also necessary. Although no studies have parametrically examined distractor similarity, many have shown that behaviorally relevant stimuli produce elevated responses in LIP, as we mentioned above. However, there is only limited evidence concerning the second issue. This is in part because most physiological studies examining overt visual search have either restricted eye movements (eg. Bichot & Schall, 1999; McPeek & Keller, 2002) or have allowed free eye movements, but have placed stimuli on the screen in an arrangement such that useful neural data can only be collected from the first saccade (Ipata et al., 2006a; Gee et al., 2010). To avoid these limitations, we performed a series of studies using an array in which stimuli were arranged so that when the animal looked at one stimulus, another was usually within the neuron’s receptive field (black lines, Fig. 4A; also see Mazer & Gallant, 2003; Bichot et al., 2005). In our task, the animals were free to look anywhere on the screen, but had to fixate a target to gain the reward (Mirpour et al., 2009; Mirpour et al., 2010). To make the animals scan through the array, the target and 4 non-target stimuli were all identical (white T shapes), so the animal did not know which one would give him the reward. Additional distractors that were clearly distinguishable from the target were also present (+s). This task, termed a foraging task, forced the animals to check each T until they received a reward. To get the reward in the fewest number of eye movements, they had to differentiate between potential targets and distractors and also remember which Ts they had already looked at. We refer to this as an efficient foraging strategy.

Fig 4.

Responses in a visual foraging task. (A) In the visual foraging task, 10 objects were presented: 5 Ts and 5 +s. To get the reward, the animal had to look at the target (one of the Ts) for 500 ms. This lead the animal to forage from T to T (cyan line) until he found the reward. The stimuli were arranged so that when the animal looked at one stimulus (small black circle) another stimulus was usually within the neuron’s receptive field (large black oval). (B) The mean normalized responses from a population of 54 neurons in two animals under 5 different conditions based on what stimulus was in the receptive field, whether it had been fixated previously within the trial and whether it was the goal of the next saccade. From Mirpour et al. (2009), Am Physiol Soc, used with permission.

The responses within LIP were sufficient to explain the efficiency of the animals’ foraging ability (Mirpour et al., 2009). The responses to potential targets were higher than responses to the task irrelevant distractors (Fig. 4B); consistent with the concept that LIP activity highlights behaviorally relevant stimuli. The highest responses were seen to stimuli that the animal was just about to look at (white columns, Fig. 4B), which is what one would expect if saccades are made to the peak of the priority map. Most interestingly, the responses to Ts that the animal had already examined during his exploration of the array were reduced compared to the responses to the same stimuli that had not been fixated, independent of whether they were the goal of the next saccade or not. These reduced responses can be thought of in two contexts. In the first, one can think of the reduced activity representing the reduction in reward likelihood of the stimulus; before the animal looks at a T, it can be seen as a potential target, but once the animal examines it without getting a reward, he realizes that it is not behaviorally relevant and has no reward likelihood. The second context is that this reduction can be thought of as the ecologically valid manifestation of inhibition of return (IOR). Although IOR was originally identified behaviorally in a reaction time task (Posner & Cohen, 1984), the interpretation of this phenomena is that it helps the mind keep track of what items have been looked at in search (Klein, 2000). Indeed, the saliency map models, on which we based our priority map model, explicitly use IOR to stop attention repeatedly jumping between the two highest peaks on the saliency map, by suppressing the response at the currently attended location (Itti & Koch, 2000). It is unclear whether the first context actually drives the change in response creating the behavioral result of the second context, but psychophysical results showing that saccadic latencies are reduced when saccades are made back to the previous point of fixation independent of reward suggest that this may not be the case (Ludwig et al., 2009).

The reduction in the response to a stimulus that has been fixated in a foraging task essentially acts as a form of short term memory that lets the priority map keep track of which potential targets have been examined. Given that we believe that saccades are made to the peak of the priority map, reducing the response of one of the peaks reduces the probability that a saccade will be made to that location. Thus, we can explain the efficiency of the animals’ foraging behavior based on the reduced responses to distractors relative to the responses to targets and to the reduced responses to Ts once they have been examined and ruled out as containing the reward. At the start of a trial, the 5 potential targets have elevated activity and the 5 distractors do not. As the animal scans through the 5 Ts, the number of elevated peaks is reduced allowing the animal to efficiently forage through the scene. To test whether this activity actually drives this behavior, we performed a microstimulation study in which neurons in LIP were stimulated after the third saccade on a subset of trials in the foraging task. The theory of this experiment was that if the reduced activity seen in LIP is the reason that the animals do not look at the stimuli, then increasing the activity should result in an increase in probability of looking at the stimulus.

Microstimulation of LIP weakly increased the probability of looking at a stimulus that was normally represented by reduced activity (Mirpour et al., 2010). In normal behavior, the animals rarely looked at distractors, which are typically represented by weak activity on the priority map. However, this probability more than doubled when a distractor was in the receptive field of neurons that were microstimulated (Fig. 5A). A similar result was seen when the stimulus in the receptive field was a T that had been examined already within the trial; the probability of going to such a stimulus was a little over 1% and increased to over 2% with microstimulation (Fig. 5B). The probability of going to the stimulated receptive field when it did not contain a stimulus did not change (Mirpour et al., 2010), suggesting that microstimulation biases performance by modulating the responses to stimuli on the priority map and not by just generating saccades. Thus, these data are consistent with the idea that, under normal conditions, the reduced activity seen in response to distractors or Ts that have been examined is the reason that saccades are rarely made to these stimuli.

Fig 5.

Effect of LIP microstimulation during the foraging task. The probability of making a saccade to the receptive field when a distractor (A) or T that has been fixated (B) is within the receptive field. Data are shown from control trials and stimulation trials in sessions in which microstimulation was performed (red columns) and in which microstimulation was not performed (sham sessions, blue columns). Increases in the probability of making a saccade to the receptive field was only seen in stimulation trials within microstimulation sessions. From Mirpour et al. (2010), Am Physiol Soc, used with permission.

Although the task described in these two studies was a foraging task in which many objects were visibly identical, the underlying conclusion is important in more traditional visual search. The results show that, in addition to identifying target stimuli from distractor stimuli, LIP activity is modulated based on whether a location has been examined or not. So whether we are looking for our keys on a cluttered kitchen bench or a face in a crowded picture, the intrinsic knowledge of where we have already looked is of critical importance (Arani et al., 1984; Horowitz, 2006).

Conclusion

We have described current thinking about the role that the lateral intraparietal area of posterior parietal cortex plays in the allocation of visual attention. It appears to act as a priority map, in which the activity at any location represents the attentional priority of the stimulus in that location. The attentional priority is related to both the object’s salience and any top-down biases that influence the relative importance of that object to the subject, including the suppression of objects that have already been examined during visual search. Overt and covert attention is guided to the peak on the priority map. As in saliency map models, this system allows subjects to methodically scan the scene starting with objects with the highest attentional priority, and is sufficient to explain efficient visual foraging strategies.