Parietal cortex regulates visual salience and salience-driven behavior

Summary

Unique stimuli stand out. In spite of an abundance of competing sensory stimuli, the detection of the most salient ones occurs without effort, and that detection contributes to the guidance of adaptive behavior. Neurons sensitive to the salience of visual stimuli are widespread throughout the primate visual system and are thought to shape the selection of visual targets. However, a neural source of salience remains elusive. In an attempt to identify a source of visual salience, we reversibly inactivated parietal cortex and simultaneously recorded salience signals in prefrontal cortex. Inactivation of parietal cortex not only caused pronounced, and selective reductions of salience signals in prefrontal cortex, but also diminished the influence of salience on visually guided behavior. These observations demonstrate a causal role of parietal cortex in regulating salience signals within the brain and in controlling salience-driven behavior.

eTOC Blurb

Chen et al. show that inactivation of parietal cortex selectively reduces salience signals within prefrontal cortex and diminishes the influence of salience on visually guided behavior. The results demonstrate a causal role of parietal cortex in regulating salience signals within the brain and in controlling salience-driven behavior.

Introduction

Throughout the brain, sensory input is continually filtered in favor of information that more adaptively shapes behavior. This filtering of sensory information is often referred to as selective attention, a basic cognitive function. Attention can be goal-driven or stimulus-driven, the former describing selective processing due to an endogenously generated signal (e.g. representation of a rule, strategy, or motivational state) and the latter describing selective processing based solely on stimulus properties (Knudsen, 2007). Much progress has been made in identifying the neural circuits controlling goal-driven attention, particularly in the primate visual system (Moore and Zirnsak, 2017). In contrast, the mechanisms controlling stimulus-driven attention remain largely unknown. In stimulus-driven attention, the selective processing of sensory stimuli occurs automatically for stimuli that are salient due to their inherent ethological relevance (e.g., a looming object) or to their uniqueness among all other stimuli (e.g., a single red stimulus among green ones) (Knudsen, 2007). In the visual modality, models of stimulus-driven attention have been developed largely from psychophysical studies of the influence of unique visual features on the allocation of attention (Egeth et al., 1972; Itti et al., 1998; Koch and Ullman, 1985; Treisman and Gelade, 1980). These models suggest that the neural mechanisms of stimulus-driven attention may be separable at some level from those controlling goal-driven attention. A key aspect of such models is the proposition that contrasts in the component features within a visual scene (e.g. color) are combined from feature-selective inputs to form maps of salience in which unique objects can be localized in space, regardless of the features that define them (Itti et al., 1998; Soltani and Koch, 2010).

In the primate brain, the control of goal-driven visual attention appears to be accomplished by neurons distributed within areas of prefrontal (Bichot et al., 2015; Buschman and Miller, 2007; Kastner et al., 1999; Moore and Fallah, 2001) and posterior parietal cortex (PPC) (Buschman and Miller, 2007; Kastner et al., 1999), along with the superior colliculus (Ignashchenkova et al., 2004; Krauzlis et al., 2013), and the pulvinar (Saalmann et al., 2012; Zhou et al., 2016). A lingering major question, however, is the extent to which any of these structures contributes causally to stimulus-driven attention. Although many studies have examined the influence of visual salience on the responses of neurons in these structures (Buschman and Miller, 2007; Constantinidis and Steinmetz, 2001; Ipata et al., 2006; Thompson and Bichot, 2005) and throughout posterior visual cortex (Allman et al., 1985; Burrows and Moore, 2009; Hegdé and Felleman, 2003; Knierim and van Essen, 1992; Motter, 1994; Reynolds and Desimone, 2003), none have identified the structures that contribute causally to the representation of salience in the brain. An abundance of neurophysiological evidence suggests a role of PPC in stimulus-driven attention and to the coding of visual salience (Gottlieb et al., 1998; Ipata et al., 2006; Kusunoki et al., 2000), particularly the lateral intraparietal area (LIP) (Bisley et al., 2011). Indeed, the emergence of visual salience signals within PPC seems to proceed faster than in prefrontal areas, suggesting that neurons there compute a map of visual salience that propagates to downstream prefrontal areas (Buschman and Miller, 2007). Yet, in spite of the strong correlative evidence of a role of PPC to visual salience, such a role has not been causally examined. Previous studies have demonstrated a causal role of PPC areas in multiple visuospatial functions (Gottlieb and Snyder, 2010), including goal-driven attention (Liu et al., 2010a; Wardak et al., 2004), eye and arm movement planning (Hwang et al., 2012; Liu et al., 2010a), and perceptual decision-making (Zhou and Freedman, 2019; although see Katz et al., 2016), but its contribution to visual salience has not been tested.

To address this question, we studied the contribution of PPC to visual salience by reversibly inactivating it in behaving monkeys. We measured the effects of reversible inactivation both on the representation of salience downstream of PPC in prefrontal cortex, and on salience-driven behavior. We first show that reversible unilateral inactivation of PPC produced behavioral effects consistent with previous studies. We then show that such inactivation led to a selective reduction in coding of unique stimuli by neural activity within prefrontal cortex. Finally, we show that PPC inactivation diminished the influence of salience on visually guided eye movements.

Results

Behavioral effects of PPC inactivation

We reversibly inactivated large portions of PPC of two behaving monkeys (J and Q) via cryoloops which were chronically implanted within the intraparietal sulcus (IPS) (Methods)(Figure S1). Cryoloops have been used extensively in the primate brain to temporarily eliminate the spiking activity of neurons within large expanses of neocortex in behaving animals (Hupé et al., 1998; Lomber et al., 1999; Ponce et al., 2008; Smolyanskaya et al., 2015). To assess the effectiveness of the inactivation, we first measured its impact on behaviors known to be affected by disruption of PPC activity in primates (Lynch and McLaren, 1989; Wardak et al., 2002). We did this in two ways. First, we measured the effects of inactivation on exploratory eye movements during free-viewing of complex images. Monkeys were allowed to freely view large images (79–98 by 49–55 degrees of visual angle, dva) for 3 seconds (Figure 1A). Consistent with previous observations, inactivation of PPC in monkeys reduced the tendency to visually explore the contralateral half of head-centered space (Figure 1B and Figure S2). To quantify this effect, we computed the density of fixations during free-viewing across all images for the two monkeys, and then compared the densities between control and inactivation (Figure 1C). For both monkeys, PPC inactivation reduced the fixation density within the contralateral visual field, resulting in a significant reduction in the proportion of fixations contralateral to the inactivation (monkey J, control_contra= 0.49, inactivation_contra= 0.37, P<10⁻³; monkey Q, control_contra= 0.71; inactivation_contra= 0.53, P <10⁻³⁴; paired t-test) and a shift in the center of mass of fixations toward the ipsilateral visual field (monkey J, shift = 5.11 dva, P<10⁻²⁸; monkey Q, shift = 4.14 dva, P<10⁻²⁸; paired t-test). Thus, even with a coarse measure of behavior, the effect of PPC inactivation was clear.

Figure 1. Behavioral effects of PPC inactivation.

(A) Free-viewing task. Images presented to the monkeys included real-word photographs, paintings, cartoons, and abstract patterns. Identical images were presented during both control (top, gold shading in IPS) and inactivation blocks (bottom, blue shading in IPS). (B) Example image presented to one monkey during a control (top) and inactivation block (bottom). Circles indicate regions of fixation and the lines indicate saccades. The origin of the coordinate system indicates the initial fixation of the monkey at the onset of the image. (C) Change in fixation densities across the population of images for Monkey J (top) and Monkey Q (bottom). The left part of the color maps corresponds to the visual field contralateral to the inactivated PPC, in head-centered coordinates. (D) Double-target, choice task. Two targets were presented at varying temporal onset asynchronies; contralateral targets could trail (-) or lead (+) ipsilateral targets. (E) Example experimental session for one monkey. Target choice functions during control and during PPC inactivation are plotted in gold and blue, respectively. Positive values denote contralateral leading targets. (F) Distribution of shifts in the PES across all sessions in the two monkeys.

Second, we used a double-target, choice task to psychophysically assess the effect of inactivation on the tendency of monkeys to choose targets in the two hemifields (Schiller and Tehovnik, 2003; Soltani et al., 2013). In this task, monkeys were rewarded for choosing between two saccadic targets, one located within the contralateral hemifield, and one in the ipsilateral hemifield. The temporal onset of the two targets was systematically varied such that the contralateral stimulus could appear earlier or later than the opposite stimulus (Figure 1D). The monkey’s tendency to select the contralateral target could then be measured as the temporal onset asynchrony required for equal probability of selecting either target. Thus, a neglect of one hemifield would result in a shift of the point of equal selection (PES) toward the ipsilateral hemifield. Indeed, that is what we observed; the PES shifted in favor of the ipsilateral target (Figure 1E). As a result, in order for contralateral targets to be chosen as frequently, they needed to appear earlier than during control blocks. This effect was reliably obtained in both monkeys (monkey J, ∆_PES=189.19 ± 76.13 ms, P < 0.04; monkey Q, ∆_PES= 85.19 ± 13.56 ms, P < 2.26 × 10⁻⁶; paired t-test) (Figure 1F). Notably, inactivation of the ventral IPS alone was sufficient to produce effects equivalent to both dorsal and ventral inactivation (Figure S3), consistent with an earlier comparison of dorsal and ventral lateral intraparietal area (LIP) (Liu et al., 2010b). The magnitude of the choice effect varied across sessions, similar to previous studies (Balan and Gottlieb, 2009). However, it was generally greater than that of studies using more localized PPC inactivations (Schiller and Tehovnik, 2003; Wardak et al., 2002; Balan and Gottlieb, 2009). Thus, by both behavioral measures, PPC inactivation produced behavioral effects that generally resembled the effects of PPC inactivation or damage in monkeys ( Balan and Gottlieb, 2009; Lynch and McLaren, 1989), effects that are thought to model hemispatial neglect in humans (Kubanek and Snyder, 2015).

Representation of salience by neurons within prefrontal cortex

To assess the effects of PPC inactivation on the representation of visual salience, we recorded neuronal spiking activity and local field potentials (LFPs) within prefrontal cortex, specifically within the frontal eye field (FEF). Neurons within the FEF receive input directly from most areas within posterior visual cortex (Schall et al., 1995), as well as strong inputs from areas within PPC, particularly the lateral intraparietal area (LIP) (Lewis and Van Essen, 2000), and FEF neurons are sensitive to visual salience (Fernandes et al., 2013; Thompson and Bichot, 2005). We recorded the activity of FEF neurons in two behaving monkeys using multichannel microelectrodes (see STAR Methods). We then assessed the representation of visual salience in the recorded neuronal (n=352) and LFP activity (n=192) that was present prior to inactivation. For both types of activity, we measured the responses to visual stimuli consisting of a single colored stimulus presented in isolation, or among an array of identically or differently colored stimuli (Figure 2A).

Figure 2. Prefrontal representation of visual salience in neuronal and LFP activity.

(A) Visual stimuli consisted of a single colored stimulus presented in isolation (Isolated), or among an array (6 ✕ 4) of identically colored stimuli (Unique). (B) Example CRFs of a single FEF neuronal recording mapped with an isolated red or green stimulus (top) and URFs of the same neuronal recording mapped with a unique red or green stimulus (bottom). Responses were normalized across stimulus conditions. Icons above each RF denote stimulus conditions, but not the full array. (C) Spiking responses from an example neuronal recording to isolated and unique stimuli presented inside the CRF/URF (dark gold), shown with responses to single and unique stimuli presented outside of the CRF/URF or to identically colored stimulus arrays (light gold) for two color polarities (left icons). (D) Response spectra of an example FEF LFP recording. Same conventions as in C. (E) High-gamma band CRFs and URFs for an example recording. Responses were normalized across stimulus conditions.

Using an isolated red or green stimulus, we mapped the region of space most sensitive to visual stimulation, i.e., the classical receptive field (CRF), for each neuronal recording. FEF neurons are not typically selective for stimulus features, including color (Bichot et al., 1996; Mohler et al., 1973), as in the example shown in Figure 2B. Nonetheless, FEF neurons are sensitive to stimuli that are unique among competing ones (Buschman and Miller, 2007; Thompson and Bichot, 2005). Thus, for each neuronal recording, we could also map the region of space most sensitive to a unique stimulus (URF). Neurons therefore signaled the location of both isolated and unique stimuli, independent of color (Figure 2B). Across our population of neurons, the difference in responses to an isolated red or green stimulus was typically small (median = 4.6%), consistent with previous studies (Bichot et al., 1996). Nonetheless, for the same population, neuronal responses were robustly enhanced by the appearance of a unique stimulus in the URF. The enhancement was evident in comparisons with responses to arrays in which the unique stimulus fell outside of the URF (Unique_In - Unique_Out). The enhancement was also evident in comparisons with responses to an array that rendered the URF stimulus identical to surrounding stimuli (Unique_In -Identical). Both enhancements were evident regardless of color polarity (Figure 2C). We quantified the two types of enhancement by computing a standard index of response enhancement, specifically the difference between the Unique_In and the Unique_Out (or Identical) responses, divided by their sum. Across the population, both types of enhancement were highly significant (median Unique_In - Unique_Out index = 0.11, P < 10⁻⁴⁵; median Unique_In-Identical index = 0.11, P < 10⁻⁴⁴; paired t-test), with approximately half of the population exhibiting significant enhancements in both of the comparisons (Unique_In - Unique_Out, 193/352; Unique_In-Identical_, 173/352).

In addition, we probed the representation of visual salience in the FEF LFPs. Information about the location of isolated visual stimuli is robustly signaled within the alpha (8–12Hz) and high-gamma (60–150Hz) bands of FEF LFPs, and clear CRFs can be derived from activity in each band (Chen et al., 2018). In the present study, we observed that activity in the high-gamma band, but not the alpha band, also robustly signaled the location of a unique visual stimulus (Extended Data Table 1). Compared to other frequency bands, responses to unique stimuli were most consistent in the high-gamma LFPs, and they were enhanced relative to responses to the appearance of unique stimuli outside of the URF and to arrays that rendered the URF stimulus identical to surrounding stimuli (Figure 2D). Activity in the beta-band showed significant reductions in responses to unique stimuli inside the RF, relative to stimuli outside or to identical stimuli. However, consistent with previous observations (Chen et al., 2018), clear CRFs and URFs could not be obtained from beta-band responses. Using the high-gamma signal, we derived visual RFs for both the isolated and the unique stimulus, similar to the spiking activity (Figure 2E). Across the population of recorded high-gamma LFPs, we observed both types of enhancement observed in the spiking responses (Unique_In - Unique_Out, median ∆Energy = 0.50 dB, P < 10⁻²⁰; Unique_In - Identical_, median ∆Energy = 0.47 dB, P < 10⁻²¹; paired t-test). Thus, similar to the spiking activity, the high-gamma LFPs were highly sensitive to visual salience.

Salience signals in prefrontal cortex during PPC inactivation

Given the clear behavioral effects we observed during PPC inactivation, we next asked whether removing parietal input alters visual responses in the FEF. We reasoned that if indeed parietal areas contribute distinctively to the representation of visual salience, then PPC inactivation should selectively reduce salience signals downstream in the FEF. Indeed, that is what we observed. First, PPC inactivation did not significantly change the selectivity of FEF neurons to color (P = 0.99, paired t-test). Second, it had relatively small effects on visual responses to isolated stimuli and the resultant CRFs derived from spiking or LFP activity. However, inactivation strongly altered visual responses to unique stimuli and resultant URFs (Figure 3). During inactivation, visually driven activity was generally reduced in proportion to the magnitude of visual responses during control trials (ANCOVA main effect, P < 10⁻⁴¹). However, the size of the reduction significantly depended on the stimulus condition (ANCOVA interaction, P < 0.002) (Extended Data Table 2), with the URF stimulus yielding the greatest reduction in visual responses. This selective reduction can be seen in the example neuron shown in Figure 4A. In this example, responses to an isolated stimulus were minimally affected by PPC inactivation. In contrast, responses to the unique stimulus were diminished compared to responses to arrays in which the unique stimulus fell outside of the URF, or an array that rendered the URF stimulus identical to surrounding stimuli. As a consequence of the selective reduction in visual responses, the two types of salience enhancement observed in FEF neurons were markedly reduced by inactivation (Control: Unique_In - Unique_Out index = 0.26, Inactivation: Unique_In - Unique_Out index = 0.10; Control: Unique_In - Identical index = 0.33, Inactivation: Unique_In - Identical index = 0.17). This pattern of results was similar across the population. For neurons modulated in at least one of the two metrics (n=193), inactivation selectively reduced responses to unique stimuli (Figure 4B; Figure S4A), and consequently, both types of enhancement (Unique_In vs. Unique_Out, P<10⁻¹¹; Unique_In vs. Identical_, P<10⁻¹⁵; paired t-test). Reductions in the two types of enhancement indices were ~38% (Control: median Unique_In - Unique_Out index = 0.21, Inactivation: median Unique_In - Unique_Out index = 0.13, P < 10⁻⁹; Control: Unique_In-Identical index = 0.18, Inactivation: Unique_In-Identical index = 0.11, P < 10⁻¹³; paired t-test) (Figure S4B and C).

Figure 3. Prefrontal CRFs and URFs during PPC inactivation.

(A) CRFs and URFs for three example neuronal recordings during control (left) and PPC inactivation (middle), shown together with their respective difference maps (inactivation - control) (right). Activity for each recording was normalized across all stimulus and experimental conditions (B) CRFs and URFs for three example LFP recordings (high-gamma band). Same organization and notation as in A.

Figure 4. Representation of salience in prefrontal neuronal activity during PPC inactivation.

(A) Mean responses of an example neuron to different stimulus conditions during control (gold) and PPC inactivation (blue). Left, responses to Unique_In (dark) and Unique_Out (light) stimuli. Middle, responses of the same neuron to Unique_In (dark) and Identical (light). Right, responses of the same neuron to the isolated stimuli presented inside (dark) and outside of the CRF (light). Shading around the response denotes ±SEM. (B) Mean responses for all modulated neurons (n = 193) during control and inactivation blocks for each of the stimulus comparisons (Unique_In vs. Unique_Out, Unique_In vs. Identical, Isolated_In vs. Isolated_Out). Same notation as in A. Gray scatterplots show the reduction in response differences for all recordings (n = 352). (C) Accuracy of classifiers trained on neuronal spiking activity to discriminate between different stimulus conditions during control and PPC inactivation. Left, accuracy of classifiers trained to discriminate between Unique_In and Unique_Out stimuli. Middle, accuracy of classifiers trained to discriminate between Unique_In and Identical stimuli. Right, accuracy of classifiers trained to discriminate between Isolated stimuli appearing inside and outside of the CRF. Scatter plots and marginal distributions compare discrimination accuracies across stimulus selective neuronal recordings (n = 193) during control and inactivation. Gray scatterplots show the reduction in enhancement indices during inactivation as a function of enhancement indices measured during control for all recordings (n = 352). Black lines show the linear regression fits.

To further quantify the effects of PPC inactivation across the population of FEF neurons, we measured the accuracy of a linear classifier in discriminating between visual stimulus conditions using the trial-by-trial responses of each individual neuronal recording. We focused our analysis on 193 FEF neuronal recordings with significant response differences between the inside and outside RF conditions for both isolated and unique stimuli (see STAR Methods) (Figure 4C). For these neurons, sensitivity to visual salience was selectively reduced. During control trials, the classifier performed above chance in discriminating the unique inside and outside conditions (Unique_In vs. Unique_Out) in 147 neuronal recordings. However, during inactivation that number was reduced by 40% to 88 (McNemar’s chi-square = 43.7, P < 10⁻¹⁰), and the median classifier performance was reduced significantly by 7.5% (P < 10⁻¹⁶; paired t-test), a 39% reduction in above-chance performance (see STAR Methods). Similarly, the classifier performed above chance in discriminating between the unique RF stimulus and the identical array (Unique_In vs. Identical) in 135 neuronal recordings during control trials. Yet during inactivation that number was reduced by 44% to 75 (McNemar’s chi-square = 37.8, P < 10⁻⁹), and the median classifier performance was reduced significantly by 7.5% (P < 10⁻¹⁶; paired t-test), a 43% reduction in above-chance performance. This reduction in discrimination performance was accompanied by a reduction in the two types of salience enhancement, a reduction that was correlated with enhancement during control trials (Unique_In - Unique_Out:_, r = −0.40, P < 10⁻¹⁷; Unique_In – Identical: r = −0.44, P < 10⁻²⁰; paired t-test). The slopes of both correlations were significantly steeper than that observed in responses to isolated stimuli (Unique_In - Unique_Out vs. Isolated: ∆slope = −0.21, P < 10⁻³; Unique_In - Identical vs. Isolated: ∆slope = −0.23, P < 10⁻⁵, ANCOVA), which again indicates that the reduction in selectivity was larger for unique stimuli. Correspondingly, the reduction in performance during inactivation for classifiers trained to discriminate an isolated stimulus inside versus outside of the CRF was 2.9% (P < 10⁻⁷; paired t-test), an 8% reduction in above-chance performance, which was significantly smaller than the reduction observed for unique stimuli (Unique_In - Unique_Out vs. Isolated: P < 10⁻⁴; Unique_In - Identical vs. Isolated: P < 10⁻⁴; paired t-test). Thus, PPC inactivation selectively reduced the representation of visual salience by neurons in prefrontal cortex.

Next, we examined the effects of PPC inactivation on the salience-driven enhancement of FEF LFP activity. During control trials, the enhancement in high-gamma band LFP responses to unique stimuli emerged ~100 ms after the visual onset response and was evident in both Unique_In - Unique_Out and Unique_In - Identical comparisons (Figure 5). During PPC inactivation, we found that both types of enhancement were reduced. As with the spiking activity, the reduction in high-gamma band responses to visual stimulation was largest when the unique stimulus appeared in the URF (Figure S5). Consequently, both types of enhancement observed in high-gamma responses to unique stimuli were reduced during PPC inactivation. The two types of enhancement were reduced significantly by 35–37% (Control: median Unique_In - Unique_Out = 0.56 dB, Inactivation: median Unique_In - Unique_Out = 0.34 dB, P < 0.003; Control: Unique_In-Identical = 0.47 dB, Inactivation: Unique_In-Identical = 0.31 dB, P < 0.004; paired t-test). As with the spiking activity, smaller changes were observed in responses to isolated stimuli. Unlike responses to unique RF stimuli, in which primarily the high-gamma band responses discriminated between the inside and outside RF conditions, both the alpha and high-gamma band responses discriminated between inside and outside conditions for isolated stimuli. During inactivation, both signals remained. But more importantly, the difference between inside and outside RF responses in the high-gamma band was reduced to a lesser extent than that observed for unique RF stimuli (15%, P < 0.006; Unique_In - Unique_Out vs. Isolated: P < 0.02; Unique_In - Identical vs. Isolated: P < 0.03; paired t-test). So, as with the spiking responses, PPC inactivation selectively reduced the representation of visual salience in prefrontal LFPs.

Figure 5. Representation of salience in prefrontal LFPs during PPC inactivation.

Comparison of average LFP time-frequency power spectrograms during control (left) and PPC inactivation (right) (N=192). The first row compares the spectrograms of responses to Unique_In and Unique_Out stimuli, and their differences (∆) during control and inactivation. The second row compares responses to Unique_In and Identical stimuli, and the third row compares responses to Identical stimuli presented inside or outside of the CRF. Difference plots only show time-frequency bins with significant energy differences.

Changes in Salience-driven behavior during PPC inactivation

Given the selective reduction in the sensitivity of FEF neurons to visual salience during PPC inactivation, we wondered if there might be corresponding changes in salience-driven behavior. Since the FEF has a well-established role in the programming and triggering of visually guided saccadic eye movements (Schall, 2004; Schiller et al., 1979), we considered that the inactivation might alter the influence of salience on this behavior. Our initial behavioral results with the free-viewing and double-target tasks indeed produced reliable effects of PPC inactivation on visually guided eye movements. However, the free-viewing task provided an additional opportunity to assess whether the inactivation altered the influence of visual salience on eye movements. Beginning with the earliest model (Koch and Ullman, 1985), a wealth of models have been developed to quantify physical salience within images based on the contrast across various feature dimensions (e.g., color) (Borji et al., 2013; Harel et al., 2007; Itti et al., 1998; Wang et al., 2016), thereby identifying points of relative salience within an image. Moreover, these models can be used to predict where in the image human observers fixate with varying accuracy (Borji et al., 2013). We leveraged this approach to quantify the distribution of salience within the images our monkeys freely viewed, and to assess the influence of salience on eye movements. Salience ‘maps’ were computed from each of the 487 images viewed by the two monkeys (65, monkey J; 431, monkey Q) using the Graph-based Visual Salience model (GBVS) (Harel et al., 2007) (Figure 6A; Figure S6A). Next, as in human studies, we measured the 2D correlation between the distribution of fixations and the salience map of each image, before and after PPC inactivation (see STAR Methods) (Figure 6B). Prior to inactivation, as in human observers, fixations were weakly, but significantly, correlated with image salience (Borji et al., 2013) (Monkey J, r_median= 0.11, P <10⁻²⁸; Monkey Q, r_median= 0.15, P < 10⁻¹⁷²), and these correlations were significantly greater than correlations obtained with shuffled images (Koehler et al., 2014) (see STAR Methods) (Monkey J, P <10⁻³; Monkey Q, P < 10⁻⁹; paired t-test). Moreover, for both monkeys, PPC inactivation significantly reduced the correlations for fixations made throughout the freely viewed images (Monkey J, ∆r_median = −0.03, P < 10⁻⁴; Monkey Q, ∆r_median = −0.01, P < 10⁻⁵; paired t-test), indicating that inactivation diminished the influence of salience on visually guided eye movements. More importantly, the reduced correlations with salience were observed within the contralateral space in both monkeys. We examined the change in correlations separately for ipsilateral and contralateral fixations, defined either in eye-centered or in head-centered coordinates (Figure 6B). In the eye-centered analysis, we divided fixations within each image into those resulting from movements made in a direction contralateral or ipsilateral to the PPC inactivation, and we computed 2D correlations separately for the two sets of fixations. This analysis revealed that PPC inactivation reduced correlations for contralaterally directed fixations in both monkeys (Monkey J, ∆r_median = −0.02, P < 0.008; Monkey Q, ∆r_median = −0.02, P < 10⁻¹³; paired t-test) (Figure 6C). In the head-centered analysis, we divided fixations within each image into those that landed within the contralateral or ipsilateral side of the image, regardless of the movement direction (Figure 6B). Similar to the eye-centered results, the image-centered analysis revealed that PPC inactivation reduced correlations for fixations within the contralateral half of images in both monkeys (Monkey J, ∆r_median = −0.02, P < 0.002; Monkey Q, ∆r_median = −0.03, P < 10⁻¹⁷; paired t-test) (Figure 6C). By comparison, we observed no consistent changes within the ipsilateral hemifield (Figure S6B). The pattern of results was similar when image salience was computed with another popular model (Itti et al., 1998) (Figure S6C). Importantly, the consistent decrease in contralateral correlation coefficients we observed was not a result of decreased saccadic accuracy during inactivation, as we did not observe such an effect (Figure S6D). Moreover, sham control sessions showed that the effect was not due to the repetition of images across blocks (Figure S6E). Instead, the decreased correlations appeared to result from a reduced influence of visual salience on the pattern of fixations directed toward the contralateral visual space, and fixations made within the contralateral half of images during PPC inactivation, consistent with the neurophysiological results.

Figure 6. Eye-centered and head-centered changes in salience-driven fixations during PPC inactivation.

(A) Example image from the free viewing task (top left) and corresponding salience map (top right). (B) Correspondence between salience and fixations made in the example image before and after PPC inactivation. Top row fixations are labelled in eye-centered coordinates as contralaterally (triangles) or ipsilaterally (circles) directed movements. Bottom row shows the same fixations labelled in head-centered coordinates as landing in the contralateral or ipsilateral half of the image. (C) Distribution of changes in fixation-salience map correlation coefficients (r_inactivation – r_control) across the population of images for the two monkeys. Left histograms show distributions based on coefficients measured from fixations across the full image. Right histograms show distributions based on contralateral fixations, defined in eye-centered (Contra_E) or head-centered (Contra_H) coordinates.

Discussion

Our observations demonstrate that neural activity within PPC is causally involved in the emergence of salience signals in prefrontal cortex and in the influence of salience on behavior. During PPC inactivation, we observed that neural responses to unique visual stimuli were reduced relative to responses to non-unique or isolated stimuli. Furthermore, we found that these reductions in neural signals were accompanied by impairments in salience-driven behavior. Parietal cortex, which is extensively evolved and enlarged in primates, consists of a constellation of multimodal, integrative cortical areas involved in the transformation of sensory and motor signals across different coordinate frames and motor effectors (Goldring and Krubitzer, 2017). A number of previous studies have examined the effects of PPC inactivation on visually guided behavior, and have demonstrated a causal role of PPC areas in multiple visuospatial functions (Gottlieb and Snyder, 2010), including goal-driven attention (Liu et al., 2010a; Wardak et al., 2004), eye and arm movement planning (Hwang et al., 2012; Liu et al., 2010a), and perceptual decision-making (Zhou and Freedman, 2019; although see Katz et al., 2016). Visual areas within PPC, such as area LIP, are heavily interconnected with feature-selective areas within extrastriate visual cortex, where information about the salience in each feature dimension is thought to be derived (Soltani and Koch, 2010). Thus, areas like LIP might integrate salience across multiple features in order to select unique stimuli and guide stimulus-driven attention and behavior.

Although both the neurophysiological and behavioral impairments we observed were robust, they were not absolute, as is often the case with studies using inactivation or lesions to probe mechanisms of visual perception (De Weerd et al., 2003; Gregoriou et al., 2014; Newsome and Paré, 1988; Ponce et al., 2008). Thus, it is important to consider which mechanisms or structures might underlie the residual sensitivity to visual salience. The coding of salience in the FEF clearly depends on input from PPC, but the residual representation of salience there could in principle be computed de novo in the FEF, particularly given the direct connections of FEF neurons with each of the many feature-selective visual areas in extrastriate cortex (Schall et al., 1995). Indeed, it remains possible that salience signals within parietal cortex mutually depend on input from the FEF. In addition, recent studies have identified representations of visual salience with very short latencies within the superficial, visual layers of the superior colliculus (White et al., 2017), which is heavily connected with the FEF, and is involved in the control of visually guided eye movements. Indeed, studies in birds reveal an important role of the midbrain in the representation of visual salience (Mysore and Knudsen, 2013). Neurons in the pulvinar might also provide an important contribution (Saalmann et al., 2012). An important goal of future studies should be to test the contribution of each of these cortical and subcortical structures to visual salience.

A collection of past studies has identified causal roles of several key structures in the control of goal-directed attention, including PPC (Moore and Zirnsak, 2017). Future studies may similarly reveal contributions of the same set of structures to stimulus-driven attention. In addition, such studies might seek to address the relative contributions of suspected structures in stimulus-driven and goal-driven attention and thus to elucidate the functional architecture of the primate visual attentional systems. For example, although the clear causal contribution of parietal cortex to stimulus-driven attention is consistent with a dominant role of parietal cortex in this form of attention (Buschman and Miller, 2007), the relative contributions of parietal and prefrontal (e.g. the FEF) cortex to either forms of attention remains an open question. Inactivation of parietal cortex might yield similar reductions in goal-driven modulation within prefrontal cortex as with stimulus-driven modulation, a result that would contradict a division of labor between parietal and prefrontal. Alternatively, parietal inactivation might produce little or no effects on goal-driven modulation within prefrontal cortex. This latter possibility would be consistent with previous evidence of more dramatic effects of inactivation (or lesions) of prefrontal cortex on behavior in goal-driven tasks, when compared directly with parietal perturbations (Lynch and McLaren, 1989; Suzuki and Gottlieb, 2013). However, given the apparent involvement of neurons within other structures in both stimulus-driven and goal-driven attention, e.g. superior colliculus (Krauzlis et al., 2013; White et al., 2017), a number of direct comparisons of the effects of inactivation might be needed to potentially identify dominant source(s) of both forms of attention within the primate brain. Further studies might also seek to determine whether the control of either form of attention is largely shared by similar or different distributed populations of neurons contained within each of the key structures.

Biologically plausible models of the computation of visual salience highlight the necessary role of feature-selective areas in generating feature contrast (Itti et al., 1998; Koch and Ullman, 1985; Soltani and Koch, 2010). In these models, feature-selective inputs are combined across multiple feature dimensions by neurons in non-selective structures (e.g. PPC) to form salience maps. Neurons within visual areas of PPC, such as LIP, receive direct inputs from feature-selective extrastriate areas (Felleman and Van, 1991), and indeed this is a common feature among other structures thought to contain salience maps, e.g. the FEF (Schall et al., 1995) and the superior colliculus (Cerkevich et al., 2014). Thus, as discussed above, it is likely that multiple structures contribute to the computation of visual salience. Our results identify PPC as being one of them.

STAR METHODS

EXPERIMENTAL MODEL AND SUBJECT DETAILS

All experimental procedures were in accordance with National Institutes of Health Guide for the Care and Use of Laboratory Animals, the Society for Neuroscience Guidelines and Policies, and Stanford University Animal Care and Use Committee. Two healthy male rhesus monkeys (Macaca mulatta, 17 and 16 kg), monkey J and monkey Q, were used in these experiments. The number of animals used is typical for primate neurophysiological experiments.

METHOD DETAILS

General and Surgical Procedures

Surgery was conducted using aseptic techniques under general anesthesia (isoflurane) and analgesics were provided during postsurgical recovery. Each animal was surgically implanted with a titanium head post and a cylindrical titanium recording chamber (20 mm diameter) overlaying the arcuate sulcus. A craniotomy was then performed in the chambers on each animal, allowing access to the FEF.

Cryoloops surgery and reversible inactivation of PPC

Each animal was surgically implanted with two stainless steel cryoloops within the intraparietal sulcus of one hemisphere. The size and shape of the cryoloops were customized to fit the contours of the IPS and to completely fill the sulcus. One longer loop (2.2–2.4 x 0.4 cm) was placed ventrally, and one shorter loop was placed dorsally (1.7–1.8 x 0.3 cm) (Extended Figure 1). During the cryoloop surgery, unilateral craniotomies were made over the intraparietal sulcus. Cryoloops were then placed beneath the dura and upon the surface of the arachnoid membrane in the dorsal and ventral intraparietal sulcus. The loops were secured to the skull with bone screws and dental acrylic. The dura was replaced and bone defects around the implanted cooling loops were repaired with original bone, Gelfoam (Pfizer) and dental acrylic. For detailed cryoloop implantation procedures, see Lomber and Payne (Lomber and Payne, 2000).

Inactivation procedures

During each experimental session, cortex within the IPS was cooled by pumping chilled methanol through the loop tubing. Loop temperature was monitored and accurately regulated within 1°C of the desired value by controlling the rate of methanol flow. A stable loop temperature (~5°C) was reached in ~5–10 min of initiating cooling, and normal brain temperature was regained within ~2 minutes after the cessation of cooling, consistent with previous studies (Lomber et al., 1994, 1996). Loop temperatures ~5 °C reliably deactivate neuronal activity across the full thickness of the underlying cortex (Lomber and Payne, 2000). During experimental sessions, blocks of inactivation lasted 30–60 minutes, and each session consisted of those in which only neurophysiological recordings or behavioral tests were done, as well as some sessions with both. In 8 neurophysiological sessions, data were collected during a control block first, followed by an inactivation block (control-inactivation). In an additional 6 sessions, we collected neurophysiological data during 2 pairs of sequential control-inactivation blocks (i.e. control-inactivation-control-inactivation). These latter sessions allowed us to test for any possible effects of time and/or block order (e.g. Figures S4B, S4C). Tests of the effects of inactivation on behavior were carried out almost exclusively on separate sessions. These behaviors included tests of inactivation effects on double-target choices (24 separate sessions of 28 total) and on free-viewing of natural images (8 separate sessions of 8 total). In the behavior-only sessions, data were collected during a control block first, followed by an inactivation block (control-inactivation). In addition, to control for the effects of block sequence on free-viewing, we collected data during an additional 4 sessions in which no inactivation was performed in the second block (sham control).

Neurophysiological recording procedures

Recording sites within the FEF were identified by eliciting short-latency, fixed vector saccadic eye movements with trains (50–100ms) of biphasic current pulses (≤50 µA; 250 Hz; 0.25 ms duration) as in previous studies (Bruce and Goldberg, 1985). Single-neuron and local field potential (LFP) recordings were obtained with 16 or 32-channel linear array electrodes with contacts spaced 150 µm apart (V and S-Probes, Plexon, Inc). Electrodes were lowered into the cortex using a hydraulic microdrive (Narishige International). Neural activity was measured against a local reference, a stainless guide tube, which was close to the electrode contacts. At the preamplifier stage, signals were processed with 0.5 Hz 1-pole high-pass and 8 kHz 4-pole low-pass anti-aliasing Bessel filters, and then divided into two streams for the recording of LFPs and spiking activity. The stream used for LFP recording was amplified (×500 – 2000), processed by a 4-pole 200 Hz low-pass Bessel filter and sampled at 1000 Hz. No other filters were used in the analyses. The stream used for spike detection was processed by a 4-pole Bessel high-pass filter (300 Hz) a 2-pole Bessel low-passed filter (6000 Hz), and was sampled at 40 kHz. Extracellular waveforms were classified as single neurons or multi-units using online-template-matching and subsequently confirmed using offline sorting (Plexon). Overall, we recorded 352 units with visual activity, of which 68 were well-isolated single units. Among 14 sessions, LFP data was recorded in 11 sessions.

CRF and URF measurements

We measured LFP and spiking activity derived CRFs within the FEF by randomly presenting a single isolated probe stimulus out of a 6 x 4 probe grid extending 75 x 45 dva (Isolated stimulus condition). In each recording session, we placed the probe grid so as to cover the area where we expected to find most RF locations based on the saccade vectors evoked by electrical stimulation at a given recording site. The probes consisted of fully saturated red or green 7 x 7 dva squares. Similarly, we measured LFP and spiking activity derived URFs within the FEF by randomly presenting a uniquely colored probe stimulus among an array of differently colored stimuli, either a single green among 23 red or a single red among 23 green stimuli (Unique stimulus condition). In addition, we also measured neural response (LFP and spiking activity) to an identically colored (24 red or 24 green) stimulus array (Identical stimulus condition). Each stimulus condition was repeated at least 8 times during both control and inactivation conditions. Different stimulus conditions were pseudo-randomly interleaved.

In each trial, monkeys were required to fixate a central fixation point (1 x 1 dva fixation window) on a gray background (60 cd/m²) for 500 ms to initiate the trial. Subsequently, either an Isolated, Unique, or Identical stimulus was presented for 500 ms while the monkey maintained fixation. Following stimulus offset, the monkey received a juice reward after an additional 300ms of fixation.

Behavioral procedures: free-viewing

During all behavioral measurements, eye position was monitored and stored at 1000 Hz (Eyelink 1000, SR Research). While seated and head-restrained, monkeys were rewarded for freely viewing complex images, similar to a previous study (Killian et al., 2012). Images (Monkey J: 79 x 49 dva; Monkey Q: 98 x 55 dva;) were presented on a display (Monkey J: Samsung 2233RZ, 120 Hz refresh rate, 1680 × 1050 pixel resolution; Monkey Q: ASUS VS228; 75 Hz refresh rate, 1920 × 1080 pixel resolution) positioned 28–30 cm in front of the animal. A novel set of 100 images was used for each experimental session. In each trial, monkeys fixated a central fixation point (1 x 1 dva fixation window) on a gray background (60 cd/m²) for 500 ms to initiate the image presentation. Each image was displayed for 3 seconds and was shown in both control and inactivation blocks. Monkeys were rewarded at the end of each trial for exploring the image for the full presentation time.

Behavioral procedures: choice task

To measure the effects of PPC inactivation on target selection, we quantified the monkey’s tendency to select stimuli at a particular location as the target of a saccadic eye movement. We employed a double-target, choice task similar to one used previously (Noudoost and Moore, 2011). In the task, the monkey was rewarded for making saccades to either one of two visual stimuli (1 dva diameter) appearing at diametrically opposed locations on the same display as used in the free-viewing task. One of the stimuli was positioned within the contralateral hemifield, and the other in the ipsilateral hemifield. The appearance of the two stimuli on a given trial occurred within a range of temporal onset asynchronies (TOAs), from trials in which the contralateral target appeared first (positive TOAs) to trials in which the contralateral target appeared second (negative TOAs). The range of TOAs for a given block of trials was −800 to 800 ms, with 7–9 discrete TOAs evenly spaced within that range, including zero. Trials were randomly interleaved such that on any given trial the monkey could not predict the TOA. In a given experimental session, at least 2 blocks of trials were collected, one prior to PPC inactivation, and one following it. Each block consisted of at least 10 trials per TOA. Each pair of pre- and post-inactivation target selection blocks could be used to compare the probability that the monkey would choose one target over the other as a function of TOA.

QUANTIFICATION AND STATISTICAL ANALYSIS

Salience map and correlation analysis of fixations during free-viewing

Both the graph based visual salience model (GBVS) (Harel et al., 2007) and the Itti-Koch-Niebur model (Itti et al., 1998) were used to compute the salience map for each image. For both models, feature channels including color, luminance, and orientation were used in the computation of the salience map.

Similar to human free viewing studies (Borji et al., 2013), a 2-D Pearson correlation coefficient was computed to quantify the relationship between the salience map SM_i of image i and the fixation density map FDM_i of image i. Raw salience maps (32× 18, ∆x = {2.5, 3.0} dva, ∆y = {2.7, 3.0} dva, depending on the display ) were used without interpolation. Fixation density maps were calculated in the same spatial resolution as the salience maps. The correlation for the full SM and full FDM is defined as

$$r_i=\frac{{\displaystyle {\sum }_{x=1}^{32}{\displaystyle {\sum }_{y=1}^{18}\left(S{M}_{i,x,y}-{\overline{SM}}_{î}\right)\left(FD{M}_{i,x,y}-{\overline{FDM}}_{î}\right)}}}{\sqrt{\left({\displaystyle {\sum }_{x=1}^{32}{\displaystyle {\sum }_{y=1}^{18}{\left(S{M}_{i,x,y}-{\overline{SM}}_{î}\right)}^2}}\right)\left({\displaystyle {\sum }_{x=1}^{32}{\displaystyle {\sum }_{y=1}^{18}{\left(FD{M}_{i,x,y}-{\overline{FDM}}_{î}\right)}^2}}\right)}}$$ (1)

where SM_i,x,y denotes the salience map for the ith image at the location (x,y), FDM_i,x,y denotes the fixation density at the location (x,y) when the monkey was viewing the ith image, $\overline{S{M}_l}$ denotes the mean salience across the whole image, and $\overline{FD{M}_l}$ denotes the mean fixation density across the whole image. For correlations in eye-centered coordinates, the FDM was computed separately for all eye movements that had a contralateral and ipsilateral component. For correlations in head-centered coordinates, the SM and FDM were computed separately for contralateral and ipsilateral halves of each image. Results of comparisons of correlations across control and inactivation blocks yielded similar results when using all fixations or matched numbers of fixations between the two blocks.

Point of equal selection in Choice task analysis

We used logistic regression, on a trial-by-trial basis (Chen and Stuphorn, 2018) to estimate the point of equal selection (PES) (Noudoost and Moore, 2011). The PES is the estimated TOA for which the selection of either target has equal probability.

Receptive Fields and Enhancement Index

For each neural recording, we normalized both the CRF and URF by linearly scaling the activity from 0 to 1 for visualization purposes, with 0 corresponding to the minimum stimulus-driven activity across all stimulus and experimental conditions and 1 corresponding to the maximum stimulus-driven activity across all stimulus and experimental conditions.

We used two indices to quantify the enhancement of neuronal responses to Unique stimuli appearing inside the URF,

$${\text{Unique}}_{\text{In}}-{\text{Unique}}_{\text{Out}}\text{\hspace{0.17em}index\hspace{0.33em}:=}\frac{R_{\text{In}}^{\text{U}}-R_{\text{Out}}^{\text{U}}}{R_{\text{In}}^{\text{U}}+R_{\text{Out}}^{\text{U}}}$$ (2)

$${\text{Unique}}_{\text{In}}-\text{Identical\hspace{0.17em}index\hspace{0.33em}:=}\frac{R_{\text{In}}^{\text{U}}-R^{\text{Id}}}{R_{\text{In}}^{\text{U}}+R^{\text{Id}}},$$ (3)

with $R_{\text{In}}^{\text{U}}$ denoting the mean neuronal response to Unique stimuli presented inside the URF, [0, 500) ms relative to stimulus onset, $R_{\text{Out}}^{\text{U}}$ denoting the mean response to Unique stimuli presented outside the URF, and R^ID denoting the mean response to Identical stimulus arrays.

Support Vector Machine (SVM) linear classifier

We used a linear support vector machine (SVM) (Chang and Lin, 2011) to quantify the selectivity of neurons to Unique and Isolated stimuli. A classifier was trained to discriminate between neuronal responses to Unique stimuli presented inside the URF and responses to Unique stimuli presented outside the URF, and between responses to Identical stimuli, on a trial-by-trial basis. Similarly, a classifier was trained to discriminate neuronal responses to Isolated stimuli presented inside the CRF from responses to Isolated stimuli presented outside the CRF, on a trial-by-trial basis. Before training, spike counts for each neuronal recording were normalized across all stimulus conditions. All reported discrimination accuracies are based on four-fold cross-validation. Permutation tests (1000 repetitions) were used to determine whether the discrimination accuracy of a given neuronal recording was significantly greater than that expected by chance. Specifically, we computed the difference in performance between the observed and the label-shuffled mean performance (~50%) to determine the above-chance performance.

Time-frequency analysis

Matching pursuit (MP) decomposition was used in calculating the spectrogram to optimize temporal and frequency resolutions (Chandran K S et al., 2016; Chen et al., 2010). This multiscale decomposition allows sharp transients in the LFP signal to be represented by functions that have narrow temporal support, rather than oscillatory functions with a temporal support of hundreds of milliseconds. The algorithm is an iterative procedure that selects a set of Gabor functions (atoms) from a redundant dictionary of functions that constitute the best possible description of the original signal. Time-frequency plots were then obtained by calculating the Wigner distribution of every atom and taking the weighted sum. We performed the MP computation using custom MATLAB (MathWorks) scripts and the MP toolbox (Chandran K S et al., 2016). Permutation tests (N=1000) with multiple correction were used to determine whether the energy distribution at selected times and frequencies was significantly different between stimulus conditions. The mean LFP power for each frequency band (alpha, 8–12 Hz; beta, 12–30 Hz; low gamma, 30–60 Hz; and high gamma, 60–150 Hz) was calculated as the mean of the energy [0, 500) ms after visual stimulus onset.

DATA AND SOFTWARE AVAILABLITY

The datasets and code supporting the current study have not been deposited in a public repository due to their volume and complexity, but will be made available by the lead contact (T.M.) upon request.

Key Resource Table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Experimental Models: Organisms/Strains
Macaca mulatta	Davis Primate Center	N/A
Software and Algorithms
MATLAB Signal Processing, Statistics and Machine Learning Toolbox, R2018a, R2019a	Mathworks	https://www.mathworks.com/products/matlab.html
LIBSVM	GitHub	https://github.com/cjlin1/libsvm
Offline Sorter, Version 3.0	Plexon	https://plexon.com/products/offline-sorter/
Data Acquisition
OmniPlex System	Plexon	https://plexon.com/products/omniplex-d-neural-data-acquisition-system-1/
Other
SR Research Eyelink Eye Tracker, Eyelink 1000 Plus	Eyelink	http://www.sr-research.com

Highlights.

• Reversible inactivation of PPC reduces contralateral visual exploration and choices

• PPC inactivation selectively reduces the coding of salience by prefrontal neurons

• Encoding of salience by prefrontal LFPs is selectively reduced by PPC inactivation

• PPC inactivation diminishes the influence of salience on eye movements