Attentional Modulation of Neuromagnetic Evoked Responses in Early Human Visual Cortex and Parietal Lobe following a Rank-Order Rule

Abstract

Top-down voluntary attention modulates the amplitude of magnetic evoked fields in the human visual cortex. Whether such modulation is flexible enough to adapt to the demands of complex tasks in which abstract rules must be applied to select a target in the presence of distracters remains unclear. We recorded brain neuromagnetic activity using whole-head magnetoencephalography in 14 human subjects during a rule-guided target selection task, and applied event-related Synthetic Aperture Magnetometry to image instantaneous changes in neuromagnetic source activity throughout the brain. During the task subjects selected one of two stimuli (the target) and ignored the other (the distracter) based on a color-rank rule (color 1 > color 2 > color 3). Our results revealed that in early visual color-sensitive areas and the parietal cortex visual stimuli evoke activity that scaled following the rank-order rule. This effect was stronger and occurred later in the parietal lobe (∼200 ms after target/distracter onset) relative to early visual areas (∼180 ms). Moreover, we found that transient changes in the target's motion direction evoked stronger responses relative to similar changes in the distracter at ∼180 ms from change onset in contralateral areas hMT+/V5. These results suggest that during target selection and allocation of attention to a stimulus, top-down signals adjust their intensity following complex selection rules according to the organism's priorities, thereby differentially modulating neuromagnetic activity across visual cortical areas.

Introduction

Previous imaging and event-related potential (ERP) studies have demonstrated that attention modulates neural activity in human visual cortex evoked by stimuli at attended spatial locations (Heinze et al., 1994; Mangun et al., 1997, 2001; Woldorff et al., 1997; Tootell et al., 1998; Müller and Kleinschmidt, 2003), as well as by stimuli possessing attended visual features, such as color, motion, and shape (Corbetta et al., 1990; Anllo-Vento and Hillyard, 1996; Beauchamp et al., 1997; Clark et al., 1997; O'Craven et al., 1997; Chawla et al., 1999; Bartels and Zeki, 2000; Saenz et al., 2002; Liu et al., 2003; Poghosyan et al., 2005). Current theories of attentional control suggest that these effects are due to the modulatory influence of top down signals—likely originating within a frontoparietal brain network (Corbetta, 1998)—on the sensitivity of visual neurons to sensory inputs (Desimone and Duncan, 1995; Moore, 2006).

It remains poorly investigated whether top-down modulatory signals flexibly adapt to the demands of tasks in which complex rules must be applied to determine the relevant stimulus. For example, when identifying target and distracter stimuli according to a rank rule where “stimulus #1 > stimulus #2 > stimulus #3,” stimulus #2 is the attended target when paired with stimulus #3, but becomes an ignored distracter when paired with stimulus #1. Thus, the target identity cannot be defined in advance but is based on the comparison between the ranks of the available options. Here, enhancing the sensitivity of visual neurons encoding a set of features and/or suppressing the rest in anticipation to the stimulus presentation would not be the optimal strategy. Rather, it would be more advantageous for attentional systems to adjust the intensity of top-down signals following the probability of a given feature to be the target according to the rank-rule; i.e., stronger increases in sensitivity for neurons encoding the features of the stimulus with the highest rank (always a target), followed by those encoding the features of the stimulus with rank 2 (50% of the times the target), and no increase or even a suppression in the sensitivity of neurons encoding the features of the stimulus with rank 3 (always a distracter).

To investigate this hypothesis, we recorded magnetoencephalographic (MEG) responses in 14 human subjects and applied event-related Synthetic Aperture Magnetometry (erSAM) analysis to image instantaneous changes in neuromagnetic source activity throughout the brain. We examined the time course, location, and amplitude of event-related fields (ERFs) during a task in which subjects selected a target stimulus and ignored a distracter based on a color-rank-order selection rule (color #1 > color #2 > color #3). Our results revealed that the amplitude of ERFs in early extrastriate color-sensitive visual areas and the posteromedial parietal cortex scaled following the stimulus rank. We observed stronger effects in the parietal compared with early visual cortex. Additionally, we found that sustained attention to the target modulated the activity evoked by motion direction changes in that stimulus in area hMT+/V5.

Materials and Methods

Subjects and recordings

Fourteen adult subjects (5 women, aged 21–40 years) participated in the experiment with their informed consent. All subjects had normal or corrected-to-normal vision and no signs of neurological or psychiatric disease.

We recorded neuromagnetic activity while subjects performed a visual task (see Experimental design and stimuli) using a whole-head 275 channel CTF MEG system (VSM MedTech). It used synthetic third-order gradient noise cancellation and was located in a magnetically shielded room. The data were collected at a sample rate of 625 samples/s with an on-line bandpass filter of 0–200 Hz. Before MEG data acquisition, we fitted each subject with coils placed at three fiducial landmarks (nasion and preauricular points) that were recognized by the MEG acquisition hardware to establish the position of the subject's head relative to the MEG sensors. Coil placements were carefully measured and photographed for off-line coregistration of the recorded MEG data to structural MR images obtained for each subject. Two additional electrodes were used, one of which was attached ∼2 cm below the left collarbone for recording the electrocardiogram, while the other was attached below the left eye (on the lower orbital portion of the orbicularis oculi muscle) for detection of eyeblink artifacts. To minimize head movements 13 of 14 recordings were made with subjects lying supine on a bed with the eyes looking at a fixation cross presented on a flat projection screen. One recording was made with the subject in sitting position.

We obtained structural MR images (T1-weighted) for each subject using a Siemens 3T MRI scanner. Coregistration of the MEG head-based coordinate system with the MRI was achieved by identifying the positions of the three head localization coils on orthogonal slices of each subject's MRI scan.

Experimental design and stimuli

We presented visual stimuli on a flat rear-projection screen, which was positioned at a distance of 75 cm from the subject's eyes and had a frame rate of 60 Hz (1 frame = 16.7 ms). A photo-resistor circuit detected a luminance change on the back of the projection screen (not visible to the subject) and sent a transistor logic pulse to the acquisition computer thereby marking the onset of each trial. Experimental trials consisted of the presentation of two white moving random-dot patterns (RDPs; white dots on a black background within a circular aperture of 7.6° radius) appearing to the right and left of a central fixation cross, at an eccentricity of 17.7° (Fig. 1A). The dots had 100% coherence with a density of 5 dots per degree². The size of individual dots was 2 pixels². After 40 frames the color of both RDPs changed from white to red, blue, or green. The two stimuli always changed to two different colors (e.g., one RDP red and the other blue). The colors served as a cue indicating the target and distracter stimulus, which were defined using one of two color rank rules (Fig. 1B). Rule #1 placed red at the highest rank followed by blue and then green (red > blue > green). Rule #2 inverted the color ranks to green > blue > red. For each trial of rule #1, red was a target with a probability of 1 and green with a probability of 0. For trials of rule #2 this relationship was reversed. In both rules, blue was a target with a probability of 0.5 because it was a target when paired with the low-rank color, but a distracter when paired with the high-rank color. All trial combinations (color pair and left-right target positions) had the same number of trials. We randomized the presentation of the different trial types and assigned rules #1 and #2 to different subjects (7 subjects rule #1, and 7 different subjects rule #2). All subjects received a short test session before the recordings. Any given subject followed either rule #1 or rule #2 during the entire recording session. To perform within-subject comparisons, we tested two of the subjects using both rules. These data are presented below in Figure 4B. However, to maintain a counterbalanced design, we did not include these data into the general dataset.

Figure 1.

Stimulus and task. A, Trial sequence: the black panels represent the screen and stimuli. In blocks of trials, the response event was either a transient direction or color change (see Materials and Methods for details). B, Color-rank selection rules: Rule #1: red > blue > green, rule #2: green > blue > red. For both rules, target probability scaled from 100% to 50% to 0% along the scale from left to right.

Figure 4.

Target selection: ERFs as a function of color. A, Average normalized activity combined across hemispheres and subjects (7 subjects for each rule) as a function of time from color cue onset. Panels from left to right represent contralateral stimulus colors red, blue (when paired with green), blue (when paired with red), and green; solid lines indicate rule #1, dashed lines rule #2. Note that the ranks of the red and green stimuli vary with rule. Top row, Middle occipital gyrus; bottom row, parietal cortex. B, The same as in A across two subjects who had participated at sessions of rule #1 and were then retested using rule #2.

After the onset of the stimulus colors (“color cue”), the subjects' task was to select covertly the target RDP, sustain attention on it, and discriminate a transient change either in its motion direction (counterclockwise/ no change/ clockwise) or color (pink/ no change/ gray) while ignoring distracter changes. We asked subjects to fixate the central fixation cross throughout the entire length of the trial. After the offset of the stimuli, they indicated their response by pressing one of three buttons using their right index, middle, and ring finger, respectively. We presented trials of direction and color changes separated into blocks of 200 trials each. Thirteen of 14 subjects performed 2 blocks of each trial type amounting to a total of 800 trials. One subject performed 3 blocks of each type, resulting in a total of 1200 trials.

Direction changes had a magnitude of ±20° and lasted 10 frames (166 ms). In color trials, all dots changed color simultaneously into either gray or pink and returned after 10 frames to their previous color (i.e., red, blue, or green). All colors were approximately isoluminant (red: 305, blue: 305, green: 226; pink: 297, gray: 340, black background: 7 cd/m²). Before the recordings, we performed psychophysical tests on 3 subjects and matched the difficulty of direction and color changes by varying their magnitudes. Target and distracter change onsets occurred jittered between 90 and 150 frames after the color cue onset. Within a trial, a change could only happen in one stimulus at a time, target or distracter. Short breaks during which the subjects were instructed to keep the head still separated experimental blocks. The order of the blocks was randomized between subjects.

To avoid adaptation to the same motion direction, the RDPs could move, from trial to trial, either upward or downward (50% of the times in each direction, presented in a randomized order). To avoid overlap between the activity due to the motor response (button press) and the evoked activity due to the change in the visual stimulus, we instructed the subjects to respond always with their right hand once the stimulus presentation had finished. They were explicitly told that this was not a reaction time task. Due to these factors there was a considerable variability in the subject's reaction time. Because neuromagnetic activity preceding motor responses typically begins 500–600 ms before movement onset (Cheyne et al., 2006), we did not expect to detect significant motor area activations in our analysis time windows.

MEG data analysis

We applied the erSAM beamformer algorithm (Cheyne et al., 2006) to image instantaneous changes in source activity during a rule-guided target selection task. The SAM algorithm (Robinson and Vrba, 1999) was introduced as a source localization method for detecting the location of multiple neural sources. This method comprises an adaptive or data-driven linear inverse localization method based on minimum-variance beamforming and has been successfully used to create differential images of source power in narrow frequency bands over short time intervals (Hillebrand et al., 2005). This approach has a number of advantages over conventional equivalent current dipole modeling methods, as it does not require that the number of active sources is specified a priori and can suppress contributions from noise sources such as eye movements. erSAM involves the application of the SAM beamformer algorithm to the calculation of source power at single time points to image time-locked activity associated with evoked response activity with the ability to differentiate cortical sources separated by distances as small as 5–10 mm (Cheyne et al., 2006). The technique proved successful at measuring activations during higher order cognitive tasks related to face processing (Itier et al., 2006), verb generation (Herdman et al., 2007), error-related feedback (Bayless et al., 2006), change detection (Martinez-Trujillo et al., 2007), and memory tasks (Moses et al., 2009; Robitaille et al., 2010).

ErSAM analysis.

For each subject, we computed volumetric erSAM images by scanning a volume covering the entire brain with a grid spacing of 3 mm. Image calculation occurred at 10 ms increments during an analysis window that started 200 ms before the onset of the change event (color cue onset and response event) and ended 500 ms after. The resulting images were spatially normalized to the Montreal Neurological Institute (MNI) (T1) template brain using SPM2 (Wellcome Institute of Cognitive Neurology, London, UK). The individual coregistered T1-weighted MR scans yielded linear and nonlinear warping parameters that were used to warp the individual erSAM images' standardized stereotactic space. The averages across subjects served as the volumetric group erSAM images. We thresholded group images at selected latencies using a nonparametric permutation test (Nichols and Holmes, 2002; Singh et al., 2003) based on images created during the baseline period to avoid bias due to the nonuniform distribution of source power in the functional erSAM images (Chau et al., 2004). No spatial or variance smoothing was applied. Thresholded images were then superimposed on the MNI template brain (Collins et al., 1994) and viewed with the mri3dX program (http://www.cubric.cf.ac.uk/Documentation/mri3dX/). We determined the localization of peak activations from the normalized images using the Talairach Daemon (Lancaster et al., 2000).

Virtual sensor analysis.

We applied a search radius of 5 mm on the location of the average peak activations in the group data to obtain individual subject coordinates. We transformed the identified Talairach coordinates to locations in the individual head-based (MEG) coordinate system and generated source waveforms at these positions (“virtual sensors”). Visual inspection of the individual waveforms confirmed clear evoked responses at the single subject level. The obtained source waveforms were then normalized to the individual maximal activation, baseline corrected, and low-pass filtered at 15 Hz. We then pooled time courses across all subjects to view the average time course of the evoked neuromagnetic responses at these locations.

Statistical testing.

We conducted statistical tests of significance using Wilcoxon signed-rank tests and Student's t tests (Matlab, statistics toolbox, MathWorks) evaluated at the 5% alpha level if not stated otherwise. We performed tests on the individual subjects' peak amplitudes, which we defined as the average activation within 10 ms around maximal activation during a time window ranging from 100 to 250 ms after change event onset.

Results

We recorded MEG activity in 14 healthy human subjects while they performed the color-rank target selection task. Half of the subjects performed the task following rule #1 (red > blue > green), the others following rule #2 (green > blue > red) (Fig. 1). All subjects performed correctly in both direction and color blocks (i.e., the hit rate close to 100%). Therefore, we included all trials in the analysis.

Figure 2 gives an example of the MEG signals recorded from one subject averaged over different experimental trials of rule #1. The dark traces display the amplitude of the average magnetic field activity in femtotesla (fT) as a function of time from change event onset (color cue and transient direction change). Average waveforms from all channels are superimposed. The insets show the distribution of intensities (visible as efflux–influx configurations with red lines representing the efflux and blue lines the influx component of the field) relative to each channel location on a sketch of the subject's head. Clearly, with the target stimulus appearing on the left the color cue onset evoked a neuromagnetic response that reached its highest amplitude over occipital brain regions between 150 and 200 ms post-cue onset (top left). Similarly, the transient motion direction change in the target (top right) evoked a neuromagnetic response centered over the right occipitotemporal hemisphere (indicated by white arrow)—contralateral to the visual hemifield where the change event occurred—that peaked approximately between 180 and 300 ms after the change onset. The bottom shows average magnetic field activity in the same subject when the target was positioned on the right. The color cue onset triggered an ERF over the occipital area with latency similar to the one corresponding to targets on the left. Direction changes in the target on the right elicited ERFs localized predominantly over the left occipitotemporal region, contralateral to the response event hemifield (white arrow).

Figure 2.

Example subject MEG data. Top, Single subject's average magnetic field activity as a function of time from color change onset (left) and direction change onset (right) when the target (here, the red RDP) was in the left visual field. Activity at all sensors is overlaid. Bottom, The same as above for changes in targets that were positioned in the right visual field. MEG maps display magnetic field topographies at indicated latencies.

To obtain a more precise localization of peak activity evoked by the different task events, we performed group erSAM analyses by averaging the individual volumetric images at selected latencies from color change onset after normalization to a standard brain template. This revealed the average position of peak activation with a spatial resolution of ∼3 mm and a temporal resolution of 10 ms (Martinez-Trujillo et al., 2007).

ERF during target selection and allocation of attention

We conducted a group erSAM analysis performed on activity evoked by the color cue onset for targets positioned left (Fig. 3A) and right (Fig. 3B) and averaged across rules #1 and #2. The slices represent brain regions maximally activated (yellow) at 0.18 s (top) and 0.2 s (bottom) after color cue onset. (For a full list of peak localizations, see Table 1.) When the target was positioned on the left and distracter on the right, average peak activations at 0.18 s post-cue onset localized to visual extrastriate brain regions (x/y/z Talairach coordinates: 33/−73/4 and −36/−75/12, contra- and ipsilateral middle occipital gyri, BA19, Fig. 3A, top). Similarly, targets on the right and distracters in the left visual field (Fig. 3B, top) elicited activation in the same regions (36/−70/−2 and −33/−67/9). Shortly after, at 0.2 s from color cue onset, areas in the posteromedial parietal cortex (12/−62/42 and −3/−72/26, right and left precuneus, BA7 and BA31, respectively) became activated while the initial evoked response in the extrastriate visual cortex slowly faded (Fig. 3A,B, bottom). Activity in the right posteromedial parietal cortex was stronger in the “target left” condition compared with the same region in the left hemisphere; while in the “target right” condition both left and right parietal cortices were activated with a slight advantage on the contralateral (left) side (Table 2). Based on these group erSAM images, we defined our regions of interest as left and right early extrastriate cortex (middle occipital gyrus), and left and right posteromedial parietal cortices (precuneus). We applied a search radius of 5 mm around these group peak locations (unwarped from template coordinates to the individual's MEG coordinates) to automatically identify peak activations in individual subjects. To extract the precise time course of activation at these selected regions of interest, we computed source waveforms (virtual sensors) for each subject at these locations (see Materials and Methods for details).

Figure 3.

Target selection: group erSAM analysis. A, B, Sagittal, coronal, and axial views of brain regions maximally activated (yellow) when the target is positioned on the left (A) or right (B) at latencies of 180 ms (top) and 200 ms (bottom) from color cue onset. Insets represent example stimulus configurations.

Table 1.

Peak activations in group erSAM images after color cue onset

Talairach coordinates (mm)			Pseudo-Z	Brain region
x	y	z
Target left/0.18 s
33	−73	4	1.02	mid. occ. gyrus, BA19 (R)
−36	−75	12	0.67	mid. occ. gyrus, BA19 (L)
−36	−66	20	0.64	mid. temp. gyrus, BA39 (L)
−12	−87	13	0.63	mid. occ. gyrus, BA18 (L)
Target right/0.18 s
36	−70	−2	0.93	inf. occ. gyrus, BA19 (R)
0	−74	26	0.81	precuneus, BA31 (L)
18	−72	17	0.81	cuneus, BA18 (R)
30	−60	20	0.75	posterior cingulate gyrus, BA31 (R)
−24	−55	−2	0.72	parahippocampal gyrus, BA19 (L)
−33	−67	9	0.71	mid. occ. gyrus, BA19 (L)
Target left/0.20 s
33	−72	9	0.82	mid. occ. gyrus, BA19 (R)
27	−61	9	0.81	posterior cingulate gyrus, BA30 (R)
−33	−76	4	0.65	mid. occ. gyrus, BA19 (L)
−18	−75	20	0.64	cuneus, BA18 (L)
12	−62	42	0.62	precuneus, BA7 (L)
Target right/0.20 ms
30	−58	6	0.82	parahippocampal gyrus, BA30 (R)
18	−72	15	0.77	cuneus, BA17 (R)
33	−72	9	0.75	mid. occ. gyrus, BA19 (R)
−15	−81	12	0.71	cuneus, BA17 (L)
−3	−72	26	0.66	precuneus, BA31 (L)
24	−62	39	0.59	precuneus, BA7 (R)

Mean locations and magnitudes of the 6 largest peak activations in the group-averaged erSAM images occurring 0.18 and 0.2 s after color cue onset for targets positioned left and right of the fixation cross. Images were thresholded using a nonparametric permutation test evaluated at p = 0.05. mid., Middle; occ., occipital; temp., temporal; inf., inferior.

Table 2.

Peak activations in group erSAM images after target/distracter change onset

Talairach coordinates (mm)			Pseudo-Z	Brain region
x	y	z
Direction change left
24	−75	9	0.81	cuneus, BA17 (R)
42	−61	9	0.77	mid. temp. gyrus, BA37 (R)
Direction change right
−30	−69	15	0.91	posterior cingulate gyrus, BA30 (L)
Color change left
30	−67	1	0.98	lingual gyrus, BA19 (R)
Color change right
−36	−70	6	0.78	mid. occ. gyrus, BA19 (L)
−24	−72	26	0.76	precuneus, BA31 (L)

Mean locations and magnitudes of the 2 largest peak activations in the group-averaged erSAM images occurring 0.18 s after transient target changes (in direction and color) in the left and right visual fields. Images were thresholded using a nonparametric permutation test evaluated at p = 0.001. mid., Middle; occ., occipital; temp., temporal.

Our goal was to examine possible effects of rank on ERF amplitudes during target selection. We first tested the possibility that despite being isoluminant the different colors may evoke ERFs of different amplitudes. The color red, for example, tends to be more salient and evoke larger responses than other colors (Anllo-Vento et al., 1998). Figure 4A shows average evoked responses to red, blue, and green obtained during sessions of rule #1 (solid lines; 7 subjects) and #2 (dashed lines; 7 subjects) at the two selected regions of interest. On average, red evoked the largest response followed by blue, and green. However, comparison of ERFs corresponding to the two rules showed that red and green peak amplitudes were modulated by task demands; activity evoked by the red and green stimuli when of highest rank (rule #1 and #2, respectively) was reduced when rules reversed and the stimuli were assigned the lowest rank. ERFs evoked by the blue stimulus are shown as a reference since in this case the rank remained similar.

To further corroborate these results, we tested two subjects who had participated in sessions of rule #1, using rule #2. We compared the ERFs evoked by each color as a function of rule. The color red elicited the strongest response in both regions of interest (Fig. 4B). More importantly, peak amplitudes were markedly reduced when red had the lowest rank (rule #2) relative to when it had the highest rank (rule #1). For green this effect was reversed. Thus, these data support our conclusion that ERFs amplitude was modulated by stimulus rank.

To test for influences of rank void of the sensory property color, for each rule type we computed the distance of the response evoked by red and green to the response evoked by the invariant blue stimulus, which for both rules served as rank #2. If evoked responses were influenced by task demands as suggested by the data in Figure 4, rank should modulate the distance to blue and result in opposite effects when comparing both rules. We observed that that was indeed the case (Fig. 5); for rule #1, red evoked larger responses than blue, while activation by green was smaller. For rule #2 this relationship was reversed. Average peak distances were reached at ∼0.17–0.18 s after cue onset for both extrastriate and parietal cortices (Fig. 5A,B, top left and middle). To corroborate this finding, we extracted single subject peak amplitude values evoked by the red, blue, and green contralateral stimuli and computed the relative distances of red and green peak amplitudes to blue (Fig. 5A,B, bottom left and middle). In line with the time series data, average distances between peak amplitudes showed an inversion following our rank rules. We further quantified the modulation by rank across rules, by computing the difference in the obtained distances to the blue stimulus between rule #1 and rule #2 (Fig. 5A,B, rightmost panels). In both regions of interest, rank significantly modulated relative evoked responses to both red and green stimuli (Wilcoxon signed-rank tests, p = 0.0001). Note that here we concentrated on the differences earlier than 200 ms from cue onset, where the strongest activation occurred.

Figure 5.

Target selection: distance to blue. A, Average distance of red and green stimuli to blue (invariant rank 2) combined across left and right middle occipital (mid. occ.) gyri. Top, left and right, represents distance as a function of time from color cue onset for rule #1 and rule #2, respectively. The bar graphs below depict the distance in single subjects' peak amplitudes. Rightmost panels represent the magnitude of rank modulation across both rules (distance rule #1 minus distance rule #2). Shaded areas and black lines are SEM. B, The same as in A for the parietal cortex.

To further investigate effects of stimulus rank on the time courses of activation, we pooled the corresponding trial types of rule #1 and #2 to yield groups of 100, 50, and 0% target trials corresponding to stimuli with ranks 1, 2, and 3 respectively. The reasoning behind this analysis was that by combining trials where red (rule #1) and green (rule #2) are the 100%-targets, and conversely trials where both colors are the 0%-targets, the contribution of the purely sensory property color is counterbalanced, hence, differences between responses based on the stimuli's colors (Fig. 4) should cancel out. In the following sections, we report results on the pooled data and refer to them as 100%-, 50%-, and 0%-target conditions. Note that purely for illustration purposes, the colors used to plot the different conditions were chosen as red (100%-target), blue (50%-target), and green (0%-target), but they are not meant to refer to the color arrangement used in rule #1. For all subjects we computed source waveforms within the two regions of interest for trials in which the white RDP in the contralateral visual field changed to the 100%-target (red for rule #1, green for rule #2), the 50%-target (blue for both rules), or the 0%-target (green and red for rule #1 and #2, respectively). Figure 6(top row) depicts the resulting average waveforms corresponding to the extrastriate visual cortex contralateral to the hemifield where the stimulus with the corresponding rank was presented (first and second panels). Peak activations in the right hemisphere were generally stronger compared with activation in the left hemisphere (paired t test, p = 0.0052). We extracted the single subjects' peak amplitude values and pooled the data across both hemispheres (third panel). Here we observed a trend for the 100%-target to evoke the largest amplitude, followed by the 50%- and 0%-targets.

Figure 6.

Target selection: extrastriate visual cortex. A, Average virtual sensors for left and right middle occipital gyri and combined across both hemispheres (from left to right) as a function of time from color cue onset. The different colors represent responses to contralateral stimuli of varying target probabilities (red: 0, blue: 0.5, green: 0). Shaded areas represent SEM. B, Corresponding peak normalized amplitudes for all subjects (blue dots) as a function of target probability for the combined dataset. Regression lines are indicated in red. C, Average peak amplitudes for the combined dataset (left and right middle occipital gyri). Left, Average peak amplitudes as a function of target probability. Right, Average peak amplitudes for the 50%-target stimulus when presented as a target (dark blue) or distracter (light blue). Black lines are SEM.

We conducted a linear regression analysis on the individual subject amplitude values pooled across both hemispheres as a function of rank (Fig. 6B). Data points were identified as outliers and removed when the corresponding residual was larger than expected in 95% of observations. We found that peak amplitudes decreased as target rank decreased (1 outlier removed; slope = −0.058; lower/upper confidence intervals = −0.108/−0.007; r = 0.347, p = 0.0264). Figure 6C (left) shows the corresponding mean activations evoked by stimuli with different ranks. Indeed, we observed a larger activation for 100%-targets, followed by 50%- and 0%-targets at a significance level of 0.1 (Kruskal–Wallis one-way ANOVA, p = 0.0878). A post hoc comparison confirmed that the main effect of rank was mainly driven by a significant difference between 100%- and 0%-targets (Wilcoxon signed-rank tests, 100%- vs 0%-targets: p = 0.0327; 100%- vs 50%-targets: p = 0.079; 50%- vs 0%-targets: p = 0.455).

To test whether these effects were based on the allocation of spatial attention rather than rank per se, we analyzed the 50%-target stimulus, which served as a control, for its absolute rank remained the same while it could be either target or distracter. This configuration allowed us to disentangle absolute rank and allocation of spatial attention. We compared average peak amplitudes between conditions in which the 50%-target stimulus was a target (that is, when paired with the lower ranked color) and those where it was a distracter (when paired with the higher ranked color; Fig. 6C, right). Peak amplitudes between target and distracter trials were not significantly different (Wilcoxon signed-rank test, p = 0.4267).

These data suggest that there is an effect of the stimulus rank-order on the amplitude of peak activations in early extrastriate visual cortex, which is not based on spatial attention but reflecting the probability of a given stimulus color to be the target.

We repeated the analysis for the left and right parietal regions of interest (Fig. 7). Here again, we observed slightly stronger activation in the right hemisphere but this difference did not reach significance (Fig. 7A, left and middle; paired t test, p = 0.2385). More importantly, in both left and right parietal cortices as well as in the pooled data (right) we observed a decrease of peak amplitudes as target probability decreased. We performed a linear regression analysis and obtained significant decreases of peak amplitudes with decreases in rank in the combined dataset (2 outliers removed; slope = −0.095; lower/upper confidence intervals = −0.155/−0.035; r = 0.462, p = 0.0027; Fig. 7B). The corresponding mean activations of the pooled data revealed a significant decrease with decreases in the rank of the stimulus color and thus target probability (Kruskal–Wallis one-way ANOVA, p = 0.0319; Fig. 7C, left). All post hoc comparisons were significant at the level of 0.1 (Wilcoxon signed-rank tests, 100%- vs 0%-targets: p = 0.001; 100%- vs 50%-targets: p = 0.049; 50%- vs 0%-targets: p = 0.092).

Figure 7.

Target selection: parietal cortex. The same as in Figure 6 for left and right parietal cortex and the combined dataset.

To control for possible effects of spatial attention, we compared average peak amplitudes between the 50%-target stimulus as a target and as a distracter (Fig. 7C, right). Peak amplitudes between target and distracter trials were not significantly different (signed-rank test, p = 0.2166).

Interestingly, the slope value obtained in our regression analysis was twofold larger in magnitude in the parietal cortex than the one observed in the extrastriate visual cortex suggesting that in the parietal cortex the effect of stimulus rank was more pronounced. To corroborate this observation, we compared peak activations evoked by the 100%-target and 0%-target stimuli and computed the mean percentage increase in both areas. Indeed, the average percentage increase in early visual cortex was 28% (±14% SEM), and increased almost twofold (53 ± 27% SEM) in the parietal cortex.

In general these findings suggest that the effects observed above were predominantly based on stimulus rank. Furthermore, the effects occurred later and were considerably stronger in the parietal relative to the visual cortex.

ERFs evoked by direction and color changes in targets and distracters

An additional feature in our task is that following target selection the subjects were required to sustain attention on that stimulus and discriminate, in different trial blocks, either a transient change in its direction or in its color while ignoring changes in the distracter stimulus. We performed group erSAM analysis to identify the average location of peak activations evoked by the different changes (Fig. 8). Direction changes in targets and distracters in the right and left visual hemifields evoked activity peaks at ∼0.18 s following the change event and localized to contralateral extrastriate areas (x/y/z Talairach coordinates: −30/−69/15 and 42/−61/9) at the approximate location of hMT+/V5 (Ahlfors et al., 1999; Händel et al., 2007; Martinez-Trujillo et al., 2007). This result agrees with the ones reported by a previous MEG study (Martinez-Trujillo et al., 2007). On the other hand, color changes in the right and left hemifields evoked peak activations localized to contralateral color-sensitive extrastriate visual areas in the middle occipital gyrus (−36/−70/6 and 30/−67/1). The latter were similar to the locations of peak activations observed after color cue onset (Fig. 3). These results are in agreement with a variety of studies using different imaging methodologies and stimulus parameters to identify regions in the occipital lobe that are selectively activated by color (Corbetta et al., 1990, 1991; Zeki et al., 1991; Sakai et al., 1995; Anllo-Vento et al., 1998) (see Table 2 for a full list of mean locations and magnitudes of peak activations).

Figure 8.

Sustained attention: group erSAM analysis. A, Sagittal, coronal, and axial views of brain regions maximally activated (yellow) 180 ms after the transient direction change in the target when positioned on the left (top) or right (bottom). B, The same as in A for transient color changes. Insets represent example stimulus configurations.

To test for effects of spatial attention, we computed single subjects' source waveforms for target and distracter changes at these regions of interest. Since we did not observe any trend in the individual subjects for the amplitude to change as a function of rank (data not shown), we pooled data across different target colors and across different distracter colors. This makes sense since during this time period of the task subjects had already made the selection and they allocated attention to the target while ignoring the distracter. For example, we pooled across activations evoked by changes in red, blue and green RDPs when they were targets, as well as across activations evoked by changes in the same stimuli when they were distracters (Fig. 9A). We found that pooled mean peak amplitudes across subjects were larger for target than for distracter changes (Fig. 9B; Wilcoxon signed-rank test, p = 0.058).

Figure 9.

Sustained attention: extrastriate visual cortices. A, Average virtual sensors for left and right areas hMT+/V5 and combined across both (from left to right) as a function of time from direction change onset. Black and gray traces represent responses to contralateral target and distracter changes, respectively. Shaded areas represent SEM. B, Average peak amplitude for the combined (left and right area hMT+/V5) for target (black) and distracter (gray) changes. Black lines are SEM. Wilcoxon signed-rank tests were performed and evaluated at p = 0.1. C, D, The same as in A and B for activity evoked by transient color changes in the left and right middle occipital (mid. occ.) areas and combined across hemispheres.

On the other hand, we found that the amplitude of the activations evoked by color changes in the target and the distracter were similar for left and right middle occipital gyri (Fig. 9C). Quantification of mean peak amplitudes for the pooled dataset did not reveal a significant difference from each other (Wilcoxon signed-rank test, p > 0.1; Fig. 9D). Interestingly, the activation in the right hemisphere was stronger than the one on the left (Student's paired t test, p = 0.0031) similar to the pattern described for color changes during target selection. We did not observe any asymmetries in amplitude between left and right areas hMT+/V5 for direction changes (p > 0.5), suggesting preferential processing of color signals in the right hemisphere.

Unilateral versus bilateral representation of space

To examine the mechanisms of target selection in extrastriate visual and parietal cortices, the analyses we presented thus far considered ERFs evoked by stimuli located in the contralateral visual field. This was based on the general notion that early visual brain areas process predominantly inputs originating from the contralateral visual hemifield. We support this notion by the lack of any significant ipsilateral activation in response to the transient color/direction changes when evaluated at p < 0.001 (Fig. 8, Table 2). In both areas hMT+/V5 and the middle occipital gyri activity was evoked exclusively in the contralateral hemisphere. Note that the latter area was also activated during target selection. We therefore consider it unlikely that the ispilateral stimulus modulated activity in the extrastriate visual cortex during target selection.

The parietal cortex on the other hand, especially the right hemisphere, has been suggested to possess a bilateral representation of space (Heilman and Van Den Abell, 1980; Mesulam, 1981). To test whether the ipsilateral stimulus modulated activity during target selection, we analyzed peak activations evoked by red and green contralateral stimuli as a function of the stimulus on the ipsilateral side. Red and green were either 100%- or 0%-targets and thus each was presented in two different color arrangements, i.e., paired with either blue or green and blue or red, respectively. Figure 10 compares ERF peak amplitudes when keeping constant the contralateral stimulus (i.e., red or green) and varying the ipsilateral paired RDP. Corroborating the above results, in the left and right middle occipital gyri we did not find any significant effects of the ipsilateral stimulus on peak responses to red or green at the Bonferroni-corrected significance level of p = 0.0125. Peak amplitudes clustered along the unity line (Wilcoxon signed-rank tests; left hemisphere, red: p = 0.017, green: p = 1; right hemisphere, red: p = 0.035, green: p = 0.67). Similarly, we did not observe a significant influence for neither color in the left parietal cortex (red: p = 0.173, green: p=0.268). Interestingly, the right parietal cortex showed a significant effect of color combination for the red stimulus configurations (red: p = 0.0031 versus green: p = 0.025) suggesting that the ipsilateral stimulus influenced evoked responses to the contralateral, red RDP. We do not think that such influences question our findings on the effect of rank on ERFs; first, because we averaged across the two possible color arrangements in the above analysis, and second, because 100%- compared with 0%-targets evoked asymmetric responses in the right parietal cortex (Fig. 7A, middle). It hence appeared as if there was more weight on processing the contralateral visual field with small influence of the ipsilateral side. These findings add further evidence to the suggested bilateral representation of space in the right parietal cortex.

Figure 10.

Unilateral versus bilateral representation of space. A, Peak normalized activations in the left and right middle occipital gyri for red (red circles) and green (green circles) contralateral stimuli. Depicted are peak activations during trials where the contralateral stimulus is kept constant (red or green) while the ipsilateral stimulus varies (color combination 1 vs 2; see text for details and p values). Closed circles represent rule #1, open circles rule #2. The dashed line is the unity line. B, The same as in A for the parietal cortex.

Discussion

Our results show that activity evoked by a visual stimulus in early extrastriate visual cortex and few milliseconds later in the posteromedial parietal cortex scaled following a rank-order selection rule. The magnitude of this effect was stronger and more robust in the parietal relative to the visual cortex. We also found that sustained attention to the target increases the activation evoked by direction changes in that stimulus relative to similar changes in the distracter.

Scaling of ERFs following stimulus rank

Previous studies of ERPs have demonstrated attentional modulation of brain activity evoked by visual stimuli in early sensory cortices (Martínez et al., 1999; Noesselt et al., 2002; Zhang and Luck, 2009). In general, visual signals originating at the attended location are preferentially processed relative to other signals. Imaging studies have reported similar results (Corbetta et al., 1990, 1991). It has been suggested that these effects are due to top-down signals from prefrontal and parietal cortices that create a tonic change in the response gain of visual extrastriate neurons with receptive fields at the attended location (Chawla et al., 1999). This would yield enhanced feedforward transmission when a stimulus appears at that location (Hillyard and Münte, 1984; Luck et al., 1997). Top-down signals related to the control of feature-based attention have also been reported in frontal and parietal regions (Giesbrecht et al., 2003). Our results support this hypothesis.

Moreover, our results demonstrate that the influence of top-down signals on the activity evoked by visual stimuli can follow complex rules. In our task trials, the attended location or feature could not be defined in advance, instead subjects had to compare the rank of the two alternatives and choose the one with the highest. We observed that the activity evoked by a stimulus scaled with its rank as early as 0.18 s after the color change onset. This relatively short latency may suggest that the effect of top-down signals was already present when visual inputs carrying information about the stimuli's color reached extrastriate visual cortex. Twenty milliseconds later, when the peak activity was registered in the parietal cortex we observed a stronger rank-dependent scaling of activity, suggesting that the modulation becomes stronger as the signal travels from extrastriate visual areas toward the parietal lobe. This agrees with observations made by single cell studies in monkeys (Cook and Maunsell, 2002; Saalmann et al., 2007) and imaging studies in humans (Kastner and Ungerleider, 2000) of stronger attentional modulation of sensory signals as one progresses downstream in the hierarchy of visual processing. This may reflect an accumulation of attentional effects along the different stages of processing with a similar top-down bias across all areas, or a stronger top-down bias in areas downstream in the hierarchy of processing, or both.

Previous single cell studies in non-human primates have suggested that single neurons in area LIP of the parietal lobe are modulated by bottom-up as well as top-down attention (Buschman and Miller, 2007; Arcizet et al., 2011), and that they are sensitive to signals such as probability of reward (Sugrue et al., 2004). It has been proposed that the parietal cortex possesses a saliency map of visual space for target selection (Koch and Ullman, 1985; Gottlieb et al., 1998; Itti and Koch, 2000; Treue, 2003). In this map, a given visual input becomes salient when it differs from other inputs in physical attributes such as color, shape, or size (bottom-up saliency). Top-down attentional signals can then increase or weaken this bottom-up saliency depending on the behavioral relevance of the inputs. As a result, attention is allocated to, and remains on, the inputs (object or location) with the highest saliency (activity) within the map (Bisley and Goldberg, 2003).

Our findings support the idea that saliency computations can occur at different levels in the hierarchy of visual processing. The effects in extrastriate visual areas may favor computations in the parietal lobe by biasing the signal at previous stages of processing. Once the target is selected within the parietal map, parietal neurons could send feedback signals to extrastriate visual areas further biasing the sensitivity of neurons to sensory inputs (Saalmann et al., 2007).

Asymmetries in evoked activity

In our study, the onset of the bilateral color cue evoked activity in the left and right color-sensitive extrastriate visual cortex. However, we observed higher peak activation in the right hemisphere. Interestingly, a study by Poghosyan et al. (2005) reported that the attentional modulation arose first in color-sensitive areas of the right hemisphere (105–132 ms), and ∼60 ms later in corresponding areas of the left hemisphere. Activity in the parietal cortices in our study showed a similar trend: average responses in the right hemisphere were higher and attentional modulation of peak amplitudes was stronger. An involvement of the right parietal cortex in attentional control processes has been reported (Kim et al., 1999; Corbetta et al., 2000; Yantis et al., 2002), being indicative of lateralized hemispheric function. Moreover, we found that the ipsilateral stimulus modulated evoked activity in the right parietal cortex supporting the idea of a bilateral representation of space in that brain area. This is in line with previous MEG reports (Martinez-Trujillo et al., 2007) and is supported by clinical data showing that structural damage to the right posterior parietal cortex in humans produces unilateral visual neglect (Mesulam, 1981). This deficit is more severe after right than left parietal lesions (Morrow and Ratcliff, 1988) supporting a degree of functional asymmetry and lateralization regarding the allocation of attention between left and right hemispheres.

Sustained attention

Our results showed that motion direction changes elicited peak activation in contralateral area hMT+/V5 whereas color changes evoked activity in the contralateral middle occipital gyrus. This demonstrates that erSAM has the spatial resolution to distinguish between changes in these different areas. In agreement with previous single cell studies in the monkey (Treue and Martinez-Trujillo, 1999) and MEG studies in humans (Händel et al., 2007) we found effects of sustained spatial attention in area hMT+/V5. These effects were likely due to an attentional bias reaching neurons within the area contralateral to the target after the selection occurred, ultimately favoring the processing of direction changes in that stimulus.

We did not observe the same significant increase in response to color changes in targets relative to distracters in the middle occipital gyrus—although we saw a trend in the data. A possible explanation for this result is that color changes were easier to discriminate than motion direction changes, despite our attempts to match task difficulty between both trial types during initial testing sessions. It has been demonstrated that greater task difficulty increases the strength of attentional modulation probably by increasing attentional effort (Spitzer et al., 1988; Boudreau et al., 2006; Chen et al., 2008). Since our task was run in blocks this possibility cannot be discarded. Under more challenging experimental conditions such as multiple distracter stimuli or smaller stimulus size, the effects of sustained attention on color changes might become statistically significant.

Time course of the modulation

The spatiotemporal resolution of erSAM allowed us to detect peak activations in two different brain regions at 180 and 200 ms after change event onset. At these times, we observed the modulation following the stimulus rank in the visual and parietal cortices respectively. These latencies are in agreement with previous EEG and MEG studies that examined the mechanisms of selective attention using tasks involving stimulus discrimination. When deploying a task in which a color has to be discriminated from another, attentional modulation typically arises between 130 and 180 ms (Anllo-Vento et al., 1998; Schoenfeld et al., 2003). Similarly, when a motion direction has to be selected from another, attentional effects occur between 150 and 160 ms (Anllo-Vento et al., 1998). Experimental tasks requiring the selection between the color and motion feature dimensions may reveal attentional latencies as fast as 90–120 ms (Schoenfeld et al., 2007). Interestingly, Schoenfeld et al. have argued that attentional modulation may arise faster when selecting between feature dimensions compared with discriminating features within the same dimension.

Somewhat surprisingly, we did not observe marked differences in latencies of attentional modulation between responses following the color cue onset when attention is divided between the two stimuli, and responses after the transient response event when attention is already allocated to the target. Possible explanations for this apparent discrepancy are that we used change events of different nature (color and motion direction) and of different intensity (high contrast color cue onset vs transient color/direction change); and that our task demands were different from the ones used in other studies. This renders comparisons between the two change events difficult.

In summary, we have demonstrated that the amplitude of ERFs first in early extrastriate and then in the parietal cortex can scale following a rank-order selection rule. The scaling in the latter area was almost twofold larger than in the former. These results indicate that attentional top-down signals adjust their intensity according to abstract rules to differentially modulate the activity of neurons across visual cortical areas and flexibly drive target selection.