Attention Improves Object Representation in Visual Cortical Field Potentials

Abstract

Selective attention improves perception and modulates neuronal responses, but how attention-dependent changes of cortical activity improve the processing of attended objects is an open question. Changes in total signal strength or enhancements in signal-to-noise ratio have been proposed as putative mechanisms. However, it is still not clear whether, and to what extent, these processes contribute to the large perceptual improvements. We studied the ability to discriminate states of activity in visual cortex evoked by differently shaped objects depending on selective attention in monkeys. We found that gamma-band activity from V4 and V1 contains a high amount of information about stimulus shape, which increases for V4 recordings considerably with attention in successful trials, but not in case of behavioral errors. This effect resulted from enhanced differences between the stimulus-specific distributions of power spectral amplitudes. It could be explained neither by enhancements of signal-to-noise ratios, nor by changes in total signal power. Instead our results indicate that attention causes underlying cortical network states to become more distinct for different stimuli, providing a new neurophysiological explanation for improvements of behavioral performance by attention. The absence of the enhancement in discriminability in trials with behavioral errors demonstrates the relevance of this novel neural mechanism for perception.

Introduction

Attending an object within a complex sensory scene is known to improve various aspects of its perception. This includes lower thresholds, faster responses, and better discriminability for attended compared with nonattended objects. Such attention-dependent enhancements of perceptual capabilities suggest an improvement of the underlying neuronal representations and processing of attended objects. Selective attention has been shown to result in modulations—often increments—of neuronal firing rate (Moran and Desimone, 1985; Motter, 1993; Treue et al., 1996; McAdams and Maunsell, 1999a; Reynolds et al., 1999, 2000). Furthermore, neurons engaged in processing of an attended object tend to organize their responses into synchronous firing patterns (Steinmetz et al., 2000) with oscillation frequencies in the gamma band (Fries et al., 2001, 2008; Bichot et al., 2005; Taylor et al., 2005; Womelsdorf et al., 2006a,b; Lakatos et al., 2008). Modulations of firing rate have been proposed to contribute to an improved representation in several ways. In case of multiple, nearby stimuli positioned within the same receptive field (RF), neurons tend to respond as if only the attended stimulus was present (Moran and Desimone, 1985; Treue and Maunsell, 1996). While this helps to disambiguate the representation with respect to multiple nearby stimuli, it does not necessarily imply an improved representation of a single attended stimulus compared with a single nonattended stimulus without other, competing stimuli within the RF. Another approach using spike rate modulations to explain perceptual improvements by attention is based on attention-dependent increases of mean firing rates observed in several studies. These response increments were found to improve signal-to-noise ratios, which could provide a basis for enhancing the discriminability of representations of different stimuli (McAdams and Maunsell, 1999b). However, the corresponding improvements of representations appear to be limited since attention-dependent increments of firing rates f are often small or even missing (Reynolds and Chelazzi, 2004) if attention is moved from a stimulus outside the RF to the stimulus within. Therefore we here investigated whether additional mechanisms are present that could explain the rather large perceptual effects of attention (Rock et al., 1992; Wolfe and Bennett, 1997).

Specifically, we investigated how selective attention influences the discriminability of neuronal activity patterns in the visual cortex of macaque monkeys that attend to one of two spatially well separated shapes placed in the right and left visual hemifields. Field potential signals were recorded from an epidural electrode array and decomposed into their frequency components. We used support vector machines as state of the art classifiers (Schölkopf et al., 2000, 2001) to identify the presented objects based on the observed neuronal activity patterns. We found a clear increase of the classification performance for attended compared with nonattended stimuli. This increase was only to a minor extent explained by an improved signal-to-noise ratio (SNR), but mainly by stimulus-specific changes of the power spectral amplitudes for different gamma-frequency components, leading to a better separability of the neural activity patterns evoked by different shapes. Finally, we analyzed error trials in which the monkeys failed to perform the shape recognition task correctly. We found that in such cases the discriminability under attention was in most cases reduced to a level similar to or even lower than the discriminability in the nonattended condition.

Materials and Methods

The datasets for the study were taken from the control experiment presented in Taylor et al. (2005) (see their Fig. 8).

Behavioral training and visual stimulation.

Two male rhesus monkeys (Macaca mulatta), monkey F and monkey M, were trained to an extended version of a delayed-match-to-sample-task. For training and recording sessions they sat in a primate chair with the head restrained. Visual stimuli were presented with a frame rate of 100 Hz on a 21 inch cathode ray tube screen 81 cm in front of the monkeys' eyes.

The monkeys started a trial (see Fig. 1) by pressing a lever after the appearance of a central fixation point on the screen. After 650 ms, two different stimuli with complex shape appeared at fixed positions in the left and right visual hemifield (time period T1). The sample stimulus of the behaviorally relevant stimulus sequence (i.e., the target sequence) was cued within the first 200 ms of stimulus presentation by green coloring, which faded into white within the subsequent 400 ms. The initial sample stimuli appeared for 1550 ms, followed by a sequence of test stimuli presented statically for 500 ms and separated by 900 ms delay intervals (blank screen with fixation point only). The test stimuli in each stimulus sequence differed from each other and were randomly selected from a set of 10. The sample stimuli were selected from a subset of six, and also varied from trial to trial. The matching test stimuli of the target and the distracter sequence appeared in different, randomly selected periods T3 to T6 (see Fig. 1a). Appearance of the matching stimulus in the initially cued target sequence required the monkey to release the lever within 1000 ms after test stimulus onset, to be rewarded with a drop of fruit juice. A reappearance of the initial shape in the stimulus sequence of the distracter had to be ignored. If monkeys broke fixation (rectangular fixation window extending vertically and horizontally by 0.75° from the fixation point), or responded too early or too late, the trial was aborted without reward.

Figure 1.

a, Schematic illustration of the shape-tracking task. Two sequences of static shapes were presented in the left and right hemifield of a computer screen (upper left rectangle). While the monkey was fixating on a dot in the center of the screen, attention of the monkey was directed by a 200 ms green coloring fading out within further 400 ms to one of the initial shapes presented during period T1—in this example, to the right-hand shape. The task for the animal was to signal the reoccurrence of the initial shape in the attended hemifield during one of the following periods T3–T5. Here, a correct response would be during or after presentation of shape s(T5). b, Regions of interest in V1 and V4 where the stimuli caused substantial activation (see Materials and Methods). c, d, Examples for typical time–frequency plots on the same time axis as in a, displaying the trial-averaged, normalized power spectral density A(t, f₀) in the attended condition for an electrode over V1 (c) and over V4 (d) from monkey F.

Each stimulus shape was defined by 12 nonvisible points interconnected by a 0.3° wide smooth Bezier curve. They were centered to the left and to the right of the fixation point, 0.9° below the horizontal meridian and 2.9° aside the vertical meridian, and covered a region of ∼4° × 4°.

During recording we presented blocks of 60–90 trials with the target stimulus on the same side to support target cueing. Monkey M was tested with the target position chosen randomly for each trial. For partial retinotopic mapping of the lower visual field representation of V1 and V4, small white squares (0.4° × 0.4°) were flashed at different positions in the visual field while monkeys were engaged in a fixation task and kept their gaze within ±0.75° around the fixation point.

Surgical preparation.

Using standard surgical techniques the monkeys were implanted with a headpost and a thin gold ring placed between the conjunctiva and the sclera of one eye for measurement of gaze direction using the indirect search coil method (Bour et al., 1984). After recovery and completion of the subsequent behavioral training, an epidural array of platinum–iridium electrodes was placed over area V4 and parts of area V1 close to the sulcus lunatus. Based on maps of the monkey brain (Gattass et al., 1981, 1988; Paxinos et al., 2000), the intended position of the array was determined relative to anatomical landmarks. Stereotactic coordinates of these landmarks were derived from structural magnetic resonance images obtained for each animal from a 4.7 T Bruker Biospec scanner. The precise location of the implanted electrode array was estimated postoperatively by the stereotactic coordinates determined during implantation, their comparison with structural magnetic resonance images obtained after implantation and morphological confirmation in one of the monkeys. The localization of the array was further improved and confirmed by the construction of a partial retinotopic map of area V4 and, in one monkey, area V1, based on recordings of γ-band responses to the small test stimuli described above with the implanted electrode array (see also Taylor et al., 2005).

Recording.

Using the chronically implanted epidural electrode arrays we recorded field potentials from parts of area V4 and V1 at 36 electrode positions in monkey M and 37 positions in monkey F. The electrode array consisted of Teflon-coated platinum–iridium (90 Pt/10 Ir) wires (diameter 50 μm, Science Products) inserted with a regular spacing of 3 mm into a 0.1-mm-thick sheet of silicone (Goodfellow). An uninsulated loop (diameter 210–220 μm) positioned parallel to the dura formed the electrode contact. The electrode impedance was typically 25 kΩ at 100 Hz. Two reference electrodes (platinum–iridium wire, 150 μm diameter) were placed frontally. In monkey F a third reference was attached to the backside of the electrode array (0.1 mm platinum–iridium foil, 4.5 mm diameter). Recordings were referenced to the latter electrode in monkey F and a frontal electrode for monkey M. Signals were amplified (40,000× in monkey F, 30,000× in monkey M, 1–150 Hz bandwidth) and continuously recorded at a sampling rate of 1 kHz.

The datasets used for the present study come from the animals and recording arrays used by Taylor et al. (2005) and overlap with data used in the previous study to control for memory-related effects. All surgical and experimental procedures were performed in accordance with the European Communities Council Directive of November 24, 1986 (86/609/EEC) and with the regulations for the welfare of experimental animals issued by the Federal Government of Germany and had been approved by the local authorities.

Data analysis.

For data analysis we rejected all trials in which the monkeys made fixation errors. For one of the monkeys some trials had to be rejected because the signal saturated the amplifier. The field potential signals were high-pass filtered with a digital filter (Butterworth IIR filter, cutoff frequency 0.65 Hz at 3 dB, forward and backward filtering to avoid phase shifts) to eliminate DC offset. To suppress the effect of the common reference and to minimize spatial smearing (Nunez et al., 1997), the current source density (CSD) (Gevins, 1984) with unit volts per square meter was computed. For each time bin, the second spatial derivative of the field potentials was computed with the Laplacian operator (Perrin et al., 1987), using Gaussian radial basis functions for interpolation (Moody and Darken, 1989). For each electrode, the CSD yields the signals v_j(t) with j denoting the trial number. These data were convoluted with complex Morlet's wavelets w(t, f₀) (Kronlandt-Martinet et al., 1987) to obtain the wavelet power coefficients a_j(t, f₀) via

The spacing of frequency bands was logarithmic between 5 and 200 Hz, chosen as f₀(k) = Ω^k−1f₀(1) for k = 1,…,17 frequency bands starting at f₀(1) = 4.84 Hz. For a sufficiently tight coverage of frequency space, we set Ω = 1.206.

For Figure 1, c and d, we computed normalized mean spectra

using normalization coefficients quantifying the background activity n(f₀) obtained from n(f₀) = 1/(t₂ − t₁)∫_t1^t₂〈a_j(t, f₀)〉_jdt with t₁ = 300 ms and t₂ = 350 ms. For all other purposes, the original, non-normalized values were taken.

In Taylor et al. (2005), we already noticed pronounced differences in spectral power between the attended and nonattended state. Here we want to investigate quantitatively how well different shapes are represented in the recorded signals, and how this representation is influenced by attention. For this purpose, classification is a quantitative method for establishing a lower boundary for the information contained in the signals, and for identifying characteristic differences between signals caused by different shapes. We performed classification on the power spectra, which were averaged over suitable time intervals (detailed procedure described later).

Classification of stimuli based on neuronal signals has been performed in various experimental settings differing in species and neuronal measures ranging from single-unit studies to EEG recordings. Hereby the usage of support vector machines (SVMs) (Schölkopf et al., 2000, 2001) was established as a standard method that delivers performances superior to linear methods, and consumes typically less resources than mathematically simpler methods or brute-force approaches like nearest-neighbor classifiers. In short, the SVM allows to segregate the multidimensional data space (e.g., where each axis of the data space represents the absolute power of one frequency component of one electrode) by a hyperplane. The resulting isolated data regions are then assigned to different classes (like the shape or the condition of attention). Given a new data point with unknown class, it is now possible to estimate the underlying class by calculating to which data region this new data point belongs. One SVM separates data space into two regions. Combining the results from more then one SVM allows to distinguish between more then two classes. Classification errors occur if regions spanned by the data points overlap and it is not sure to which region one point really belongs. For extending the possibilities of separating the whole data space into two regions by a hyperplane, the data space is transformed into “feature space,” typically by a nonlinear mapping using so-called kernel functions. For implementing the SVMs, we used the widely used libsvm software package (Chang and Lin, 2001), which provides convenient data preprocessing routines automatically searching the parameter space of possible SVM realizations, while calibrating the input data to match the abilities of the classification algorithm. A radial basis function kernel was chosen for classification. The software package optimizes its parameters only on the training set of the data and tests it performance on a separate test dataset. These datasets are never mixed.

For classifying the data according to the presented shapes, we used two procedures. For monkey F, we divided each of the analyzed datasets corresponding to a behavioral condition like “attended” or “nonattended” with N trials into two subsets of approximately equal size, by alternately assigning trials to one training subset and to one test subset in an interleaved manner. Typically test and training sets contained up to ∼1400 trials each. The data were measured within 11 session for monkey F and five sessions for monkey M. Data from different sessions were pooled for each monkey separately. Given the good stability of recordings from day to day, we did not attempt to remove putative nonstationarities between sessions. For monkey M (except for the section “Characterization of stimulus-specific signals” because of computational reasons), we used a “leave-one-out” scheme, in which the SVM was trained on N − 1 trials, with up to ∼800 trials, and with the remaining trial acting as the test set. This procedure was repeated such that each single trial was used once as the test trial, and the resulting single classification performances were averaged. It provides better generalization properties of the learned SVM at the price of being numerically more expensive, by a factor of ∼4N. The reason for using two different procedures was the lower number of total trials of monkey M (approximately one-third). The classes to be learned were defined by the shapes s_j(k) presented to the monkeys in selected intervals k ∈ {T1, T2, T3,…} of the stimulus sequence (see Fig. 1a) displayed in the visual hemifield contralateral to the recordings during trial j. In total, there were six shape classes s ∈ {1,…,6} used as targets in the trials. In the following description, where necessary, we will use the index s to distinguish variables, which were computed using only wavelet coefficients from trials j in which the shape shown in interval k = T1 was s, i.e., s_j(T1) = s. Likewise, we will distinguish data from trials where attention was directed to the visual hemifield represented in the recorded brain region with a superscript A (attended), while using a superscript N (not attended) otherwise.

We then trained SVMs on the training sets, and estimated their classification performance on the test sets. Performance P was measured as the total percentage of shapes classified correctly by the SVM in the test sets. The chance level P_chance was computed as the ratio of the occurrences of the most frequently presented pattern in the training set to the total number of trials in this set. An increase (decrease) in performance P above chance level was considered to be significant as soon as the probability to obtain an equal or higher (equal or lower) performance by drawing from a binomial distribution around P_binom = P_chance was smaller than p = 0.02, respectively. A difference in performance P^A − P^N was considered significant as soon as the probability to draw P^A and P^N in two binomial experiments was smaller than p for any putative underlying probability P_binom.

From all coefficients a_j(t, f₀) obtained within a period T for the center frequency f₀, we selected a subset of a's equally spaced in time and computed averaged coefficients ā_j(f₀). The spacing was adjusted to approximately twice the period 1/f₀, which is sufficient to capture the typical rate of change in wavelet-analyzed signals. Averaging led to a large decrease in computational complexity for the training of the SVMs. Numerous test simulations with the original, full set of coefficients yielded no substantial difference in classification performance (data not shown); thus, we proceeded using average coefficients only.

Analysis of the residual eye movements within the fixation area revealed small statistical differences between the attended and the nonattended condition. Further tests on selected subsets of our data (see supplemental material, available at www.jneurosci.org) excluded the possibility that differences in classification performances could be explained by these unequal statistical properties.

To determine the influence of the SNR on the classification performance, the SNR η = μ/σ was computed as the quotient of mean μ = 〈ā_j〉 and standard deviation σ = $\sqrt{\langle \overline{a_j}-\mu \rangle }$. Using the individual means μ_s and standard deviations σ_s computed for the different shapes s, we used two procedures to created surrogate data (Theiler et al., 1992; Schreiber and Schmitz, 1996; Kaplan, 1997) by scaling the original wavelet coefficients such that either (I) only the changes in SNR were reproduced, or (II) only the changes in mean values were reproduced, when going from one to another attentional condition. This scaling could be applied in two directions, creating a quasi-attended condition from nonattended data, or creating a quasi-nonattended condition from attended data, as explained in the following paragraphs.

Procedure I.

We computed the first “quasi”-attended dataset a_j,s^A(I)(f₀), having the same means as the real nonattended dataset, but the original SNRs from the real attended dataset, via

Similarly, the first “quasi”-nonattended dataset a_j,s^N(I)(f₀), having the same means as the real attended dataset, but the original SNRs from the real nonattended dataset, was computed via

Procedure II.

We computed the second “quasi”-attended dataset a_j,s^A(II)(f₀) having the same SNRs as the real nonattended dataset, but the original means from the real attended dataset, via

Similarly, the second procedure yields the second “quasi”-nonattended dataset a_j,s^N(II)(f₀) having the same SNRs as the real attended dataset, but the original means from the real nonattended dataset, via

Results

To investigate the effect of selective attention on stimulus specificity of neural activity patterns in visual cortex, two macaque monkeys were trained to perform a delayed-match-to-sample task. Multiple test stimuli were to be compared with an initial sample stimulus (Fig. 1a). This task required animals to direct their covert attention to one of two stimulus sequences simultaneously presented to the right and to the left of the fixation spot. During the 1550 ms of the initial sample presentation period, the shape at the behaviorally relevant location was cued by an initial green coloring lasting 200 ms that faded out within 600 ms after stimulus onset. Subsequently, on each side two to five test stimuli were shown sequentially for 500 ms periods, each preceded by a 900 ms delay period in which only the fixation point remained visible. The animals were required to release a lever as soon as a test stimulus identical to the cued sample appeared at the previously cued location. During task performance the epidural electrode array recorded field potentials from V4 and V1 (Fig. 1b). We refer to recorded activity patterns induced by the cued stimulus sequence as the “attended condition” (a-C) in contrast to the “nonattended condition” (n-C), where the recorded activity patterns were induced by the uncued stimulus sequence.

Average behavioral performance of monkeys during recording sessions was estimated from all but the longest trials in which a response would have been always correct. Disregarding fixation errors, the monkeys performed correct for 83.1% (monkey M) and 73.4% (monkey F) of the trials. Correct responses occurred 467 ms (M) and 418 ms (F) after target stimulus onset (median values). Errors were distributed approximately similarly over different initial figures.

Stimulus specificity of responses

In both monkeys the stimuli presented in the visual hemifield contralateral to the implanted array induced local field-potential responses recorded from visual areas V4 and V1 (Fig. 1b). The time–frequency plots in Figure 1, c and d, show that the increase of normalized power due to the stimuli depends on the frequency band and is most pronounced between 40 and 100 Hz.

To estimate the amount of information about the presented stimuli contained in the recorded data, we trained SVMs to discriminate the stimuli from the neuronal signals in individual trials (a-C). We based our estimate on the signal strengths computed by a wavelet analysis in 17 different logarithmically scaled frequency bands between 5 and 200 Hz (In the following we will denote the number of used frequencies by N_Freq and the number of used electrodes by N_Elec. Furthermore we denote by p_Chance the probability that the shown performance is explained by chance). Using from each electrode the wavelet coefficient averaged over the sample period (650–2200 ms after trial start) allowed us to identify 93.1% (N_Freq = 17; N_Elec = 37; p_Chance < 1 × 10⁻³²⁵) of the initial stimuli correctly for monkey F and 89.1% (N_Freq = 17; N_Elec = 36; p_Chance < 1 × 10⁻³²⁵) for monkey M in test datasets (see Materials and Methods). The chance levels for six different initial shapes were 18.1% for monkey F and 18.5% for monkey M, respectively. Inspection of classification performance of signals from individual electrodes revealed major contributions from two clusters of electrodes (Fig. 2; supplemental Fig. S1, available at www.jneurosci.org as supplemental material). One was located over area V4 and the other over area V1. Separate analysis of both clusters in the attended condition revealed that the most discriminative set of four electrodes in V4 (marked by yellow crosses in Fig. 2a) provided a classification rate of 64.2% (N_Freq = 17; N_Elec = 4; p_Chance < 5 × 10⁻³¹³) for monkey F and 54.8% (N_Freq = 17; N_Elec = 4; p_Chance < 5 × 10⁻²²⁹) for monkey M. Selecting the electrode cluster over V4 according to anatomical properties (by excluding the electrodes covering V1 and its proximity as defined by the measured retinotopic map) led to similar results. Within V1 a corresponding set of three electrodes (marked by green crosses in Fig. 2a) allowed for classification rates of 85.6% for monkey F (N_Freq = 17; N_Elec = 3; p_Chance < 1 × 10⁻³²⁵) and 81.6% for monkey M (N_Freq = 17; N_Elec = 3; p_Chance < 1 × 10⁻³²⁵). Signals of individual electrodes in V4 reached classification rates of up to 42.2% (N_Freq = 17; N_Elec = 1; p_Chance < 1 × 10⁻⁹⁶; monkey F) and 35.8% (N_Freq = 17; N_Elec = 1; p_Chance < 2 × 10⁻⁵⁹; monkey M), and in V1 up to 67.8% (N_Freq = 17; N_Elec = 1; p_Chance < 1 × 10⁻³²⁵; monkey F) and 49.1% (N_Freq = 17; N_Elec = 1; p_Chance < 2 × 10⁻¹⁶⁸; monkey M). Similar results were observed during presentation of the test stimuli following the sample stimulus.

Figure 2.

a, Classification performance P of the initial shape s(T1) (presented from t = 650 ms to t = 2200 ms, cf. Fig. 1a) for monkey F in the attended condition. P is shown in dependence on the position of the electrodes in the array (small circles). The performance level is color coded according to the bar shown to the right of the array. For the gray colored squares, classification performance did not differ significantly (p = 0.02) from the chance level of 18% (indicated by the black horizontal line in the color bar). Classification performance reaches peak values of 67.8% (N_Freq = 17; N_Elec = 1; p_Chance < 1 × 10⁻³²⁵) and 42.4% (N_Freq = 17; N_Elec = 1; p_Chance < 2 × 10⁻⁹⁶) in two regions corresponding to areas V1 and V4, respectively (cf. Fig. 1b). Gray arrows mark the “main” V4 electrode showing the highest performance. The combinations of electrodes in V4 selected for our further computational analysis are marked with yellow crosses (V1, green crosses). b, Difference in classification performance between attended and nonattended stimuli, same display as in a. The gray squares indicate electrodes where either the differences in performance deviated not significantly from zero, or where the performance under attention was not significantly different from chance level (p = 0.02).

Training the SVMs to classify whether the stimulus was attended or not resulted in a performance of 77.4% (N_Freq = 17; N_Elec = 4; p_Chance < 6 × 10⁻¹⁹¹; monkey F) and 77.0% (N_Freq = 17; N_Elec = 4; p_Chance < 7 × 10⁻⁹⁹; monkey M) for the V4 electrode combination (chance levels were 50% for monkey F, and 51% for monkey M). We used only data from 1400 to 2250 ms after trial onset for excluding influences from tagging the behaviorally relevant shape with green color in the beginning of the initial period. Taking these results together, the approach using SVMs demonstrates that field potentials recorded at the surface of the dura are highly specific for the individual stimuli processed in the cortical columns underneath the electrodes.

Attention improves classification performance

Comparison of the classification performance obtained for attended and nonattended stimuli revealed a clear difference in favor of the attended condition in area V4. For the selected combination of four electrodes, the classification rate raised significantly with attention during the initial period, from 55.5% (N_Freq = 17; N_Elec = 4; p_Chance < 3 × 10⁻²²⁰) to 64.2% (N_Freq = 17; N_Elec = 4; p_Chance < 5 × 10⁻³¹³) (p_Difference < 3 × 10⁻⁶, binomial test) and from 45.8% (N_Freq = 17; N_Elec = 4; p_Chance < 4 × 10⁻¹¹⁹) to 54.8% (N_Freq = 17; N_Elec = 4; p_Chance < 5 × 10⁻²²⁹) (p_Difference < 5 × 10⁻⁴) in monkeys F and M, respectively. For the most discriminative signal from a single V4 electrode, classification performance increased from 35.4% (N_Freq = 17; N_Elec = 1; p_Chance < 4 × 10⁻⁵⁶) to 42.2% (N_Freq = 17; N_Elec = 1; p_Chance < 1 × 10⁻⁹⁶) (p_Difference < 3 × 10⁻⁴) in monkey F. In monkey M, classification increased from 29.2% (N_Freq = 17; N_Elec = 1; p_Chance < 1 × 10⁻¹⁹) by 6.65% (p_Difference < 6 × 10⁻³) at one V4 electrode providing a peak performance of 35.8% (N_Freq = 17; N_Elec = 1; p_Chance < 2 × 10⁻⁵⁹). The absolute difference in classification performance for all single electrodes is shown in Figure 2b (and for monkey M, in supplemental Fig. S2a, available at www.jneurosci.org as supplemental material). Significant differences cluster around the highly discriminative region in V4. A few scattered electrodes also reach significant differences, but only for very low classification rates close to chance levels. No significant differences were observed for electrodes located over V1.

In line with the enduring requirement to attend to all stimuli in a sequence, an attention-dependent increment of classification performance in V4 was present not only in the initial period, but was found similarly during all test stimulus presentations. For the final test period, which is associated with a correct response, classification performance (V4 electrode combination) estimated in a 400 ms window starting with stimulus onset rose from 40.6% (N_Freq = 13; N_Elec = 4; p_Chance < 3 × 10⁻⁴⁶) to 49.1% (N_Freq = 13; N_Elec = 4; p_Chance < 8 × 10⁻¹⁵²; monkey F) (p_Difference < 2 × 10⁻⁴) and from 27.7% (N_Freq = 13; N_Elec = 4; p_Chance < 2 × 10⁻⁹) to 40.5% (N_Freq = 13; N_Elec = 4; p_Chance < 8 × 10⁻⁹²; monkey M) (p_Difference < 2 × 10⁻⁵). Chance levels were similar to those in the initial period. No such differences were found during delay periods.

The stability of stimulus-specific information and its attention-dependent enhancement over time is demonstrated by the time course of classification performance (see Fig. 3; supplemental Fig. S2, available at www.jneurosci.org as supplemental material). In particular, for the attended condition there is little indication that stimulus onset or offset contain much more information on stimulus identity than the static periods between them. This shows that stimulus-specific activity patterns occur continuously while stimuli are presented and are not specifically related to the transient caused by stimulus onset or offset. This raises the question of whether the specific signal patterns, which support identification of a presented stimulus from the recorded activity, are similar throughout the trial or whether they change over time. We investigated this issue by testing whether SVMs can successfully classify data from a period of the trial, which was not their training period. Therefore a first SVM was trained for the first 400 ms following initial stimulus onset, and a second one for a corresponding period following the onset of the last test stimulus preceding the repetition of the initial stimulus, which required the monkey's response. The results (Table 1) show that successful classification is possible not only for test data taken from the same period as the training data, but also for data from trial periods far apart in time. In general, the performance for different training and classification periods is comparable but somewhat smaller compared with the performance achieved with test and training data from the same period. This indicates that the characteristics of the signals supporting stimulus identification are stable over time.

Figure 3.

a, Time course of classification performance for the selected electrode combination above V4 (cf. Fig. 2a, yellow crosses), shown for the attended (red dotted line) and for the nonattended condition (blue dotted line) in monkey F. Data for the power coefficients in a frequency range between 5 and 200 Hz was taken from a range starting 200 ms before, and ending 200 ms after the times marked with the red and blue crosses, respectively. The black circles indicate a significant difference between the performances in both conditions (p = 0.02), while solid lines depict the chance level for the corresponding condition. In a, the SVMs were trained to classify the initial shape s(T1) presented to the monkeys during the period T1 shaded in light gray. Time t is measured relative to trial onset. In b and c, the SVMs were trained to classify the second-to-last and the last shape (target) displayed in the sequence, respectively (stimulus display periods are again shaded in light gray). Time t is measured relative to the onset of the second-to-last shape in b, and relative to the onset of the target shape in c.

Table 1.

Similarity of stimulus-specific activity patterns supporting classification along trials

The table shows classification performance for data from the period for which the SVM was trained, compared with classification performance on data from a different period (shaded in grey) obtained from the selected V4 electrode combination (a-C, monkey F). Corresponding values for monkey M (a-C) are shown in parentheses.

If this attention-dependent enhancement of the discriminability of cortical states is of behavioral relevance, we would expect that behavioral errors may result from failures of such enhancements. We tested this hypothesis by training a SVM and estimating the classification performance in trials, which ended with a behavioral error. For trials in which monkeys responded to a wrong test stimulus or failed to respond to the test stimulus matching the sample, classification performance in the stimulus period immediately preceding the erroneous response fell significantly under the level achieved in correct trials. In monkey F, classification performance for the electrode combination in V4 was reduced significantly from 49.1% (N_Freq = 13; N_Elec = 4; p_Chance < 8 × 10⁻¹⁵²) to 36.2% (N_Freq = 13; N_Elec = 4; p_Chance < 1 × 10⁻⁹) (p_Difference < 1 × 10⁻⁵), which is even less than the 40.6% observed for correct trials in which no attention was paid to the stimulus. Similarly in monkey M, classification performance fell from 40.5% (N_Freq = 13; N_Elec = 4; p_Chance < 8 × 10⁻⁹²) to 21.6% (N_Freq = 13; N_Elec = 4; p_Chance < 4 × 10⁻¹) (p_Difference < 2 × 10⁻⁵). No significant difference in classifying nonattended shapes was found between the trials with correct and wrong responses (monkey F). Performance for nonattended stimuli in trials with a correct response in monkey M was 27.7% (N_Freq = 13; N_Elec = 4; p_Chance < 8 × 10⁻¹⁵) and again there was no significant difference to performance in error trials.

A similar reduction of the classification performance in error trials was found also for the temporally much earlier initial period. Here performance reduced from 64.2% (N_Freq = 17; N_Elec = 4; p_Chance < 5 × 10⁻³¹³) to 51.9% (N_Freq = 17; N_Elec = 4; p_Chance < 1 × 10⁻⁴³) (p_Difference < 5 × 10⁻⁶) and from 54.8% (N_Freq = 17; N_Elec = 4; p_Chance < 1 × 10⁻³²⁵) to 41.5% (N_Freq = 17; N_Elec = 4; p_Chance < 6 × 10⁻¹¹) (p_Difference < 2 × 10⁻³) in monkeys F and M, respectively. These findings indicate a close relation between attention-dependent enhancements of the discriminability of the cortical states associated with different stimuli and the behavioral performance in the delayed-match-to-sample-task.

Characterization of stimulus-specific signals

To identify the signal components, which allow for discrimination between different stimuli we analyzed their distribution over different frequency bands. Based on data from the selected electrode combination over V4 of monkey F during presentation of the initial stimulus, we analyzed how the accuracy of classification depends on a selected interval of the frequency spectrum. Each entry in the matrix in Figure 4a and supplemental Figure S3a (available at www.jneurosci.org as supplemental material) reports the classification performance achieved for a given number of wavelet-coefficients (vertical axis) from a continuous frequency band terminating at the frequency indicated on the horizontal axis. The analysis reveals that most information on stimulus shape is contained in the frequency range above 40 Hz, while almost no information is contained in lower frequency bands. Close to maximal classification rates of 64.6% (N_Freq = 6; N_Elec = 4; p_Chance < 1 × 10⁻³¹⁷) correct are already possible if only six wavelet coefficients between 38 Hz and 122 Hz are used, and >59% (N_Freq = 2; N_Elec = 4; p_Chance < 1 × 10⁻²⁵¹) correct classification is still possible with the three coefficients having their center frequencies at 61, 76, and 96 Hz. The same pattern of results was found in both animals for the attended as well as for the nonattended condition (see supplemental data). The attention-dependent enhancement (Fig. 4b; supplemental Fig. S3b, available at www.jneurosci.org as supplemental material) is largest within the same frequency range.

Figure 4.

a, Classification performance P for monkey F, using different subsets of the power coefficients from the selected V4 electrode combination obtained during the initial period T1 (650–2200 ms after trial onset) under attention. Each square shows in color code the SVM's performance on a combination of successive frequency bins, whose total number is indicated by the index on the vertical axis, while their highest frequency bin is indicated with the horizontal axis. For example, the performance value shown in the square marked with a white circle was obtained using data from six frequency bins starting at 38 Hz, and ending with 122 Hz (white rectangle). The performance shown in the square marked with a gray cross was obtained using data from only three frequency bins at 61, 76, and 96 Hz (gray rectangle). b, Percentage of increase in classification performance under attention, for the same combinations of frequency bands as in a.

To test whether most of the information about different stimuli is contained in signal energy, we normalized the wavelet power coefficients from the V4 main electrode by their sum over all frequencies. In monkey F, classifying on the remaining features reduced performance from 42.2% (N_Freq = 17; N_Elec = 1; p_Chance < 6 × 10⁻⁹⁶) to 40.7% (N_Freq = 17, normalized; N_Elec = 1; p_Chance < 8 × 10⁻⁸⁶) in the attended condition [n-C: 35.4% (N_Freq = 17; N_Elec = 1; p_Chance < 1 × 10⁻⁵⁵) to 35.4% (N_Freq = 17, normalized; N_Elec = 1; p_Chance < 1 × 10⁻⁵⁵)]. In monkey M, performance was reduced from 35.8% (N_Freq = 17; N_Elec = 1; p_Chance < 6 × 10⁻⁵⁹) to 30.3% (N_Freq = 17, normalized; N_Elec = 1; p_Chance < 1 × 10⁻²⁹) in the attended condition [n-C: 29.2% (N_Freq = 17; N_Elec = 1; p_Chance < 7 × 10⁻²⁰) to 24.3% (N_Freq = 17, normalized; N_Elec = 1; p_Chance < 3 × 10⁻⁶)]. These numbers demonstrate that the major part of information about the shapes is preserved when removing all information about total signal energy. This leaves the possibilities that this information is either contained in total signal power of a specific frequency band or distributed differentially in spectral signal power.

We therefore quantified next how much information is contained in signal energy when selecting the most informative frequency range in terms of this feature. Thoroughly exploring all frequency intervals like in Figure 4, this range turned out to span from 61 to 96 Hz in monkey F, and from 76 to 122 Hz in monkey M. Classifying only on the total energy within these ranges reduced performance for monkey F from 42.2% (N_Freq = 17; N_Elec = 1; p_Chance < 6 × 10⁻⁹⁶) to 28.8% (N_Freq = 1; after summation N_Elec = 1; p_Chance < 1 × 10⁻²²) in the attended condition [n-C: 35.4% (N_Freq = 17; N_Elec = 1; p_Chance < 1 × 10⁻⁵⁵) to 29.0% (N_Freq = 1; after summation N_Elec = 1; p_Chance < 2 × 10⁻²⁵)]. No such reduction was observed for both conditions in monkey M.

In summary, these results demonstrate that the discernability of field potentials caused by different stimuli is based on stimulus-dependent differences of spectral activity patterns and overall energy in the gamma-band. In monkey F, information in the spectral patterns is even predominant over information in signal energy, while in monkey M, total signal energy selected from the frequency band 76 to 122 Hz is sufficient to explain shape classification performance.

In addition to the spectral distribution of information about shape, we also find a spatial distribution of information over electrode positions, as can be expected from a retinotopic mapping of a stimulus. To quantify this effect, we first removed spectral information by computing the average power amplitudes over all frequencies in selected frequency bands and classified the resulting data. For this purpose we whitened the power spectra by normalizing the power amplitudes by their grand averages over all experimental conditions. This prevents noise from the low-frequency channels to mask information in higher frequencies with lower spectral amplitudes. Second, we removed spatial information by computing the average power amplitudes over all electrodes in our selected set and again classified the resulting data. For monkey F, removing spectral information decreased classification performance from 64.2% to 35.7% (monkey M: 54.0% to 29.9%), while removing spatial information lead to a decrease from 64.2% to 40.8% (monkey M: 54.0% to 34.2%). Thus information is contained in both frequency and space, in an approximately equal proportion. The difference in information between the attended and nonattended condition follows the same scheme: For monkey F, removing spectral information decreased attentional classification gain from 8.7% to 5.2%, while removing spatial information lead to a decrease to 5.9%. For monkey M, the loss of spectral information decreased attentional classification gain from 12.4% to 6.2%, while removing spatial information lead to a decrease to 4.7%.

How did the signals change to become more distinct under conditions of selective attention? Distinguishing between two stimulus categories (termed “classes” in machine learning literature) is only possible when the recorded signals evoked by theses classes occupy different regions in data space. The number of errors made during a classification is related to the relative overlap between these regions (Fig. 5a). Errors can be reduced when the regions shrink in diameter (Fig. 5b), or when the distance between the centers of the regions increases (Fig. 5c). The first effect can be mediated by increasing the SNR, while the second effect can be achieved by a class-specific multiplicative scaling of the data during which the SNR remains constant. Of course these two effects and also more complicated statistical changes in the data could be combined to further reduce the number of classification errors, but it turns out that it suffices to quantify these two basic effects to explain most of the effects of attention in the data.

Figure 5.

Examples for classification problems in a two-dimensional data space spanned by the variables a(f₁) and a(f₂), which could represent the wavelet coefficients for two different frequency bands. The regions indicated by the blue ellipsoids in a symbolize data from two shapes A and B. In our case, these two shapes would correspond to ensembles of data points obtained by the repeated presentation of two shapes A and B, respectively. When a new data point in the shaded region is observed (green cross), any classifier trained on the previously observed data is likely to make an error because the data may belong to either of the two shapes. The total number of errors thus corresponds to the relative size of the shaded region where these shapes overlap. b, If attention would decrease the SNR, as indicated by the shape boundaries shrinking around their centers (red ellipsoids), the same observation could now unambiguously be attributed to shape A, thus reducing the classification errors (shaded region). c, If instead attention shifts the region centers (arrows), this change can also disambiguate the classification problem and reduce the number of errors (shaded region)—even when the SNR remains constant.

Investigating the wavelet coefficients we observe both small changes in the SNRs, η, and in the mean values μ. For the SNRs, we find an average change of 8η^A/η^N − 1〉 = 4 ± 7% (F) and −1 ± 9% (M), and for the mean values an absolute average change of 8|μ^A/μ^N − 1|〉 = 14.9 ± 10.4% (F) and 3.3 ± 4.2% (M) (A: attended, N: not attended; averages 8…〉 are taken over all frequency bands and shape classes) (compare also to supplemental Fig. S4a–c, available at www.jneurosci.org as supplemental material). Following our theoretical argument outlined above (cf. Fig. 5), both changes may be responsible for the enhanced performance under attention. However, from the average values it is not clear to which extent the small changes in mean SNR can explain the full attentional gain, and whether the class regions are really shifted away from each other rather than toward each other.

To quantify the effects of the changes in SNR and mean values separately, we thus performed two tests on the nonattended dataset: In the first test, we only changed the SNRs such that they matched the SNRs of the attended dataset, while holding the mean values of the dataset constant. A SVM was then trained and tested on this “quasi”-attended dataset to quantify how the separation of the shape classes improved through this transformation. In the second test, we only changed the mean values such that they matched the mean values of the attended dataset, while holding the SNRs of the dataset constant. Again we trained and tested a SVM on this second, “quasi”-attended dataset for quantifying the resulting improvement in shape discrimination (for details of this procedure, see Materials and Methods). For confirmation, a similar, “inverse” test was performed on the attended dataset. These data were transformed into two “quasi”-nonattended datasets, for which a successive SVM classification then quantified the resulting degradation in performance. Improvements and degradations in performance were finally compared with the real difference in classification performance on the original data, and expressed in percentages of these original differences being explained by the two scaling procedures.

For the V4 electrode combination scaling the SNR explained only 1.6% (F) and 28.7% (M) of the original increase in performance under attention, while scaling the mean values was capable to explain 108.4% (F) and 50.2% (M), respectively (compare also to the frequency-resolved plots in supplemental Figs. S6, S7, available at www.jneurosci.org as supplemental material). For the used epidural local field potentials, the result clearly indicates that attentional gain in performance is only to a minor extent caused by changes in SNR, but is to a large extent explained by shape-specific differential scaling of frequency components rendering the neural activity for different shapes more distinct from each other (cf. supplemental Fig. S4d–f, available at www.jneurosci.org as supplemental material). This finding reveals a new effect of attention acting on coherent neuronal activity in area V4.

Discussion

The present study has three main results: (1) Processing of different shape stimuli results in activity patterns that in single trials are surprisingly well distinguishable in the local field potential of area V4. (2) Selective attention substantially enhanced the stimulus-dependent differences of these neural activity patterns for the attended stimulus. (3) Behavioral failures went along with a reduction of classification performance. The components of the signal most discriminative for different shapes were contained in the γ-band above 40 Hz and their stimulus-specific characteristics stayed similar during different stimulation periods in a trial. The attention-dependent enhancement of stimulus discriminability cannot be explained by a simple increase of the SNR, but turns out to be most strongly related to a stimulus-specific differential scaling of the frequency components. This scaling results in an enhanced separation between the characteristic frequency patterns in the γ-band for different stimuli.

The enhanced discriminability under conditions of attention could in principle be traced back to two different changes in the signals. First, there is a small but statistically significant improvement of the signal-to-noise ratio. This finding is in line with a study by Mitchell et al. (2007) finding a small decrease in variance for inhibitory neurons in macaque area V4, and a study by McAdams and Maunsell (1999b) describing an attention-dependent improvement of the SNR for spike count data from area V4. In their data, the rise of the SNR resulted from the attention-dependent increase of stimulus responses together with a less than proportional increase of the standard deviation, which rises approximately only as the square root of the response. Together with the enhanced absolute difference between the responses for different stimuli being amplified by a stimulus-independent gain factor, the increased SNR resulted in an improved orientation discriminability. In contrast, increases of the SNR for the field potential data in the present study explained only a minor part of the entire enhancement of shape discriminability under attention. The major part of the effect is based on an attention-dependent increase of differences between responses to different stimuli, which allow for improved stimulus discriminability even though SNRs increased only very little.

Our results indicate that attention changes the spectral composition and spatial distribution of the coherent neural signals in different ways for different stimuli. Not only were these changes different for different stimuli, but in addition their direction was such as to increase distinctiveness of the respective neural activity patterns (in spatial and frequency composition) (see Fig. 5 for illustration). Arbitrary directions of changes would not necessarily have caused an improvement of stimulus discriminability. Thus the major part of the effect is therefore not explained by a uniform, stimulus-independent effect as in gain models for single-cell firing rate data, but by more complex changes in the composition and distribution of neural activity. While the underlying neuronal mechanisms are not known, two attention-dependent effects might be considered to contribute to such changes: (1) Feature-specific changes in attentional modulation have been observed in area MT (Martinez-Trujillo and Treue, 2004), which could play a significant role in scaling neural signals differentially depending on stimulus shape. However, such feature-based attention effects were measured for a different experimental design analyzing solely neuronal responses to the distracter (Martinez-Trujillo and Treue, 2004). In contrast to these experiments, our experimental design (but not the pattern of results) directly corresponds to studies, which observed explicitly no differential effects but a homogeneous gain effect for the orientation tuning of responses in the spike rate of single neurons (McAdams and Maunsell, 1999a). Consequently these known feature-based effects at the single neuron level are not expected to occur in the present paradigm and are therefore unlikely to be the bases of the differential effects observed. (2) Recruiting additional neuronal populations to encode a stimulus by dynamically changing their receptive field properties (Connor et al., 1997; Womelsdorf et al., 2006a) could also render signals from the neural population for different shapes more distinct from each other and change dynamic properties of the activated network. In any case, it must be noted that changes of firing rate of individual neurons do not simply translate into changes of oscillatory frequency in the local population (Gray et al., 1990).

What do these findings imply with respect to cortical stimulus processing and its dependence on attention? Classification in the present study depends on the pattern of frequencies in the local field potential caused by neural processing of different stimuli. While oscillatory field potential responses clearly lack the specificity of information contained in the activity of the contributing single neurons, they reflect a synchronized part of neuronal activity patterns (Elul, 1971; Nunez, 1995; Robbe et al., 2006). Synchronous activity is known to be particularly effective in driving other neurons (Segev and Rall, 1998; Usrey et al., 2000; Azouz and Gray, 2003; Bruno and Sakmann, 2006) and has therefore been implicated in structuring effective connectivity (Aertsen et al., 1989; Fries, 2005; Womelsdorf et al., 2007) and defining in a transient and flexible manner neuronal assemblies (von der Malsburg, 1985; Aertsen et al., 1986; Abeles, 1991; Singer and Gray, 1995; Kreiter and Singer, 1996). Previous studies already demonstrated that attention enhances specifically such oscillatory activity in the visual system (Fries et al., 2001, 2008; Bichot et al., 2005; Taylor et al., 2005; Womelsdorf et al., 2006a,b; Lakatos et al., 2008). In addition, our results show that with attention the structural organization of field potentials systematically changes, indicating an attention-dependent change of the dynamic state in the network of synchronized neurons processing the stimulus in area V4. The more distinct patterns of neural activity associated with processing of an attended compared with nonattended stimuli suggest a more differentiated and specific composition and state of synchronous neuronal assemblies if they process shape under conditions of attention. Such indications for enhanced modes of processing of attended stimuli are well in line with psychophysical findings demonstrating improved processing of attended stimuli and the particular importance of attention for shape perception (Rock and Gutman, 1981; Rock et al., 1992).

Further evidence for the behavioral relevance of the attention-dependent changes of cortical processing reflected by the observed changes in the pattern of field potentials comes from the reduced discriminability of signals preceding behavioral errors. In fact, our findings imply that one could predict the occurrence of behavioral errors from such less distinct signals. In summary, the present data provide evidence that selective attention improves processing of attended stimuli by enhancing the distinctiveness and discriminability of cortical network states involved in the representation and processing of individual stimuli already in cortical area V4.

While attention caused a clear improvement of the stimulus classification achieved for field potentials recorded from area V4, no significant effect was found for area V1. A likely reason for this lack of effect is the very high classification performance observed for the V1 recording sites already without attention. This very high performance is based on the much higher resolution in visual space achieved also by recording gamma-band responses in field potentials recorded over V1 compared with V4. Since all shape stimuli approximately fit into the RFs of V4 neurons, different stimuli cause only moderate differences of average activity in the stimulus driven, local V4 population. In contrast the RF size for gamma-band responses in the local field potentials of V1 (Eckhorn et al., 1993) is much smaller than the stimuli, even if recorded above the cortex (Rols et al., 2001). Together with the large cortical magnification factor of ∼3 mm/deg for V1, the activity induced by the different loops of a stimulus is spread out over multiple, spatially well separated columns with separate RFs for which independent gamma-band responses can be recorded from V1. Evidence for high spatial specificity of field potentials and in particular gamma-band responses has been provided previously with intracortical recordings indicating an extent of only a few hundred micrometer over which field-potentials are integrated (Engel et al., 1990; Liu and Newsome, 2006; Katzner et al., 2009; Nauhaus et al., 2009). It is therefore not surprising that epicortical recordings with a spacing of 3 mm can provide largely independent signals. This high spatial resolution of the shape stimuli in V1 allows for massive, shape-dependent differences in the coverage of separate RFs by different stimuli. Consequently strong differences of the overall activation of the cortical columns underneath a V1 electrode allow distinguishing stimuli very reliably, even if they are not attended. This high stimulus discriminability was in addition confirmed by a receiver operating characteristic analysis testing how well pairs of two stimuli can be distinguished by a single frequency component from one electrode (for details see supplemental Fig. S5, available at www.jneurosci.org as supplemental material). Performing this analysis for all possible combinations of stimulus pairs and frequency components for V1 revealed that almost 30% (monkey F) or 17% (monkey M) of combinations permitted a 90–100% correct differentiation between two stimuli. The presence of these simple, almost perfectly discriminative signals already in the nonattended condition strongly reduces the possibility to observe in V1 any further attention-dependent classification enhancements based on different, more complex indicators of attention-dependent changes in cortical network states. In contrast, in V4 there was no combination, which allowed for such a high classification performance, which leaves room to observe substantial attention-dependent improvements of classification. Thus our results do not exclude the possibility that similar changes as in V4 could also be observed for V1, if stimulus size would be as similar with respect to the small V1 RFs as it was with respect to the large RFs of V4.

The high stimulus discriminability achieved with local field potentials recorded from the surface of the dura is also of interest in the context of brain–machine interfaces (BMIs). In our study the electrode arrays were carried over years and recordings were pooled from recording sessions over several weeks. While BMIs based on single-unit or multiunit recordings (Wessberg et al., 2000; Taylor et al., 2002) typically need an initial calibration for each session, the recordings for the present study are sufficiently stable to allow for demanding stimulus discriminations with the same classifier over months. This is even more remarkable since the stimuli were not constructed to be easily distinguishable, but to require considerable effort by the monkeys for successful discrimination. The findings therefore suggest that the comparatively simple shapes of letters and many symbols could be detected with high reliability in the spatial distribution of field potentials recorded from visual cortex.