Spatial summation revealed in the earliest visual evoked component C1 and the effect of attention on its linearity

Abstract

In natural scenes, multiple objects are usually presented simultaneously. How do specific areas of the brain respond to multiple objects based on their responses to each individual object? Previous functional magnetic resonance imaging (fMRI) studies have shown that the activity induced by a multiobject stimulus in the primary visual cortex (V1) can be predicted by the linear or nonlinear sum of the activities induced by its component objects. However, there has been little evidence from electroencephelogram (EEG) studies so far. Here we explored how V1 responded to multiple objects by comparing the EEG signals evoked by a three-grating stimulus with those evoked by its two components (the central grating and 2 flanking gratings). We focused on the earliest visual component C1 (onset latency of ∼50 ms) because it has been shown to reflect the feedforward responses of neurons in V1. We found that when the stimulus was unattended, the amplitude of the C1 evoked by the three-grating stimulus roughly equaled the sum of the amplitudes of the C1s evoked by its two components, regardless of the distances between these gratings. When the stimulus was attended, this linear spatial summation existed only when the three gratings were far apart from each other. When the three gratings were close to each other, the spatial summation became compressed. These results suggest that the earliest visual responses in V1 follow a linear summation rule when attention is not involved and that attention can affect the earliest interactions between multiple objects.

object recognition is a basic function of the visual system. In natural scenes, objects are usually clustered. Despite this, the majority of previously conducted studies have focused only on how a single object is processed in the visual system. As a result, although important to computational neuroscience, it is still unclear how the visual system responds to multiple objects based on its responses to each individual object. When computational models were established to predict the response of neurons or brain areas to a multiobject stimulus, they often assumed that the response to this multiobject stimulus could be predicted as the sum of the individual responses to each of its components (Dumoulin and Wandell 2008; Walther et al. 2005). However, there has been little evidence for this summation, even in the earliest visual cortical area V1. Single-unit recording studies have focused on extrastriate areas (Gawne and Martin 2002; Luck et al. 1997; Oleksiak et al. 2011; Recanzone et al. 1997; Zoccolan et al. 2005) and showed that in these areas neuronal responses to a multiobject stimulus could be predicted by either the weighted average or the maximum response to its constituent components. However, because the receptive fields of neurons in V1 are too small to cover multiple objects, very few electrophysiological studies have tried to tackle how V1 responds to multiple objects at the single unit level. Some researchers have tried to address this question at the neuronal population level with functional magnetic resonance imaging (fMRI) (Hansen et al. 2004; Kay et al. 2013; Pihlaja et al. 2008; Vanni et al. 2005), but their results were conflicting: while some of them found that the voxel responses in V1 were well predicted by linear spatial summation (Hansen et al. 2004), others reported compressive spatial summation in V1 (Kay et al. 2013; Pihlaja et al. 2008; Vanni et al. 2005).

In this study, we examined whether the activation of V1 follows a linear spatial summation rule at the neuronal population level by comparing the electroencephelogram (EEG) signals evoked by three gratings with the summed EEG signals evoked separately by the central grating and two flanking gratings. As V1 is more sensitive to simple visual features such as orientation, using gratings as stimuli will help us tackle the linear summation problem at the early stage of visual processing. The results with gratings will shed light on the results with high-level objects in natural scenes. Similar to fMRI, EEG reflects the postsynaptic potentials of a population of neurons (Luck 2005). However, as EEG has a higher temporal resolution than fMRI, it can provide more precise temporal information about the evolution of spatial summation [although fMRI can also be used to detect subtle difference in the onset timing with careful designs (e.g., Chen et al. 2010)]. We focused on the earliest visual evoked event-related potential (ERP) component C1 (Jeffreys and Axford 1972) because it has been shown to reflect the feedforward signals in V1 (Bao et al. 2010; Clark et al. 1994; Di Russo et al. 2003; Martinez et al. 1999). Since voltages in the brain can directly summate (Nunez and Srinivasan 2006), if the population responses of neurons in V1 follow linear spatial summation, the C1 amplitude of three gratings should equal the sum of the C1 amplitudes of its components. Otherwise, the former would be smaller because of mutual suppression between neighboring gratings (Kastner et al. 1998; Luck et al. 1997).

Considering that spatial attention may influence the potential suppressive interaction between neighboring objects (Chen et al. 2014), we conducted the EEG study in both attended (i.e., participants attended to the stimulus position) and unattended (i.e., participants attended away from the stimulus position) conditions. One important property of C1 is that the C1 evoked by a stimulus in the upper visual field has a negative magnitude whereas the C1 evoked by a stimulus in the lower visual field has a positive magnitude. To confirm the validity of the ERP component C1 we examined and the generalizability of our effect, we performed the same test in both the upper (experiment 1) and lower visual fields (experiment 2).

METHODS

Participants

Twenty-five participants (12 males, 13 females) participated in experiment 1, and 21 participants (13 males, 8 females) participated in experiment 2. One participant's data (male) in experiment 1 and two participants' data (1 male and 1 female) in experiment 2 were discarded due to strong alpha waves in their EEG signals (Luck 2005). All participants were right-handed and reported normal or corrected-to-normal vision. Ages ranged from 18 to 25. All participants gave written informed consent in accordance with the procedures and protocols approved by the human participants review committee of Peking University.

Stimuli and Procedure

Experiment 1.

All stimuli consisted of circular sinusoidal gratings (diameter = 2.36°; spatial frequency = 2.54 c/d; full contrast; mean luminance = 61.47 cd/m²). The background had the same luminance as the mean luminance of the gratings. The orientation of the grating in the center was either +45° or −45° while the orientations of the two flanking gratings were independently and randomly selected from 0° to 180° for each trial.

Five stimulus configurations were used: one grating (One), two close gratings (Two_close), two distant gratings (Two_distant), three close gratings (Three_close, created by combining the one grating with the two close gratings in space), and three distant gratings (Three_distant, created by combining the one grating with the two distant gratings in space) (Fig. 1A). The center-to-center distance between close gratings was 2.48°, and the distance between distant gratings was 5.07°. The stimulus was centered at 8° eccentricity in the upper left visual quadrant. The fixation was a point at the center of the screen. A chin rest was used to stabilize the head position. All visual stimuli were displayed on a ViewSonic color graphic monitor (refresh rate: 75 Hz; resolution: 1,024 × 768; size: 22 in.) with a gray background at a viewing distance of 73 cm.

Fig. 1.

Stimuli and procedures of experiment 1 and experiment 2. A: 5 figure configurations were used as stimuli. B: procedure of a trial in both the attended (i.e., attended to the stimulus quadrant) and unattended (i.e., attended away from the stimulus quadrant) sessions of experiment 1. In the attended session, participants were instructed to judge the orientation of the grating in the upper left visual field of the second stimulus, while in the unattended session, participants judged the orientation of the grating in the lower right visual field of the second stimulus. C: procedure of experiment 2 was identical to that of experiment 1. The only difference was in the positions of the first and second stimuli.

Each trial began with a stimulus (the first stimulus) presented in the upper left visual field for 500 ms. This stimulus was randomly selected from the five stimulus configurations. After a blank interval (jittered between 200 and 400 ms), two gratings (the second stimulus) with orientations slightly different from the vertical were presented for 100 ms. One was presented in the same position as the central grating of the first stimulus (i.e., upper left visual field), and the other was presented in the diagonally opposite position to the central grating of the first stimulus (i.e., lower right visual field). These two gratings were to attract participants' attention to the quadrant that was the same as the quadrant of the first stimulus or to the quadrant that was diagonally opposite to the quadrant of the first stimulus. Specifically, in the attended session, participants were instructed to judge the orientation of the upper left grating (left or right, relative to vertical orientation) of the second stimulus, which attracted their attention to the same quadrant as the first stimulus. In the unattended session, participants were instructed to judge the orientation of the lower right grating of the second stimulus, which attracted their attention to the diagonally opposite quadrant of the first stimulus. Two dashed circles were always presented on the screen to indicate the positions of the two gratings (Fig. 1B). It should be noted that the procedures in both sessions were identical. The tasks in both sessions were stimulus-irrelevant (i.e., irrelevant to the first stimulus), preventing participants from selectively attending to a specific stimulus configuration. The difference between the orientation of task-relevant grating (the upper left grating in the attended session or the lower right grating in the unattended session) of the second stimulus and the vertical orientation was adjusted to keep participants' performance level at ∼80% correct.

The attended and unattended sessions were performed on different days in a counterbalanced order across participants. The color of the fixation point was red or green to indicate whether a session was attended or unattended (also counterbalanced across participants), respectively. There were 20 blocks in each session. Each block consisted of 100 trials, 20 trials for each of the 5 stimulus configurations, presented in a random order. Therefore, for each stimulus configuration, there were 400 trials in total. Although we did not record the orientation of the flanking gratings for each condition, the orientations of the flanking gratings in the two-grating conditions (i.e., Two_close and Two_distant) and those in the three-grating conditions (i.e., Three_close and Three_distant) should have been balanced, given that the orientation of the two flanking gratings was independently and randomly selected from 0 to 180° on each trial and there were 400 trials for each condition. To prevent eye movements towards the target location, all subjects were trained to maintain fixation before they started the EEG experiments. We repeatedly emphasized the importance of maintaining fixation throughout the experiment. The eye movement data of four naïve subjects were collected when they performed the same experiment with the same procedure. The maximum deviation from the fixation point for all subjects was <1°, which suggests that even naïve subjects can well maintain their gaze position at the center of the screen.

Experiment 2.

The aim of this experiment was to replicate the results of experiment 1 with stimuli in the lower visual field. Therefore, the stimuli and procedure of experiment 2 were identical to those of experiment 1, and only the stimulus positions differed. That is, in experiment 2, the first stimulus was in the lower left visual quadrant. One of the gratings of the second stimulus was in the lower left visual field. The other was in the upper right visual field (Fig. 1C).

Recording

Scalp EEG was recorded from 64 Ag/AgCl electrodes positioned according to the extended international 10–20 EEG system. Vertical electro-oculogram (VEOG) was recorded from an electrode placed above the right eye. Horizontal EOG (HEOG) was recorded from an electrode placed at the outer canthus of the left eye. Electrode impedance was kept below 5 kΩ. EEG was amplified with a gain of 500 K, band pass filtered at 0.05–100 Hz, and digitized at a sampling rate of 1,000 Hz. The signals on these electrodes were referenced online to the nose and were re-referenced offline to the average of two mastoids.

EEG Analysis

Only the EEG signals induced by the first stimulus were analyzed. Offline data analysis was performed with Brain Vision Analyzer (Brain Products, Munich, Germany). The EEG data were first low-pass filtered at 30 Hz and then epoched starting at 100 ms before the stimulus onset and ending 300 ms after stimulus onset. Each epoch was baseline-corrected against the mean voltage of the 100-ms prestimulus interval. The epochs contaminated by eye blinks, eye movements, or muscle potentials exceeding ±50 μV at any electrode were excluded from the average. The remaining epochs were averaged for each stimulus configuration. To select electrodes for the amplitude and latency analyses, grand-averaged ERPs were made by averaging signals across participants and stimulus configurations but separately for the attended and unattended sessions. The five electrodes with the largest C1 amplitudes were chosen for further analysis. To quantify the C1 amplitude and latency of each stimulus configuration for each participant, the waveforms across these five electrodes were first averaged to acquire an average waveform. Then, the mean amplitude of the 11 sampling points around the C1 peak of the averaged waveform was measured as the C1 amplitude. The peak time point of the averaged waveform between 50 and 90 ms was measured as the C1 latency.

Estimation of the dipole sources was performed with the BESA algorithm (BESA research 6.0), as described by Clark and Hillyard (1994). The C1 component was modeled based jointly on the grand-averaged waveforms elicited by all five stimulus configurations. The waveform in the 5-ms interval around the peak point (between 80 and 84 ms in both experiments) was simulated with one dipole with free location and orientation.

For comparison, we also examined the spatial summation effect in the ERP component following C1. When the stimulus was in the upper left visual field (experiment 1), the following component was P1 with its peak amplitude in the right parietal occipital scalp sites. It is believed that P1 reflects extrastriate activation (Di Russo et al. 2002; Martinez et al. 1999). When the stimulus was in the lower left visual field (experiment 2), the following component in posterior scalp sites was N150, which has been shown to have a source in the ventral extrastirate cortex (Di Russo et al. 2002). The same method was used to measure the amplitudes and latencies of P1 and N150.

RESULTS

Experiment 1: Upper Visual Field

Behavioral results.

In the attended session, participants discriminated the orientation of the upper left grating of the second stimulus. This was to attract participants' attention to the quadrant where the first stimulus was presented. We did not ask participants to respond to the first stimulus directly because in that case their attention level might differ due to stimulus complexity difference. The response accuracies of the five configuration conditions were as follows: One, 77.4 ± 0.89%; Two_close, 82.3 ± 0.82%; Three_close, 80.3 ± 0.71%; Two_distant, 83.2 ± 0.86%; and Three_distant, 80.4 ± 0.82%. The main effect of the stimulus was significant [repeated-measures ANOVA, F_(4,92) = 4.36, P = 0.003]. The accuracies in stimulus conditions with one grating in the center (One, Three_close, and Three_distant) were significantly smaller than those without a grating in the center (Two_close and Two_distant) [paired t-test, all t(23) > 2.43, P < 0.03]. This is probably because the stimuli with a central grating served as forward masks to the upper left grating of the second stimulus. However, the main effect of distance was not significant [repeated-measures ANOVA, F_(1,23) = 0.127, P = 0.725], which suggests that participants were equally involved in the task in both the close and the distant grating conditions.

In the unattended session, participants discriminated the orientation of the lower right grating of the second stimulus. The response accuracies of the five configuration conditions were as follows: One, 81.4 ± 0.87%; Two_close, 82.5 ± 0.86%; Three_close, 82.3 ± 0.85%; Two_distant, 81.8 ± 0.95%; and Three_distant, 82.3 ± 0.93%. The main effect of the stimulus was not significant [repeated-measures ANOVA, F_(4,92) = 1.44, P = 0.227], which suggests that participants were equally involved in all conditions. Taken together, these behavioral results suggest that any ERP difference between close and distant grating conditions cannot be attributed to different levels of cognitive involvement.

ERP results.

The second stimulus was to attract participants' attention to a specific quadrant. We only analyzed signals evoked by the first stimulus. To get the topography of C1, we averaged the ERPs of all five stimulus configurations for the attended and unattended sessions separately. Consistent with previous studies (Bao et al. 2010; Clark et al. 1994), the C1 evoked by stimuli in the upper left visual field had the largest amplitude in the left occipital parietal scalp sites (Fig. 2A, upper left quadrant of unattended and attended panels). The five electrodes with the largest C1 were chosen for further analysis. They were CP1, CP3, P1, P3, and P5 in both the attended and unattended conditions (Fig. 2A, within the black ellipse). Figure 2B shows the waveforms for each of the five stimulus conditions separately, averaged across all participants and five electrodes. The C1 peak latency was between 80 and 84 ms after stimulus onset.

Fig. 2.

Event-related potential (ERP) results for the attended and unattended sessions in experiment 1. A: upper left quadrant of each panel shows the C1 topographies in response to the 1st stimulus averaged over all 5 stimulus conditions and participants. Posterior electrodes, including CP1, CP3, P1, P3, and P5 (within the black ellipse), had the largest C1 amplitudes. The other 3 quadrants show the location of a single dipole that best accounted for the variance in the C1 scalp voltage distribution. B: ERPs averaged over the 5 electrodes and all participants for each stimulus condition. C1s are indicated by black arrows.

To examine whether linear spatial summation existed for close and distant gratings in the attended and unattended sessions, we added peak amplitude of the C1 induced by one grating (i.e., One) to that induced by two gratings (i.e., Two_close or Two_distant) and compared the summed peak with the peak amplitude of the C1 induced by three gratings (Three_close or Three_distant; Fig. 3A). It should be noted that these three gratings overlapped the positions of the one grating and two gratings. In the unattended session, regardless of the distance between gratings, the summed amplitude (C1_One + C1_Two) was not significantly different from the C1 amplitude of three gratings (C1_Three) [C1_One + C1_Two vs. C1_Three: close, t(23) = −1.69, P = 0.10; distant, t(23) = −0.53, P = 0.60], which suggests that in the unattended condition, C1 follows a linear spatial summation rule. However, in the attended session, although the linear summation still existed for distant gratings [C1_One + C1_Two vs. C1_Three, t(23) = −1.51, P = 0.14], C1_Three was significantly smaller than the sum for close gratings [C1_One + C1_Two vs. C1_Three, t(23) = −5.71, P < 10⁻⁶], indicating suppressive interactions between close gratings. An alternative way, however, is to sum up the waveforms of one grating and two gratings and then compare the peak of the C1 of the summed waveforms with the peak of C1 induced by the three gratings (Miller et al. 2015). In the current study, there was no significant latency difference between different conditions [paired t-test, all t(23) < 1.76, P > 0.092]; therefore, this method should produce similar results to our method. We also analyzed the data with this method, and indeed, similar results were observed.

Fig. 3.

Attentional modulation on the interactions between gratings in experiment 1. A: overview of the C1 amplitudes and the comparisons of C1_One + C1_Two and C1_Three in all conditions. The difference between C1_One + C1_Two and C1_Three was only significant for close gratings in the attended session. B: suppression index, defined as (C1_One + C1_Two) − C1_Three in all conditions. Attention increased the suppressive interactions between close gratings, but not distant gratings. **P < 0.01, statistically significant difference between stimulus conditions. Error bars denote means ± 1SE across participants.

We defined suppression index as (C1_One + C1_Two) − C1_Three to further examine how distance and attention influence the interactions between gratings (Fig. 3B). The suppression index should be zero if the signals follow a linear summation rule and should be negative if the signals are subadditive. Seventeen out of the 24 participants showed a negative suppression index in the close condition when the stimuli were attended, but fewer participants showed a negative suppression index in the other three conditions (14 in the distant condition when the stimuli were attended, 11 and 12 in the close and distant conditions, respectively, when the stimuli were not attended). Repeated-measures ANOVA showed that the interaction between attention and distance was significant [F_(1,20) = 18.83, P = 0.003]. Paired t-test showed that attention increased the suppressive interactions between close gratings [t(23) = −2.91, P = 0.008] but not distant gratings [t(23) = −0.58, P = 0.56]. As C1 has a peak latency of 80 - 84 ms after stimulus onset, these results suggest that spatial attention increased the suppressive interactions between close objects, but not distant objects, as early as 80 ms after stimulus onset.

To examine exactly when the linear summation ended and subadditivity started to emerge, we first calculated the suppression index at each time point for each condition and each subject separately, then compared the suppression index with zero at each time point across subjects. We adopted a criterion of at least 20 consecutive time points (i.e., 20 ms) that are smaller than 0 at a P < 0.05 level (one-tailed t-test; Chen et al. 2009) to detect intervals that showed a significant difference between suppression index and zero. This criterion corresponds to a P value of <0.038 corrected with the method proposed by Guthrie and Buchwald (1991). The first time point of such intervals indicated the emergence of subadditivity. We found that when the stimuli were attended, suppressive interaction emerged at 40 ms after stimulus onset in the close condition but did not emerge until 140 ms after stimulus onset in the other conditions (i.e., when the stimuli were not attended or when the distance between nearby gratings was distant). These results suggest that subadditivity emerges as early as 40 ms after stimulus onset depending on both the attentional modulation and the distance between the nearby gratings.

Although previous results have provided strong evidence that C1 originates mainly from V1, we performed dipole source analysis with BESA to confirm the cortical source of C1 in our experiment. The analysis was performed on the C1s averaged over all five stimulus configurations but for the attended and unattended sessions separately. We found that a single dipole in right V1 (Talairach coordinate: 16, −88, 7) accounted for 94.3% of the variance of C1 in the attended session, and another dipole also in right V1 (Talairach coordinate: 13, −82, 9) accounted for 95.9% of the variance of C1 in the unattended session. Therefore, the C1s observed here were mainly generated by V1. The positions of both dipoles are shown in the upper right, lower left, and lower right quadrants of unattended and attended panels of Fig. 2A. Taken together, these results suggest that when the stimulus is not attended, spatial summation in V1 at the population level is linear. However, when the stimulus is attended, spatial summation in V1 is linear only when the components of the stimulus are far from each other. In other words, when the stimulus is attended, whether or not the spatial summation in V1 is linear depends on the distance between the components of the stimulus.

As there has been much controversy as to whether or not attention can modulate the earliest visual activity reflected in C1 (Kelly et al. 2008; Martinez et al. 1999; Rauss et al. 2009), we also examined the influence of attention on the amplitude and latency of C1 evoked by a single stimulus. The main effect of attention on C1 amplitudes of the five stimulus configurations was not significant, and there was no significant amplitude difference between C1s obtained in the unattended and attended sessions for any stimulus configurations [repeated-measures ANOVA, F_(1,23) = 0.41, P = 0.52; paired t-test, all t(23) < 1.76, P > 0.092]. This was also the case for the main effect of attention on C1 latencies [repeated-measures ANOVA, F_(1,23) = 3.37, P = 0.08; paired t-test, all t(23) < 1.95, P > 0.063]. The lack of attentional modulation on the activity of a single stimulus was consistent with previous results (Luck et al. 1997; Moran and Desimone 1985). However, given that attention did modulate the suppressive interaction between gratings that were close to each other, the nonsignificant effect of attention on individual stimulus could be a result of insufficient statistical power. It is very possible to obtain a significant effect if we include more subjects or more trials.

For comparison, we also investigated whether or not the P1 component, which appears right after C1 and is believed to reflect extrastriate activation (Martinez et al. 1999), also follows linear summation. The five electrodes with the largest P1 amplitude were P4, P6, P8, PO4, and PO8. The P1 peak latency was between 106 and 122 ms after stimulus onset (Fig. 4). The main effect of attention on P1 latency was marginally significant [F_(1,23) = 4.181, P = 0.052], and the main effect of stimulus configuration on P1 latency was significant [F_(4,92) = 4.051, P = 0.005]. Because there were latency differences between conditions, it was not reasonable to sum up the peak amplitudes of P1s induced by one grating and those induced by two gratings. Instead, we summed up the waveforms first and then compared the peak of P1 component of the summed waveform (P1_sum) with the peak of the waveform induced by three gratings (P1_Three) as Miller et al. (2015) did. In the unattended condition, P1_Three was smaller than P1_sum for far gratings but not for near gratings [P1_sum vs. P1_Three: close, t(23) = 1.645, P = 0.11; distant, t(23) = 2.69, P = 0.013]. However, as the amplitude of P1_one was very small and the differences between P1s induced by two and three grating conditions were also very small, it is unclear whether or not the spatial summation reflected in P1 also followed a linear summation rule when the stimuli were not attended. In the attended condition, P1_Three was smaller than P1_sum regardless of the distance between gratings [P1_sum vs. P1_Three: close, t(23) = 5.24, P < 0.001; distant, t(23) = 3.63, P = 0.001]. Therefore, the linear spatial summation relationship of P1 did not exist when the stimuli were attended. In addition, consistent with previous results (Di Russo et al. 2003; Fu et al. 2010; Heinze et al. 1994; Mangun et al. 1998; Martinez et al. 1999; Woldorff et al. 1997), we found that the amplitude of P1 evoked by a single stimulus was significantly enhanced by attention [main effect of attention, F_(1,23) = 10.25, P = 0.004; paired t-test, all P < 0.02 except for the Two_distant condition, t(23) = 1.00, P = 0.32].

Fig. 4.

Waveforms averaged across the 5 electrodes with the biggest P1 amplitudes (P4, P6, P8, PO4, and PO8) for each of the 5 stimulus conditions in both the unattended and attended sessions in experiment 1. P1s are indicated by black arrows.

Experiment 2: Lower Visual Field

One typical property of C1 is that its polarity reverses when the stimulus location changes from one visual field to another (upper vs. lower). That is, a stimulus in the upper visual field evokes a negative C1 while a stimulus in the lower visual field evokes a positive C1. To confirm that the conclusions of experiment 1 were not specific to the upper visual field, we replicated experiment 2 in the lower visual field. Specifically, in experiment 2, the first stimulus was in the lower left visual field; the two gratings of the second stimulus were in the lower left and upper right visual fields, respectively (Fig. 1C).

Behavioral results.

In line with experiment 1, we compared the orientation judgment accuracies in all conditions to confirm that participants did not selectively attend to specific stimulus distance conditions (close vs. distant grating conditions). In the attended session, participants discriminated the orientation of the grating of the second stimulus in the lower left visual field. The accuracies in the five configuration conditions were as follows: One, 80.3 ± 1.71%; Two_close, 83.4 ± 1.29%; Three_close, 82.1 ± 1.64%; Two_distant, 83.5 ± 1.37%; and Three_distant, 80.7 ± 1.65%. The accuracies in stimulus conditions with the central grating (One, Three_close, and Three_distant) were significantly smaller than those without the central grating (Two_close and Two_distant; paired t-test, all P < 0.04). However, as predicted, the main effect of distance (close or distant) was not significant [F_(1,23) = 1.39, P = 0.25].

In the unattended session, participants discriminated the orientation of the cuing grating in the upper right visual field. The accuracies in the five configuration conditions were as follows: One, 85.4 ± 1.93%; Two_close, 85.7 ± 1.90%; Three_close, 85.6 ± 1.79%; Two_distant, 86.6 ± 1.95%; and Three_distant, 86.7 ± 1.74%. The main effect of distance (close or distant) was not significant [F_(1,23) = 0.85, P = 0.37].

ERP results.

Consistent with previous studies (Bao et al. 2010; Clark et al. 1994), the C1 of the stimulus in the lower left visual field had the largest amplitude in the right posterior occipital scalp sites and the amplitude was positive (color map in Fig. 5A). The five electrodes with the largest C1 amplitudes were P2, P4, P6, PO4, and PO8 (indicated by the black ellipses in Fig. 5A). The peak latencies of the C1s averaged across participants for five stimulus configurations were between 80 and 84 ms after stimulus onset.

Fig. 5.

ERP results for the attended and unattended sessions in experiment 2. A: upper left quadrant of each panel shows the C1 topographies in response to the 1st stimulus averaged over all 5 stimulus conditions and participants. Posterior electrodes, including P2, P4, P6, PO4, and PO8 (within the black ellipse), had the largest C1 amplitudes. The other three quadrants show the location of a single dipole that best accounted for the variance in the C1 scalp voltage distribution. B: ERPs averaged over the 5 electrodes and all participants for each stimulus condition. C1s are indicated by black arrows. N150s are indicated by gray arrows.

Due to the fact that neither the main effect of attention [F_(1,18) = 0.06, P = 0.809] nor the main effect of stimulus configuration [F_(4,72) = 0.805, P = 0.526] on C1 latency was significant, we analyzed the data using similar methods as in experiment 1. We found that when the stimuli were not attended, C1 followed linear spatial summation regardless of the distance between gratings [C1_One + C1_Two vs. C1_Three: close, t(18) = 1.42, P = 0.17; distant, t(18) = 1.10, P = 0.29]. However, when the stimuli were attended, C1_Three was significantly smaller than C1_One + C1_Two for close gratings [t(18) = 3.63, P = 0.002] but not for distant gratings [t(18) = 0.24, P = 0.81]. This suggests that there were suppressive interactions between close gratings but not between distant gratings when the stimuli were attended (Fig. 6A). The suppression index was also defined to examine how distance and attention influence the interactions between gratings (Fig. 6B). Unlike experiment 1, the suppression index should be positive because the C1 was positive. Fifteen out of the 19 participants showed a positive suppression index in the close condition when the stimuli were attended, but fewer participants showed a positive suppression index in the other three conditions (10 in the distant condition when the stimuli were attended, 12 and 9 in the close and distant conditions, respectively, when the stimuli were not attended). Repeated-measures ANOVA showed that the interactions between attention and distance was significant [F_(1,18) = 4.57, P = 0.046]. A paired t-test showed that the increase of suppressive interactions caused by attention was close to significant between close gratings [t(18) = 2.08, P = 0.051] but was far from significant between distant gratings [t(18) = −0.669, P = 0.512]. When the stimuli were attended, significant suppressive interaction (i.e., subadditivity) emerged at 58 ms after stimulus onset in the close condition but at 95 ms in the distant condition. When the stimuli were not attended, regardless of the distance between nearby gratings, there was no suppressive interaction until 150 ms after stimulus onset. All of these results replicated the main results that we reported in experiment 1.

Fig. 6.

Attentional modulation on the interactions between gratings in experiment 2. A: overview of the C1 amplitudes and the comparisons of C1_One + C1_Two and C1_Three in all conditions. The difference between C1_One + C1_Two and C1_Three was only significant for close gratings in the attended session. B: suppression index, defined as (C1_One + C1_Two) − C1_Three, in all conditions. Attention increased the suppressive interactions between close gratings, but not distant gratings. **P < 0.01, statistically significant difference between stimulus conditions. Error bars denote means ± 1SE across participants.

The cortical sources of the C1 in both attended and unattended sessions were on right V1 (unattended session: Talairach coordinate, 19, −90, −9; 87.8% of the C1 variance was accounted for; attended session: Talairach coordinate, 18, −92, −10; 86.5% of the C1 variance was accounted for). The positions of both dipoles are shown in the upper right, lower left, and lower right quadrants of unattended and attended panels in Fig. 5A.

The main effect of attention on C1 amplitude was not significant [F_(1,18) = 1.90, P = 0.19]. No amplitude difference for any stimulus configuration between the attended and unattended conditions was significant [all pairs, t(18) < 2.03, P > 0.05]. Again, because attention did modulate the suppressive interaction between gratings that were close to each other, the nonsignificant effect of attention on individual stimulus could be a result of insufficient statistical power.

With lower visual field stimuli, the component following C1 was a negative component N150 with its peak amplitude in the right posterior occipital cortex. The generator of this component is believed to be in the ventral extrastriate cortex (Di Russo et al. 2002). The latency of N150 was between 140 and 180 ms (Fig. 5B, indicated by gray arrows). The main effect of attention on the latency of N150 was not significant [F_(1,18) = 1.322, P = 0.265], but the main effect of stimulus configuration was significant [F_(4,72) = 12.42, P < 0.001]. On average, the latency in One condition was ∼10 ms later than in the other conditions. The N150 in Three_distant condition had the shortest latency. In this case, it is not reasonable to sum up the peak amplitudes of single grating and two gratings. Therefore, we summed up the waveforms first and then compared the peak of the summed waveform (N150_sum) with the peak of the N150 induced by three gratings (N150_three). In both the attended and unattended conditions, N150_three was less negative than N150_sum regardless of the distance between gratings [all t(18) >3.25, P < 0.005].

DISCUSSION

By comparing the C1 induced by a three-grating stimulus with the sum of the C1s induced by its components (i.e., the central grating of this three-grating stimulus and the two flanking gratings of this three-grating stimulus), we found that when the stimulus was not attended, the C1 evoked by a three-grating stimulus was roughly identical to the sum of the C1s evoked by its components. Moreover, cuing attention to the stimuli's positions made the C1 amplitudes in the close condition subadditive but did not affect the linear summation in the distant condition. These findings were observed in both the upper and lower visual fields. The subadditivity emerged as early as 40 ms after stimulus onset when the stimuli were presented in the upper visual field and as early as 58 ms when the stimuli were presented in the lower visual field. In addition, BESA source analysis revealed that the variance of C1 could be largely accounted for by a single dipole in V1, suggesting that C1 was mainly generated in V1. Taken together, we conclude that 1) the earliest visual evoked component C1, which reflects the population responses of neurons in V1, follows linear spatial summation when the stimulus is not attended; and 2) attention can modulate the interactions between objects in V1 as early as 80 ms after stimulus onset, especially when the objects are close to each other in space.

It should be noted that although a similar design has been used in our previous study (Chen et al. 2014), in which we also provided evidence that spatial attention can modulate the earliest interactions between multiple gratings, the current study is not a simple replication of our previous study. The current study was designed to examine whether the earliest visual signal reflected in C1 follows a linear spatial summation rule, whereas the previous study was designed to investigate the neural mechanisms of crowding. Due to the purpose difference, we asked participants to perform different tasks in these two studies. As the previous study was designed to examine the neural mechanisms of crowding (i.e., the deleterious influence of the flankers on the recognition of a target), participants were asked to perform a target-related task (i.e., responding to the target orientation) in the attended session. The task was more difficult for the close condition than for the distant condition. Although, to our knowledge, no evidence has shown that task difficulty influences the earliest visual signals, it is still worth testing a stimulus-irrelevant task (such as what we used in the current study that participants respond to the second instead of the first stimulus). In this case, the task difficulty difference between different conditions would not affect our results. Moreover, in the current study, we conducted experiments in both the upper and lower visual fields, which provided more convincing support to our conclusions.

Implications for Spatial Summation in V1

Our results have important implications in understanding how the visual system integrates its responses to individual objects to generate responses to a multiobject stimulus (i.e., spatial summation). In previous research, most of the single-unit studies have focused on extrastriate areas because the receptive fields of V1 neurons are too small to cover multiple objects. They showed that in V2 (Luck et al. 1997), V4 (Gawne and Martin 2002), V7a (Oleksiak et al. 2011), IT (Zoccolan et al. 2005), and MT (Recanzone et al. 1997), neuronal responses to multiple stimuli can be predicted by either the weighted average or the maximum of the responses of the constituent stimuli. Some researchers have proposed even more complicated algorithms, such as divisive inhibition (Britten and Heuer 1999; Simoncelli and Heeger 1998). In any case, these results suggest that spatial summation in extrastriate areas follows nonlinear rules (maximum, weighted average, or divisive inhibition).

Although it is difficult to explore how an individual neuron in V1 responds to multiple objects, we can examine how neurons in V1 respond to multiple objects at the neuronal population level with fMRI. Hansen et al. (2004) assessed the linearity of spatial summation by comparing the activations to checkerboard wedges and rings with sums of activations to their component patches and found that the responses of voxels in V1 were well predicted by linear spatial summation (but also see Pihlaja et al. 2008 and Vanni et al. 2005). However, a recent study (Kay et al. 2013) found that suppressive spatial summation was observed in V1 and grew more pronounced in relatively anterior extrastriate areas. This is consistent with the previous fMRI finding that V1 showed the smallest difference between sequential presentation and simultaneous presentation among V1–V4 (Kastner et al. 1998). In the higher-level category-selective visual areas, such as Fusiform Face Area (FFA) and Parahippocampal Place Area (PPA), Reddy et al. (2009) found that the fMRI signals to simultaneously presented categories can be predicted by the weighted average of signals to two individually presented categories. To sum up, although conflicting, among areas from V1 to V4 and other higher level visual areas, V1 has been shown to have the most similar response patterns to linear spatial summation.

Our linear summation results revealed in C1 are consistent with the aforementioned fMRI result (Hansen et al. 2004). This in turn provides compelling support to consider C1 as a measure of early visual signals in V1. Moreover, the high temporal resolution of EEG ensures that our results are less likely to be caused by feedback signals from higher level cortical areas, compared with the fMRI results. Our results showed that although linear spatial summation does exist in V1, this linear relationship is conditional: it depends on both the attentional state of the participants and the spatial layout of the stimuli. When attention is not involved, or when the attended objects are far from each other, V1 exhibits linear summation behavior; however, when the attended objects are close to each other, linear summation disappears.

In addition, our results showed that linear summation occurs as early as 80 ms after stimulus onset but does not persist after 122 ms, i.e., the linear summation results were not observed in P1 or N150 in our experiments. As C1 reflects the activity in V1, and P1 and N150 reflects the activity in extrastriate visual cortex (V2, V3, etc.), this difference again suggests that the linearity of spatial summation disappears gradually from striate to extrastriate cortex, which is consistent with previous evidence (Miller et al. 2015). Our results are also consistent with previous magnetoencephalography (MEG) (Supek et al. 1999) and electrocorticography (ECoG) results (Winawer et al. 2013). Specifically, Supek et al. (1999) observed linear spatial summation up to 150 ms after stimulus onset with MEG. Winawer et al. (2013) reported that the stimulus-locked component of ECoG responses has an approximate linear spatial summation, but the broadband asynchronous component of ECoG responses is subadditive. They suggested that the stimulus-locked component of ECoG reflects a brief, transient response to contrast, similar to our C1 response, whereas the broadband component reflects a longer sustained response that covers several transient periods, similar with our later ERP components such as P1 and N150.

Implications for the Neural Mechanisms of Attention

Our results also have important implications for the neural mechanisms of spatial attention. On one hand, whether or not attention can modulate C1 amplitude has long been a controversy (Frey et al. 2010; Kelly et al. 2008; Martinez et al. 1999). The weak influence of attention on C1 amplitude found in our study is consistent with the previous results (Frey et al. 2010; Kelly et al. 2008; Martinez et al. 1999). However, given that attention did modulate the suppressive interaction between gratings that were close to each other, the nonsignificant effect of attention on individual stimulus might just be a result of insufficient statistical power. On the other hand, the strong modulation of attention on early interactions between multiple objects revealed in C1 suggests that attention can modulate interactions between objects in V1 at a very early stage. Most previous electrophysiological and human fMRI studies only showed that interactions in extrastriate areas could be modulated by attention (Kastner et al. 1998). Although our previous study (Chen et al. 2014) and a recent study by Miller et al. (2015) provided evidence that attention can modulate the interaction between objects, there are limitations in these studies. For example, Miller et al.(2015) did not include an unattended condition in their study; therefore, their results could not address whether or not suppressive interactions exists between objects when the stimuli were not attended. In addition, as we explained earlier, compared with our previous study, our current study has a more appropriate design for examining the effects of attention (i.e., there were no task difficulty differences between different stimulus configurations) and provided converging evidence from both the upper and lower visual fields that the interactions between neighboring objects can be modulated by attention as early as 80 ms in V1. Therefore, our findings will add to the current understanding of attention modulation.

It should also be noted that our finding on increased interactions between close gratings by spatial attention does not run contrary to the previous finding that selective attention decreases sensory interactions (Desimone and Duncan 1995; Kastner et al. 1998). One significant difference in our study was that participants did not attempt to suppress the influence of flanking gratings because they responded to the second stimulus instead of the first stimulus. It is possible that the previous finding on selective attention (i.e., selective attention decreases the interactions between multiple stimuli) and our finding regarding spatial attention (i.e., spatial attention increases interactions) revealed independent processes of attention. For example, they might explain why we have difficulty in identifying a central target among multiple objects at first glance (because spatial attention increases interactions at the earliest stage of visual cortical processing), but after focusing on the target position for a while, we can easily identify the target (because selective attention is employed to filter out irrelevant information at a later stage).

In sum, we provide converging evidence through the earliest ERP component C1 that the early visual signals at the population level follow a linear spatial summation rule and that spatial attention can affect the linearity of this summation when the multiple objects are close to each other. One limitation of our experiment is that we only tested three objects. It is worth testing how visual signals pool when there are more objects. In addition, we did not manipulate the relationship between the orientations of nearby gratings. It has been shown that the direction (inhibition or facilitation) of interactions between nearby gratings depends on their collinearity and contrast (Polat et al. 1998). In our study, the orientation of the target and all flankers were randomly selected for each trial and all gratings had full contrast. We did not analyze how collinearity affected the direction of interactions. Instead, we pooled the effects of inhibition and facilitation, which showed that the overall effect between nearby full-contrast gratings was inhibition. Nevertheless, our comparison between the amplitude of C1 induced by a multiobject stimulus and the sum of the amplitudes of C1s induced by its component objects provides a novel method for measuring cortical interactions between multiple items using ERPs. In the future, researchers could test how collinearity and contrast of gratings affect the timing of facilitation or inhibition between nearby gratings. One could also use this method to explore interactions between high-level visual stimuli such as animals, faces, or houses.

GRANTS

This work was supported by National Natural Science Foundation of China (NSFC) Grant 31230029, Ministry of Science and Technology Grant 2015CB351800, and NSFC Grant 31421003.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

Author contributions: J.C., Q.Y., and F.F. conception and design of research; J.C., Q.Y., Z.Z., and Y.P. performed experiments; J.C. and Q.Y. analyzed data; J.C. and F.F. interpreted results of experiments; J.C. prepared figures; J.C., Q.Y., and F.F. drafted manuscript; J.C., Q.Y., and F.F. edited and revised manuscript; J.C., Q.Y., Z.Z., Y.P., and F.F. approved final version of manuscript.