N2ac: An ERP Component Associated with the Focusing of Attention Within an Auditory Scene

Abstract

Humans must often focus attention onto relevant sensory signals in the presence of simultaneous irrelevant signals. This type of attention has been explored in vision with the N2pc component, and the present study sought to find an analogous auditory effect. In Experiment 1, two 750-ms sounds were presented simultaneously, one from each of two lateral speakers. On each trial, participants indicated whether one of the two sounds was a pre-defined target. We found that targets elicited an N2ac component: a negativity in the N2 latency range at anterior contralateral electrodes. We also observed a later and more posterior contralateral positivity. Experiment 2 replicated these effects and demonstrated that they arose from competition between attended and unattended tones rather than reflecting lateralized effects of attention for individual tones. The N2ac component may provide a useful tool for studying selective attention within auditory scenes.

Introduction

The environment provides us with an enormous amount of simultaneous sensory information. The computational challenges involved with processing multiple simultaneous inputs has been well recognized in the visual attention literature, but it is also a significant problem for the auditory environment. Although we may not take note of the buzzing electronics, the scuffing of the shoes, the clicking of a mouse across the room, the whistling of an individual as he exits the elevator, and the interchange between the barista and her patron, all of these sounds may be simultaneously present in a typical auditory scene. But what happens if we want to focus on a single conversation and ignore all of the other acoustic signals that are mixed together with this conversation? Pulling one signal out of a mixture of simultaneous competing signals may involve attentional mechanisms that are simply unnecessary under the conditions usually used to study auditory attention, in which the attended and ignored stimuli are presented sequentially. However, the problem of attending to one of several simultaneous sources of information may be the most vexing problem faced by individuals as they age (Tun, 1998), and it is not easily solved by a hearing aid that simply adjusts the gain or frequency composition of the input reaching the cochlea (Verschuure & van Benthem, 1992).

These general issues has been extensively studied in the visual attention literature, in which several common behavioral tasks involve the simultaneous presentation of attended and unattended stimuli (e.g., Eriksen, 1995; Navon, 1977; Wolfe, 1998) and in which theories focus extensively on solving the computational problems that arise in these tasks (Desimone & Duncan, 1995; Treisman, 1988; Tsotsos, 1995; Wolfe, 2007). Event-related potentials (ERPs) have also been useful for studying this issue, because a specific ERP component has been identified that reflects the focusing of attention onto one item in the presence of simultaneous distractors. Specifically, the N2pc (N2-posterior-contralateral) component is observed over the visual cortex contralateral to the visual field containing the attended stimulus (Luck & Hillyard 1994a, 1994b; Luck, in press).

Figure 1 A shows the basic setup of an N2pc experiment. The voltage over the left hemisphere is more negative from approximately 200-300 ms when the target appears in the contralateral (right) visual field than when the target appears in the ipsilateral (left) hemisphere, and a complementary pattern is observed over the right hemisphere. By collapsing across the left and right hemispheres and forming a contralateral-minus-ipsilateral difference wave, it is possible to isolate brain activity that reflects the focusing of attention independent of any overall differences between left and right targets or between the left and right hemispheres (just as an analogous subtraction is used to isolate the motor preparation associated with the lateralized readiness potential, Smulders & Miller, in press). The contralateral nature of the N2pc is key: without this contralaterality, it would be impossible to isolate attention-related activity from the mixture of components elicited by the simultaneously presented attended and unattended objects in the stimulus array.

Figure 1.

A) Typical N2pc paradigm and results. With the stimuli shown here, subjects would be instructed either to detect the presence of a specific color (e.g., the color red) or to discriminate a feature of the object drawn in that color (e.g., the position of the gap on the red square). For either of these instructions, an N2pc component is observed contralateral to the item of the attended color. That is, the voltage is more negative when the attended item is contralateral to the electrode site compared to when it is ipsilateral. Eye movements are not permitted, so the N2pc reflects a covert shift of attention rather than an overt shift of gaze. B) Translation of the visual N2pc paradigm to an analogous auditory paradigm. In the auditory paradigm, pairs of stimuli were presented simultaneously, with one sound from each speaker, and subjects listened for a specific target sound that could occur in either speaker. The waveforms shown here are grand averages over the anterior cluster of electrode sites (F3, F7, C3, T7, F4, F8, C4, T8). For visual clarity, these waveforms were filtered offline with a low-pass Gaussian filter with a half-amplitude cutoff at 50 Hz.

The N2pc component has been used widely to study a variety of different visual spatial attentional topics. These include capture of attention by salient task-irrelevant stimuli (Eimer & Kiss, 2007), effects of reward on the orienting of attention (Kiss, Driver, & Eimer, 2009), the binding problem (Luck, Chelazzi, Hillyard, & Desimone, 1997), inhibition of return (McDonald, Hickey, Green, & Whitman, 2009), and dual-task interference (Dell'Acqua, Sessa, Jolicoeur, & Robitaille, 2006). An analogous ERP component for auditory stimuli could be similarly useful in understanding how attention influences the perception of auditory scenes, but the appropriate experiments have not been conducted to identify this component. The current experiments were designed to search for this auditory analog of the N2pc.

It is not clear from previous research whether an auditory analog of the N2pc component might exist. First, almost all previous ERP studies of auditory attention have involved sequential rather than simultaneous presentation of attended and ignored stimuli, and the mechanism of attention reflected by the N2pc appears to operate primarily or solely under conditions of simultaneous competition (see Luck & Hillyard, 1994b). Though some studies in the last decade have started to look at complex sound scenes and even simultaneously presented sounds (Rao, Zhang, & Miller, 2010), many of these studies have done so by looking at complex harmonics or mistuned harmonics (Alain, Arnott, & Picton 2001; Johnson, Hautus, & Clapp, 2003; Dyson, Alain & He, 2005; McDonald & Alain, 2005), or concurrent steady-state responses (Bidet-Caulet & Bertrand, 2005, 2009; Bidet-Caulet et al. 2007;). Most sounds in these experiments were presented to both ears and lack the spatial separation that one would need to see an auditory analog of the N2pc effect (but see Deouell, Bentin, & Giard, 1998; Bidet-Caulet & Bertrand, 2005; 2009; Bidet-Caulet et al. 2007 for exceptions) or did not involve searching for a target in an unknown location (Paavilainen, Saarinen, Tervaniemi, & Näätänen, 1995; Näätänen, Paavilainen, Tiitinen, Jiang, & Alho,1993). Second, the N2pc component is defined as the difference in amplitude between the focusing of attention on contralateral and ipsilateral stimuli (relative to a given electrode site), and it cannot be isolated from the rest of the ERP waveform unless the processing is sufficiently contralateral. The auditory cortex is not as lateralized as the visual cortex, so a contralateral-minus ipsilateral-difference may not be visible in the auditory modality. Third, it simply may not be the case that selective attention behaves in a similar way in the auditory system as the visual system: The two modalities encounter very different types of stimuli (both spatially and temporally) and may require different methods of attention. Thus, the presence or absence of an auditory analog of the N2pc component is a completely open question.

Some evidence suggests that the auditory system may be lateralized sufficiently to show an N2pc-like effect. In a PET study by Alho et al. (1999), participants were instructed to pay attention to a specific ear (either the right or left) in a standard dichotic listening task and indicate when they heard a deviant tone. Alho et al. (1999) found large activation clusters in the hemisphere contralateral to the attended ear, regardless of which ear was being attended. Jäncke, Specht, Shah, & Hugdahl (2003) found a similar effect in an fMRI study, showing increased activation in the auditory cortex contralateral to the attended ear. Although these studies suggest the kind of lateralization pattern in the auditory cortex that would be necessary to find an N2pc-like effect, the lateralized effects in these studies may not reflect an N2pc-like process. In particular, these effects may have consisted of a sustained change in contralateral activity over the entire block reflecting preparatory processes rather than a transient contralateral response that occurred when a potential target was detected in the stimuli.

Many other imaging studies have failed to observe this sort of lateralization, instead observing a left hemisphere dominance regardless of which ear received the sounds (Devlin et al., 2003). Others have shown substantial contralaterality for monaurally presented stimuli but not for binaurally presented stimuli (Woldorff et al., 1999). It has also been suggested that the location of processing in the auditory pathway may be a determinate of lateralization (Schönwiesner, Krumbholz, Rubsamen, Fink, & Von Cramon, 2007).

The above evidence suggests that there is some potential for a contralaterally dominant response to auditory stimuli. Findings in imaging studies may not translate well to ERPs, but some studies using monaural stimuli have shown enhanced responses in the contralateral hemisphere (Wolpaw & Penry, 1977; Peronnet, Giard, Bertrand, & Pernier, 1984; Mäkelä et al., 1993). Many of these studies used passive task conditions, in which subjects did not actively attend to any of the stimuli, but Perrault & Picton (1984) used an attention task and found that attention increased the amplitude of what they called the N1c component, contralateral to the ear of stimulation. Thus, although the data are mixed, there is good reason to believe that the auditory system exhibits sufficient contralaterality with monaural stimuli that an N2pc-like effect could potentially be observed with binaural stimuli. Indeed, Johnson et al. (2003), using a dichotic masking technique, found enhanced activity contralateral to the perceived pitch location, even though there was no actual lateralization of the presented complex sounds. Similarly, Deouell et al. (1998) found strong contralateral responses in a passive mismatch negativity (MMN) study in which two distinct sounds were presented simultaneously to two ears.

Whereas most previous ERP studies of auditory attention have cued participants to attend to a particular location, simulating the natural situation in which one sustains attention over time to sounds arising from a given spatial location, we have simulated a different natural situation in which an observer is searching for a particular type of sound (e.g., a child's voice) and does not know in advance where the sound is located. Specifically, each stimulus contained two sounds that began and ended simultaneously (one from the left speaker, one from the right), with the location of the target varying unpredictably across trials (see Figure 1B). This is analogous to the standard N2pc design shown in Figure 1A, except that a single item was present on each side (as in some previous N2pc studies, such as Eimer, 1996 and Luck, Girelli, McDermott & Ford, 1997).

Two experiments were conducted. In both experiments, the two stimuli presented on each trial were randomly selected from a set of four sounds, one of which was defined as the target for a given block of trials. Participants were asked to report whether the target sound was present in each bilateral stimulus pair. Different types of sounds were used in the two experiments so that we could assess the replicability and generality of the results. In addition, the second experiment also contained unilateral stimuli so that we could assess whether the contralateral responses obtained for bilateral stimuli reflected additional attentional processes that are needed to resolve competition between simultaneously presented stimuli. Both experiments yielded a negativity in the N2 latency range at anterior contralateral electrode sites, which we are calling N2ac. This pattern of lateralization was not observed for the unilateral stimuli, and it appears to reflect attentional processes that arise when an attended stimulus faces competition from a simultaneous unattended stimulus. In addition, we observed a later contralateral positivity over posterior scalp regions.

Experiment 1

Methods

Participants

We recorded from 14 participants between the ages of 18 and 30 years; all participants reported normal hearing and no history of neurological disorders or diseases. Individuals provided informed consent before starting the experiment and were paid for their participation.

Stimuli and Procedure

Stimuli were created digitally at a 44100 Hz sampling rate using Audacity software and were presented on Cambridge Soundworks Ensemble III speakers. Four different stimuli were used, each 750 ms in duration: a pure tone of 440 Hz, a white noise burst, a click train (impulses at 10 Hz), and a frequency sweep (from 100 Hz to 800 Hz). All stimuli, except the click train, had a 10-ms rise time and fall time. Two of these stimuli were presented simultaneously on each trial, selected at random without replacement from the set of four possible sounds. Each sound was presented on a separate speaker, resulting in one sound presented on the right and a different sound on the left. The stimulus onset asynchrony was 1500 ± 150 ms (rectangular distribution). Stimulus intensity was approximately 50 dB SPL for each individual sound, as measured with an Extech Instruments 407730 sound meter. Testing occurred in an Eckel C-15A sound attenuating chamber with an electrical shielding package.

For each block of trials, one of the four stimuli was randomly designated the target. Because the four stimulus types were combined with independent probabilities, with two of the four stimuli presented simultaneously from different speakers on each trial, the target was present in 50% of the stimulus pairs. On each trial, the subject pressed one of two buttons to indicate whether the target sound was present or absent. Speed and accuracy were equally stressed. The location of the target was not explicitly task relevant but it was assumed that subjects would focus their attention onto the location of the target in the course of identifying it.

Each stimulus was the target in two different blocks, leading to a total of 8 blocks that were run in random order. Each block consisted of 156 trials, yielding a total of 1248 trials across the entire experiment (including 312 targets presented on each side).

Participants sat in an electrically shielded chamber 70 cm away from a computer screen, which was encased in a Faraday cage, and 171 cm away from each of two laterally placed speakers. The speakers were located at approximately ±35 degrees of azimuth and approximately 19 degrees of elevation relative to the subject's head. Participants were instructed to keep their eyes on a fixation cross presented on the computer screen throughout the experiment.

Recording and Analysis

EEG was recorded using the Biosemi ActiveTwo system. We recorded from 32 Ag/AgCl scalp sites (Fp1/2, Fz, F3/4, F7/8, Cz, C3/4, T7/8, Pz, P1/2, P3/4, P5/6, P7/8, P9/10, POz, PO3/4, PO7/8, Oz, O1/2, Iz), along with left and right mastoid sites, horizontal electrooculogram (HEOG) sites at the outer right and left canthi, and vertical electrooculogram (VEOG) sites above and below the right eye. All channels were recorded in single-ended mode, low-pass filtered using a fifth order sinc filter with a half-power cutoff at 204.8 Hz, and then digitized at 1024 Hz with 24 bits of resolution. The data were high-pass filtered offline using a zero phase-shift Butterworth filter with a half-amplitude cutoff at 0.01 Hz and re-referenced to the average of the left and right mastoids.

Per standard lab procedure, we excluded subjects for whom more than 25% of trials were contaminated by artifacts; two subjects were excluded for this reason. An average of 18.5% of trials were removed because of artifacts in the remaining 12 participants.

We collapsed the waveforms across target type (click train, pure tone, white noise, and FM sweep) to avoid physical stimulus confounds in the analyses presented here. Visual inspection of the waveforms for the individual target types revealed that the results were similar across target types, except for the FM sweep, which was slightly delayed in time.

The contralateral attention effect was isolated by creating a difference wave, consisting of the average of the contralateral waveforms (right hemisphere electrodes for targets on the left, and left hemisphere electrodes for targets on the right) minus the average of the ipsilateral waveforms (right hemisphere electrodes for targets on the right, left hemisphere electrodes for targets on the left) for target-present trials only. Trials with incorrect behavioral responses were excluded.

Because this is a new paradigm, and we had no a priori information about the timing and scalp distribution of the effects, our statistical analyses began with an omnibus test that included multiple electrode sites and multiple time periods. One set of analyses included the data from a cluster of anterior electrode sites (F3/4, F7/8, C3/4, T7/8), where auditory system ERPs are typically largest; another included the data from a cluster of posterior electrode sites (P1/2, P3/4 P5/6, P7/8, P9/10, PO3/4, PO7/8, O1/2), where attentional control signals and crossmodal effects might be observed. From each of these electrodes, we measured the mean amplitude from the contralateral-minus-ipsilateral difference waves over consecutive 100-ms time intervals between 200 and 600 ms. The ANOVA for a given group of electrodes (the anterior set or the posterior set) thus included factors of Time (100-200, 200-300, 300-400, 400-500, 500-600), Hemisphere (left, right), and Site (the different sites within a cluster, as defined earlier). The data were measured from contralateral-minus-ipsilateral difference waves; therefore the significance of the overall attention effect was tested as the difference between the overall mean amplitude of the contralateral-minus-ipsilateral difference relative to zero rather than as an explicit ANOVA factor. The Huynh-Feldt correction for nonsphericity was used in all analyses; all p values reported here have been corrected.

Results and Discussion

Behavioral Results

Participants reported the target as present or absent with 93.7% accuracy, with an average response time of 539.6 ms (SD 106.5 ms) for correct target-present responses and 610.6 ms (SD 111 ms) for correct target-absent responses. A paired sample t-test indicated that this difference was significant (t(12)=-7.89, p<0.001).

Electrophysiology at the Anterior Electrode Cluster

Figure 1B shows the left hemisphere, right hemisphere, and combined waveforms averaged across the anterior electrode cluster. Figure 2 shows the contralateral, ipsilateral, and contralateral-minus-ipsilateral difference waveforms, with the upper waveforms averaged across the anterior electrode cluster and the lower waveforms averaged across the posterior electrode cluster. We will first consider the anterior cluster. In this cluster, the voltage contralateral to the target was more negative than the voltage ipsilateral to the target from approximately 200-500 ms after stimulus onset. This contralateral-minus-ipsilateral difference mainly overlapped the N2 peak, and we therefore call it the N2ac (N2-anterior-contralateral) component. This is meant as a purely descriptive term, denoting the timing and scalp distribution of the effect and highlighting the fact that it was isolated using the same approach used to isolate the N2pc component. This label is not meant to imply that the N2ac component reflects the same type of computational function as the N2pc component (although this is a possibility).

Figure 2.

Grand average of contralateral, ipsilateral, and contralateral-minus-ipsilateral waveforms for Experiment 1. Anterior waveforms are collapsed across electrode sites F3/4, F7/8, C3/4, T7/8. Posterior waveforms are collapsed across electrode sites P1/2, P3/4, P5/6, P7/8, P9/10, PO3/4, PO7/8, O1/2. For visual clarity, the waveforms shown here were filtered offline with a low-pass Gaussian filter with a half-amplitude cutoff at 50 Hz. The raw contralateral and ipsilateral waveforms use the 2.0 μV scale. The contralateral-minus-ipsilateral difference waveforms use the 1.0 μV scale.

The detailed results of the statistical analyses of the anterior and posterior electrode clusters are presented in Tables 1 and 2, respectively. These tables provide the F and p values, so the main text will provide these values only for follow-up analyses not shown in these tables.

Table 1.

Summary of statistical analyses for the anterior electrode cluster in Experiments 1 and 2, including Huynh-Feldt corrected p values

	Experiment 1		Experiment 2
	Bilateral Targets		Bilateral Targets		Bilateral Nontargets		Unilateral Targets		Unilateral Nontargets
	F Value	P Value	F Value	P Value	F Value	P Value	F Value	P Value	F Value	P Value
Overall N2ac (difference from zero)	-14.01	0.003*	-1.79	0.204	0.570	0.464	0.586	0.457	1.85	0.197
Time	3.944	0.031*	5.108	0.004*	2.081	0.130	0.643	0.560	1.831	0.181
Hemisphere	0.384	0.548	0.040	0.920	4.599	0.51	0.325	0.579	6.495	0.024*
Time × Hemisphere	0.525	0.640	0.085	0.935	0.479	0.652	0.415	0.663	0.668	0.524
Electrode site	0.631	0.600	0.801	0.574	1.647	0.129	0.513	0.804	0.967	0.465
Time × electrode site	1.510	0.155	1.002	0.443	0.777	0.647	0.541	0.804	0.705	0.630
Hemisphere × Electrode site	1.006	0.399	0.984	0.446	0.684	0.667	1.764	0.100	1.562	0.148
Time × Hemisphere × Electrode site	3.74	0.859	0.797	0.656	0.664	0.760	0.562	0.805	1.628	0.099

Table 2.

Summary of statistical analyses for the posterior electrode cluster in Experiments 1 and 2, including Huynh-Feldt corrected p values

	Experiment 1		Experiment 2
	Bilateral Targets		Bilateral Targets		Bilateral Nontargets		Unilateral Targets		Unilateral Nontargets
	F Value	P Value	F Value	P Value	F Value	P Value	F Value	P Value	F Value	P Value
Overall N2ac (difference from zero)	2.715	0.128	3.87	0.071	10.523	0.006*	0.727	0.409	1.72	0.212
Time	7.103	0.004*	15.319	0.000*	5.513	0.009*	2.262	0.108	0.824	0.429
Hemisphere	0.220	0.649	0.000	0.988	5.221	0.040*	3.293	0.093	3.708	0.076
Time × Hemisphere	0.070	0.947	0.365	0.662	2.671	0.067	0.041	0.953	0.717	0.494
Electrode site	2.718	0.019*	2.313	0.069	0.452	0.815	0.604	0.734	1.774	0.112
Time × electrode site	1.435	0.209	1.471	0.171	0.692	0.693	0.825	0.621	1.392	0.192
Hemisphere × Electrode site	0.432	0.836	0.300	0.844	1.955	0.100	1.058	0.377	1.353	0.253
Time × Hemisphere × Electrode site	0.580	.768	0.986	0.459	1.064	0.393	0.490	0.842	1.250	0.284

The overall N2ac contralaterality effect was significantly different from zero in the anterior electrode cluster, indicating that attending to a particular target sound within a bilateral pair led to a significantly more negative potential over the hemisphere contralateral to the target than over the ipsilateral hemisphere. Note that this effect did not reflect a sensory lateralization because the target could be either on the left or on the right for a given physical stimulus pair. Instead, this contralaterality reflected an attention-based interaction between the observer's task-defined attentional set (i.e., the representation of the target for that trial block) and the location of a given physical stimulus within a stimulus pair.

It should be noted that it is not possible to determine from this study whether the N2ac effect reflects a greater negativity over the contralateral hemisphere, a greater positivity over the ipsilateral hemisphere, or both. Given the general contralateral organization of the brain, however, it is reasonable to provisionally assume that this effect is primarily a contralateral negativity until contrary evidence is obtained.

A significant main effect of Time was also obtained, indicating that the contralateralipsilateral difference was significantly larger during some intervals than others. Specifically, this difference was most pronounced in the 200-300 and 300-400 ms time periods, falling off later in time. Follow-up ANOVAs conducted on the individual 100-ms time periods indicated that the difference was significant (p < .05) from 200-300, 300-400, and 400-500 ms, but not from 500-600 ms. Some contralateral-ipsilateral difference was also evident in the waveforms from 100-200 ms, but this difference did not approach significance in a test of this interval (F < 1). Thus, the N2ac effect occurs at an intermediate stage of processing, beginning after the N1 wave but peaking prior to the P3 wave. This timing indicates that the brain is able to detect and localize the target sound from the mixed bilateral input by the 200-300 ms latency range.

We found no significant hemispheric effects or interactions; that is, although the voltage was more negative for contralateral than ipsilateral stimuli at a given electrode site, the magnitude of this difference did not vary significantly between the left and right hemispheres.

We also found no significant electrode site effects or interactions. In other words, we were unable to detect any differences among the anterior electrodes in the amplitude of the contralateral-ipsilateral difference. A more detailed description of scalp topography will be presented following Experiment 2.

Electrophysiology at the Posterior Electrode Cluster

The bottom set of waveforms in Figure 2 shows the results collapsed across the posterior set of electrode sites. At these sites, the contralateral and ipsilateral waveforms did not diverge until approximately 400 ms, at which time the contralateral waveform became more positive than the ipsilateral waveform. This effect lasted until approximately 600 ms.

In the ANOVA across time windows at these electrode sites (see Table 2), the overall contralateral-ipsilateral difference was not significantly different from zero. The main effect of Time was significant, however, indicating that the contralateral-minus-ipsilateral difference was larger for some time periods than other, consistent with the observation that the waveforms did not begin to diverge until approximately 400 ms. Separate ANOVAs for the individual time windows showed that the contralateral-ipsilateral difference was significant (p < .05) in the 400-500 and 500-600 ms time periods but not in the 200-300 and 300-400 ms time periods. The main effect of Electrode site was also significant in the analysis including all time periods, indicating that some electrode sites showed a larger contralateral-ipsilateral difference than others (the voltage topography will be described in detail later). Thus, although the posterior contralateral-ipsilateral difference was not robust enough to be significant when collapsed across all sites and time periods, it was significant in several individual time periods and varied significantly over time and over electrode sites.

Experiment 2

Experiment 2 was conducted to assess the replicability and generality of the effects observed in Experiment 1. In addition, we recorded from additional electrodes over anterior sites to obtain more detailed scalp distribution information. We also explored whether the laterality effects observed in Experiment 1 are unique to the simultaneous presentation of attended and unattended stimuli or whether comparable effects are also present when the stimuli are presented in isolation.

The stimuli and task were very similar to those of Experiment 1, with two significant changes. Instead of having four completely distinct sounds as stimuli, the four sounds were created by factorially combining two different pitches with the presence or absence of amplitude modulation. The pitch difference was designed to be more discriminable than the amplitude modulation difference, making it possible to determine whether the N2ac would be observed for a nontarget stimulus containing the target value along the easy-to-discriminate dimension that did not contain the target value along the difficult-to-discriminate dimension. Such a finding would indicate that the N2ac is elicited prior to the completion of target discrimination. Luck and Hillyard (1994a) performed an analogous visual experiment, and found that nontargets containing an easy-to-discriminate target feature elicited a significant N2pc response. This suggested that the N2pc reflects a process that is interposed between initial featural analysis and complete target identification.

We also added unilateral trials on which one of the four stimulus types was presented by itself on one side. This allowed us to ask whether the contralateral-ipsilateral difference observed for bilateral stimuli is simply a result of the summated contralateral-ipsilateral differences observed for unilateral stimuli or whether the N2ac effect reflects processes that are recruited specifically when multiple simultaneous stimuli compete with each other. In the visual domain, the N2pc has been shown to reflect competitive processes: it is largely absent when stimuli are presented one at a time, and it becomes larger when a distractor is placed in close proximity (see review by Luck, in press). The present experiment sought to address this same issue for the N2ac component.

To answer this question, we borrowed an analytical approach that has been used extensively to ask whether early auditory sensory components reflect independent processing of stimuli presented to the two ears or whether they reflect a stage at which information from the two ears has been combined. This approach takes advantage of the fact that voltages summate linearly, and the ERPs produced by the simultaneous but independent processing of two stimuli (e.g. two different sounds presented simultaneously to the right and left ears) with no interaction should therefore be exactly equal to the sum of the ERPs produced by the same stimuli presented at different times (e.g. one sound presented to the right ear and then, at a different time, the other sound being presented to the left ear). In contrast, when the two simultaneous sounds are not processed independently but somehow interact, the sum of the ERP responses will not equal that of simultaneously presented stimuli (Dobie & Norton, 1980). This technique has been used to look at the binaural interaction in brainstem evoked responses (Dobie & Norton, 1980; Decker & Howe, 1981; Wrege & Starr, 1981; McPherson & Starr, 1993; Polyakov & Pratt, 1996), auditory mid-latency responses (Dobie & Norton, 1980; Woods & Clayworth, 1985; Polyakov & Pratt 1995) ,as well as later components such as the MMN (Lavikainen, Tiitinen, May, Näätänen, 1997; McPherson & Starr, 1993). Research using this approach has shown that binaural interactions are small at early stages in the auditory pathway but become quite substantial by the time of the midlatency responses (for a review, see Pratt, in press).

In the present context, we asked whether the attention-related lateralization observed for simultaneous stimulus pairs was identical to the sum of the attention-related lateralizations observed for the same stimuli when presented individually. Specifically, we summed together the ERPs elicited by two different unilateral stimuli and compared the attention-related lateralization observed for this artificially summated pair with the attention-related lateralization observed for the same two stimuli when they were presented simultaneously. For example, we summed the waveform elicited by a high pitch, unmodulated tone presented by itself on the left side and the waveform elicited by a low pitch, modulated tone presented by itself on the right side, and we examined the contralateral-ipsilateral difference when the left tone or the right tone was the target. This was then compared with the contralateral-ipsilateral difference observed in the waveform recorded when these two tones were presented simultaneously (the different physical stimulus combinations were collapsed first to eliminate any effects based on physical stimulus differences). If the N2ac reflects processes that are recruited exclusively or more heavily when the attended and unattended tones are presented simultaneously, then it should be larger in the ERPs elicited by actual bilateral stimuli than in the artificially summated ERPs elicited by unilateral stimuli.

Methods

Participants

We recorded from 15 new participants (ages 18-30) who did not participate in Experiment 1. All subjects reported normal hearing and no history of neurological disorders or disease. All gave informed consent before the experiment and were compensated for their participation afterwards.

Stimuli and Procedure

The stimuli, task, and procedures were identical to those used in Experiment 1, except as noted. The four individual sounds consisted of an unmodulated 179-Hz sine wave, a 179-Hz sine wave modulated at 25 Hz with a modulation depth of .4, an unmodulated 711-Hz sine wave, and a 711-Hz sine wave modulated at 55 Hz with a depth of .25. Pilot testing indicated that the modulation parameters were approximately equally discriminable for the two pitches. Stimulus duration was 750 ms, with a 20-ms rise time and fall time, and stimulus intensity was approximately 55 dB SPL for each individual sound presented unilaterally. The sound intensity was measured with an Extech Instruments #407730 sound meter for each individual sound measured from the approximate location of the participant's head. One of the four stimuli was designated the target at the beginning of each trial block.

Bilateral stimuli consisted of two tones presented simultaneously, one from each of the two speakers. One tone was always high-pitched and the other was always low-pitched, but the assignment of pitches to speakers was random. In addition, the presence or absence of amplitude modulation was randomly and independently determined for the two tones. Thus, one side or the other always contained the target pitch, and this pitch was equally likely to be modulated or unmodulated. For unilateral stimuli, one of the four combinations of pitch and amplitude modulation was presented from either the left or right speaker, with equal probability for all eight combinations of pitch, amplitude, and side. As in Experiment 1, subjects were instructed to press one button if the target tone was present and another button if it was absent.

There were 304 trials per block and a total of eight blocks, presented in random order, with each of the four sounds serving as the target in two blocks. Half of the stimuli within a block were unilateral and half were bilateral. Because each of the four sounds was equally likely to be presented, the target was present on 25% of the unilateral trials. The independent combination of two sounds in each bilateral pair necessarily led to the target being present on 50% of the bilateral trials.

Recording & Analysis

We recorded from 64 electrodes (Fp1/2, Fpz, AF3/4, AF7/8, AFz, F1/2, Fz, F3/4, F5/6, F7/8, Cz, C3/4, T7/8, Pz, P1/2, P3/4, P5/6, P7/8, P9/10, POz. PO3/4, PO7/8, Oz, O1/2, Iz) in addition to the HEOG, VEOG, and mastoid sites. One participant was excluded due to excessive artifacts (<25%). An average of 20.3% of trials were excluded due to artifacts in the remaining 14 subjects.

As in Experiment 1, we collapsed across target type for all analyses to avoid any physical stimulus confounds. Visual inspection prior to collapsing indicated that the waveforms for each target type were highly similar.

The ERPs elicited by unilateral stimuli were summed together to create artificial bilateral waveforms prior to all analyses. For example, to simulate a bilateral trial with a target on the left and a nontarget on the right, we summed together the ERP elicited by a unilateral target presented on the left and the ERP elicited by a unilaterally presented nontarget presented on the right. Once these summed waveforms were created, they were analyzed in the same manner as the real bilateral stimuli.

All measurements were taken from contralateral-minus-ipsilateral waveforms for correct target-present trials. Separate ANOVAs were again performed on anterior (F1/2, F3/4, F5/6, F7/8, F9/10, FC1/2, FC3/4, FC5/6, FC7/8, FT7/8, C1/2, C3/4, C5/6, T7/8) and posterior (P1/2, P3/4, P5/6, P7/8, P9/10, PO3/4, PO7/8, O1/2) electrode clusters. To be consistent with Experiment 1, individual ANOVAs (Time × Hemisphere × Electrode site) were run for target-present trials and target-absent trials, separately for the unilateral and bilateral trials, and separately for the anterior and posterior electrode clusters.

Results and Discussion

Behavioral Results

Participants accurately identified the target as present or absent 92% of the time. On bilateral trials, participants responded with an average response time of 667.2 ms (SD 71.3 ms) when the target was present and 680.9 ms (SD 76.9 ms) when the target was absent. Response times to unilaterally presented stimuli were faster, with an average response time of 590 ms (SD 63.5) for targets and 593 ms (SD 77 ms) for nontargets. A two-way ANOVA with factors of bilateral/unilateral X target present/absent indicated that the difference between unilateral and bilateral was significant (F(1,13)=113.8, p<0.001), whereas overall the present versus absent difference was not significant (F(1,13)=1.49, n.s.). A significant interaction between bilateral/unilateral factor and target present/absent (F(1,13)=4.72, p<0.05) indicated that participants were slower for target-absent trials, but only if two sounds were presented simultaneously.

Electrophysiology: Bilateral Targets

Our first goal was to determine whether the results of Experiment 2 replicated those of Experiment 1, limiting the analyses to bilateral target-present stimuli. Figure 3A shows the contralateral and ipsilateral waveforms elicited by bilateral target-present waveforms at the anterior electrode sites, along with the contralateral-minus-ipsilateral difference. The N1 waves for these stimuli were considerably larger than those observed in Experiment 1, making the contralateral-ipsilateral difference somewhat more difficult to see. However, the difference wave contained a clear negativity from approximately 200-400 ms post-stimulus. This effect was generally similar to the N2ac effect observed in Experiment 1, but it was somewhat smaller and shorter in duration in the present experiment.

Figure 3.

Grand average of contralateral ipsilateral and contralateral minus ipsilateral waveforms for Experiment 2. A) Bilaterally presented Targets at anterior electrodes. B) Bilaterally presented nontargets at anterior electrodes. C) Unilaterally presented targets at anterior electrodes. D) Unilaterally presented nontargets at anterior electrodes. E) Bilaterally presented targets at posterior electrodes. F) Bilaterally presented nontargets at posterior electrodes. G) Unilaterally presented targets at posterior electrodes. H) Unilaterally presented nontargets at posterior electrodes. Anterior waveforms are collapsed across electrode sites F1/2, F3/4, F5/6, F7/8, FC1/2, FC3/4, FC5/6, FT7/8, C1/2, C3/4, C5/6, T7/T8. Posterior waveforms are collapsed across electrode sites. P1/2, P3/4, P5/6, P7/8, P9/10, PO3/4, PO7/8, O1/2. For visual clarity, the waveforms shown here were filtered offline with a low-pass Gaussian filter with a half-amplitude cutoff at 50 Hz. The raw contralateral and ipsilateral waveforms use the 2.0 μV scale. The contralateral-minus-ipsilateral difference waveforms use the 1.0 μV scale.

Because of the relatively short duration of the N2ac effect, the overall contralateralipsilateral difference in the anterior electrode cluster was not significantly different from zero across the entire 200-600 ms time range (see Table 1). However, the main effect of Time was significant, indicating that the N2ac effect was present in some intervals and varied in amplitude over time. Separate analyses of the individual time windows indicated that the contralateralipsilateral difference was significantly greater that zero in the 200-300 ms time period (p < .05), marginally significant in the 300-400 ms time period (p = .066) and did not approach significance in the 400-500 and 500-600 ms time periods. As in Experiment 1, there were no significant effects or interactions involving Hemisphere or Electrode in the anterior electrode cluster. Thus, these results replicate the basic N2ac pattern observed in Experiment 1.

At the posterior electrode sites, the contralateral waveform became more positive than the ipsilateral waveform beginning around 400 ms (see Figure 3E), just as in Experiment 1. The overall contralateral-ipsilateral difference did not quite reach significance across the entire 200-600 ms time range (see Table 2). However, a significant effect of Time indicated that this effect varied significantly across time periods. Follow-up analyses indicated that the contralateralipsilateral difference was significantly greater than zero in the 400-500 and 500-600 ms time periods (p < .05) but not in the 200-300 and 300-400 ms time periods. These results replicate the late posterior contralateral-ipsilateral difference observed in Experiment 1.

Electrophysiology: Bilateral Nontargets

On target-absent trials, one side always contained a tone that matched the target in the easy-to-discriminate pitch dimension but mismatched in the amplitude modulation dimension, and the other side always contained a tone with the nontarget pitch (which might be either modulated or unmodulated). This made it possible compared the nontarget waveforms contralateral versus ipsilateral to the side containing the matching pitch. As shown in Figure 3B, a small negativity was visible in the contralateral-minus-ipsilateral difference waveforms for nontargets at the anterior electrode sites, but none of the main effects or interactions approached significance. However, a clear contralateral positivity was observed from approximately 300-600 ms in the posterior electrode sites. This difference was significantly greater than zero when collapsed across the entire 200-600 ms time window and also yielded a significant main effect of time. This effect was also larger in the right hemisphere than the left hemisphere, leading to a significant main effect of hemisphere. Thus, unlike the visual N2pc, the N2ac was present only for targets and not for nontargets that shared a feature with the target, but the late posterior contralateral-ipsilateral difference was present for both targets and nontargets sharing the target's pitch. This suggests that the pitch and modulation features were processed together rather than sequentially in this experiment; this may simply mean that the modulation dimension was sufficiently discriminable that subjects were able to process it concurrently with the pitch dimension.

Electrophysiology: Unilateral Targets

Figures 3C and 3G show the ERPs from the unilateral stimuli after they were summed together to simulate a bilateral trial with a target on one side and a nontarget on the other side. The summation led to a very large N1 wave, which replicates previous research showing that the N1 component does not represent independent processing of stimuli presented in the left and right ears (Lavikainen et al., 1997). That is, the N1 response to a bilateral stimulus is not equal to the sum of the responses to the unilateral stimuli.

In contrast to the enlarged N1 response in the summated unilateral ERPs compared to the bilateral ERPs, the contralateral-ipsilateral difference that was observed in the bilateral ERPs on target-present trials was smaller or absent for the summated unilateral ERPs (compare Figure 3A and Figure 3C). None of the contralateral-ipsilateral difference effects for target-present unilateral stimuli approached significance for either the anterior or posterior electrode clusters (see Tables 1 and 2, respectively). Thus, there was no evidence of either an N2ac or a late posterior contralateral-ipsilateral difference in the unilateral stimuli. This indicates that the N2ac reflects activity that is triggered (or enhanced) by competition between simultaneous attended and unattended auditory stimuli rather than reflecting lateralized processes that are equally present for isolated and simultaneous stimuli.

Electrophysiology: Unilateral Nontargets

Figures 3D and 3H show the waveforms created by summing together unilateral nontarget stimuli, with the target matching pitch on one side and the nonmatching pitch on the other side. A contralateral positivity can be seen in these waveforms starting around 300 ms, especially at the anterior electrode sites. Although the overall contralateral-ipsilateral difference was not significantly greater than zero, the difference was larger in the right hemisphere than in the left hemisphere, leading to a significant main effect of Hemisphere at the anterior sites (Table 1). This hemispheric difference did not quite reach significance at the posterior sites (Table 2). Thus, rather than producing an N2ac, unilateral nontargets containing the attended pitch elicited a contralateral positivity, especially over the right hemisphere and especially over anterior scalp sites.

Scalp Distributions for Experiments 1 and 2

It can be misleading to plot the scalp topography for contralateral-minus-ipsilateral difference waves, because this difference necessarily falls to zero at the midline. We therefore plotted the scalp topography of difference waves created by subtracting trials with the target on the left minus trials with the target on the right (Figure 4). In these plots, a contralateral-ipsilateral difference will have opposite polarities over the left and right hemispheres. For example, a more negative voltage for a contralateral stimulus will appear as a negative voltage over the right hemisphere and as a positive voltage over the left hemisphere. This is exactly what was observed over anterior scalp sites between 200 and 600 ms, for both Experiment 1 and Experiment 2. Interestingly, the positive voltage over the left hemisphere was somewhat anterior (peaking near the F3 electrode site) whereas the negative voltage over the right hemisphere was more central (peaking near the C4 electrode site). This difference in distribution between the left and right hemispheres was not statistically significant (see the Hemisphere x Electrode Site interactions in Table 1), but it was observed in both experiments. Further research is needed to determine whether this is a real difference between the left and right hemispheres.

Figure 4.

Voltage maps for bilateral target-present trials, based on left-target-minus-right-target difference waveforms from Experiments 1 and 2.

The late posterior contralateral-ipsilateral difference can be seen in the rear views (bottom two rows of Figure 4) as a positive voltage over the right hemisphere and a negative voltage over the left hemisphere. This difference tended to be somewhat more ventral and lateral for Experiment 1 (peaking near the TO3 and TO4 electrodes) and more dorsal and medial for Experiment 2 (peaking between the P1/PO3 and P2/PO4 electrodes). It should be noted that this effect does not appear to be an N2pc component, which has the opposite polarity and a more ventral-lateral distribution (cf. Figure 11 in Luck & Hillyard, 1994a). In addition, it should not be assumed that this effect is generated in visual cortex; for example, it may instead be generated in the attentional control areas of the posterior parietal lobe.

General Discussion

The main goal of these experiments was to examine whether an ERP component can be identified that is a consequence of the allocation of selective attention within bilateral auditory stimulus arrays, analogous (but not necessarily homologous) to the N2pc. Whereas the vast majority of ERP studies of auditory spatial attention have directed attention to a given location prior to stimulus onset, the present study examined an analog of the common real-world situation in which people must search for a specific sound within an environment containing multiple concurrent sounds. In this situation, we observed an N2ac component that was quite analogous to the N2pc component, consisting of a greater negative potential contralateral to the location of the target sound beginning approximately 200 ms after the onset of the bilateral stimulus array. Though we suspect that the N2ac may directly reflect the allocation of selective attention to a target, the current experiment cannot separate whether it is a reflection of attention or a downstream consequence of shifting attention and identifying a target. Additional research is needed to clarify this.

It should be noted that we cannot be certain from these data whether this effect reflects an increased negative potential triggered by the target tone over the contralateral hemisphere, an increased positive potential triggered by the target tone over the ipsilateral hemisphere, an increased positive component triggered by the nontarget tone over the hemisphere contralateral to the nontarget, an increased negative component triggered by the nontarget tone over the hemisphere ipsilateral to the nontarget, or some combination. Indeed, under some circumstances the N2pc component may reflect both a negativity contralateral to the target and a positivity contralateral to the distractor (Hickey, Di Lollo, & McDonald, 2009). The term N2ac is intended to be used descriptively, not to indicate which of these is likely to be true.

The N2ac observed in the present study was smaller in amplitude than a typical N2pc (0.5–1 μV for N2ac versus 1-2 μV for a typical N2pc). It is possible that this reflects the lesser degree of contralaterality in the auditory system than in the visual system. The N2ac was also longer in duration than a typical N2pc. This may reflect differences in the time course of attentive processing across modalities. However, both the amplitude and the duration of the N2pc depend on the nature of the stimuli and task, and the same is presumably true of the N2ac, so it would be premature at this point to conclude that the N2ac and N2pc components are intrinsically different along these dimensions.

Just as the N2pc component is small or absent when visual objects are presented in isolation, without concurrent distractor stimuli, no clear evidence of an N2ac component was observed for unilateral stimuli in Experiment 2. This suggests that N2ac, like N2pc, reflects a process involved in resolving competition between simultaneously present stimuli. In the visual modality, most neurons have spatially defined receptive fields, and the presence of multiple stimuli within a neuron's receptive field leads to ambiguity in the neuron's output. For example, if a red-selective neuron has two objects in its receptive field and fires at a high rate, it is ambiguous which of the two stimuli is actually red (see Luck & Beach, 1998). This problem can be solved by focusing attention onto one of the two objects so that the neuron primarily codes the color of only the attended item (see Luck et al., 1997). Indeed, single-unit studies of visual attention in monkeys have demonstrated that the largest effects of attention occur under these conditions of competition between stimuli presented simultaneously within a receptive field (Reynolds, Chelazzi, & Desimone, 1999; Moran & Desimone, 1985; Luck et al., 1997). Given that neurons in auditory cortex typically respond to stimuli over a broad spatial range, it is likely that the left and right tones used in the present experiment provided simultaneous inputs to the many of the same neurons, creating the same general type of competition that has been studied in the visual modality. Thus, the N2ac component may reflect the same general type of mechanism for biasing neural coding toward the attended stimulus that has been observed in the visual modality. Further research is necessary to test this conjecture.

Studying the competition between simultaneously audible sounds is likely to be of practical as well as theoretical importance, because the most common complaint of people who suffer hearing loss over the course of aging is that they have trouble understanding speech in a noisy environment (Edwards, 2000). This problem involves attentional dysfunction as well as sensory dysfunction (Best, Marrone, Mason, Kidd & Shinn-Cunningham, 2009), and the N2ac component may be useful in studying the nature of this problem. The use of simultaneous stimulus onsets and offset in the present study is an exaggeration of this “cocktail party” situation, because attended and unattended sounds in real-world environments typically have different amplitude envelopes. That is, even though two sounds may be present at the same time, they may onset at different times, offset at different times, and generally vary in intensity over time in an uncorrelated manner. For example, when two people are speaking simultaneously, the words spoken by the two individuals do not usually onset simultaneously. However, much of the information from the attended and unattended sources does occur simultaneously in this situation (e.g., the onset of a word spoken by one individual may occur during the middle of a word spoken by another individual). Thus, the simultaneous stimuli used in the present study may reveal processes that are important even under less synchronized real-world settings. Future research is needed to determine whether the N2ac would be observed when the attended and unattended stimuli do not onset simultaneously but do overlap in time.

It is unclear what process is reflected by the late posterior contralateral-ipsilateral difference, observed from 400-600 ms in the present study. Given that many of the behavioral responses occurred prior to 600 ms, this ERP effect probably does not reflect a process that was needed to identify the target sound. Moreover, although it is difficult to determine the location of an ERP generator from the scalp distribution alone, it is unlikely that the focal parietal-occipital distribution of this effect was a result of a generator within unimodal auditory cortex. One possibility is that this effect reflects intermodal processing, such as a shift of visual attention toward the location of the auditory target. Participants were able to see the speakers, so they could potential focus visual attention onto the origin of the sounds. Alternatively, this late effect could represent a general reorienting of attention, either vision-specific or potentially supramodal, back to a neutral or central point, preparing the participant for the next trial. Further research is necessary to determine the nature of the processes reflected by this effect.

We would like to conclude by discussing the relationship between the contralateralipsilateral differences observed here and the ERP components that are present in the original waveforms from which the difference waves were constructed. It is extremely difficult to definitively isolate a single ERP component from the mixture of components that is present at every time point and at every electrode site in an ERP waveform (see Chapter 2 in Luck, 2005 for an extended discussion). Difference waves can be used to isolate individual components by subtracting one waveform from a second waveform when all but one (or a few) components are the same in the two waveforms. One of the main limitations of this approach is that many components may vary with a given manipulation, and the difference will then contain multiple components (although fewer than the original waveforms). The kind of contralateral-minus-ipsilateral difference wave used here tends to minimize the number of components that vary across the waveforms being subtracted, because everything is the same in the two waveforms except for the location of the target relative to the location of the electrode site. Moreover, even if the component is present in both the contralateral and ipsilateral hemispheres but larger in one than in the other, the amplitude of the difference between contralateral and ipsilateral will be proportional to the amplitude of the component under most conditions (see Luck, in press). This subtractive approach does not guarantee that a single component is isolated (e.g., both the N2pc and the contralateral delay activity appear in contralateral-minus-ipsilateral difference waveforms under some conditions in visual working memory experiments; see Drew & Vogel, 2008). However, this approach has been extremely useful for several components that have been used to provide definitive answers to important scientific questions, including the N2pc component (Luck, in press), the contralateral delay activity (Perez & Vogel, in press), and the lateralized readiness potential (Smulders & Miller, in press). Thus, there is reason to expect that the N2ac, which is isolated in the same manner as these other components, could be used to provide definitive answers to important questions about auditory selective attention.