Sounds Activate Visual Cortex and Improve Visual Discrimination

Abstract

A recent study in humans (McDonald et al., 2013) found that peripheral, task-irrelevant sounds activated contralateral visual cortex automatically as revealed by an auditory-evoked contralateral occipital positivity (ACOP) recorded from the scalp. The present study investigated the functional significance of this cross-modal activation of visual cortex, in particular whether the sound-evoked ACOP is predictive of improved perceptual processing of a subsequent visual target. A trial-by-trial analysis showed that the ACOP amplitude was markedly larger preceding correct than incorrect pattern discriminations of visual targets that were colocalized with the preceding sound. Dipole modeling of the scalp topography of the ACOP localized its neural generators to the ventrolateral extrastriate visual cortex. These results provide direct evidence that the cross-modal activation of contralateral visual cortex by a spatially nonpredictive but salient sound facilitates the discriminative processing of a subsequent visual target event at the location of the sound. Recordings of event-related potentials to the targets support the hypothesis that the ACOP is a neural consequence of the automatic orienting of visual attention to the location of the sound.

Introduction

It is well documented that a salient sound may affect the perceptual and neural processing of a subsequent visual stimulus that appears at the same location (for review, see McDonald et al., 2012). For example, a lateralized auditory cue can improve the detection and discrimination of a colocalized visual target (Dufour, 1999; McDonald et al., 2000; Frassinetti et al., 2002; for review, see Spence and Driver, 2004; Leo et al., 2011), hasten perceptual awareness of the target (McDonald et al., 2005), and increase the apparent luminance contrast of the target (Störmer et al., 2009). These cross-modal cueing effects may be a consequence of the orienting of visual attention toward the location of the salient sound (McDonald et al., 2012).

The neural mechanisms responsible for this cross-modal facilitation of visual processing are not well understood (Macaluso et al., 2001; McDonald et al., 2001). Previous neuroimaging studies using functional magnetic resonance imaging have reported that auditory stimuli may activate the visual cortex when they are task relevant and attended but not when unattended (Zimmer et al., 2004; Wu et al., 2007; Cate et al., 2009; Bueti and Macaluso, 2010). However, a recent study using event-related potential (ERP) recordings found that salient sounds activated the visual cortex regardless of their task relevance (McDonald et al., 2013). In four different experiments, it was found that a lateralized sound elicited an enlarged contralateral positive potential in the interval of 200–450 ms after sound onset with neural generators localized to the ventral extrastriate visual cortex. This auditory-evoked contralateral occipital positivity (ACOP) was elicited by salient sounds even when the tasks were entirely auditory and no visual stimuli were presented at all and when the sounds were both spatially and temporally nonpredictive of relevant target events. It was concluded that salient sounds automatically activate the visual cortex regardless of their relevance or cue value.

The functional significance of the ACOP remains to be determined. McDonald et al. (2013) found that individuals with larger ACOP amplitudes to a lateralized sound tended to report colocalized visual targets as having higher luminance contrast, but it is unclear whether the ACOP signifies a more general improvement of visual processing that would lead, for example, to more accurate pattern discriminations. The present study examined this question by recording ERPs in a task in which a lateralized sound was followed after 400 ms by a visual target consisting of a masked letter “T” or “L,” which was presented unpredictably at either the same location (valid trial) or on the opposite side (invalid trial) as the preceding sound. Trial-to-trial variations in the ACOP were found to be predictive of target discrimination accuracy on valid trials, consistent with the view that the ACOP reflects a cross-modal priming of the visual cortex that improves the discriminative processing of colocalized visual events.

Materials and Methods

Participants.

Sixteen subjects (10 females; mean age of 22.5 years) participated in the study after giving informed written consent as approved by the Human Research Protections Program of the University of California, San Diego. All participants had normal or corrected-to-normal vision and normal hearing.

Stimuli and procedure.

The experiment was conducted in a dimly lit, electrically shielded and sound-attenuated chamber. Visual stimuli were presented on a 27-inch computer monitor, and auditory stimuli were delivered by a pair of external loudspeakers positioned at the left and right sides of the monitor (Fig. 1). Auditory stimuli were pink noise bursts (500–15,000 Hz) with 5 ms rise and fall ramps (78 dB SPL, 83 ms duration) delivered unilaterally from one of the speakers. On each trial, an auditory cue was presented at random from either the left or right speaker (50% left; 50% right). After a stimulus-onset asynchrony (SOA; onset to onset) of 400 ms, either the letter T or L (2.4° letter height, 50 ms duration) was presented equiprobably at an eccentricity of 17.5° lateral to fixation on either the same side as the sound cue (valid) or the opposite side (invalid) on 57% of the trials (28.5% valid, 28.5% invalid in random order). Thus, the side of the sound cue was nonpredictive of the location of the subsequent visual target letter. The letter was immediately followed by a mask of superimposed grid lines (7 × 7° of visual angle) that was presented for 100 ms. On another 28.5% of the trials, the grid lines mask was presented to the left or right side without any preceding letter (mask-only “catch” trials; 450 ms SOA). On the remaining 14.5% of the trials, no visual stimulus was presented after the sound cue (cue-only trials). The cue-only trials allowed visualization of the ACOP waveform by itself without contamination from target-evoked activity. All these types of trials were presented in random order with intertrial intervals of 1200–1450 ms.

Figure 1.

A, Schematic illustration of the stimulus sequence. Example shown is for a valid trial in which a target letter L was presented on the same side as the auditory cue. B, ACOP waveforms averaged over six pairs of posterior electrode sites ipsilateral and contralateral to the sound stimulus (see Materials and Methods) on sound-only (cue-only) trials. The ACOP (contralateral minus ipsilateral difference) begins at ∼200–250 ms and extends to ∼450 ms. The shaded area (300–400 ms) shows the interval in which the ACOP was quantified on the letter-present trials, which had a cue–target SOA of 400 ms. The ACOP difference amplitude averaged over the 300–400 ms interval was highly significant on these cue-only trials (contralateral vs ipsilateral, F_(1,15) = 15.83, p < 0.002). Topographical map shows ACOP distribution with lateral occipital maximum. C, ACOP difference amplitudes mapped over successive 20 ms intervals within the 300–400 ms measurement window. The successive scalp distributions are highly similar, indicative of stable neural generators over the 300–400 ms interval.

During the experiment, participants were required to maintain their eyes fixated on a central cross (0.7 × 0.7° of visual angle). They were informed that a letter target would usually follow the sound, but its left–right position would be unpredictable; their assigned task was to discriminate the letter target (if presented) by pressing one of two buttons on a game pad device. The response buttons for T and L were counterbalanced across participants. No responses were required when letters were absent on cue-only and catch trials. Discrimination accuracy was quantified in terms of hit rate (equivalent to percentage correct) and in terms of discrimination sensitivity (d′) for the T versus L responses on the letter-present trials. The experiment comprised 15 blocks of 126 trials each.

Electrophysiological recording and analysis.

The electroencephalogram (EEG) was recorded continuously from 60 electrode sites using a modified 10-10 system montage (McDonald et al., 2003). Horizontal eye movements were monitored via electrodes at the left and right outer canthi (horizontal EOG). Vertical eye movements and blinks were monitored by an electrode below the left eye (vertical EOG). The electrode impedances were kept below 5 kΩ. The EEG and EOG signals were digitized at 250 Hz, amplified with a gain of 10,000, and filtered with an amplifier bandpass of 0.01–80 Hz. The right mastoid electrode served as the reference during data acquisition. The EEG was averaged in 500 ms epochs time-locked to the auditory stimulus onset, including a 100 ms prestimulus baseline. Epochs contaminated by eye movements, eyeblinks, muscle activity, or amplifier blocking were rejected. The resulting averaged ERP waveforms were then digitally low-pass filtered (−3 dB cutoff at 25 Hz) and re-referenced to the average of the left and right mastoids.

ERPs recorded on the letter-present trials were analyzed in relation to discrimination accuracy. A trial-by-trial analysis was performed in which ERPs were averaged separately for trials with validly and invalidly cued target locations and with correct and incorrect target discrimination responses. For analysis purposes, ERP waveforms were collapsed across auditory stimulus location (left, right) and hemisphere of recording (left, right) to obtain ERPs recorded on the contralateral hemisphere and on the ipsilateral hemisphere with respect to the auditory stimulus location. The ACOP component was measured as the mean amplitude over six pairs of posterior electrode sites (P7/P8, PO3/PO4, PO7/PO8, O1/O₂, I3/I4, and I5/I6) in which its amplitude was greatest over the time window of 300–400 ms after stimulus onset with respect to a 100 ms prestimulus baseline (Fig. 1B). This measurement interval was chosen after an analysis of ACOP amplitudes (contralateral minus ipsilateral) over successive 20 ms intervals in the cue-only waveforms. The ACOP was significantly more positive contralaterally (p < 0.05 by t tests) over all intervals between 260 and 440 ms. The interval 300–400 ms was chosen because of the following: (1) it included the largest ACOP amplitudes; (2) it was sufficiently broad to achieve good signal-to-noise ratios; and (3) it immediately preceded the onset of the visual target. Importantly, the scalp topography of the ACOP was consistent throughout the 100 ms measurement interval (Fig. 1C).

The ACOP amplitudes (mean amplitudes over the 300–400 ms interval) on letter-present trials were first analyzed in a three-way repeated-measures ANOVA with factors of hemisphere (contralateral/ipsilateral to side of cue), cue validity (valid/invalid cue–target locations), and letter discrimination accuracy (correct/incorrect). After finding a highly significant three-way interaction, follow-up analyses were performed in which the lateralized ACOP was quantified as the difference between the amplitudes at contralateral minus ipsilateral homologous sites for each trial type (valid/invalid × correct/incorrect). The resulting ACOP difference amplitudes were analyzed with specific contrasts between correct trials versus incorrect trials under each validity condition. A similar three-way ANOVA was performed on the amplitudes of the ERPs elicited by the visual target stimuli. Separate analyses were performed for the mean amplitudes of the successive components of the target-elicited ERP over the following intervals and electrodes: (1) P1 (120–140 ms, measured over the same electrodes as ACOP); (2) N1 (160–190 ms, measured over the same electrodes as ACOP); and (3) P3/P300 (400–500 ms, CP1/CP2, P1/P2).

For purposes of comparison, the mean amplitude of the auditory-evoked N1 component (92–140 ms) was also measured for each subject over a cluster of 14 anterior electrode sites (FC1/FC2, FC3/FC4, FC5/FC6, C1/C2, C3/C4, C5/C6, and T7/T8) in which its amplitude was largest. N1 amplitudes were quantified and analyzed by repeated-measures ANOVAs using the same contralateral versus ipsilateral comparisons as were performed on the ACOP.

Topographical mapping and source localization.

To visualize the lateral asymmetry of the N1 and ACOP components, topographical voltage distributions over specific time latencies were calculated for difference waveforms obtained by subtracting the ERPs recorded ipsilaterally from the ERPs recorded contralaterally. These contralateral minus ipsilateral voltage distributions (grand-averaged over all subjects) were projected to the right hemisphere on each map (Fig. 1B). Anatomical sources of the contralateral minus ipsilateral ACOP difference component on letter-present trials were estimated using the Brain Electrical Source Analysis program (BESA, version 5.0) within the same latency as used in statistical testing. Symmetrical dipole pairs of equivalent current dipoles were fit to the symmetrical voltage maps of the grand-averaged waveforms within specified time intervals (for general dipole-modeling methods, see McDonald et al., 2005). A standardized head model was used for source analysis (BESA version 5.3). Source locations were also estimated from the grand-averaged voltage topographies by distributed linear inverse solutions based on a local autoregressive average (LAURA) (Grave de Peralta Menendez et al., 2004). LAURA estimates 3D current density distributions using a realistic head model with a solution space of 4024 nodes equally distributed within the gray matter of the average template brain of the Montreal Neurological Institute. LAURA analyses were implemented using Cartool software. The resulting source estimations were transformed into the standardized coordinate system of Talairach and Tournoux (1988) and projected onto a structural brain image supplied by MRIcro (Rorden and Brett, 2000) using the AFNI (Analysis of Functional NeuroImaging) software (Cox, 1996).

Results

Behavioral data

The hit rates, d′ values, and reaction times for the letter discrimination task were compared between the valid and invalid cueing conditions by ANOVA. Both the mean hit rates [valid, 72.5 ± 12.3% (mean ± SE); invalid, 69.6 ± 12.5%; F_(1,15) = 37.14, p < 0.0001] and d′ values (valid, 1.57 ± 0.70; invalid, 1.44 ± 0.67; F_(1,15) = 34.36, p < 0.001) were significantly higher for valid than invalid trials. Reaction times were also faster for the validly cued targets (valid, 545 ± 65 ms; invalid, 553 ± 64 ms; F_(1,15) = 23.86, p < 0.001). These behavioral results show that sound cues improved the discrimination of letters that were presented at the cued (valid) location.

ERPs to auditory cues

The typical auditory-evoked N1 component peaking at ∼100–110 ms (Picton 2011) was observed in the fronto-temporal ERP waveforms elicited by the lateralized sounds under all conditions (Figs. 2, 3, left columns). As in many previous studies (McDonald et al., 2013), the N1 amplitude was larger over the hemisphere contralateral to the sound location (contralateral vs ipsilateral fronto-temporal amplitudes over 92–140 ms: valid correct, F_(1,15) = 26.20, p < 0.0005; valid incorrect, F_(1,15) = 13.08, p < 0.005; invalid correct, F_(1,15) = 39.57, p < 0.00005; invalid incorrect, F_(1,15) = 14.71, p < 0.005). The contralateral minus ipsilateral difference amplitudes for the N1 did not differ significantly between correct trials and incorrect trials under either condition (valid, F_(1,15) = 0.17, p = 0.69; invalid, F_(1,15) = 0.69, p = 0.42). These results suggest that the N1 asymmetries were not associated with the enhanced discriminability of validly cued visual targets.

Figure 2.

ERP waveforms elicited by lateralized sounds and topographical distributions of lateralized N1 (92–140 ms) and ACOP (300–400 ms) components under valid conditions. N1 waveforms (left column) were averaged over seven homologous pairs of anterior temporal sites, and ACOP waveforms were averaged over six homologous pairs of occipital electrodes. ERP waveforms were collapsed over left and right sound locations and left and right hemispheres to form averaged waveforms recorded contralaterally and ipsilaterally with respect to sound location. Topographical voltage distributions are shown for the contralateral minus ipsilateral difference amplitude, projected onto the right hemisphere. Enlarged N1 negativities were observed for both correct trials (A) and incorrect trials (B), but the ACOP was only elicited on correct trials. The voltage scale shown for the top left pair of waveforms applies to all.

Figure 3.

Same as Figure 2 but for invalid cueing condition. Enlarged N1 negativities and significant ACOPs were observed for both correct trials (A) and incorrect trials (B).

The ACOP amplitude values averaged over the 300–400 ms time window were first subjected to a three-way ANOVA with factors of hemisphere (contralateral/ipsilateral with respect to side of cue), validity (valid/invalid), and discrimination accuracy (correct/incorrect). There was a significant main effect of hemisphere (F_(1,15) = 10.99, p < 0.005), with greater positivity over the contralateral occipital scalp, but the main effects of validity (F_(1,15) = 0.93, p = 0.350) and accuracy (F_(1,15) = 0.09, p = 0.775) were not significant. Importantly, there was a significant three-way interaction of hemisphere × validity × accuracy (F_(1,15) = 11.12, p < 0.005), indicating that the ACOP amplitude (contralateral minus ipsilateral) was affected by accuracy in a different way on valid and invalid trials. To investigate this interaction, additional specific contrasts were performed as described below.

In contrast with the auditory-evoked N1, the ACOP (measured as the contralateral minus ipsilateral difference at occipital sites over the time window 300–400 ms on letter-present trials) had a highly significant amplitude on valid-correct trials (0.37 ± 0.06 μV; contralateral vs ipsilateral, F_(1,15) = 68.64, p < 0.000001) but was not evident on valid-incorrect trials (0.06 ± 0.18 μV; contralateral vs ipsilateral, F_(1,15) = 0.12, p = 0.73; Fig. 2, right column). However, for invalidly cued trials, significant ACOPs of intermediate amplitude were observed for both correct (0.28 ± 0.11 μV; contralateral vs ipsilateral, F_(1,15) = 12.04, p < 0.005) and incorrect (0.27 ± 0.09 μV; contralateral vs ipsilateral, F_(1,15) = 19.03, p < 0.001) trials (Fig. 3, right column). As follows from the above, ACOP amplitudes were significantly larger for correct trials than for incorrect trials in the validly cued condition (correct vs incorrect, F_(1,15) = 10.80, p < 0.005) but not in the invalidly cued condition (correct vs incorrect, F_(1,15) = 0.24, p = 0.63; Fig. 4A). Moreover, the ACOP amplitude on the valid-correct trials was larger than on either the invalid-correct (F_(1,15) = 6.46, p < 0.05) or invalid-incorrect (F_(1,15) = 10.07, p < 0.01) trials.

Figure 4.

A, Mean ACOP difference amplitudes (contralateral minus ipsilateral) within the time window of 300–400 ms averaged over a cluster of six occipital electrodes. Note that the ACOP was significantly larger for correct than for incorrect trials under the valid cueing condition (p < 0.005) but not for the invalid condition. B, Estimated dipole sources of the ACOP difference amplitudes (300–400 ms), derived from the grand-averaged voltage topographies, were localized to the ventrolateral extrastriate visual cortex (Brodmann's area 19). Colored areas represent maximal current densities of the grand-averaged ACOP difference as determined by LAURA.

For both the N1 and ACOP components, topographical voltage maps were created of the contralateral minus ipsilateral difference amplitudes over all pairs of homologous electrodes (e.g., PO3 and PO4). The N1 (92–140 ms) was found to have a contralateral negative distribution over anterior fronto-temporal regions under all conditions (Figs. 2, 3, left columns). In contrast, the ACOP difference amplitude had a lateral occipital focus on the cue-only trials (Fig. 1B) and on the valid-correct trials as well as on the invalid-correct and incorrect trials (Figs. 2, 3, right columns). However, on valid-incorrect trials, the topographical map in the ACOP interval shows no sign of occipital activity (Fig. 2B). The neural generators of the ACOP on letter-present trials were estimated by dipole modeling of the contralateral minus ipsilateral difference topographies within the 300–400 ms time window. The best-fitting dipole sources were localized to ventrolateral extrastriate visual cortex of the hemisphere contralateral to the sound position (Fig. 4B). The Talairach coordinates (x, y, z) and residual variance (RV) values for these dipole estimations are as follows: valid correct, (±35, −69, −7), RV of 8.5%; invalid correct, (±38, −72, −5), RV of 10.2%; invalid incorrect, (±43, −71, −15), RV of 9.7%. These source localizations were situated in the ventrolateral occipital cortex (Brodmann's area 19) and are in good agreement with those reported previously for the ACOP (McDonald et al., 2013). A parallel analysis of the sources of the ACOP using LAURA showed maximal regions of current density that overlapped with the dipole positions estimated by BESA (Fig. 4). The congruence between the results of these two very different source estimation techniques (LAURA and BESA) and the congruence with the ACOP sources previously estimated by McDonald et al. (2013) increases our confidence in the ventrolateral occipital localization for the ACOP generators. However, it must be cautioned that inverse source estimations such as these can only serve as approximations of the true neural generator configuration because of the inherently ill-posed nature of the inverse problem (Nunez and Srinivasan 2006).

ERPs to visual targets

The ERPs to the visual target stimuli differed as a function of the accuracy (correctness) of the subjects' target discrimination responses on a trial-by-trial basis, but different early ERP components were modulated for validly and invalidly cued targets (Fig. 5). The occipitally distributed P1 component (with maximal amplitude at 120–140 ms latency) did not show a significant overall modulation as a function of either validity (valid vs invalid, F_(1,15) = 3.82, p = 0.07) or accuracy (correct vs incorrect, F_(1,15) = 3.43, p = 0.08), but the validity × accuracy interaction was highly significant (F_(1,15) = 13.47, p < 0.003). Specific comparisons showed that this interaction was the result of the P1 amplitude being larger on correct than incorrect trials for validly cued targets (F_(1,15) = 11.26, p < 0.005) but not for invalidly cued targets (F_(1,15) = 0.04, p = 0.838). In contrast, the subsequent N1 component (160–190 ms) showed a significant overall main effect for accuracy, with correct amplitudes greater than incorrect (F_(1,15) = 7.27, p < 0.05). Specific contrasts showed that the N1 was significantly enlarged for correct trials only for invalidly cued targets (correct vs incorrect, F_(1,15) = 11.73, p < 0.005) and not for validly cued targets (F_(1,15) = 1.99, p = 0.178). As can be seen in the topographical maps of Figure 5, there was a tendency for the P1 amplitude to be larger over the hemisphere ipsilateral to the target location (contralateral vs ipsilateral, F_(1,15) = 5.12, p < 0.05), in accordance with previous reports of P1 topography at longer latencies (Di Russo et al., 2002, 2003). In contrast, the N1 component was largest over the hemisphere contralateral to the side of the target (contralateral vs ipsilateral, F_(1,15) = 10.50, p < 0.006).

Figure 5.

Left, ERPs to visual targets (letters), recorded at occipital sites I5/I6 contralateral to the target location. These waveforms were obtained by collapsing ERPs recorded from left and right hemispheres elicited by stimuli in the right and left visual fields, respectively. Measurement intervals for the P1 (120–140 ms), N1 (160–190 ms), and P3/P300 (400–500 ms) components are shaded. Right, Topographical voltage distributions for the correct minus incorrect difference amplitudes for each component. Maps depict voltages contralateral to the stimulus on the right side and ipsilateral on the left.

A late slow positivity extending over 350–600 ms in the target waveforms (Fig. 5) most likely corresponds to a P3/P300 component associated with detections of task-relevant events (Hillyard and Picton, 1987). This slow positivity, quantified over 400–500 ms at which its amplitude was maximal, was much larger for correct than incorrect trials over both validity conditions (F_(1,15) = 35.81, p < 0.0001), with no significant accuracy × validity interaction (F_(1,15) = 0.10, p < 0.746). The broad amplitude distribution of this positivity with a parietal maximum is consistent with that reported previously for the P300 (Hillyard and Picton 1987).

Discussion

A previous study (McDonald et al., 2013) identified a late positive ERP that was elicited in the ventrolateral visual cortex in the hemisphere contralateral to the side of a salient sound, regardless of whether the sound was relevant to the ongoing task assignment. This ACOP was interpreted as an automatic cross-modal activation of the visual cortex by the sound. The present study explored the functional significance of the ACOP in a task in which lateralized, nonpredictive sounds preceded masked letters (T or L) that had to be discriminated. Consistent with previous studies (Dufour, 1999; McDonald et al., 2000; Frassinetti et al., 2002), letters presented on the same side as the preceding sound (valid condition) could be discriminated faster and more accurately than letters presented on the opposite side (invalid trials). Most importantly, when ERPs were averaged separately on a trial-by-trial basis according to the correctness of the subject's letter discrimination, it was found that the ACOP elicited before correct discriminations was substantially larger than preceding incorrect discriminations for the validly cued letters. In fact, the ACOP was completely absent on valid trials that resulted in an incorrect discrimination. The ACOP preceding correct discriminations was localized to the ventrolateral extrastriate visual cortex (Brodmann's area 19), as reported previously (McDonald et al., 2013). The present results extend the previous finding of McDonald et al. (2013) that larger ACOP amplitudes were associated with enhancement of the simple sensory feature of luminance contrast. The present trial-by-trial analysis demonstrates that the ACOP reflects an automatic cross-modal priming of the visual cortex that facilitates the perceptual processing of a subsequent visual event at the cued location and enables more effective discrimination of its detailed pattern characteristics.

It is noteworthy that the lateralized ACOP amplitude was not predictive of correct versus incorrect letter discriminations on invalidly cued trials. The implication is that trial-by-trial fluctuations in ACOP-related neural activity in the occipital cortex contralateral to the auditory cue influence discriminative processing of visual targets at the cued location but not at the uncued location. It should be noted that the ACOP is measured before the validity of the cue is known to the subject, and therefore the pretarget ACOP fluctuations must have the same trial-to-trial amplitude distribution on valid and invalid trials. The finding of much larger ACOPs on correct versus incorrect trials for valid trials but intermediate amplitudes on both correct and incorrect invalid trials is exactly what would be expected if the ACOP elicited by the auditory cue resulted in enhanced processing at the cued location without any suppression of visual processing at the uncued location. In the latter case, we would expect to find a larger ACOP (more positive in the hemisphere contralateral to the auditory cue) preceding incorrect than correct discriminations of invalidly cued targets.

A more specific account of these invalid trial results would follow from the hypothesis that the ACOP reflects the orienting of attention toward the cued location (McDonald et al., 2013), which improves perceptual discriminations at that location. According to this account, correct performance on invalid trials would depend on a reorienting of attention to the target appearing at the uncued location, and the present ACOP results suggest that the effectiveness of the reorienting (indexed by the accuracy of discrimination of invalid targets) does not depend on the strength of the original orienting (as indexed by the ACOP). This proposal that correct invalid discriminations are associated with a reorienting of attention is consistent with the pattern of associated modulations of the ERPs to the visual targets (see below).

The ERPs to the visual targets were also differentiated according to whether they occurred on the same side as the auditory cue (valid trials) or the opposite side (invalid trials). For validly cued targets, the amplitude of the occipital P1 component (120–140 ms) was larger on trials having correct target discriminations than incorrect. In contrast, for the invalidly cued targets, it was the N1 component alone that was enlarged on trials with correct versus incorrect target discriminations. Such dissociations between attention effects on the P1 and N1 components have been observed in previous studies, suggesting that they reflect different aspects of attentional selection (for review, see Hopfinger et al., 2004; Hopf et al., 2009). A prominent hypothesis has been that the P1 amplitude modulations with attention reflect an early gain control mechanism (Heinze et al., 1990; Hillyard et al., 1998), which in the present study would be associated with larger anticipatory ACOP amplitudes on valid trials. This correlation suggests that the ACOP may reflect the lateralized orienting of an attentional process that increases the sensory gain of visual targets at the cued location. In contrast, previous studies found the N1 to be enlarged when attention was oriented or reoriented to a relevant stimulus location for additional discriminative processing (Luck et al., 1990; Vogel and Luck 2000). In the present study, these ERP modulations suggest that the invalidly cued targets did not receive the early gain control boost (indexed by P1), but discrimination performance was better for those invalid targets when subjects reoriented their attention to the actual target location (indexed by N1). The absence of a P1 modulation with correctness on the invalid trials is in accord with the hypothesis (based on the ACOP results) that the attentional process associated with ACOP is an early sensory gain enhancement that only affects processing at the cued location.

This linkage of the ACOP with facilitated perceptual processing at the cued location may be contrasted with the properties of other EEG/ERP components elicited during voluntary shifts of attention. Specifically, previous EEG studies of voluntary attention have revealed that attention-directing symbolic cues trigger a lateralized ERP component called the late directing attention positivity (LDAP), as well as lateralized changes in alpha-band (8–14 Hz) oscillatory activity (Harter et al., 1989, Hopf and Mangun, 2000; Worden et al., 2000; Kelly et al., 2006; Rihs et al., 2007; Hanslmayr et al., 2011). However, both the LDAP and the cue-triggered changes in alpha-band activity are typically manifested as increased activity over the occipital hemisphere ipsilateral to the cued hemifield. These ipsilateral changes have been interpreted in terms of active suppression of the to-be-ignored visual hemifield (Worden et al., 2000; Kelly et al., 2006; Rihs et al., 2007; McDonald and Green, 2008; Green and McDonald, 2010; Foxe and Snyder, 2011). Thus, there may be a fundamental distinction between the automatic, involuntary orienting of attention triggered by a nonpredictive auditory cue (which primarily facilitates the cued hemifield) and the voluntary shifting of attention by a symbolic, predictive cue (which primarily suppresses the uncued hemifield).

In a recent series of studies, the presentation of a sound was found to increase the rate of perceiving phosphenes that were induced by transcranial magnetic stimulation over the visual cortex (Romei et al., 2007, 2009, 2013; Bolognini et al., 2010; Spierer et al., 2013). This sound-induced facilitatory effect on cortical excitability begins as early as 30 ms after sound onset (Spierer et al., 2013) and was associated with phase-locking of the alpha rhythm in visual areas (Romei et al., 2012). Additional research is needed to determine whether the ACOP is associated with such EEG phase modulations and whether it depends on very early auditory input to the visual cortex.

In other recent studies, target sounds delivered to the left or right ear elicited a long-latency (300–600 ms) contralateral positivity with a scalp topography similar to that of the ACOP (Gamble and Luck, 2011; Gamble and Woldorff, 2014). Whereas this late contralateral positivity was triggered only by task-relevant sounds, the ACOP is readily elicited by task-irrelevant and nonpredictive sounds (McDonald et al., 2013); accordingly, McDonald et al. hypothesized that the ACOP represents an automatic, exogenously driven cross-modal activation of visual cortex. The present results are consistent with this hypothesis of automatic activation in that the spatially nonpredictive sounds nonetheless elicited a lateralized ACOP in visual cortex over 200–450 ms that covaried with improved processing of a spatially coincident visual target that followed the sound by 400 ms. Additional studies are needed to investigate whether the posterior positivity reported by Gamble and colleagues represents a delayed activation of the visual cortex by a relevant sound and whether it may correlate with visual perception.

In summary, the present study found that the automatic activation of the ventrolateral visual cortex by a salient sound has important consequences for perception of a subsequent visual target. The finding that sound-induced ACOP amplitudes were larger preceding correct than incorrect letter discriminations on validly cued trials provides the first direct neurophysiological evidence that the lateralized activation of visual cortex by a spatially nonpredictive auditory cue results in more effective perceptual processing of visual pattern information at the cued location. The cross-modal influences of nonpredictive auditory cues on visual perception reported here and in previous studies (for review, see McDonald et al., 2012) may well be a consequence of the automatic orienting of spatial attention to the location of the salient sound.