Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Oct 1.
Published in final edited form as: Cognition. 2011 Jul 8;121(1):133–139. doi: 10.1016/j.cognition.2011.06.003

Changes in auditory frequency guide visual-spatial attention

Julia A Mossbridge 1, Marcia Grabowecky 1,2, Satoru Suzuki 1,2
PMCID: PMC3149771  NIHMSID: NIHMS305847  PMID: 21741633

Abstract

How do the characteristics of sounds influence the allocation of visual-spatial attention? Natural sounds typically change in frequency. Here we demonstrate that the direction of frequency change guides visual-spatial attention more strongly than the average or ending frequency, and provide evidence suggesting that this cross-modal effect may be mediated by perceptual experience. We used a Go/No-Go color-matching task to avoid response compatibility confounds. Participants performed the task either with their heads upright or tilted by 90°, misaligning the head-centered and environmental axes. The first of two colored circles was presented at fixation and the second was presented in one of four surrounding positions in a cardinal or diagonal direction. Either an ascending or descending auditory-frequency sweep was presented coincident with the first circle. Participants were instructed to respond to the color match between the two circles and to ignore the uninformative sounds. Ascending frequency sweeps facilitated performance (response time and/or sensitivity) when the second circle was presented at the cardinal top position and descending sweeps facilitated performance when the second circle was presented at the cardinal bottom position; there were no effects of the average or ending frequency. The sweeps had no effects when circles were presented at diagonal locations, and head tilt entirely eliminated the effect. Thus, visual-spatial cueing by pitch change is narrowly tuned to vertical directions and dominates any effect of average or ending frequency. Because this cross-modal cueing is dependent on the alignment of head-centered and environmental axes, it may develop through associative learning during waking upright experience.

Keywords: cross-modal perception, auditory-visual interactions, visual-spatial attention, implicit attentional processing, multi-modal cognition

Introduction

In everyday experience, hearing a sound guides our visual attention to the location of the sound source (e.g., Bolognini et al., 2005; Driver & Spence, 1998; Frassinetti et al., 2002; Stein et al., 1989; for review see Talsma et al. 2010). More surprisingly, even when sounds come from a single location, different sounds can guide visual attention in distinct ways. For example, characteristic sounds (e.g., a dog bark) speed and guide eye movements toward congruous pictures (e.g., a dog) in a visual-search paradigm even when the sounds provide no location information (Iordanescu et al., 2008; 2010).

Sounds with no object-specific referent but with differing qualities, such as sounds with high versus low steady pitch or high versus low intensity, direct attention to the top or the bottom of the visual field, respectively (Ben-Artzi & Marks, 1995; Bernstein & Edelstein, 1971; Evans & Treisman, 2010; Melara & O'Brien, 1987; Patching & Quinlan, 2002; Pratt, 1930; Widmann et al., 2007; Widmann et al., 2004). It is not yet clear whether such implicit sound-quality effects on spatial attention are the result of innate wiring constraints or are learned associations (Marks, 1987; Melara & O'Brien, 1987). Although this question could be informed by clarifying the crucial parameters that modulate these phenomena, there has been little parametric investigation beyond the initial description of the effects themselves. Furthermore, naturally occurring sounds typically change in pitch. Here we demonstrate that for sounds with changing pitch the direction of frequency change guides visual-spatial attention more strongly than average or ending frequency, and we present a careful investigation of this cross-modal effect to elucidate whether it is mediated by perceptual experience.

In four experiments, we (1) compared the effect of the direction of frequency modulation (ascending or descending) with the effect of average and ending frequencies themselves (high or low), (2) determined whether these sounds guided attention in specific directions (e.g., upward or downward) or to general regions (e.g., the upper field or the lower field), and (3) investigated whether these cross-modal effects depended on perceptual experience by testing a condition in which retinotopic and environmental directions were misaligned.

Experiment One: Does frequency-modulation direction influence visual-spatial attention?

We orthogonally varied the direction of frequency modulation (ascending or descending) and the average frequency (high or low) of a sound presented immediately prior to a visual probe. In this way, we determined how frequency-modulation direction and overall frequency influenced visual-spatial attention.

Method

Participants

Thirty-three Northwestern undergraduates (20 women, 18–21 years of age, 1 left-handed) gave informed consent to participate in the experiment for course credit. All participants had normal or corrected-to-normal visual acuity, color vision, and hearing.

Stimuli

The visual stimuli are illustrated in Figure 1A. A central fixation marker “+” (.41° by .41° visual angle) and the four surrounding squares (2.3° by 2.3° visual angle) were drawn with dark lines (5.9 cd/m2) against a white background (112 cd/m2). Each square was equidistant from the central fixation marker (6.3° from fixation to center of square). The colors used for the reference and probe circles (1.8° visual angle in diameter) were blue (CIE[.240,.248], 68.1 cd/m2), green (CIE[.287,.342], 77.9 cd/m2), pink (CIE[.305,.316], 80.7 cd/m2), and orange (CIE[.331,.358], 85.5 cd/m2). Viewing distance was approximately 70 cm. Auditory stimuli were four 500 ms linear frequency-modulated sound sweeps (~70 db SPL): two ascending sweeps of either lower (changing from 300 Hz to 450 Hz) or higher (450 to 600 Hz) average frequency, and two descending sweeps with either lower (450 to 300 Hz) or higher (600 to 450 Hz) average frequency. On each trial, one of the four sweeps was randomly selected with equal probability.

Figure 1.

Figure 1

Open in a new tab

Schematic of Experiments One, Two and Four, all of which used the same Go/No-Go color-matching task (see Methods for details). On each trial, two colored circles were sequentially presented, the first (the reference circle) at fixation and the second (the probe circle) in one of four surrounding squares; on a “Go” trial the colors of the reference and probe circles matched. An ascending or descending frequency-modulated sound coincided with the reference circle. The three experiments differed in the arrangement of the surrounding squares and the participant's head orientation: (A) In Experiment One, the squares were at cardinal locations and the participant's head was upright. (B) In Experiment Two, the squares were at diagonal locations and the participant's head was upright. Experiment Three (not shown) was similar to Experiments One and Two, but only two squares were presented in either the vertically aligned or diagonally aligned locations (see text for details). (C) In Experiment Four, the squares were at cardinal locations and the participant's head was tilted 90° to the right.

Apparatus

The visual stimuli were presented on an LCD color monitor and the auditory stimuli were binaurally presented using Sennheiser HD280pro headphones. The experiment was controlled using Psychtoolbox 3.0.8 (Brainard, 1997; Pelli, 1997) in MATLAB 7.4.0 (Mathworks, MA) with a Dell desktop computer.

Procedure and Conditions

Participants performed 200 trials of a Go/No-Go color-matching task (Figure 1A), similar to that used by Pratt and Hommel (2003). Participants were informed that they would hear sounds during each trial, but that these sounds were uninformative. They were instructed to ignore the sounds and concentrate on fixating on the fixation marker until the probe circle was presented in one of the four peripheral squares.

Each trial began with the display of central fixation marker and four surrounding squares presented at top, bottom, left, and right locations; this display lasted 1 sec. The reference circle then appeared at the center for 500 ms, accompanied by one of the four frequency-modulated sound sweeps. Upon offset of the reference circle and sound, the probe circle appeared in one of the four squares for 750 ms. Participants were asked to press the space bar on a computer keyboard as quickly as possible if the colors of the probe and reference circles matched; this was the case on 75% of trials. If the colors did not match, participants were instructed to withhold response. We used a Go/No-Go paradigm to avoid response-compatibility effects. There was no delay between the offset of the reference circle and the onset of the probe circle, thus any working-memory load would have been minimal. Response times (RT) and accuracy (hits and false alarms) were recorded as the dependent measures. Following the participant's response (or after the 750 ms response deadline if no response), the trial ended and the next trial began after 1 sec.

Results

Frequency-modulated sounds produced direction-specific effects on visual responses. When collapsed across average auditory frequency, there was a significant auditory-sweep-direction (ascending or descending) by visual-target-location (top, bottom, left, or right) interaction in RT, F(3,96) = 3.371, p < 0.022 (Figure 2A, large plot). Follow-up analyses showed that, along the vertical axis, responses to the top location were faster following ascending frequency sweeps than following descending frequency sweeps, and vice versa for responses to the bottom location, F(1,32) = 16.654, p < 0.0003 (sweep-direction by visual-target-location [top or bottom] interaction). Along the horizontal axis, there was a trend that responses to the right location were faster following ascending frequency sweeps than following descending frequency sweeps, and vice versa for the left location, F(1,32) = 3.931, p < 0.057 (sweep-direction by visual-target-location [right or left] interaction).

Figure 2.

Figure 2

Open in a new tab

Mean response times (RT; large plots) for correct “Go” trials and sensitivity values (d; small plots) in Experiment One; faster RTs and greater sensitivity values are plotted toward the outer ring of the polar plots. Error bars represent ± 1 SEM adjusted for the within-participant design. (A) Frequency-modulation effect (collapsed across frequency). Dashed lines connect mean values for trials with ascending-frequency sounds; solid lines connect mean values for trials with descending-frequency sounds. (B) Frequency effect (collapsed across frequency-modulation direction). Dashed lines connect mean values for trials with higher-frequency sounds; solid lines connect mean values for trials with lower-frequency sounds.

For vertical directions, sensitivity (d, MacMillan & Creelman, 2005) followed the same pattern as RT, though the d effect was primarily driven by the higher sensitivity at the bottom location following descending sweep (Figure 2A, small plot), F(1,32) = 4.677, p < 0.04 (sweep-direction by visual-target-location [top or bottom] interaction, with no significant interaction for criterion [c]; F[1,32] = 0.000, n.s.). The similar effects on RT and sensitivity for vertical locations suggests that ascending and descending auditory frequency sweeps guide attention to visual objects presented above and below fixation.

For horizontal directions, sensitivity followed the opposite pattern to RT, higher sensitivity at the left location following ascending frequency sweeps than following descending frequency sweeps, and vice versa at the right location, F(1,32) = 6.577, p < 0.016 (sweep-direction by visual-target-location [right or left] interaction, with no significant interaction for c; F(1,32) = 0.466, n.s.). This indicates a speed-sensitivity trade-off. Ascending and descending auditory frequency sweeps speeded responses to the right and left locations at the expense of sensitivity.

Compared to the robust cross-modal effects of ascending and descending frequency sweeps, average auditory frequency (higher or lower, collapsed across sweep direction) had little effect (Figure 2B). It appears that responses to the top location were faster following higher than with lower auditory frequencies. However, for both RT and sensitivity, there were no significant main effects or interactions involving average frequency (RT: Fs < 1.560; d': Fs < 0.408; c: Fs < 0.441). This result indicates that there were also no effects involving ending frequency, because the higher average frequency stimuli had higher ending frequencies (450 and 600 Hz), and the lower average frequency stimuli had lower ending frequencies (450 and 300 Hz).

Taken together, the response-time and sensitivity results suggest that auditory frequency modulation (beyond average and ending frequency) produces a cross-modal effect in which ascending and descending auditory frequency sweeps guide visual-spatial attention in upward and downward directions, respectively.

Experiment Two: Are influences of frequency-modulation direction on attention limited to the vertical axis?

The aim of the second experiment was to test the potential vertical tuning of this cross-modal effect by using diagonal directions. If auditory frequency sweeps were strongly associated with upward and downward visual directions, they should produce little effect in the diagonal directions.

Method

The methods for this experiment were the same as in Experiment One, except that the four boxes surrounding the central fixation marker were arranged in diagonal directions (Figure 1B). A new group of thirty-three Northwestern undergraduates (23 women, 18–21 years old, all right handed) gave informed consent to participate in the experiment for course credit. All participants had normal or corrected-to-normal visual acuity, color vision, and hearing.

Results

In contrast to the results for cardinal directions in Experiment One, auditory frequency sweeps did not produce cross-modal effects for diagonal directions (Figure 3A). For both RT and d' the data revealed no significant main effects or interactions involving auditory frequency-sweep direction (RT: Fs < 1.870; d': Fs < 0.642; c: Fs < 0.353). Similarly, average auditory frequency had little effect on either measure (RT: Fs < 1.870; d': Fs < 1.807; c: Fs < 0.573) (Figure 3B).

Figure 3.

Figure 3

Open in a new tab

Mean response times (RT; large plots) for correct “Go” trials and sensitivity values (d•; small plots) in Experiment Two; faster RTs and greater sensitivity values are plotted toward the outer ring of the polar plots. Error bars represent ± 1 SEM adjusted for the within-participant design. (A) Frequency-modulation effect (collapsed across frequency). Dashed lines connect mean values for trials with ascending-frequency sounds; solid lines connect mean values for trials with descending-frequency sounds. (B) Frequency effect (collapsed across frequency-modulation direction). Dashed lines connect mean values for trials with higher-frequency sounds; solid lines connect mean values for trials with lower-frequency sounds.

One possible explanation for the lack of a cross-modal effect in this experiment is that the ascending and descending frequency sweeps broadly guided attention in the regions above and below the fixation marker, respectively. Because there were two potential probe locations within each region in this experiment (compared to only one in Experiment One), the null effect of frequency sweeps could have resulted from spreading attention across potential target locations. To resolve this issue, we performed a control experiment using only two possible target locations, one above and one below the fixation marker, arranged either vertically or diagonally.

Experiment Three: Is the lack of an influence of frequency-modulation direction on diagonal locations due to diffuse spread of attention?

Method

The methods for this experiment were the same as in Experiments One and Two, except that three blocked conditions were presented, each with only two possible target locations: (1) top and bottom (vertical), (2) top left and bottom right (diagonal), and (3) top right and bottom left (diagonal). The condition order was counterbalanced. A new group of thirty-six Northwestern undergraduates (18 women, 18–21 years old, all right handed) gave informed consent to participate in the experiment for course credit. All participants had normal or corrected-to-normal visual acuity, color vision and hearing.

Results

Auditory frequency sweeps again induced a cross-modal attentional effect, but only for the vertical condition (Table 1). The vertical condition produced a significant interaction between frequency-sweep direction and probe location for RT (F[1,35]=7.791, p<0.009). Sensitivity was not significantly affected (F[1,35]=2.808, n.s.), but criterion shifted towards the optimal value (equivalent to β = 1/3 for our 3:1 Go to No-Go ratio) in the congruent auditory-visual pairings (probe at the top location following ascending sweep and probe at the bottom location following descending sweep) compared with the incongruent auditory-visual pairings (probe at the top location following descending sweep and probe at the bottom location following ascending sweep) (F[1,35]=5.057, p<0.04). Neither diagonal condition produced significant interactions for RT, d', or c, nor was the average frequency effect significant for any measure from any of the three conditions (Table 1). The lack of a cross-modal effect at diagonal locations in this experiment, where potential spreading of attention was equivalent for the vertical and diagonal conditions, indicates that the association between auditory frequency-sweep direction and visual-spatial attention is specifically tuned to vertical directions.

Table 1.

Means, standard deviations (sd; corrected for within-participant comparisons), and interaction statistics for reaction time (RT) and sensitivity (d') measures from the three blocked conditions of Experiment Three. A significant interaction indicates that frequency sweeps (or average frequencies) guide attention to the upper and lower locations over and above any overall effects of specific sounds or probe location. (A) Potential probe locations at the top and bottom of the screen only. (B) Potential probe locations at the top left and bottom right only. (C) Potential probe locations at the top right and bottom left only.

graphic file with name nihms-305847-t0001.jpg freq. sweeps avg. freq.
ascend. descend. higher lower
mean RT in ms (sd) Top 439 (15) 446 (15) 441 (17) 444 (12)
Bottom 438 (16) 430 (16) 434 (14) 435 (14)
freq. sweep (or avg. freq.) × position interaction F(1,35)=7.791, p<0.009 F(1,35)=0.581, n.s.
mean d' (sd) Top 3.0 (.6) 3.2 (.3) 3.1 (.3) 3.1 (.4)
Bottom 3.1 (.4) 3.0 (.4) 3.1 (.3) 3.0 (.4)
freq. sweep (or avg. freq.) × position interaction F(1,35)=2.808, n.s. F(1,35)=1.841, n.s.
graphic file with name nihms-305847-t0002.jpg freq. sweeps avg. freq.
ascend. descend. higher lower
mean RT in ms (sd) Top Left 430 (17) 436 (13) 431 (13) 434 (15)
Bottom Right 426 (17) 424 (11) 427 (13) 424 (13)
freq. sweep (or avg. freq.) × position interaction F(1,35)=1.615, n.s. F(1,35)=2.860, n.s.
mean d' (sd) Top Left 3.1 (.4) 3.1 (.4) 3.1 (.4) 3.2 (.4)
Bottom Right 3.1 (.5) 3.0 (.5) 3.0 (.5) 3.1 (.4)
freq. sweep (or avg. freq.) × position interaction F(1,35)=0.690, n.s. F(1,35)=0.037, n.s.
graphic file with name nihms-305847-t0003.jpg freq. sweeps avg. freq.
ascend. descend. higher lower
mean RT in ms (sd) Top Right 432 (15) 429 (15) 428 (11) 433 (17)
Bottom Left 428 (15) 426 (13) 424 (14) 430 (14)
freq. sweep (or avg. freq.) × position interaction F(1,35)=0.031, n.s. F(1,35)=0.093, n.s.
mean d' (sd) Top Right 3.2 (.4) 3.1 (.4) 3.2 (.4) 3.1 (.5)
Bottom Left 3.1 (.4) 3.0 (.4) 3.1 (.4) 3.1 (.4)
freq. sweep (or avg. freq.) × position interaction F(1,35)=0.037, n.s. F(1,35)=2.466, n.s.
Open in a new tab

Experiment Four: Is the effect of frequency-modulation direction retinotopic?

Participants tilted their heads by 90° to misalign the head-centered and environmental axes. If the cross-modal mapping is based on a retinotopic representation, tilting the head by 90° should shift the response pattern by 90°, but if the mapping is based on environmental coordinates, tilting the head should make no difference. Alternatively, if the cross-modal mapping is closely associated with typical upright experience, tilting the head by 90° might eliminate the cross-modal attention effect.

Method

The methods for this experiment were the same as in experiment one, except that the participant's head was tilted 90° to the right and rested on a pillow (Figure 1C). A new group of thirty-three Northwestern undergraduates (18 women, 18–21 years old, all right handed) gave informed consent to participate in the experiment for course credit. All participants had normal or corrected-to-normal visual acuity, color vision and hearing.

Results

When a 90° head tilt was used to separate the head-centered axis from the environmental axis, the direction-specific effect of auditory frequency sweeps disappeared (Figure 4A). There was no significant main effect of auditory frequency-sweep direction on RT or d' (RT: F < 1.176; d': F < 1.851), nor did it interact with probe location (RT: F < 1.116; d': F < 0.494). Although ascending frequency sweeps shifted c closer to the optimal value relative to descending sweeps (F[1, 32]=7.121, p<0.012), this effect did not interact with probe location (F[3, 96]=0.646, n.s.). In terms of average frequency, higher frequency speeded responses at all locations (Figure 4B) as indicated by a main effect of average frequency on RT, F(1,32)=5.328, p<0.028, but this effect did not interact with probe location, F(1,32)=0.391, n.s. It is possible that listening to ascending frequency sweeps and higher-pitched sounds caused increased arousal for this group of participants. There were no main effects or interactions involving average frequency on d' (Fs < 0.573) or c (Fs <2.424).

Figure 4.

Figure 4

Open in a new tab

Mean response times (RT; large plots) for correct “Go” trials and sensitivity values (d'; small plots) in Experiment Four; faster RTs and greater sensitivity values are plotted toward the outer ring of the polar plots. Note that in this experiment, the retinotopic vertical and environmental vertical were orthogonal. Error bars represent ± 1 SEM adjusted for the within-participant design. (A) Frequency-modulation effect (collapsed across frequency). Dashed lines connect mean values for trials with ascending-frequency sounds; solid lines connect mean values for trials with descending-frequency sounds. (B) Frequency effect (collapsed across frequency-modulation direction). Dashed lines connect mean values for trials with higher-frequency sounds; solid lines connect mean values for trials with lower-frequency sounds.

Curiously, responses to all stimuli were faster at the environmental bottom, as revealed by a main effect of location, F(3,96)=8.562, p<0.00005. Pilot data from a head-tilt experiment using no auditory stimuli show the same downward bias, suggesting that this is a non-auditory effect.

Discussion

Natural sounds tend to be frequency modulated. Our results suggest that sounds with ascending frequency modulation guide visual spatial attention upwards whereas sounds with descending frequency modulation direct attention downwards. These cross-modal attention effects are likely automatic rather than volitional because the sounds were uninformative about the upcoming probe location, spatial information was irrelevant to the color-matching task, and participants were instructed to ignore the sounds. Interestingly, we found no location-specific effect of average or ending frequency in any of the four experiments. One possibility is that the presence of frequency modulation in the stimuli superceded the previously demonstrated association between static auditory frequency and visual elevation (Ben-Artzi & Marks, 1995; Bernstein & Edelstein, 1971; Evans & Treisman, 2010; Melara & O'Brien, 1987; Patching & Quinlan, 2002; Pratt, 1930; Widmann et al., 2007; Widmann et al., 2004).

Surprisingly, the cross-modal attention-cueing effect is not present for diagonal directions, indicating its narrow directional tuning (less than 45°). This narrow directional tuning suggests that ascending and descending tones may guide attention by interacting with visual motion mechanisms, which are known to be directionally tuned (e.g., Priebe et al., 2003). This interpretation echoes previous work showing that sounds with ascending and descending frequency modulations are judged to be associated with upward and downward motion, respectively, across cultures by both musicians and non-musicians (Deutsch et al., 2007; Eitan & Granot, 2006; Walker, 1987), and that these associations also induce a visual illusion of upward and downward motion (Jain et al., 2008; Maeda et al., 2004; Miller et al., 1958).

Regardless of the mechanisms underlying the association between frequency modulation and visual attention, the association seems to be learned rather than innate. Whereas the cross-modal attention effect disappears when the head-centered and environmental axes are misaligned by tilting participants' heads by 90°, effects of innate auditory-visual neural connections are unlikely to depend on specific posture relative to the environment. For example, we recently reported that a lifelong synesthete who hears higher/lower static pitches in association with upward-/downward-moving sinusoidal gratings maintains this association retinotopically when her head is tilted 90° (Noble et al., 2010). Thus, the fact that the current cross-modal attention-cueing effect depends on upright head orientation suggests that the underlying associations between frequency modulated sounds and environmental vertical directions develop through audiovisual experience that occurs predominantly in an upright posture. There remains, however, open questions as to which auditory-visual experiences form this association and what adaptive purposes it may serve.

  • >

    Sounds ascending/descending in pitch guide visual-spatial attention upward/downward.

  • >

    The effect dominates the attentional effects of unchanging pitch.

  • >

    This cross-modal attentional effect is specific to vertical (not diagonal) visual locations.

  • >

    The effect is obliterated when the head axis is not aligned with the body axis.

Acknowledgments

We thank John Middlebrooks for helpful dialogue. Funding sources: NSF BCS0643191, NIH R01 EY018197, NIH 5T32NS047987-05.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Ben-Artzi E, Marks LE. Visual-auditory interaction in speeded classification: role of stimulus difference. Percept Psychophys. 1995;57(8):1151–1162. doi: 10.3758/bf03208371. [DOI] [PubMed] [Google Scholar]
  2. Bernstein IH, Edelstein BA. Effects of some variations in auditory input upon visual choice reaction time. J Exp Psychol. 1971;87(2):241–247. doi: 10.1037/h0030524. [DOI] [PubMed] [Google Scholar]
  3. Bolognini N, Frassinetti F, Serino A, Ladavas E. “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs. Exp Brain Res. 2005;160(3):273–282. doi: 10.1007/s00221-004-2005-z. [DOI] [PubMed] [Google Scholar]
  4. Brainard D. The psychophysics toolbox. Spatial Vision. 1997;10:433–446. [PubMed] [Google Scholar]
  5. Deutsch D, Hamaoui K, Henthorn T. The glissando illusion and handedness. Neuropsychologia. 2007;45(13):2981–2988. doi: 10.1016/j.neuropsychologia.2007.05.015. [DOI] [PubMed] [Google Scholar]
  6. Driver J, Spence C. Attention and the crossmodal construction of space. Trends in Cognitive Sciences. 1998;2:254–262. doi: 10.1016/S1364-6613(98)01188-7. [DOI] [PubMed] [Google Scholar]
  7. Eitan Z, Granot RY. HOW MUSIC MOVES: Musical Parameters and Listeners' Images of Motion. Music Perception. 2006;23(3):27. [Google Scholar]
  8. Evans KK, Treisman A. Natural cross-modal mappings between visual and auditory features. J Vis. 2010;10(1):1–12. doi: 10.1167/10.1.6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Frassinetti F, Bolognini N, Ladavas E. Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res. 2002;147(3):332–343. doi: 10.1007/s00221-002-1262-y. [DOI] [PubMed] [Google Scholar]
  10. Iordanescu L, Guzman-Martinez E, Grabowecky M, Suzuki S. Characteristic sounds facilitate visual search. Psychon Bull Rev. 2008;15(3):548–554. doi: 10.3758/pbr.15.3.548. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Iordanescu L, Grabowecky M, Franconeri S, Theeuwes J, Suzuki S. Characteristic sounds make you look at target objects more quickly. Attention, Perception & Psychophysics. 2010;72(7):1736–1741. doi: 10.3758/APP.72.7.1736. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Jain A, Sally SL, Papathomas TV. Audiovisual short-term influences and aftereffects in motion: examination across three sets of directional pairings. J Vis. 2008;8(15):7, 1–13. doi: 10.1167/8.15.7. [DOI] [PubMed] [Google Scholar]
  13. MacMillan NA, Creelman CD. Detection Theory: A User's Guide. Laurence Erlbaum Associates; New York: 2005. [Google Scholar]
  14. Maeda F, Kanai R, Shimojo S. Changing pitch induced visual motion illusion. Curr Biol. 2004;14(23):R990–991. doi: 10.1016/j.cub.2004.11.018. [DOI] [PubMed] [Google Scholar]
  15. Marks LE. On cross-modal similarity: auditory-visual interactions in speeded discrimination. J Exp Psychol Hum Percept Perform. 1987;13(3):384–394. doi: 10.1037//0096-1523.13.3.384. [DOI] [PubMed] [Google Scholar]
  16. Melara R, O'Brien T. Interaction between synesthetically corresponding dimensions. Journal of Experimental Psychology: General. 1987;116:323–336. [Google Scholar]
  17. Miller A, Wener H, Wapner S. Studies in physiognomic perception: V. Effect of ascending and descending gliding tones on audokinetic motion. Journal of Psychology. 1958;46:101–105. [Google Scholar]
  18. Noble C, Mossbridge J, Iordanescu L, Sherman A, List A, Grabowecky M, Suzuki S. Motion induced pitch: A case of visual-auditory synesthesia. Journal of Vision. 2010 [Google Scholar]
  19. Patching GR, Quinlan PT. Garner and congruence effects in the speeded classification of bimodal signals. J Exp Psychol Hum Percept Perform. 2002;28(4):755–775. [PubMed] [Google Scholar]
  20. Pelli D. The videotoolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision. 1997;10:437–442. [PubMed] [Google Scholar]
  21. Pratt C. The spatial character of high and low tones. Journal of Experimental Psychology. 1930;13:278–285. [Google Scholar]
  22. Pratt J, Hommel B. Symbolic control of visual attention: The role of working memory and attentional control settings. Journal of Experiment Psychology: Human Perception and Performance. 2003;29:835–45. doi: 10.1037/0096-1523.29.5.835. [DOI] [PubMed] [Google Scholar]
  23. Priebe NJ, Cassanello CR, Lisberger SG. The neural representation of speed in macaque area MT. Journal of Neuroscience. 2003;23(13):5650–5661. doi: 10.1523/JNEUROSCI.23-13-05650.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Stein B, Meredith M, Huneycutt W, McDade L. Behavioral indices of multisensory integration: orientation to visual cues is affected by auditory stimuli. Journal of Cognitive Neuroscience. 1989;1(1):12–24. doi: 10.1162/jocn.1989.1.1.12. [DOI] [PubMed] [Google Scholar]
  25. Talsma D, Senkowski D, Soto-Faraco S, Woldorff MG. The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences. 2010;14:400–10. doi: 10.1016/j.tics.2010.06.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Wagner S, Winner E, Cicchetti D, Gardner H. “Metaphorical” mapping in human infants. Child Development. 1981;52(2):728–731. [Google Scholar]
  27. Walker P, Bremmer J, Mason U, Spring J, Mattock K, Slater A, Johnson S. Preverbal infants' sensitivity to synaesthetic cross-modality correspondences. Psychological Science. 2010;21(1):21–25. doi: 10.1177/0956797609354734. [DOI] [PubMed] [Google Scholar]
  28. Walker R. The effects of culture, environment, age, and musical training on choices of visual metaphors for sound. Perception and Psychophysics. 1987;42(5):491–502. doi: 10.3758/bf03209757. [DOI] [PubMed] [Google Scholar]
  29. Widmann A, Gruber T, Kujala T, Tervaniemi M, Schroger E. Binding symbols and sounds: evidence from event-related oscillatory gamma-band activity. Cereb Cortex. 2007;17(11):2696–2702. doi: 10.1093/cercor/bhl178. [DOI] [PubMed] [Google Scholar]
  30. Widmann A, Kujala T, Tervaniemi M, Kujala A, Schroger E. From symbols to sounds: visual symbolic information activates sound representations. Psychophysiology. 2004;41(5):709–715. doi: 10.1111/j.1469-8986.2004.00208.x. [DOI] [PubMed] [Google Scholar]

RESOURCES