Intersensory Redundancy Guides Attentional Selectivity and Perceptual Learning in Infancy

Abstract

This study assessed an intersensory redundancy hypothesis, which holds that in early infancy information presented redundantly and in temporal synchrony across two sense modalities selectively recruits attention and facilitates perceptual differentiation more effectively than does the same information presented unimodally. Five-month-old infants’ sensitivity to the amodal property of rhythm was examined in 3 experiments. Results revealed that habituation to a bimodal (auditory and visual) rhythm resulted in discrimination of a novel rhythm, whereas habituation to the same rhythm presented unimodally (auditory or visual) resulted in no evidence of discrimination. Also, temporal synchrony between the bimodal auditory and visual information was necessary for rhythm discrimination. These findings support an intersensory redundancy hypothesis and provide further evidence for the importance of redundancy for guiding and constraining early perceptual learning.

The newborn infant encounters a world of objects and events that present a richly structured array of stimulation to all the senses. Recent research demonstrates that young infants are adept perceivers of this multimodal stimulation (see Lewkowicz & Lickliter, 1994; Rose & Ruff, 1987). Infants do not perceive disparate sensations through the various sense modalities; rather, they are able to select information that is meaningful and relevant to their actions and to perceive coherent, unitary multimodal events even in the first months of life. For example, 2- to 5-month-old infants are able to perceive a relationship between a face and a voice on the basis of temporal synchrony, shared rhythm, and spectral information between the movements of the mouth and the timing and nature of the speech sounds (Dodd, 1979; Kuhl & Meltzoff, 1984; Mendelson & Ferland, 1982). By 5–7 months, infants can also match faces and voices on the basis of the age and gender of the speaker as well as the speaker’s affective expression (Bahrick, Netto, & Hernandez-Reif, 1998; Walker-Andrews, 1982; Walker-Andrews, Bahrick, Raglioni, & Diaz, 1991). Infants of 3–6 months are also able to match a soundtrack to the appropriate one of two objects hitting a surface on the basis of the object’s substance and composition (Bahrick, 1983, 1987, 1988, 1992). Although there is now a solid base of data demonstrating that infants are adept perceivers of multimodal stimulation across a variety of natural events, as yet little is currently known about how infants accomplish this. How and on what basis do infants begin to parse, perceive, and derive meaning from the flux of multimodal stimulation in a manner that lays a foundation for the perceptual world of the adult?

Consistent with Gibson’s (1969) invariant detection view, Bahrick (1992, 1994; Bahrick & Pickens, 1994) has proposed that amodal information initially guides this developmental process. Amodal information is information that is not specific to a particular sense modality but is redundant across two or more senses (i.e., bimodally or multimodally specified). Detection of amodal information directs initial perceptual learning. It focuses the infant’s attention on meaningful, unitary multimodal events (events in which stimulation from different modalities emanates from a single source) and at the same time serves as a buffer against the learning of incongruent or arbitrary intersensory relations (Bahrick, 1992, 1994). For example, detection of amodal properties such as temporal synchrony, common rhythm, and rate of motion may focus infant attention on the relation between the sights and sounds of a person speaking. This sensitivity to amodal properties also ensures that infants do not relate the speech sounds to the color or appearance of the person’s clothing, to the face of an unrelated person, or to a nearby object.

How might the detection of amodal information lead to the rapid development of veridical perception and intermodal knowledge consistent with that of adult perceivers? We propose an intersensory redundancy hypothesis. Intersensory redundancy refers to spatially coordinated and concurrent presentation of the same information (e.g., rate, rhythm, intensity) across two or more sense modalities. For the auditory and visual modalities, it also entails the temporally synchronous alignment of the bimodal information. The hypothesis consists of three components: (a) Intersensory redundancy recruits infant attention, causing amodal properties of events to become “foreground.” (b) This leads to perceptual processing, learning, and eventually memory for bimodally or multimodally specified properties before other properties. (c) The perceptual precedence of amodal information ensures unitary perception of single multimodal events and constrains further processing. Thus, infants’ responsiveness to intersensory redundancy leads to intermodal learning that is veridical and efficient and captures the salient dimensions of the adult’s perceptual world. We conducted three experiments testing infant discrimination of audiovisual rhythm information in order to evaluate predictions of the intersensory redundancy hypothesis. We describe the three components of the hypothesis in more detail below.

First, we argue that the impact of intermodal redundancy operates at the level of attention. Prior research has emphasized the important role of the infant’s capabilities for detecting amodal relations (Bahrick, 1988; Bahrick & Pickens, 1994; Gibson, 1969; Morrongiello, Fenwick, & Chance, 1998). However, no research has focused directly on the attentional salience of amodal information (as contrasted with unimodal information) and its consequences for perception. We propose that when the same information is spatially coordinated and simultaneously presented across two or more modalities, it is highly salient to infants and directs attentional selectivity at the expense of information that is not redundant. This hypothesis has not been tested and is addressed by the research presented here.

The assumption that amodal information is salient to infants is implicit in several methods used to investigate intermodal functioning in infancy. For example, the success of the intermodal preference procedure (e.g., Bahrick, 1983; Spelke, 1976, 1979; Walker-Andrews, 1982) is based on the fact that acoustic information guides visual exploration in infancy. Infants tend to look more to the one of two simultaneously visible films that belongs with the soundtrack they are heating. Further, the salience of intersensory redundancy in the audiovisual domain appears to be greatly enhanced by the synchronous presentation of the auditory and visual stimulation. The importance of synchrony for infant perception has been well documented (e.g., see Bahrick & Pickens, 1994; Lewkowicz, 1994, 1996; Morrongiello, Fenwick, & Nutley, 1998) and is also reflected in the intermodal preference procedure, in which the bimodal information is typically presented synchronously, as in the normal environment. This tight temporal coupling causes the redundant properties to stand out from other nonredundant properties. However, young infants are also able to detect amodal properties across vision and audition when the visual stimulation and the acoustic stimulation are presented successively rather than synchronously (e.g., Bahrick, 1983; Spelke, 1981). Detecting amodal properties in unimodal stimulation should be more feasible after infants have had some experience detecting a given property when it was previously bimodally and synchronously available. This facilitation could occur through experience with similar past events or in a single episode in a manner similar to a priming effect. For example, because the property in question (e.g., the rhythm of a ball bouncing) would be highly salient when it was experienced bimodally, then later, when it was available unimodally (e.g., bouncing sounds coming from a nearby room), it is likely that attention would continue to be focused on the property of rhythm. In contrast, asynchronous presentations specify a lack of relation between the auditory stimulation and the visual stimulation (Bahrick, 1983; Bahrick & Pickens, 1994). For example, hearing a speaking voice out of synchrony with the movements of a person’s face tells one that the voice and face do not belong together. The role of synchrony across other sense modalities, however, is not as clear as in the audiovisual domain. For example, visual–tactual processing (e.g., shape and texture) typically involves concurrent and spatially coordinated visual and tactual information, but haptic information is obtained sequentially and does not typically require a tight temporal coupling across modalities for the experience of unity. Thus, for the visual–tactual domain, intersensory redundancy may typically involve concurrent, spatially coordinated visual and tactual exploration but may not necessarily be synchronous, in the sense used above.

A study on infant selective attention of audiovisual information (Bahrick, Walker, & Neisser, 1981) illustrates the salience of intersensory redundancy for attentional deployment in a powerful way. In this study, infants viewed films of two superimposed events (e.g., a toy slinky moving and a hand-clapping game). For the adult viewer, when the two superimposed events were viewed silently, they appeared to be an amalgamation of ghostly images passing through one another. However, as soon as the soundtrack to one was turned on, the sound-specified event suddenly stood out from the other event, creating a strong impression of figure and ground. Infants also appeared to be affected in this way by the addition of the soundtrack. By playing the synchronous soundtrack to one of the superimposed events, we were able to direct infants’ attentional selectivity to that event and to cause them to ignore the silent event. However, once the soundtrack was turned off and the films were separated, appearing side by side, infants took the opportunity to view the novel, previously silent film. A second study confirmed this interpretation. Infants were presented with only one centrally projected event with sound, followed by silent trials of two events side by side (the familiar event and a novel one). Again, infants viewed the novel, previously not visible film, and the effect was of the same magnitude as in the superimposed condition. Together these results demonstrate that the intersensory redundancy provided by the natural, synchronous soundtrack to a visible event can guide infants’ visual selectivity, even when another event occupies the same spatial location. The impact of the attentional salience of redundant information on subsequent processing and learning in infancy appears to be profound.

A second component of the intersensory redundancy hypothesis is that the attentional salience of redundant information leads to perceptual processing and learning about properties of objects that are bimodally specified. This essentially creates a processing priority (both developmentally and within an episode of exploration) for certain kinds of information over other kinds. Specifically, under conditions of bimodal stimulation, properties of events that are conveyed in two or more modalities will be detected, processed, learned, and thus remembered prior to properties that are available in only one sense modality. Multimodal information tends to be spatial or temporal in nature and at a relatively more global or abstract level. When multimodal information is available, these amodal properties (e.g., intensity, rhythm, tempo, synchrony, shape, substance) will generally be differentiated first, whereas information that is modality-specific (e.g., color, pattern, pitch, timber) will generally be differentiated later. Conversely, when only unimodal stimulation is available, unimodal properties may be differentiated at the expense of bimodal properties. Thus, the rhythm or tempo of a unimodal event is less likely to be noticed than the rhythm or tempo of a bimodal event.

A third component of the intersensory redundancy hypothesis is that the perceptual precedence of amodal information should have powerful effects on the development of perception. It can set the initial conditions for perceptual learning and thus organize and constrain subsequent learning in several ways. First, it ensures that perception of information available to the different senses will be coordinated properly and that infants will attend to streams of light and sound emanating from a single, unitary event. For example, sights and sounds that occur in synchrony and share a temporal pattern will be perceived as belonging together (e.g., the sights and sounds of a ball bouncing or of a person speaking). The temporal synchrony between visual and acoustic stimulation has been found to guide multimodal exploration in the first months of life (e.g., Bahrick, in press; Dodd, 1979; Lewkowicz, 1994, 1996), ensuring that infants will attend to sights and sounds that belong together rather than to disparate streams of visual and acoustic stimulation. Second, attention to unitary sights and sounds allows further differentiation of information about the unitary event to proceed. Third, by selectively attending to sights and sounds from unitary events, infants do not learn inappropriate or incongruent relations. Thus, the young infant would not attempt to unite the sight of one person’s face with the audible voice of a different, nearby person or object. In this manner, intersensory redundancy can facilitate coordinated perception of multimodal events.

In the experiments reported in this article, we tested several key aspects of the intersensory redundancy hypothesis. Specifically, we tested the notion that a property that was presented across two modalities in a temporally coordinated manner would recruit attention and lead to perceptual differentiation, whereas the same property presented unimodally would recruit less attention and lead to attenuated discrimination. We chose the temporal property of rhythm for our present purposes. Prior research (Pickens & Bahrick, 1995) had demonstrated that 7-month-old infants were capable of discriminating among different irregular rhythms when they were presented bimodally. Because we wanted a task that was difficult for infants but one in which discrimination would be possible, we chose to present younger infants (5-month-olds) with these bimodal rhythms. We thus tested the hypothesis that presenting a rhythm bimodally in a temporally coordinated manner would lead to discrimination of the rhythm information (Experiment 1), whereas presenting the same rhythm unimodally would lead to attenuated discrimination of the rhythm (Experiment 2). Further, we hypothesized that if the salience of intersensory redundancy is dependent on temporal synchrony, then presenting the rhythm bimodally but in a temporally asynchronous manner (Experiment 3) would lead to a failure to discriminate the rhythm.

Experiment 1: Perceptual Differentiation of Rhythm Presented Bimodally

Rhythm can be defined as the temporal pattern of elements and intervals between elements of a repeating pattern. A number of studies have demonstrated infants’ sensitivity to rhythm information in unimodal stimulation, either acoustic (Chang & Trehub, 1977; Demany, McKenzie, & Vurpillot, 1977; Morrongiello, 1984, 1986, 1988) or visual (Mendelson, 1986), in the first half year of life. In general, infants’ temporal discrimination skills appear to improve with development during the 1st year. For example, Morrongiello (1984) reported that although 6-month-olds detected absolute timing shifts (in which the duration of an element was changed) in a series of white noise bursts, infants did not detect the more difficult relative timing shifts (in which the order of identical elements was changed) until 12 months. Only a few studies, however, have investigated infants’ bimodal perception of rhythm (Allen, Walker, Symonds, & Marcell, 1977; Mendelson & Ferland, 1982; Pickens & Bahrick, 1995, 1997). For example, Pickens and Bahrick (1995) demonstrated that 7-month-old infants could discriminate a change (relative timing shift) in the rhythm of a bimodal event played at different tempos (rates) as well as a change in the tempo of the event played at different rhythms. However, infants were unable to abstract the invariant rhythm across changes in tempo (Pickens & Bahrick, 1997). Although a handful of studies have explored infants’ perception of bimodally presented rhythms, no studies to date have investigated the relative importance of bimodal versus unimodal stimulation for rhythm discrimination. Experiment 1 served as a point of departure for this comparison.

The present experiment assessed whether infants at the age of 5 months were capable of discriminating the relatively difficult bimodally presented rhythm contrasts (shifts in relative timing) used by Pickens and Bahrick (1995, 1997) with 7-month-olds. Infants were habituated to a bimodally presented rhythm depicted by a videotape of a red hammer tapping on a wooden surface accompanied by its natural sounds. Following habituation, half the infants received two visual test trials depicting a change in rhythm, whereas half received no change in rhythm. We expected that if infants detected the change in rhythm, they would show visual recovery (increased looking) in response to the novel rhythm and not in response to the familiar rhythm.

Method

Participants

Sixteen 5-month-old infants (9 boys and 7 girls) participated. Their mean age was 151.1 (SD = 3.2) days. The data from 1 additional infant were eliminated because of the infant’s excessive fussiness.

Stimulus events

Videotaped audiovisual events depicted a bright red hammer moving up and down, striking a wooden surface, and creating one of four distinctive rhythms (see Pickens & Bahrick, 1995, 1997, for details). The rhythms were irregular (having elements and intervals of different durations) and differed only in terms of the arrangement of elements in the sequence. These rhythms were presented bimodally in prior research and 7-month-old infants were able to discriminate between them (Pickens & Bahrick, 1995). Each sequence consisted of a repeating pattern of a four-beat measure with four impacts alternating with a four-beat measure of rests. The rhythms were as follows: (a) x xx o x, (b) x o xx x, (c) x x o xx, and (d) xx o xx (where x represents a whole-beat impact; xx, two half-beat impacts; and o, a whole-beat rest). The rhythms were presented at a rate of 55 impacts per minute (taking into account the four-beat rest alternating with the four-beat rhythm). Each four-beat rhythm was approximately 2.18 s long, as was each four-beat measure of rests. A control stimulus consisted of a videotaped display of a green and white toy turtle spinning its forearms, creating a whirring sound.

Apparatus

Infants were seated in a standard infant seat approximately 55 cm from a 19-in. (Panasonic BT-S1900N) video monitor that was surrounded by black posterboard. Two apertures in the posterboard, one toward the upper right and the other toward the upper left side of the monitor, allowed observers to monitor the infants’ visual fixations. A mechanical toy dog and a set of colored Christmas tree lights were positioned above and to the right of the video display. They were used to attract the infant’s attention between trials if necessary.

The audiovisual events were videotaped with a Panasonic (WV3170) color video camera and a Sony (EMC-150T) remote microphone. They were edited and presented with a Panasonic (VHS NV-A500) edit controller that was connected to three Panasonic video decks (AG-6300 and AG 7750s). With three video decks, we were able to switch between the habituation, test, and control displays without the extra time or noise that would have resulted from changing cassettes. All soundtracks were presented from a speaker just beneath the video screen at a level of approximately 65 dB (A) measured in the area near the infant seat.

A trained observer (unaware of the hypotheses of the study and unable to see the visual displays) monitored infants’ visual fixations by depressing a button while the infant fixated the video image. The button box was connected to a PC that was programmed to record visual fixations online, to signal as soon as the infant had looked away for 1.5 s, and to signal when the habituation criterion had been reached. The signal was transmitted to the experimenter through a small speaker and headphone. A permanent record of the infant’s visual fixations was created on the computer during the experiment. The observations of the primary observer controlled the audiovisual presentations, whereas those of the secondary observer were simply recorded for later calculation of interobserver reliability.

Procedure

Infants were tested with an infant-control habituation procedure (Horowitz, Paden, Bhana, & Self, 1972) to determine whether they could detect a change in rhythm following bimodal familiarization. Infants were randomly assigned to either the experimental condition (rhythm change; AV–V change) or the control condition (no rhythm change; AV–V control). For habituation, they received an audible and visible (AV) presentation of the hammer tapping out a distinctive four-beat rhythm. For test, they received a visual (V) presentation of the hammer tapping out a distinctive four-beat rhythm with no sound audible. Infants in the experimental condition received two visual test trials depicting a change in rhythm, whereas those in the control condition received two visual test trials depicting no change in rhythm. Within each condition, two infants were randomly assigned to a given rhythm (1, 2, 3, or 4) for the habituation sequence. Then, for the test trials of the experimental condition, those who were habituated with Rhythm 1 received Rhythm 3 for the test and vice versa, and those who were habituated with Rhythm 2 received Rhythm 4 for the test and vice versa (see Pickens & Bahrick, 1995).

In general, the habituation sequence consisted of an initial control trial and four mandatory habituation trials and was terminated after the infant reached the habituation criterion and two subsequent (no-change) posthabituation trials. Each trial began when the infant fixated the visual display and was terminated after the infant looked away for a period of 1.5 s. Further, a ceiling of 60 s was set for the maximum trial length, and 20 trials was set as the maximum number of trials. No infants were rejected for failure to habituate within this number of trials. The criterion for habituation was defined as a decrement of 50% or greater, on two consecutive trials, relative to the infant’s initial fixation level (the average number of seconds of fixation during the first two habituation trials). After the habituation criterion was met, two no-change posthabituation trials were presented. They served to establish a more conservative criterion for habituation, reducing chance habituation and taking into account spontaneous regression effects by assessing visual recovery in relation to these trials (see Bertenthal, Haith, & Campos, 1983, for a discussion of regression effects). Following the habituation sequence, infants received two test trials to assess visual recovery and then a final control trial. The initial and final control trials depicted a toy turtle with spinning arms that created a whirring sound.

We examined the infants’ data to determine whether two criteria had been met (see Bahrick, 1992, 1994). First, to ensure that infants had, in fact, habituated to the events, we determined whose mean posthabituation fixation levels exceeded their mean initial fixation levels (baseline) and excluded their data from the study (N = 0). Second, to exclude the data of infants who were overly fatigued and unable to show visual recovery, we compared the duration of fixation to the moving turtle on the final control trial with the duration of fixation to the habituation stimuli on the posthabituation trials. If the control-trial looking was less than 150% of the posthabituation fixation level, we rejected those data (N = 1). The remaining infants showed substantial visual recovery on the final control trial (Mdn = 408% of the posthabituation fixation level).

A secondary observer monitored visual fixation for 6 infants (38% of the sample) for the purpose of assessing interobserver reliability. Total fixation time was calculated independently for each trial for each infant on the basis of observations made by the primary and the secondary observers. A Pearson product-moment correlation between these observations (number of seconds spent looking per trial) served as the measure of reliability and averaged.99 (SD =.005).

Results

Primary analyses

Figure 1A displays the mean visual fixation as a function of trial type (baseline, posthabituation, or test) for the rhythm-change (AV–V change) and the no-change (AV–V control) conditions. The primary dependent variable was visual recovery in response to the test displays, a difference score between fixation on test trials and fixation on posthabituation trials (see Figure 1B). Visual recovery in response to the test displays was calculated by subtracting the mean fixation time across the two posthabituation trials from the mean fixation time across the two test trials for each infant; infants’ recovery scores were then averaged across condition. To determine whether infants showed significant recovery in response to the change in rhythm, we conducted a single-sample t test on the mean recovery score of the experimental group against the chance value of 0. The results revealed that the experimental group showed significant visual recovery in response to the change in rhythm, t(7) = 3.37, p =.012. Thus, infants who received a bimodal presentation of a rhythm (AV) followed by a unimodal (V) test depicting a new rhythm showed evidence of discriminating the rhythms by looking longer at the new rhythm. Because there was also a change from bimodal to unimodal rhythm presentation from habituation to test, it was important to ascertain that the visual recovery was due not to this shift (i.e., the change from sound to silence from habituation to test) but rather to the perception of a rhythm change. Thus, a single-sample t test was also conducted on the results of the control group, who received no change in rhythm but received a change from bimodal (AV) presentation of the rhythm during habituation to unimodal (V) presentation of the rhythm during test. Results indicated no significant visual recovery in response to the change from bimodal to unimodal presentation of the rhythm, t(7) = −1.14, p >.1. In fact, the mean recovery was negative, in the opposite direction from what one would expect if infants responded to the change from bimodal to unimodal presentation. Thus, infants showed no evidence of responding to the loss of sound from habituation to test when the same rhythm was presented. These findings confirm that the visual recovery observed in the experimental group was in fact due to the infants’ detecting a change in rhythm.

Figure 1.

Experiment 1: Mean visual fixation as a function of trial type (A) and mean visual recovery (B) following bimodal habituation to the rhythm-change (AV–V [audiovisual-visual] change) and the no-change (AV–V control) conditions. Baseline is the mean visual fixation during the first two habituation trials, reflecting initial interest level in the habituation event. Posthabituation is the mean visual fixation during the two no-change trials just after the habituation criterion was met, reflecting final interest level in the habituation event. Visual recovery is a difference score between visual fixation during the test trials and visual fixation during the posthabituation trials. Standard deviations appear in parentheses.

In addition, we compared the results of the experimental and control groups in a two-sample t test to determine whether visual recovery under the rhythm-change condition was significantly greater than visual recovery under the no-change condition. The results indicated a significant difference, t(14) = 2.73, p =.016. Thus, infants showed significantly more visual recovery in response to a change in rhythm than in response to no change in rhythm.

Secondary analyses

We conducted additional analyses to determine whether initial interest level (baseline), final interest level (posthabituation; see Figure 1), or total processing time (number of seconds to habituation) for the bimodal displays differed as a function of condition. Three one-way analyses of variance were conducted, one for baseline, one for posthabituation, and the third for seconds to habituation, with condition (experimental or control) as the main factor. They revealed no significant effect for the baseline, F(1, 14) = 0.07, p >.1, or the seconds-to-habituation, F(1, 14) = 0.38, p >.1, measures, indicating that infants in the two groups did not differ in their initial interest or total processing time for the bimodal events. However, for the posthabituation measure, there was a significant main effect, F(1, 14) = 4.83, p =.045, indicating that following habituation, infants in the control condition (M = 10.24 s) were more interested in the bimodal events than were infants in the experimental condition (M = 5.39 s). This difference may potentially indicate a difference in the degree of encoding by the two groups; however, it is not likely to have an impact on the main findings. The primary analyses used visual recovery scores (the within-subject change in looking from posthabituation to test trials) to assess detection of a change in rhythm. Further, the experimental condition itself provided evidence for detection of the change in rhythm. Recall that the control study was conducted primarily to assess whether infants would show visual recovery in response to a change from an audiovisual presentation to a visual-only presentation of the same rhythm.

Discussion

Overall, these results indicate that following habituation to bimodal presentations of an irregular five-element rhythm, infants showed differential visual recovery in response to the test displays. Specifically, visual recovery was significantly greater when the test depicted a novel rhythm than when it depicted the familiar rhythm. Thus, 5-month-olds were able to detect the rhythm change in these events when they were presented bimodally.

These findings also demonstrate that even 5-month-olds can discriminate changes in the relative timing (the order of elements) of complex rhythms when the rhythms are presented bimodally. These findings are in contrast to the results of prior studies of unimodal rhythm perception, which suggested that discrimination of more difficult contrasts in relative timing occurs later in development (e.g., Morrongiello, 1984).

On what basis might infants have detected the change in rhythm? In a manner consistent with the intersensory redundancy hypothesis, infants may have abstracted the bimodal information for rhythm during habituation. During the test trials, they then abstracted either the same or a different rhythm presented visually. In contrast, infants may have abstracted unimodal (visual and/or acoustic) information for rhythm during habituation but failed to detect the redundancy across the senses. If they had detected unimodal visual rhythm during habituation, then during the visual test trials, they could also have detected the change in the visual rhythm. However, if infants detected only unimodal acoustic information for rhythm during habituation, they would have been unable to detect the change in rhythm during the test, because it was presented visually. In Experiment 2 we tested whether 5-month-olds were capable of detecting rhythm information on the basis of exposure to unimodal visual or unimodal auditory information.

Experiment 2: Perceptual Differentiation of Rhythm Presented Unimodally

If intersensory redundancy directs attentional selectivity to the redundant property, then attention should be directed to the property of rhythm to a greater extent under bimodal presentation conditions than under unimodal conditions. Consequently, abstraction of rhythm and detection of a change in rhythm should be impaired under unimodal relative to bimodal presentation conditions. In contrast, if unimodal exposure is just as effective as bimodal exposure for highlighting rhythm, and if detection of a rhythm change in Experiment 1 was accomplished by detecting unimodal information, then infants should show significant visual recovery in response to a change in rhythm under unimodal habituation conditions as well. In this experiment, we presented 5-month-old infants with the same rhythms as in Experiment 1, but we presented the rhythms unimodally. In accordance with the intersensory redundancy hypothesis, we expected that discrimination of rhythm information would be attenuated relative to that in the bimodal condition.

Method

Participants

Sixteen 5-month-old infants (8 boys and 8 girls) participated. Their mean age was 153.1 (SD = 4.25) days. The data from 5 additional infants were rejected because of excessive fussiness (N = 2) and equipment failure or experimenter error (N = 3).

Stimulus events and apparatus

Infants were presented with either the visual or the acoustic portions of the bimodal events used in Experiment 1. The visual portions depicted the red hammer tapping out the irregular rhythms. The acoustic portions consisted of the soundtracks of the hammer tapping out the irregular rhythms. The soundtracks were played along with a static image of the red hammer. Thus, looking directed to the still hammer in the presence of sound was used as a measure of attention to the sound (see Walker-Andrews & Grolnick, 1983; Walker-Andrews & Lennon, 1991). The apparatus was the same as that used in Experiment 1.

Procedure

The procedures were similar to those used for the rhythm-change group of Experiment 1. Infants were randomly assigned to either the auditory or the visual condition. All habituation and counterbalancing procedures were identical to those of Experiment 1 except that the rhythms were presented unimodally. In the visual condition, infants were habituated to a silent videotape of the red hammer tapping out one of the four rhythms. Then they received two visual test trials depicting the red hammer tapping out a new rhythm. In the auditory condition, infants were habituated to the soundtrack of the hammer tapping out one of the four rhythms while they viewed a still image of the red hammer. During the test trial, they were presented with the soundtrack of the hammer tapping out a new rhythm along with the still image of the red hammer.

The same criteria for eliminating the data of infants who were fatigued or who failed to habituate were maintained from Experiment 1; however, no participants were excluded on these bases. Infants showed substantial visual recovery on the final control trials relative to their posthabituation fixation levels (Mdn = 599% of their posthabituation fixation level). All participants habituated within 20 trials. A secondary observer monitored visual fixation behavior for 4 of the 16 infants (25%). A Pearson product-moment correlation between observations of the primary and secondary observers averaged.99 (SD =.001).

Results

Primary analyses

Figure 2A displays the mean visual fixation as a function of trial type (baseline, posthabituation, or test) for the unimodal visual (V–V) and acoustic (A–A) conditions. Figure 2B presents the visual recovery scores for the unimodal visual and acoustic conditions. Visual recovery was calculated as before, and single-sample t tests were conducted to determine whether infants showed significant visual recovery in response to the change in rhythm. Results indicated no significant recovery in either the visual, t(7) = − 0.58, p >.1, or the auditory, t(7) = 1.6, p >.1, condition. Thus, infants showed no evidence of detecting a change in rhythm under unimodal presentation conditions.

Figure 2.

Experiment 2: Mean visual fixation as a function of trial type (A) and mean visual recovery (B) following unimodal habituation to the visual (V–V change) and auditory (A–A change) conditions. Baseline is the mean visual fixation during the first two habituation trials, reflecting initial interest level in the habituation event. Posthabituation is the mean visual fixation during the two no-change trials just after the habituation criterion was met, reflecting final interest level in the habituation event. Visual recovery is a difference score between visual fixation during the test trials and visual fixation during the posthabituation trials. Standard deviations appear in parentheses.

We also compared the results of this experiment with those of Experiment 1 to determine whether visual recovery under the bimodal condition was different from that under the unimodal conditions. A one-way analysis of variance was conducted on visual recovery scores, with condition as the main factor (AV–V change, A–A change, or V–V change). The results indicated a significant main effect of condition, F(1, 21) = 7.03, p =.005. In addition, planned comparisons (Duncan’s range test) indicated that visual recovery in the bimodal condition was significantly greater than visual recovery in either of the unimodal conditions (p <.05).

Secondary analyses

We conducted further analyses to determine whether initial interest level (baseline), final interest level (posthabituation), or total processing time (number of seconds to habituation) for the unimodal displays differed as a function of condition. Three one-way analyses of variance were conducted with condition (V–V or A–A) as the main factor, one for looking during baseline, one for looking during posthabituation, and the third for seconds to habituation. They revealed no significant main effects of condition: F(1, 14) = 0.18, p >.1; F(1, 14) = 0.54, p >.1; F(1, 14) = 1.18, p >.1, respectively. Thus there were no a priori differences between initial and final interest levels or total processing time as a function of whether infants received visual or auditory rhythm displays.

Further analyses were undertaken to compare the processing times of the infants in Experiments 1 and 2. Did the bimodal events elicit more interest and more processing time than the unimodal events? An analysis of variance was conducted on the number of seconds to habituation, with condition (AV–V change, M = 151.9 s, SD = 57.9; V–V change, M = 122.5 s, SD = 61.7; A–A change, M = 91.4 s, SD = 52.1) as the main factor. The results indicated that although the means were in the expected direction, the main effect of condition did not reach significance, F(2, 21) = 2.23, p =.13. Thus, the differences in processing time between the bimodal and unimodal conditions were not significant.

To further address any potential contribution of processing time to the facilitating effect of bimodal stimulation, we used processing time (seconds to habituation) as a covariate in an analysis of covariance on visual recovery, with condition (AV–V change, V–V change, or A–A change) as the main factor. The results indicated that processing time was not a significant predictor of visual recovery, F(1, 20) = 0.18, p >.1. Further, the main effect of condition was just as robust after the means had been adjusted for the effects of processing time, F(2, 20) = 6.63, p =.006. Thus, processing-time differences can safely be eliminated as a basis for the significant differences in visual recovery across bimodal and unimodal conditions.

Discussion

The results provided no evidence that 5-month-old infants detected a change in rhythm when the rhythm was presented unimodally. This finding contrasts with the results of Experiment 1, in which 5-month-olds detected a change in rhythm when the rhythm was presented bimodally. In fact, detection of the rhythm change was significantly greater under the bimodal conditions of Experiment 1 than under either of the unimodal conditions of Experiment 2.

The results of the present experiment rule out the hypothesis that the detection of rhythm in Experiment 1 was due to abstraction of unimodal information. Infants showed no evidence of discriminating a change in rhythm on the basis of unimodal information. Rather, the results suggest that infants detected the rhythm by abstracting amodal information common to the visual and the acoustic stimulation. These findings lend support to the intersensory redundancy hypothesis and suggest that intersensory redundancy recruits attention to properties that are bimodally specified, whereas the same properties presented unimodally do not appear to selectively recruit attention. As a result, infants should generally be poorer at detecting amodal properties such as rhythm or tempo under conditions of unimodal presentation than under conditions of bimodal presentation.

The results of the present experiments also rule out the hypothesis that longer processing time was responsible for the infants’ enhanced ability to discriminate rhythm under bimodal presentation. The differences in processing time were not significant. Further, when processing time was used as a covariate, the differences in visual recovery across bimodal and unimodal conditions were still highly significant. Thus, longer processing time does not appear to be responsible for the enhanced discrimination of amodal properties; rather, intersensory redundancy selectively recruits attention to the amodal properties and facilitates processing of those properties.

Interestingly, although it seems intuitively plausible that rhythm information might be best conveyed by the auditory modality, the results of Experiment 2 disconfirm this view. Five-month-old infants showed no evidence of discriminating the rhythms under the auditory condition. Apparently, the addition of the visual information in Experiment 1, by creating redundancy and thereby increasing the salience of the rhythms to 5-month-olds, made it possible for them to discriminate the rhythms. It appears that bimodal information was necessary for discrimination under these conditions at 5 months of age, although eventually older infants could, of course, discriminate the rhythms unimodally. For example, research has demonstrated that at 7 months, infants could discriminate rhythms presented unimodally when the rhythms were simpler and when little modality-specific information was available (i.e., flashing lights and pulsing tones, Allen et al., 1977). Further, it is possible that the intersensory information contributed something above and beyond any simple additive effect from the two separate modalities. That is, visual recovery in the visual condition was negative (M = −0.55 s), and visual recovery in the auditory condition was small (M = 1.11 s). Together, they do not sum to the level of visual recovery observed in the bimodal condition of Experiment 1 (M = 5.29 s).

The notion that multimodal stimulation is more salient than the combined contribution of stimulation from the separate modalities is consistent with recent findings from neurophysiological and behavioral research on intermodal perception that used animal subjects. A number of studies have shown that the effectiveness of a stimulus in one modality in eliciting attentive and orienting behaviors is dramatically affected by the presence of a stimulus from another modality (see Stein & Meredith, 1993).

Experiment 3: Perceptual Differentiation of Bimodal Rhythm Presented Asynchronously

Experiments 1 and 2 demonstrated that differentiation of rhythmic pattern was facilitated by bimodal presentations and impaired by unimodal presentations of the rhythmic sequences, providing support for our intersensory redundancy hypothesis. However, from these experiments it cannot be determined which aspects of the bimodal presentation are responsible for the enhanced discrimination of rhythm. According to our hypothesis, the synchronous alignment of the auditory and visual components should be a critical factor in highlighting the amodal property of rhythm. Synchrony makes numerous amodal temporal properties apparent, including the rate, rhythm, tempo, duration, composition, and substance of an impacting object, and infants are capable of detecting all of these properties (Bahrick, 1983, 1987, 1988; Bahrick & Pickens, 1994; Lewkowicz, 1994). This amodal structure may increase the salience of the bimodal stimulation and cause infants to attend to the amodal property longer and process it more deeply than other kinds of stimulation. Alternatively, it is possible that concurrent stimulation in two sense modalities per se is responsible for the increased salience of bimodal stimulation and that synchrony is not necessary. For example, bimodal stimulation may be more arousing than unimodal stimulation because there is a greater amount of overall stimulation, which therefore causes increased attention and processing of bimodal displays relative to unimodal displays. In Experiment 3 we addressed these two hypotheses.

If the salience of intersensory redundancy depends on the proper temporal alignment of the auditory stimulation and the visual stimulation rather than their co-occurrence per se, then discrimination of amodal information should be much more difficult when the auditory stimulation and the visual stimulation are presented out of synchrony. Thus, in Experiment 3 we presented the rhythm bimodally, just like in Experiment 1. However, we presented the films and soundtracks out of synchrony. By presenting the same films and soundtracks as in Experiment 1 (temporally misaligned, however), we ensured that any arousal associated with bimodal presentations would be comparable across experiments. However, if intersensory redundancy entails synchrony, then the salience of the amodal property of rhythm should be dramatically decreased when synchrony is absent. Thus, in the present experiment, we expected infants to show no evidence of discriminating the rhythm information.

Method

Participants

Eight 5-month-old infants (5 boys and 3 girls) participated. Their mean age was 151.1 (SD = 3.9) days, similar to the mean age of the infants in Experiment 1. None of the data were eliminated because of infant fussiness, equipment failure, or infants’ failure to meet the fatigue criteria.

Stimulus events, apparatus, and procedures

Infants were presented with the bimodal stimulus events from Experiment 1. The procedures were identical to those of the experimental condition (AV–V change) of Experiment 1 with the exception that during the habituation and test trials, the videotapes and soundtracks were presented out of synchrony. We accomplished this by beginning the soundtrack and the video portions of the events independently, each at a different random position in the videotape such that the sounds and the object movements were out of phase with one another and their phase relationship varied randomly from one infant to the next. All counterbalancing was identical to that in Experiment 1.

As in Experiment 1, infants showed substantial visual recovery on the final control trials relative to their posthabituation fixation levels (Mdn = 456% of their posthabituation fixation level). A secondary observer monitored visual fixation behavior for 3 of the 8 infants (37%). The Pearson product-moment correlation between the observations of the primary and secondary observers averaged.99 (SD =.01).

Results

Primary analyses

The mean visual fixation as a function of trial type is displayed in Figure 3A, and the visual recovery scores are displayed in Figure 3B, for the results of this experiment as well as those of the experimental condition of Experiment 1. We conducted a single-sample t test on the visual recovery scores of infants in Experiment 3 to determine whether they showed significant visual recovery in response to the change in rhythm. In contrast to our findings from Experiment 1, these results indicated no evidence of visual recovery in response to the change in rhythm under the bimodal, asynchronous presentation, t(7) = − 1.2, p >.1. Thus, infants showed no evidence of discriminating rhythm information when the videotapes and soundtracks were presented out of synchrony.

Figure 3.

Experiment 3: Mean visual fixation as a function of trial type (A) and mean visual recovery (B) following bimodal, asynchronous (Asynch) habituation and test (Experiment 3) versus bimodal, synchronous (Synch) habituation and test (Experiment 1) for the rhythm-change (AV–V [audiovisual-visual] change) condition. Baseline is the mean visual fixation during the first two habituation trials, reflecting initial interest level in the habituation event. Posthabituation is the mean visual fixation during the two no-change trials just after the habituation criterion was met, reflecting final interest level in the habituation event. Visual recovery is a difference score between visual fixation during the test trials and visual fixation during the posthabituation trials. Standard deviations appear in parentheses.

We also compared the results of this experiment with those of Experiment 1 to determine whether visual recovery in response to the change in rhythm was greater under the synchronous conditions of Experiment 1 than under the asynchronous conditions of Experiment 3. A two-sample t test was conducted on the visual recovery scores, and the results indicated a significant difference between the means, t(14) = 3.41, p =.004. Thus, infants showed significantly greater visual recovery in response to the change in rhythm when the films and soundtracks were presented synchronously in Experiment 1 than when they were presented asynchronously in Experiment 3.

Secondary analyses

We conducted additional analyses to assess whether initial interest level (baseline), final interest level (posthabituation), or overall amount of processing time (seconds to habituation) for the bimodal displays differed as a function of synchrony. If the synchronous alignment of audiovisual displays is important for highlighting amodal relations, infants might be expected to attend longer to synchronous displays than to asynchronous displays. Analyses of variance were conducted separately for each variable, with synchrony (Experiment 1, synchronous vs. Experiment 3, asynchronous) as the main factor. The results indicated that infants showed a marginally significant main effect of synchrony for their initial interest level in the bimodal events, F(1, 14) = 4.13, p =.06, with means of 45.6 s (SD = 16.4) of looking directed to the synchronous displays and 27.0 s (SD = 20.1) of looking directed to the asynchronous displays. Thus, the synchronous displays were more interesting to infants at first. Infants also spent more time overall processing the synchronous displays (M = 152.0 s, SD = 57.9) than the asynchronous displays (M = 100.5 s, SD = 57.1); however, this difference was also only marginally significant, F(1, 14) = 3.2, p =.095. Following habituation, however, infants showed no difference in their final interest levels (posthabituation fixation levels) to the two types of displays (for the synchronous displays, M = 5.4 s, SD = 4.1; for the asynchronous displays, M = 6.0 s, SD = 5.1). Thus, the infants appeared to be somewhat more interested in the synchronous displays than the asynchronous displays at first and to spend more time overall exploring them.

We also conducted an analysis of covariance to assess the contribution of processing-time differences to the differences in visual recovery between synchronous and asynchronous presentation conditions (Experiment 1 vs. Experiment 3). The results indicated that processing time was not a significant covariate, F(1, 13) = 1.78, p >.1, and the difference in visual recovery was still significant after the means had been adjusted for the effects of processing time, F(1, 13) = 13.94, p =.003. Thus, differential interest in the synchronous and asynchronous displays does not account for the difference in visual recovery in response to the change in rhythm that was observed for the synchronous versus asynchronous displays.

Discussion

In Experiment 3, in which the bimodal stimulation was presented out of synchrony, infants showed no evidence of detecting a change in rhythm. These findings contrast with those of Experiment 1, in which infants showed robust discrimination of the change in rhythm when the bimodal stimulation was presented in synchrony. Further, discrimination of rhythm was significantly greater in the synchronous condition of Experiment 1 than in the asynchronous condition of Experiment 3. These findings support our prediction that the salience of intersensory redundancy depends on the synchronous alignment of the audible stimulation and the visible stimulation. This alignment makes amodal relations apparent and likely causes increased arousal, attention, and processing of the amodal properties. These findings also disconfirm the hypothesis that the salience of intersensory redundancy is a result of concurrent stimulation in two moralities per se and of any increase in arousal associated with concurrent stimulation. Secondary analyses also found marginally significant evidence that infants were more interested in the synchronous bimodal displays at first and spent more time overall attending to them than to the asynchronous bimodal displays. This finding also supports the notion that synchrony between audible stimulation and visible stimulation elicits greater attention and processing than does their simple co-occurrence. In addition, the greater processing time itself was not responsible for the difference in rhythm discrimination between the synchronous and asynchronous conditions, as was indicated by an analysis of covariance.

General Discussion

Together, the results of these experiments demonstrate that 5-month-old infants showed evidence of discriminating five-element rhythms when the rhythm information was presented bimodally but not when it was presented unimodally, either visually or acoustically. To date, only a few studies have shown discrimination by young infants of rhythm presented in bimodal events (e.g., Allen et al., 1977; Mendelson & Ferland, 1982; Pickens & Bahrick, 1995). However, none has documented the superiority of bimodal over unimodal presentation for rhythm discrimination. The rhythms discriminated by infants in the present experiments were complex and depicted a shift in relative timing (the arrangement of elements). Further, the rhythms were depicted by naturalistic audiovisual events, rich in both amodal information (e.g., tempo, synchrony, and information about substance available visually and acoustically) and modality-specific information (e.g., color, pattern, pitch, and timbre). The availability of both amodal and modality-specific properties is characteristic of most events in the natural environment and may be important for fostering the superiority of bimodal over unimodal processing of rhythm information.

More important, this series of experiments highlights the functional significance of the distinction between multimodal and unimodal stimulation for guiding attention and perceptual learning in infancy. Multimodal stimulation makes overlapping, temporally coordinated information available to the different senses. This “redundancy” has a powerful impact on the deployment of attention. We have proposed an intersensory redundancy hypothesis to describe the nature of these effects on perceptual development. Specifically, intersensory redundancy recruits infant attention, causing the redundant information to become “foreground” and other information to become “background.” This fosters perceptual differentiation, learning, and memory for redundant, amodal properties before other stimulus properties. As a result, intersensory redundancy ensures coordinated perception of unitary events. Further, when the same amodal property is presented unimodally, it will not recruit comparable levels of attention and thus will not be perceived, learned, or remembered as well. Rather, unimodal stimulation may at first foster more attention to properties that are modality specific, such as color and form for the visual modality and pitch and timbre for the auditory modality.

The results of the present study support a number of aspects of the intersensory redundancy hypothesis. First, they demonstrate that intersensory redundancy has a profound effect on infants’ perceptual differentiation of amodal information (Experiments 1 and 2). Infants showed robust discrimination of a rhythm change when the rhythm was presented bimodally (Experiment 1), whereas neither a unimodal visual presentation nor a unimodal auditory presentation of the rhythm (Experiment 2) elicited evidence of sensitivity to rhythm information.

Second, the results demonstrate that the salience of intersensory redundancy depends on the proper temporal alignment of the components of the bimodal audiovisual stimulation (Experiment 3). That is, bimodal, asynchronous presentations of the audiovisual rhythm events did not foster discrimination of rhythm information. Temporal synchrony appears to be an important prerequisite for the facilitating effects of intersensory redundancy in the audiovisual domain.

Third, this research also suggests that intersensory redundancy affects infants’ selective attention to amodal versus unimodal information. Infants selectively attend to amodal properties in bimodal, temporally coordinated stimulation (Experiment 1), and they appear to attend to other stimulus properties in unimodal stimulation (Experiment 2). Further, once infants detect an amodal property in bimodal stimulation, attention to that property can be facilitated in unimodal stimulation as well. In Experiment 1, rhythm was first presented bimodally during habituation and then followed by unimodal visual test trials. Infants had no difficulty abstracting the rhythm during habituation. Moreover, they were then able to abstract the rhythm in the unimodal test trials as well. These findings thus suggest that once intersensory redundancy recruits attention to an amodal property, it can facilitate attention to that property in unimodal stimulation. Thus, amodal properties such as rhythm are not inherently difficult to abstract in unimodal stimulation. Rather, it is a matter of attentional focus. Attention is likely to be selectively focused on amodal properties (e.g., rhythm, tempo, intensity) under conditions of redundant bimodal stimulation and more focused on other features (modality-specific properties such as color or pitch) under conditions of unimodal stimulation.

Taken together, these findings thus support the intersensory redundancy hypothesis in the domains of both attention and perceptual differentiation. They suggest that intersensory redundancy directs and constrains attention to bimodal information and that this leads to selective processing of amodal properties when multimodal stimulation is available. Conversely, when multimodal stimulation is not available, attention may be more broadly focused on a variety of properties that are unique to individual modalities, and amodal properties will be less attended.

It should be emphasized that the importance of redundant stimulation for directing attentional selectivity is expected to be most pronounced during early perceptual learning in a given domain, and we are not implying that infants are unable to detect amodal properties in unimodal stimulation. In fact, a number of studies have demonstrated that infants are adept perceivers of amodal information in unimodal stimulation (e.g., Chang & Trehub, 1977; Demany et al., 1977; Mendelson, 1986; Morrongiello, 1984, 1986, 1988). Studies that used the cross-modal transfer method (see Rose & Ruff, 1987, for a review) and the successive method of presenting audible and visible components of events (e.g., Bahrick, 1983, Experiment 3) also provide compelling examples of this ability. Rather, amodal properties (such as rhythm, tempo, or intensity changes) are apparently most salient and easiest to detect when they are bimodally presented. Further, it is likely that although intersensory redundancy selectively recruits infant attention to amodal properties, it also facilitates subsequent attention and differentiation of those properties in unimodal stimulation (as suggested by the results of Experiment 1 discussed above). Recall that in the present study, fairly complex bimodal rhythms were chosen so that discrimination would be relatively difficult for 5-month-olds, thus allowing us to more easily observe the advantage of bimodal over unimodal stimulation for supporting perceptual differentiation. However, one might expect the advantage of bimodal over unimodal stimulation to be much less evident when using simpler rhythms or when testing older infants in later stages of perceptual development.

The importance of multimodal stimulation has also been reported in other work with human infants. In a series of related studies, Lewkowicz (1988a, 1988b, 1992b, 1994b, 1996) found that infants discriminated changes in amodal properties such as rate, synchrony, and duration when the changes occurred in both the auditory and visual modalities concurrently and synchronously. In contrast, when these changes occurred in only one modality, discrimination was not consistently observed. Lewkowicz (1996) suggested that there is a fundamental difference between the value of multimodal information and the value of unimodal information during early development, with multimodal information having greater perceptual salience than unimodal information.

What are the consequences of the attentional salience of redundant information for the developing human infant? One important consequence appears to be that in everyday perception of multimodal events, the chain of attention and perceptual differentiation begins with global, amodal information. Information that can be redundantly specified is by definition amodal and more global or abstract than information that cannot be specified by two or more sense modalities. The attentional salience of redundant information causes perceptual differentiation of an event to proceed from global information to increasingly more specific information, which is consistent with Gibson’s (1969) view of perception. Further, this attentional salience results in a developmental lag between infants’ detection of amodal relations and their detection of modality-specific or arbitrary relations (see Bahrick, 1994; Bahrick & Pickens, 1994; Hernandez-Reif & Bahrick, in press). Arbitrary intermodal relations are those that do not occur predictably in nature or across contexts. For example, the relation between the color of an object and the pitch of its sound or the texture of its surface and the relation between a verbal label we give an object and its appearance are arbitrary. Arbitrary relations must be learned through experience and may differ from one context or event to another (e.g., red objects only sometimes produce high-pitched sounds). This developmental lag between the detection of amodal and arbitrary relations is adaptive because it facilitates infants’ learning of consistencies and predictable relations across modalities that occur irrespective of context and in turn promotes appropriate generalizations across events (see Bahrick, 1992; Bahrick & Pickens, 1994). Only after infants have abstracted appropriate generalizations will they learn about arbitrary relations that are context specific.

In addition, recent research has demonstrated that detection of amodal relations can guide and constrain intermodal learning about arbitrary, modality-specific relations (Bahrick, 1994; Gogate & Bahrick, 1998; Hernandez-Reif & Bahrick, in press). For example, arbitrary relations between a verbal label and an object are first learned only in the presence of amodal synchrony between the movement of the object and its verbal label (Gogate & Bahrick, 1998). Similarly, the relations between an object’s color or pattern and its shape are learned only in the presence of amodal shape information concurrently available to vision and touch (Hernandez-Reif & Bahrick, in press). Thus, empirical evidence is accruing which suggests that perceptual differentiation begins with global information and proceeds to more specific levels, with detection of global relations guiding and constraining perceptual differentiation of more specific information (for a review, see Bahrick, in press). The intersensory redundancy hypothesis provides an explanation for how the perceptual system might get started along this important trajectory and suggests that selective attention to intermodal redundancy initially guides this process.

The results of the present research also call for a shift in our thinking regarding the fit between the infant and his or her multimodal environment. Previous research has emphasized that young infants are somehow “preadapted” to detect amodal relations at the expense of other types of information (Bahrick, 1992, 1994; Bahrick & Pickens, 1994; Gibson, 1969; Walker-Andrews, 1994). The present research and the intermodal redundancy hypothesis raise awareness that it is the redundancy of amodal stimulation, when it is presented bimodally and synchronously, that recruits attention and initiates perceptual processing. It is our view that infants are neither preadapted (prior to experience) to detect amodal relations nor preadapted to detect redundancy across the senses. Rather, sensitivity to amodal information is an emergent property of the relationship between the infant’s perceptual system and the nature of information in the world. This sensitivity presumably emerges as a result of the developmental history (both prenatal and postnatal) of the organism in its normal developmental context.

Results from neuroanatomical studies also indicate that bimodal sensory stimulation has greater salience than unimodal stimulation. For example, the performance of cats trained to approach a low-intensity visual stimulus was significantly enhanced when a brief, low-intensity auditory stimulus was presented at the same location, but performance was significantly depressed when the auditory stimulus was spatially or temporally disparate in relation to the visual stimulus (Stein, Meredith, Honeycutt, & McDade, 1989). These findings parallel those reported here. Further, the magnitude of these effects exceeded the level predicted by adding together the animal’s performance to each single-modality stimulus alone. This “multiplicative” nature of bimodal interactions was also observed physiologically for single nerve cells (Meredith, Nemitz, & Stein, 1987; Meredith & Stein, 1986), illustrating that the rules governing multisensory integration at the level of the cell are consistent with the behavior of the organism in response to multisensory stimuli (Stein, Meredith, & Wallace, 1994). Together, these studies indicate that given the appropriate temporal and spatial coincidence, pairing two stimuli from different modalities can often elicit a response that is greater than the sum of its unimodal parts.

Similarly, comparative studies also support the view that in early development, bimodal sensory stimulation has greater salience than unimodal stimulation. Recent work with precocial birds has highlighted the importance of multimodal stimulation and has shown different developmental consequences of bimodal versus unimodal exposure to the same information (Lickliter, Lewkowicz, & Columbus, 1996; Sleigh, Columbus, & Lickliter, 1998). In these studies, bobwhite quail chicks were reared with either normal bimodal stimulation or augmented or attenuated unimodal (auditory or visual) stimulation for the first several days following hatching. The results indicated that bimodal sensory experience is necessary for normal intersensory responsiveness to maternal auditory and visual stimulation in newly hatched chicks. In addition, augmented unimodal exposure (either auditory or visual) was found to interfere with normal intersensory responsiveness (Sleigh et al., 1998) and sensitivity to the spatial collocation of bimodal stimulation (Columbus, Sleigh, Lickliter, & Lewkowicz, 1998).

These comparative studies highlight the salience of multimodal stimulation for the development of attention and perception. They reveal a consistent pattern of findings across different measures of multimodal responsiveness, across different levels of analysis (from neural to behavioral), and across different species. This convergence across levels supports a functional distinction between unimodal and bimodal sensory stimulation in early development and suggests that the concept of intersensory redundancy is fundamental and may be meaningfully investigated as a general developmental principle applicable across a variety of avian and mammalian species.

The present research provides support for important aspects of the intersensory redundancy hypothesis and at the same time calls for future research. Additional studies should be conducted to investigate the critical factors underlying the differential attentional salience of amodal information in bimodal and unimodal stimulation. The present research demonstrated the importance of concurrent, temporally synchronous stimulation. However, in all the experiments reported here (and in the natural environment), the sounds and sights were also spatially collocated. Future research should explore the role of collocation and its interaction with temporal synchrony in the salience of intersensory redundancy. In addition, if the intersensory redundancy hypothesis reflects general developmental processes, then it should also be evident when other amodal properties are tested. Further research assessing attention and discrimination of amodal properties such as tempo, intensity, and temporal information specifying object composition, under both unimodal and bimodal presentation conditions, is needed in order to establish the generalizability of these findings. Converging work with animal subjects will also allow us to assess the extent to which the intersensory redundancy hypothesis is a general principle of development applicable across species.