Up Versus Down: The Role of Intersensory Redundancy in the Development of Infants’ Sensitivity to the Orientation of Moving Objects

Abstract

According to the intersensory redundancy hypothesis (IRH), during early development, perception of nonredundantly specified properties is facilitated in unimodal stimulation as compared with bimodal stimulation. Later in development, attention becomes more flexible and infants can detect nonredundantly specified properties in both unimodal and bimodal stimulation. This study tested these predictions by assessing the development of infants’ sensitivity to the orientation of an object striking a surface, information that is nonredundantly specified in visual and in audiovisual stimulation. Infants of 3, 5, and 8 months were habituated to unimodal visual or bimodal, synchronous, audiovisual films of a hammer tapping a rhythm in 1 of 2 orientations (upward vs. downward). Results demonstrated an Age × Condition interaction, where younger infants (3 and 5 months) detected the orientation change in unimodal but not bimodal stimulation, whereas older infants (8 months) detected the change in both types of stimulation. Further, in a control study, 3-month-olds detected the orientation change when bimodal stimulation was asynchronous, demonstrating that temporal synchrony impaired performance in the bimodal condition. These findings converge with those of prior studies and support predictions of the IRH.

We live in a world of objects and events that are dynamic and provide information to multiple senses simultaneously. Young infants must coordinate the flux of changing stimulation across different sense modalities to perceive unitary, coherent objects and events. A great deal of research now demonstrates that young infants are capable of perceiving unitary multimodal events by detecting amodal information, information that can be detected by more than one sense modality (e.g., synchrony, rhythm, tempo, intensity of stimulation; see Bahrick & Lickliter, 2002; Bahrick & Pickens, 1994; Gibson & Pick, 2000; Lewkowicz, 2000; Lickliter & Bahrick, 2000). For example, infants perceive the rhythm, tempo, and synchrony common to sights and sounds of moving objects (Bahrick, 1987, 1988; Bahrick, Flom, & Lickliter, 2002; Bahrick & Lickliter, 2000; Lewkowicz, 1992, 2000; Mendelson & Ferland, 1982). They also detect amodal information in audiovisual speech including affect, spectral information for vowel sounds, synchrony, gender, and age specified by the face and voice of the speaker (Bahrick, Netto, & Hernandez-Reif, 1998; Dodd, 1979; Kuhl & Meltzoff, 1984; Lewkowicz, 1996; Patterson & Werker, 1999; Walker, 1982; Walker-Andrews, 1997; Walker-Andrews, Bahrick, Raglioni, & Diaz, 1991). Detection of this information enables infants to perceive unitary events from the stimulation presented to different senses. Further, detection of amodal information appears to guide and constrain detection of more specific aspects of stimulation, including information that is unique to a particular sensory modality (e.g., Bahrick, 2001; Gogate & Bahrick, 1998; Hernandez-Reif & Bahrick, 2001; Slater, Quinn, Brown, & Hayes, 1999). This process of detecting amodal relations first, and differentiating stimulation in order of increasing specificity, guides attention in a manner that promotes rapid and economical development of perceptual skills in line with those of adult perceivers (see Bahrick 2000, 2001; Gibson, 1969).

Traditionally, research on the development of multimodal and unimodal perception has been studied separately, even though the topics of investigation may overlap. Most research in the area of perceptual and cognitive development has focused on the development of infant sensitivity to information in a single sense modality and concurrent stimulation to other sense modalities is typically eliminated in research designs. Thus, research abounds on topics such as the development of visual memory, the perception of speech as an auditory stream, the perception of faces separate from voices, and so forth (see Kuhn & Siegler, 1998, for examples). However, perceptual and cognitive abilitiese merge in a multimodal context of people who coordinate speech, gesture, facial movements, and touch, and in a world where objects and events can be seen, heard, and often felt (e.g., Bahrick & Lickliter, 2002; Gogate, Bahrick, & Watson, 2000; Jaffe, Beebe, Feldstein, Crown, & Jasnow, 2001;Lickliter & Bahrick, 2001; Stoffregen & Bardy, 2001). Little research has investigated how perception of objects and events emerges and develops in a world that provides both multimodal and unimodal stimulation and there continues to be a dichotomy between unimodal and multimodal developmental research.

Recently, Bahrick and Lickliter (2000) proposed and provided evidence for an intersensory redundancy hypothesis (IRH) that bridges this dichotomy. This hypothesis predicts how and under what conditions redundantly versus non-redundantly specified information is detected in a multimodal environment. Intersensory redundancy refers to the spatially and temporally coordinated presentation of the same amodal information in two or more senses. Thus, by this definition, intersensory redundancy entails temporal synchrony between the stimulation in two sense modalities. For example, the synchrony uniting the sights and sounds of impact of a bouncing ball highlights the shared amodal information such as the tempo, rhythm, and intensity conveyed by the auditory and visual stimulation. Intersensory redundancy has been shown to be highly salient to young infants and to direct attentional selectivity (Bahrick & Lickliter, 2000, 2002; Bahrick, Flom, et al., 2002, Bahrick, Lickliter, & Flom, 2004). In early development, information experienced redundantly across two or more sensory modalities (amodal information) selectively recruits attention to redundantly specified properties of events (e.g., tempo, rhythm, intensity) at the expense of nonredundantly specified properties (e.g., color, pattern, pitch, or timbre). The IRH makes two complementary predictions (see Figure 1). The multimodal prediction holds that amodal properties are detected more easily in bimodal, synchronous stimulation where they are redundantly specified than in unimodal stimulation where they are not redundantly specified (A > C; see Figure 1). In contrast, attention to nonredundantly specified properties is facilitated in unimodal stimulation as compared with bimodal stimulation because there is no competition from highly salient amodal, redundantly specified properties. Thus, the unimodal and complementary prediction of the IRH is that nonredundantly specified properties are detected more easily in unimodal stimulation as compared with the same nonredundantly specified properties in bimodal stimulation during early development (C > B, see Figure 1).

FIGURE 1.

Predictions of the intersensory redundancy hypothesis: Facilitation of attention and perceptual processing for a given event property as a function of whether the property is redundantly versus nonredundantly specified and whether the type of stimulation available for exploration is bimodal versus unimodal. Detection of a redundantly specified, amodal property is facilitated in bimodal, synchronous stimulation as compared with detection of the same property when it is nonredundantly specified in unimodal stimulation (multimodal prediction, A > C) and detection of a nonredundantly specified property is facilitated in unimodal stimulation as compared with detection of the same property when it is nonredundantly specified in bimodal synchronous stimulation (unimodal prediction, C > B). Note: For intersensory redundancy (as contrasted with intrasensory redundancy), there are no event properties that are redundantly specified in unimodal stimulation and thus this quadrant is not represented.

Prior research has supported the multimodal prediction of the IRH (e.g., Bahrick & Lickliter, 2000; Bahrick, Flom, et al., 2002; Gogate & Bahrick, 1998; Hollich, Newman, & Jusczyk, 2005; Lewkowicz, 2004a, 2004b; Lickliter, Bahrick, & Honeycutt, 2002, 2004; Walker-Andrews, 1997). For example, young infants detect the amodal properties of rhythm and tempo when they are presented bimodally and redundantly, but not when they are presented unimodally and nonredundantly. Five-month-old infants who were habituated to videos of a toy hammer tapping out a rhythm showed visual recovery to a novel rhythm in audiovisual stimulation (they could see and hear the hammer tapping), but not in audio or visual stimulation alone (Bahrick & Lickliter, 2000). Further, infants showed this intersensory facilitation only when the sights and sounds of the hammer tapping were temporally synchronous and not when they were asynchronous. Thus, intersensory facilitation is not the result of a greater amount of stimulation in two modalities than one, or of exposure to two different types of stimulation. Rather, it is the result of the synchronous alignment of two patterns of stimulation that redundantly specify the same amodal properties. Similarly, Bahrick, Flom, et al. (2002) demonstrated that 3-month-olds could detect a change in the tempo of the hammer tapping when it was seen and heard, but not when it was either seen or heard alone. Thus, intersensory redundancy in audiovisual stimulation facilitates detection of amodal properties of events for young infants. This facilitation effect has also been shown in comparative studies with animal embryos and infants. Lickliter et al. (2002, 2004) found that intersensory redundancy facilitates learning of and memory for the temporal patterning of a maternal call in bobwhite quail, even in the prenatal period. When a light was flashed in synchrony with the temporal patterning of an individual maternal call, embryos learned the call 4 times faster and remembered it 4 times longer than under conditions of unimodal auditory or asynchronous audiovisual stimulation. Thus, as with human infants, synchronous alignment of the visual and auditory stimulation appeared necessary for intersensory facilitation.

To date, however, little research has addressed the unimodal prediction of the IRH. Is the perception of nonredundantly specified properties facilitated in unimodal stimulation as compared with bimodal stimulation? Further, how does sensitivity to these properties develop? According to the unimodal prediction of the IRH, unimodal stimulation should facilitate perception of nonredundantly specified properties relative to detection of nonredundantly specified properties in bimodal stimulation in early development (C > B; see Figure 1). For example, detection of the particular sound of an individual voice should be facilitated when no face is visible, and detection of the configuration of the face should be facilitated when no voice is heard. Recent studies have supported these predictions (Bahrick, Lickliter, Shuman, Batista, & Grandez, 2003; Bahrick, Lickliter, Vaillant, Shuman, & Castellanos, 2004), indicating that when the modality of presentation is specific to the property in question (e.g., color or pattern for vision; timbre or pitch for audition), there is likely to be an initial advantage for detecting that property in unimodal stimulation over bimodal, synchronous stimulation.

The IRH also makes a developmental prediction (Bahrick & Lickliter, 2002, 2004). The facilitating effects of intersensory redundancy should be most apparent in early development when infants are first learning to perceive new information or when the task is difficult. With age and experience, perceptual differentiation progresses, and infants detect increasingly more specific information (e.g., Gibson, 1969). At the same time, perceptual processing becomes faster and more efficient (e.g., Hale, 1990; Rose, 1983; Rose, Feldman, & Janowski, 2002). This promotes attentional flexibility, allowing infants to attend to more attributes in a shorter period of time and to switch focus between these attributes more easily. Thus, with experience young infants become proficient at detecting various properties (both redundantly and nonredundantly specified) in unimodal and multimodal stimulation (Bahrick & Lickliter, 2004). Recent studies investigating infants’sensitivity to the amodal properties of rhythm and tempo support this developmental prediction. Although younger infants show sensitivity to rhythm (5 months) and tempo (3 months) in bimodal, redundant stimulation but not in unimodal, nonredundant stimulation (Bahrick & Lickliter, 2000; Bahrick, Flom, et al., 2002), older infants are able to detect a change in rhythm (7.5 months) and tempo (5 months) under conditions of both unimodal and bimodal stimulation (Bahrick & Lickliter, 2004). Thus, the early facilitation effects of redundancy become less apparent as infants become more skilled perceivers and as tasks that were initially difficult become relatively easy. It also follows, however, that if tasks were made more difficult, performance should revert to early patterns of attentional facilitation.

Is there a parallel developmental shift in infants’ perception of nonredundantly specified properties? Do infants also demonstrate a developmental shift from initial detection of nonredundantly specified properties in unimodal stimulation (quadrant C, Figure 1) to later detection of nonredundantly specified properties in bimodal stimulation (quadrant B, Figure 1)? Is the perceptual facilitation for nonredundantly specified (amodal and modality-specific) properties in unimodal contexts apparent in younger infants and attenuated in older infants as they become more proficient at processing and switching attention among various properties in unimodal and bimodal contexts?

This study tested both the unimodal prediction and the developmental prediction of the IRH. We assessed 3-, 5-, and 8-month-old infants’ detection of the orientation of an object striking a surface, information that is nonredundantly specified in unimodal visual and in bimodal, synchronous, audiovisual stimulation. The orientation of an object and its impact (upward vs. downward) was chosen because it could be conveyed visually but not acoustically.¹ Further, several studies have demonstrated young infants’ sensitivity to orientation in dynamic events. For example, infants detect the direction and trajectory of the motion of objects and events around them. Bahrick and colleagues (Bahrick, Hernandez-Reif, & Pickens, 1997; Bahrick & Pickens, 1995) found that 3-month-old infants distinguish the trajectory of a moving object (swinging vs. a circular motion) and show long-term memory for the object’s motion across a period of 3 months. Five-month-olds also distinguish and remember everyday actions such as brushing teeth, brushing hair, or blowing bubbles, across a 7-week period (Bahrick, Gogate, & Ruiz, 2002). Even infants under 1 month of age distinguish between an object approaching and one retreating, and between one approaching on a hit path versus one on a miss path, and show avoidance reactions to only those on a collision course (Ball & Tronick, 1971; Nanez & Yonas, 1994). Further, by 3 to 5 months, infants are able to detect the direction, orientation, and timing of their own limb motions (Bahrick & Watson, 1985; Rochat & Morgan, 1995; Schmuckler, 1995). This ability is fundamental for the development of self-perception. Taken together, these findings indicate that young infants are highly sensitive to the direction, orientation, and timing of their own movements and to those of the animate and inanimate objects around them.

In this research, infants of three ages were shown films of a toy hammer tapping in one of two orientations (upward vs. downward) under conditions of bimodal (synchronous auditory-visual) or unimodal (visual) exposure (Experiment 1) or bimodal (asynchronous auditory-visual) exposure (Experiment 2). Infants were then tested to determine if they could detect a change in event orientation. It was predicted that the young infants would show unimodal facilitation by detecting the change in orientation following unimodal visual exposure and bimodal, asynchronous exposure, but not following bimodal, synchronous, audiovisual exposure. During exploration of synchronous bimodal events, intersensory redundancy should be highly salient and promote attention to redundantly specified amodal properties (e.g., tempo and rhythm, as demonstrated by Bahrick & Lickliter, 2000, and Bahrick, Flom, et al., 2002, using these same events) at the expense of non-redundantly specified properties such as orientation of motion. In contrast, older infants should detect the change in orientation in both bimodal synchronous and unimodal stimulation, as their attention has become more flexible and efficient as a result of increased perceptual differentiation. Specifically, they should be more proficient at perceiving nonredundantly specified properties in bimodal stimulation even in the face of attentional competition from salient redundantly specified properties.

EXPERIMENT 1

Method

Three-, 5-, and 8-month-old infants were tested to determine if they could detect a change in the orientation of an object hitting a surface under bimodal (synchronous audiovisual) versus unimodal (visual) exposure conditions. Procedures and stimuli were similar to those of Bahrick and Lickliter (2000) and Bahrick, Flom, et al. (2002). Infants were habituated to films of a hammer tapping out a rhythm in one of two orientations (upward vs. downward). Test trials consisting of a change in event orientation followed the posthabituation trials.

Participants

Ninety-six infants participated, 32 each at ages, 3, 5, and 8 months. They had mean ages of 106.0 days (SD = 4.3), 150.3 days (SD = 5.3), and 244.8 days (SD = 9.4), respectively. There were a total of 48 boys and 48 girls, with 10 boys and 22 girls at 3 months, 24 boys and 8 girls at 5 months, and 14 boys and 18 girls at 8 months. Infants were primarily from middle-class homes and their parents had at least a high school education. Thirty-two of the infants were White, 5 were African American, 2 were Asian, and 57 were of Hispanic origin. The majority (n = 77) of the infants were tested at Florida International University in the lab of the first author, and a portion (n = 19: n = 15 at 3 months and n = 4 at 8 months) were tested at Brigham Young University in the lab of the third author. A total of 41 additional infants participated, but their data were not included in the final sample. In the 3-month group, the data of 8 infants were rejected: 2 for fussiness, 2 for equipment failure, 1 for experimenter error, 1 for failure to habituate within 20 trials, and 2 for failure to pass the fatigue criterion (see Procedure for details). At the age of 5 months, the data of 16 infants were rejected: 2 for fussiness, 3 for experimenter error, 1 for equipment failure, 3 for failure to habituate within 20 trials, and 7 for failure to pass the fatigue criterion. At the age of 8 months, the data of 17 infants were rejected: 2 for fussiness, 2 for experimenter error, 5 for equipment failure, 3 for failure to habituate within 20 trials, and 5 for failure to pass the fatigue criterion. All infants were healthy and had a gestation period of at least 38 weeks.

Stimulus events

Stimulus events (taken from Pickens & Bahrick, 1995, 1997) depicted a red toy hammer tapping out a distinctive rhythm against a light-colored wooden surface. The hammer portrayed movements in one of two orientations of motion, upward or downward. Either the hammer struck upward against a wooden ceiling or downward against a wooden floor. The films were identical in all other respects. One of two irregular rhythms was played at a tempo of 110 beats per minute (1.8 Hz). The rhythms were X-O-XX-X and XX-O-X-X where X represents a whole-beat impact, XX represents two half-beat impacts, and O represents a whole-beat rest (see Pickens & Bahrick 1995, 1997, for details). The natural impact sounds produced by the events could be heard during the bimodal presentations of the displays and the sounds were synchronized with the visual trajectory changes of the hammer at each impact. A control display depicted a green and white toy turtle whose arms spun, making a whirring sound.

Apparatus

Infants sat in an infant seat facing a 19-in. (Panasonic BT-S1900N) video monitor at a distance of approximately 55 cm. The infant seat and monitor were surrounded by black curtains. The curtains had two apertures near the upper right and upper left corners of the video monitor that allowed the observers to view the infants’ visual fixations to the video displays. A small set of bells inside plastic spheres was located near the video monitor and was used to attract the infants’ attention toward the video monitor when needed.

The stimulus events were videotaped with a Panasonic (WV3170) color video camera and a Sony (EMC-105T) microphone. The events were edited using a Panasonic (VHS NV-A500) edit controller connected to three Panasonic video decks (AG 6300 and AG 7750). The video of the hammer tapping in the upward motion was created by inverting the video image of the hammer tapping in the downward motion using a digital video mixer (Videonics MX-1NTSC). The videos were presented to infants using the three video decks and edit controller. All soundtracks were presented from a speaker located just beneath the video monitor at approximately 65dB (A scale, fast response) measured from the infant seat.

A trained observer, unaware of the infant’s condition and unable to see the video displays, monitored infants’ visual fixations by depressing a button while the infant visually fixated the display. The button box was connected to a computer that recorded and computed the durations of infants’ visual fixations to the video displays and signaled a second experimenter when it was time to commence and terminate each trial. A record of the infants’ visual fixations was created online. The observations of the primary observer controlled the audiovisual presentations. A secondary observer recorded infant visual fixations for a proportion of the participants and provided data for calculating interobserver reliability.

Procedure

Infants were tested in an infant-control habituation procedure (Horowitz, Paden, Bhana, & Self, 1972) to determine if they could detect a change in the orientation of the impact of the hammer following bimodal versus unimodal exposure. Thirty-two infants of each age were randomly assigned to the bimodal (auditory-visual) or the unimodal (visual) habituation conditions (n = 16 each condition, each age). Within each of these conditions, half the infants (n = 8) received habituation with the hammer hitting against the floor (downward) and half (n = 8) received habituation with the hammer hitting against the ceiling (upward). Thus, during the bimodal habituation infants could hear and see the hammer tapping a rhythm in one of the two orientations. During the unimodal habituation infants could see but not hear the hammer tapping a rhythm in one of the two orientations. For the test trials, all infants received a change in orientation of the hammer’s impacts (and no other changes) under their respective conditions, to assess sensitivity to event orientation.

The habituation procedure was similar to that of our prior studies (Bahrick 1992; Bahrick, Flom, et al., 2002; Bahrick & Lickliter, 2000). In general, the habituation sequence consisted of an initial control trial (thetoyturtle) and four mandatory habituation trials. Habituation was terminated after the infant reached the habituation criterion and completed two (no-change) posthabituation trials. Each trial began when the infant fixated the visual display and was terminated after the infant looked away for 1.5 consecutive sec. Further, a ceiling of 60 sec was set as the maximum trial length, and 20 trials was the maximum number of trials for habituation. The habituation criterion was defined as a decrement of 50% or greater on two consecutive trials, relative to the infant’s initial fixation level (baseline, the average number of seconds of fixation during the first two habituation trials). After the habituation criterion was met, two no-change posthabituation trials were presented where infants received two additional habituation trials. Visual recovery on subsequent test trials was assessed in relation to these posthabituation trials. This served to establish a more conservative criterion for habituation by reducing chance habituation and taking spontaneous regression effects into account (see Bertenthal, Haith, & Campos, 1983, for a discussion of the importance of addressing these effects). To ensure that infants had actually habituated to the displays (and that the criterion was not met by chance), during the habituation phase the computer program compared the infants’ mean posthabituation looking score with their habituation criterion. If the mean posthabituation looking exceeded the infant’s habituation criterion, the infant was returned to the habituation phase and given additional habituation trials until reaching criterion again (n = 3 at the age of 3 months, and n = 1 at 8 months). Then, two no-change posthabituation trials followed as before. After infants met this habituation criterion, there was no evidence of spontaneous regression during the posthabituation trials with respect to the two final habituation trials at any age (Ms =.70,.15, and −.29, for 3-, 5-, and 8-month-olds, respectively; all ps >.1). Following successful completion of the habituation phase and the two posthabituation trials, infants received two test trials that served as the basis for calculating visual recovery to a change in stimulation. Infants then received a final control trial (the toy turtle), which served as a basis for assessing fatigue.

The final control trial assessed whether infants were overly fatigued and unable to show visual recovery (see Bahrick, 1992, 1994; Bahrick, Flom, et al., 2002; Bahrick & Lickliter, 2000). The duration of fixation to the moving turtle on the final control trial was compared with the duration of fixation to the turtle on the initial control trial. A visual fixation to the turtle on the final control trial that was at least 20% of the initial trial was established for including the data. The data of 14 infants overall were rejected for failure to meet the fatigue criterion (n = 1 unimodal, n = 1 bimodal, at 3 months; n = 4 unimodal, n = 3 bimodal at 5 months; n = 3 unimodal, n = 2 bimodal, at 8 months). The remaining infants showed substantial visual fixation on the final control trial. The mean fixation level on the final control trial with respect to the initial control trial was 131.5% (SD = 103) overall, where 3-month-olds showed a mean of 102% in the bimodal and 145% in the unimodal condition, 5 month olds showed a mean of 114% in the bimodal and 134% in the unimodal condition, and 8 month olds showed a mean of 120% in the bimodal and 174% in the unimodal condition

A secondary observer monitored infant visual fixation for a total of 21 infants (22% of the sample) as a basis for calculating interobserver reliability (n = 5 at 3 months, n = 8 at 5 months, and n = 8 at 8 months). For each infant, total fixation to the display was calculated independently for the primary and the secondary observer for each trial. The fixation times were then correlated (Pearson product–moment correlation) across trials for each participant. Interobserver reliability, the mean of these correlations, was.98 (SD =.03) overall. At 3 months it was.99 (SD =.004), at 5 months it was.97 (SD =.02), and at 8 months, it was.97 (SD =.04).

Results and Discussion

Table 1 depicts the pattern of habituation for infants of each age in the unimodal and the bimodal conditions, including processing time, number of trials to habituation, and the mean visual fixation to the displays as a function of trial type (baseline, posthabituation, or test). Visual recovery to the change in event orientation in the test displays was the primary dependent variable. This was computed by subtracting the mean fixation on the test trials from the mean fixation on the posthabituation trials for each infant and averaging across infants. Visual recovery scores for infants of each age for the unimodal and bimodal conditions are depicted in Figure 2.

TABLE 1.

Amount of Processing Time (Total Number of Seconds Looking During Habituation), Number of Trials to Habituation, Mean Seconds Looking During Baseline (First Two Habituation Trials), Posthabituation (Final Two Habituation Trials), and Test (Two Test Trials Depicting a Change in Direction of Motion), for Infants of 3, 5, and 8 Months as a Function of Condition, Age, and Experiment

			Processing Time		Trials to Habituation		Baseline		Posthabituation		Test
Experiment	Age	Condition	M	SD	M	SD	M	SD	M	SD	M	SD
1	3 months	Bimodal synchronous	154.93	54.26	6.81	1.76	35.00	13.13	4.34	2.90	6.74	5.72
		Unimodal	207.74	158.19	8.56	3.20	33.12	16.47	5.77	4.60	21.06	18.04
1	5 months	Bimodal synchronous	202.64	69.38	7.25	1.95	44.36	16.54	10.54	5.82	14.49	14.99
		Unimodal	170.95	95.78	7.44	2.37	35.88	16.14	6.26	3.24	10.93	6.12
1	8 months	Bimodal synchronous	121.84	65.19	8.15	4.76	24.68	13.57	4.55	2.78	9.15	5.34
		Unimodal	93.74	73.06	8.44	3.90	13.46	13.53	4.08	3.59	8.38	4.98
2	3 months	Bimodal asynchronous	231.90	158.74	8.20	3.00	38.5	17.80	7.76	4.85	24.35	16.75

FIGURE 2.

Mean visual recovery to the change in event orientation as a function of condition (unimodal visual, bimodal audiovisual) for infants of 3, 5, and 8 months of age. Visual recovery is a difference score between visual fixation during the test trials and visual fixation during the posthabituation trials. Standard deviations appear in parentheses.

To address the main research questions, a two-way analysis of variance (ANOVA) was conducted on visual recovery with age (3, 5, or 8 months) and condition (unimodal, bimodal) as between-subject factors. Support for predictions of the IRH was revealed by a significant Age × Condition interaction, F(2, 90) = 4.21, p =.018, Cohen’s d =.75, where 3-month-olds showed detection of the change in orientation under the unimodal conditions but 8-month-olds showed detection under both the unimodal and bimodal conditions, and by a significant main effect of condition, F(1, 90) = 4.61, p =.034, Cohen’s d =.44, with visual recovery to the unimodal displays significantly higher than to the bimodal displays. Planned comparisons revealed a significantly greater visual recovery to the unimodal than the bimodal changes in orientation at 3 months, t(30) = 3.0, p =.008, Cohen’s d = 1.1, but no difference at 5 and 8 months (ps >.1). Even when overall processing time was used as a covariate and these means were adjusted for processing time differences, planned comparisons also revealed significantly greater visual recovery to the unimodal than the bimodal changes in orientation at 3 months (p =.002), but no difference at 5 and 8 months (ps >.1).

To determine at which ages and under which conditions infants showed significant evidence of detecting the change in orientation of the hammer’s impacts, single-sample t tests were calculated on the mean visual recovery scores against the chance value of 0 for each condition at each age. Results (see Figure 2) indicated significant visual recovery for infants who received unimodal habituation and tests at 3, 5, and 8 months, t(15) = 3.76, p =.002, Cohen’s d = 1.37; t(15) = 2.64, p =.018, Cohen’s d =.97; t(15) = 4.73, p =.0003, Cohen’s d = 1.74, respectively. However, under conditions of bimodal audiovisual habituation and testing, infants showed evidence of detecting a change in orientation of motion only at 8 months of age, t(15) = 3.24, p =.006, Cohen’s d = 1.18. Thus, consistent with predictions of the IRH, the younger infants (3 and 5 months) discriminated a change in the orientation of the hammer’s impacts following unimodal visual exposure but not following bimodal audiovisual exposure, whereas older infants (8 months) discriminated the change under both unimodal and bimodal exposure conditions.²

The data were also examined at the individual participant level. At 3 months, 14 of the 16 infants showed visual recovery scores that were positive in the unimodal condition (p =.002 according to a binomial test), whereas in the bimodal condition, only 9 of the 16 infants had positive visual recovery scores (p >.1). At 5 months, 15 of the 16 infants showed positive visual recovery scores in the unimodal condition (p =.0002 according to a binomial test), whereas in the bimodal condition, only 9 of the 16 infants had positive visual recovery scores (p >.1). At the age of 8 months, 15 of the 16 infants showed positive visual recovery scores in the unimodal condition (p =.0002), and 14 of the 16 infants had positive scores in the bimodal condition (p =.002). These results converge with those of the group analyses, and indicate that infants of all ages were able to discriminate a change in the orientation of the hammer and its impacts during unimodal visual exposure, whereas only the oldest infants were able to discriminate the change in orientation during the bimodal, synchronous, audiovisual exposure to the events.

Secondary analyses were conducted to assess any effects of gender or test environment on infants’ visual recovery scores. A three-way ANOVA was conducted with age, condition, and infant gender as between-subject factors and revealed no effects of gender or interaction of gender with any of the main variables (ps >.1). T tests compared the visual recovery of infants tested in the two different labs at each of the two ages (3 and 8 months). No differences in performance were observed as a function of test environment (all ps >.1).

Secondary analyses were also conducted to determine whether infants showed any differences in their patterns of habituation as a function of age or as a function of whether they received bimodal, audiovisual habituation or unimodal visual habituation. Specifically, we asked whether infants showed any differences in initial interest level (baseline fixation time), final interest level (posthabituation fixation time), number of trials to habituation, or total processing time (number of seconds exposure to the habituation events) for the bimodal versus the unimodal visual displays at each age. These data are displayed in Table 1. Separate two-way ANOVAs were conducted for each of these variables, with age (3, 5, or 8 months) and condition (bimodal, unimodal) as the main factors. Results revealed a significant main effect of condition for only the mean baseline (first two habituation trials) looking, F(1, 90) = 5.50, p =.021, Cohen’s d =.21. This reflects the fact that initial interest was higher for the bimodal than the unimodal displays. However, neither the overall processing time, the final interest level (posthabituation), nor the number of trials were significantly different for the bimodal and unimodal conditions (ps >.1). Main effects of age were found for posthabituation looking, F(2, 90) = 6.81, p =.002, Cohen’s d =.24, where final interest was higher for the 5-month-olds than the 3- or 8-month-olds, and for total processing time, F(2, 90) = 9.36, p <.001, Cohen’s d =.27. The number of seconds spent processing the events was significantly greater for the 3- and 5-month-olds than the 8-month-olds, whereas the 3- and 5-month olds did not differ from one another in processing time. This is consistent with prior research findings that younger infants require more time to process or explore events than do older infants (Bahrick, Hernandez-Reif, & Flom, 2005; Hale, 1990; Rose, 1983; Rose et al., 2002). There was also one significant Age × Condition interaction, for posthabituation looking, F(2, 90) = 4.0, p =.02, Cohen’s d =.73, where 5-month-olds had greater final interest in the bimodal display than the unimodal displays but 3- and 8-month-olds did not differ. No other effects were found (ps >.1).

Secondary analyses were also conducted to determine whether visual recovery differed as a function of whether infants were habituated to the upward or downward orientation of the hammer and its impacts, or whether the hammer depicted one rhythm or the other. Results of a three-way ANOVA for each condition with age (3, 5, or 8 months), orientation (upward vs. downward), and rhythm (1 vs. 2) as the between-subject factors, indicated no significant main effects of event orientation or rhythm, or interactions of these factors, for either the unimodal or the bimodal condition (all ps >.05). Thus, infants showed no differential effects as a function of which rhythm or event orientation they received during habituation.

EXPERIMENT 2

The results of Experiment 1 demonstrate that young infants show better detection of orientation, a nonredundantly specified property, in unimodal stimulation than bimodal synchronous stimulation. According to the IRH, this unimodal facilitation is a result of having no attentional competition from redundantly specified amodal properties. Intersensory redundancy in bimodal stimulation attracts attention to amodal properties of stimulation at the expense of other properties, impairing detection of nonredundantly specified properties such as orientation. Prior research with both human and nonhuman animal infants has supported the conclusion that it is the redundancy and not other factors such as the amount or type of stimulation that is responsible for intersensory facilitation. Bimodal, asynchronous control conditions did not promote detection of amodal properties; in contrast, bimodal synchronous conditions did (Bahrick & Lickliter, 2000; Lickliter et al., 2002, 2004).

This experiment explored the basis for the impaired detection of orientation by 3-month-olds in the bimodal condition of Experiment 1. We argue that infants’ failure to detect orientation was a consequence of the redundancy provided by the bimodal, synchronous stimulation that promotes attention to competing amodal properties. Alternatively, it is possible that bimodal stimulation interfered with detection of orientation because bimodal stimulation provides more or different information than unimodal stimulation. For example, two streams of information may be more distracting or provide a greater amount of overall stimulation than one stream of information. Thus, in this experiment, infants were exposed to asynchronous bimodal stimulation, providing the same amount and type of stimulation as in Experiment 1, but eliminating intersensory redundancy. If redundancy captures attention and is responsible for the unimodal–bimodal differences in detection of orientation at 3 months, then asynchronous bimodal stimulation should not interfere with detection of orientation. To test this hypothesis, 3-month-olds were habituated with asynchronous bimodal presentations of the hammer tapping, and then tested for detection of a change in orientation. Given that this asynchronous presentation provided no audiovisual redundancy, we predicted that infants’ performance should be comparable to that of the unimodal condition in Experiment 1.

Method

Participants

Twelve 3-month-old infants (7 boys and 5 girls) participated. They had a mean age of 106.7 days (SD = 4.3). They were selected using the same selection criteria as infants in Experiment 1. Six of the infants were White, 1 was Asian, and 5 were of Hispanic origin. Seven infants were tested at Florida International University and 5 were tested at Brigham Young University. One additional infant participated, but his data were rejected for excessive fussing.

Stimulus events, apparatus, and procedure

All events and procedures were identical to those of Experiment 1 with the exception that asynchronous films and soundtracks were presented. The asynchrony was achieved by using two identical videos and playing the soundtrack from one video deck and the video portion from another deck. The visual and auditory impacts of the hammer were thus unsystematically out of phase. Care was taken to ensure that the audio and visual impacts were misaligned, but the degree of misalignment varied across infants. Interobserver reliability, calculated for 3 of the infants tested, was.998 (SD =.002).

Results and Discussion

Figure 3 depicts the mean visual recovery to a change in orientation for infants in this study (asynchronous bimodal stimulation) along with that of the 3-month-olds in Experiment 1 (unimodal visual and synchronous bimodal stimulation). The mean visual recovery to the change in orientation under the asynchronous conditions of this study was 16.59 (SD = 15.47). This mean is significantly different from chance according to a single-sample t test, t(11)=3.72, p=.003, Cohen’s d=1.58, indicating that 3-month-old infants detected the change in orientation in the bimodal, asynchronous presentation, supporting predictions of the IRH. This finding was corroborated by individual participant analyses. Eleven of 12 infants showed positive visual recovery scores and this result is significantly greater than chance according to a binomial test (p=.003). Further, the performance of infants in the asynchronous condition (Experiment 2) was compared with that of 3-month-olds in the unimodal and bimodal synchronous conditions of Experiment 1. Results revealed a significant main effect of condition, F(2, 41) = 5.36, p =.009, Cohen’s d =.51, with significantly greater visual recovery to the change in orientation for infants in the bimodal asynchronous and unimodal conditions than those in the bimodal synchronous condition (p =.007, p =.008, respectively). These results support the interpretation that intersensory redundancy available in the bimodal synchronous presentation of Experiment 1 impaired infants’ detection of orientation. Further, asynchronous, bimodal exposure appeared comparable to unimodal exposure in facilitating detection of nonredundantly specified properties of stimulation.

FIGURE 3.

Mean visual recovery to the change in event orientation for 3-month-olds in Experiment 1 who received unimodal visual versus synchronous, bimodal audiovisual stimulation and for 3-month-olds in Experiment 2 who received asynchronous, bimodal audiovisual stimulation. Visual recovery is a difference score between visual fixation during the test trials and visual fixation during the posthabituation trials. Standard deviations appear in parentheses.

Secondary analyses on the data of Experiment 2 revealed no differences in visual recovery as a function of gender, t(10) =.43, p >.1, or whether infants were tested in one laboratory or the other, t(10) =.19, p >.1. Further, a two-way ANOVA revealed that there were no differences as a function of whether infants were habituated to the upward or downward orientation or to rhythm 1 versus rhythm 2 (all ps >.1).

Infants’ pattern of habituation in the experiment was compared with that of the 3-month-olds in Experiment 1. A one-way ANOVA with condition (bimodal asynchronous, Experiment 2; bimodal synchronous, Experiment 1; unimodal visual, Experiment 1) as a between-subject factor was conducted for each of four measures, baseline fixation, posthabituation fixation, processing time, and trials to habituation. Results revealed no significant effects of any of these factors (all ps >.1), indicating no difference in the patterns of habituation across Experiments 1 and 2.

GENERAL DISCUSSION

This research tested two predictions of the IRH concerning the development of selective attention and perception of unimodal and multimodal events. We explored whether, in early infancy, detection of nonredundantly specified properties of events is facilitated in unimodal stimulation as compared with bimodal stimulation (the unimodal prediction of the IRH) and whether a developmental shift occurs as infants gain experience with objects and events and develop more efficient and flexible patterns of attentional allocation. That is, would older infants show detection of nonredundantly specified properties in both unimodal and bimodal stimulation (the developmental prediction of the IRH)? The experiments reported here evaluated these two predictions by assessing infants’detection of the orientation of an object’s impacts, information that can be detected visually but not acoustically, and therefore is nonredundantly specified in both unimodal visual and bimodal audiovisual stimulation.

In Experiment 1, infants of 3, 5, and 8 months viewed films of a hammer tapping in one of two orientations (upward vs. downward) under unimodal visual or bimodal synchronous audiovisual conditions, and then received test trials with a change in orientation. Results supported both predictions of the IRH. Infants of 3, 5, and 8 months showed significant visual recovery, detecting the change in orientation in unimodal visual stimulation at all ages. In contrast, only the oldest infants detected the change in orientation in bimodal, synchronous stimulation, where there was attentional competition for redundantly specified properties. Comparisons across groups revealed an Age × Condition interaction with 3-month-olds showing greater visual recovery to the change in orientation under the unimodal than the bimodal condition, but older infants showing no differences. Planned comparisons demonstrated greater visual recovery to the unimodal than the bimodal changes in orientation at 3 months, but no differences at 5 and 8 months, even when visual recovery scores were equated for (nonsignificant) differences in processing time. Individual participant analyses also corroborated the pattern revealed by the group analyses. Thus, for younger infants (aged 3 and 5 months) detection of orientation (information that was nonredundantly specified in both unimodal and bimodal stimulation) was facilitated when stimulation was unimodal (visual) and it was attenuated when stimulation was bimodal (synchronous audiovisual).

These findings provide direct support for the unimodal prediction of the IRH, which holds that detection of nonredundantly specified properties is facilitated under conditions of unimodal and attenuated under conditions of bimodal stimulation in early development (Bahrick & Lickliter, 2000, 2002; Bahrick, Lickliter, & Flom, 2004). Additional support for this prediction is provided by studies of face and voice perception that show facilitation of acoustically specified properties (e.g., pitch and timbre of voices) as well as visually specified properties (e.g., configuration of the face) in unimodal displays in young infants (Bahrick, Lickliter, Vaillant, et al., 2004; Bahrick et al., 2003).

Why might detection of nonredundantly specified properties be selectively advantaged in unimodal stimulation in early infancy? We hypothesized that with unimodal stimulation there is no competition from highly salient redundantly specified properties. Bimodal or multimodal stimulation creates intersensory redundancy, causing redundantly specified amodal properties (e.g., rhythm, tempo, synchrony) to be highlighted and compete for attention with nonredundantly specified properties (e.g., orientation, color, shape). For example, we previously found that 3- and 5-month-old infants detect the rhythm and tempo of the present hammer events in bimodal, redundant stimulation but not in unimodal stimulation (Bahrick, Flom, et al., 2002; Bahrick & Lickliter, 2000). In contrast, when synchronized sounds are not present and only the visual stimulation from an event is available, infant attention is selectively directed to properties that are visually detectable and nonredundantly specified (in this study, the orientation of the hammer and its visual impacts).

This hypothesis, that unimodal facilitation results from lack of attentional competition from salient intersensory redundancy, was tested in Experiment 2. Was the failure of 3-month-olds to detect orientation changes in Experiment 1 a result of the salience of intersensory redundancy provided by the synchronous bimodal stimulation? By presenting asynchronous bimodal stimulation we held constant the amount and type of stimulation while eliminating intersensory redundancy for competing amodal properties. Under these conditions of asynchronous bimodal stimulation, we predicted that 3-month-olds would detect a change in orientation and they would do so better than under conditions of synchronous bimodal stimulation where orientation was nonredundantly specified. Results supported our predictions and demonstrated significant visual recovery to the change in orientation when the soundtrack was asynchronous and greater visual recovery in the asynchronous and unimodal conditions than the synchronous bimodal condition. These findings suggest that detection of nonredundantly specified properties of events is promoted when no intersensory redundancy competes for attention, supporting the unimodal prediction of the IRH.

How does the facilitation of nonredundantly specified properties in unimodal stimulation change across development? As infants become more experienced, perceptual differentiation progresses and perceptual capabilities become more flexible and extend to a variety of contexts. The processing advantages seen in early infancy are no longer as apparent when perceivers are more skilled. We have recently provided support for this principle regarding infants’ detection of rhythm and tempo (Bahrick & Lickliter, 2004). Although young infants detected the rhythm and tempo of the hammers tapping only in bimodal, redundant stimulation, older infants detected the rhythm and tempo even in unimodal, nonredundant stimulation. Results of this experiment corroborated this developmental shift for infants’ detection of nonredundantly specified information. Although 3- and 5-month-olds detected the change in orientation of the hammer’s impacts only in unimodal visual stimulation, by 8 months of age infants were able to detect the change, even in bimodal stimulation. The 8-month-olds showed visual recovery to the change in the orientation of the hammer’s impacts (upward vs. downward) in the synchronous audiovisual displays, despite the fact that these displays recruit attention to other redundantly specified amodal properties such as rhythm, tempo, and synchrony.

Taken together with our prior findings, this study suggests several developmental changes in attention and perception of multimodal events across the first 8 months of life. First, older infants extend their detection of properties of events to new contexts, from unimodal to bimodal (this study), or from bimodal to unimodal (Bahrick & Lickliter, 2004). Nonredundantly specified properties appear to be detected first in unimodal stimulation and later extended to bimodal and multimodal stimulation, and redundantly specified properties appear to be detected first in bimodal and multimodal stimulation and then extended to unimodal stimulation. Second, older infants also seem to detect a larger range of event properties (both redundantly and nonredundantly specified) than younger infants, as demonstrated by the findings of our prior study, where 8-month-olds detected the rhythm and tempo of the hammer events (Bahrick & Lickliter, 2004), and by these findings, where 8-month-olds detected the orientation of the hammer’s impacts. In contrast, younger infants detected either the redundantly specified rhythm and tempo or the nonredundantly specified orientation, but not both, depending on whether stimulation was unimodal or multimodal.

Interestingly, results of this study indicate that infants also showed greater initial interest in the bimodal rather than the unimodal displays, and comparable processing time of the bimodal and unimodal displays; however, at the younger ages discrimination of orientation was apparent only for the unimodal displays in Experiment 1. This suggests that either young infants were attending to properties other than orientation in the bimodal displays, or alternatively, that information for orientation is more difficult to abstract in bimodal than unimodal displays. The results of Experiment 2 indicating that redundancy hinders infants’detection of orientation in bimodal displays, together with results of our prior studies using these same events demonstrating that young infants attend to redundantly specified properties of rhythm and tempo in the bimodal synchronous presentations (Bahrick, Flom, et al., 2002; Bahrick & Lickliter, 2000) support the former view. Infants appear to attend to different properties of stimulation in unimodal and bimodal synchronous displays. Thus, orientation information is likely not especially difficult to abstract in bimodal displays; rather, it is a matter of attentional selectivity.

Taken together with our prior studies (Bahrick, Flom, et al., 2002; Bahrick & Lickliter, 2000, 2004), this study portrays a picture of how exploration of objects and events that provide unimodal and multimodal stimulation in the natural environment might interact to promote integrated and coherent knowledge about both amodal and modality-specific properties of events during infancy. Exploration in the everyday environment of the infant likely consists of continuously shifting cycles of attention to various redundantly and nonredundantly specified properties of objects and events as a function of the changing nature of stimulation (unimodal vs. multimodal) encountered. For example, in close face-to-face interaction when an adult is speaking, infants may selectively attend to amodal, redundantly specified properties of the face and voice such as the synchrony of audiovisual speech and its common rhythm, tempo, or prosody. Similarly, when the infant is held and moved by the caretaker, intermodal visual-proprioceptive or auditory-proprioceptive contingency may recruit attention to temporally coordinated motion, touch, sound, or visual stimulation. In contrast, nonredundantly specified properties may be attended when a nearby adult is still or silent. Thus, infant attention may be recruited to aspects of the face and body that are specific to vision such as the configuration of the face, hair-style, pattern and color of clothing, or direction and orientation of motion. Similarly, when the caretaker speaks from a nearby room, attention to nonredundantly specified and unique properties of the voice such as pitch and timbre may be promoted. In this manner, perception of objects and events in natural contexts, where bouts of unimodal and multimodal exploration of events typically occur, can promote the development of sensitivity to both amodal and modality-specific properties of objects and events in an intercoordinated manner.

In a more general sense, the IRH and the research findings generated by it describe how attention to various properties of objects and events (amodal and redundantly specified vs. modality-specific and nonredundantly specified) shifts as a function of the type of stimulation encountered (unimodal vs. multimodal) and how this changes developmentally. In early development, infant attention is captured by intersensory redundancy in multimodal stimulation, which highlights amodal properties of events (Bahrick, Flom, et al., 2002; Bahrick & Lickliter, 2000, 2002; Lickliter et al., 2002, 2004). Because most events are multimodal, detection of redundantly specified amodal properties of stimulation is promoted to a greater extent than detection of modality-specific properties. This promotes perception, learning, and memory for amodal aspects of stimulation prior to other aspects. These initial conditions can provide the basis for a cascading set of influences on perceptual development that may continue to influence perception, learning, and memory into later stages of development (see Bahrick & Lickliter, 2002; Lickliter & Bahrick, 2001, for further discussion). For example, this early detection of amodal, redundantly specified properties of stimulation can serve to guide and organize perceptual development, such that sights and sounds that belong together are perceived together and are thus experienced as unitary, coherent events. Further, early detection of amodal relations can guide and constrain perception of modality-specific information. Temporal synchrony serves to bind auditory and visual stimulation from unitary events, promote attention to amodal properties, and then foster further differentiation of event properties in order of increasing specificity (see Bahrick, 2001; Gibson, 1969; Gogate & Bahrick 1998; Hernandez-Reif & Bahrick, 2001). Later in development, as infant attention becomes more flexible and processing becomes more efficient (and tasks become relatively easier with experience), infants become capable of detecting both redundantly specified properties and nonredundantly specified properties in stimulation of various types. Further research is needed to determine to what extent and under what conditions intersensory redundancy continues to facilitate perception of amodal properties of stimulation in later development.

To date, research testing predictions of the IRH has demonstrated the facilitating effects of intersensory redundancy for detection of amodal properties across a variety of events. However it is not yet known the extent to which perceptual facilitation might occur as a result of other types of redundancy. For example, in early development might intrasensory redundancy (redundancy within a single sense modality such as that provided by two different objects tapping the same rhythm, or a light flashing in synchrony with an object’s movements) also facilitate detection of amodal properties (e.g., rhythm, tempo, and intensity) and consequently also hinder detection of nonredundantly specified properties? Studies are currently in progress to address these important questions.

From a methodological perspective, the research reported here suggests that the current dichotomy characterizing research on unimodal versus multimodal perception is in need of revision. In particular, our findings highlight the need for more integration between unimodal and multimodal research investigating attention, perception, cognition, and memory. More ecologically relevant theories and a more complete understanding of development can be achieved by studying perception of the young organism in an environment that provides both unimodal and multimodal stimulation for a variety of properties of events.