The origins of cortical multisensory dynamics: Evidence from human infants

Abstract

Classic views of multisensory processing suggest that cortical sensory regions are specialized. More recent views argue that cortical sensory regions are inherently multisensory. To date, there are no published neuroimaging data that directly test these claims in infancy. Here we used fNIRS to show that temporal and occipital cortex are functionally coupled in 3.5-5-month-old infants (N = 65), and that the extent of this coupling during a synchronous, but not an asynchronous, audiovisual event predicted whether occipital cortex would subsequently respond to sound-only information. These data suggest that multisensory experience may shape cortical dynamics to adapt to the ubiquity of synchronous multisensory information in the environment, and invoke the possibility that adaptation to the environment can also reflect broadening of the computational range of sensory systems.

1. Introduction

Multisensory processing is critical to perception and attention (Amso and Scerif, 2015; Macaluso et al., 2016) and to complex learned skills, including speech production and language comprehension (Bishop and Miller, 2009; Rosenblum, 2008; Skipper et al., 2007). Until recently, the classic view of multisensory processing has been that cortical regions are inherently specialized, for example sound information is only processed in temporal cortex (see Macaluso, 2006 for review). In this view, multisensory processing only occurs when modality-specific information reaches higher-order association areas. This view has been challenged by the alternative that the neocortex is largely a multisensory organ (Ghazanfar and Schroeder, 2006). This newer view is based on several strands of evidence. First, occipital and temporal cortices have been found to respond to both auditory and visual stimulation in adults (see Murray et al., 2016 for review). Second, multisensory interactions occur not only in higher-order association areas, but also and concurrently in the midbrain and in sensory-specific cortices (Stein and Stanford, 2008). Finally, multisensory interactions in sensory cortical areas occur as early as 40 ms following sensory input, suggesting that feedforward mechanisms from low-level sensory regions are as likely to support multisensory processing as feedback from higher-level association cortex (Foxe and Schroeder, 2005; Schroeder and Foxe, 2005).

What remains unclear is the developmental origin of these effects. To our knowledge, there are no cortical neuroimaging studies in human infants that have uniquely tested the inherent versus experience-dependent origins of multisensory processing. This is despite the fact that infants begin life with powerful, though relatively rudimentary, behavioral multisensory processing abilities (Lewkowicz and Ghazanfar, 2009; Lewkowicz, 2014). For instance, infants use temporal synchrony to perceive multisensory coherence from birth (Lewkowicz et al., 2010; Lewkowicz, 1996, 2010), as well as to support learning in general (Bahrick and Lickliter, 2000). The relevancy of synchrony for multisensory processing is evident at the neural level in adult findings. For instance, prior work has shown that synchronous multisensory experience drives functional connectivity among occipital and temporal cortices, which then enhances unisensory processing (Lewis and Noppeney, 2010; Tyll et al., 2013). Yet, the bulk of the data on multisensory processing in the brain comes from adult data, and, unfortunately, those data reflect the bias of years of exclusive exposure to synchronous multisensory experiences. Thus, a consensus about the inherent or experience-driven multisensory nature of cortex cannot be assumed from adult data because they represent a far more mature developmental state.

Prior work examining multisensory processing using neuroimaging in infancy was not designed to determine the unisensory or multisensory nature of cortex and, indeed, offers conflicting evidence on this issue. For example, Bortfeld et al. (2007) used fNIRS to examine 6-9-month-old infants’ occipital and temporal responses to visual animations either in isolation or when paired with speech sounds. They found that left temporal cortex activation was specific to speech sounds and that occipital cortex was specific to the visual animations. Similarly, Taga and Asakawa (2007) presented 2-4-month-old infants with speech sounds in alternation with checkerboard patterns. They found that visual events do not affect unrelated auditory processing in infancy. In contrast, other studies using fNIRS have shown that presentation of audiovisual information increases activation in both areas relative to when the sound was removed (Watanabe et al., 2013; Emberson et al., 2015). These findings indicate that occipital cortex may be multisensory because it responds in an additive fashion to the audio + visual information. Finally, it has been found that 5-month-old infants have event related potentials associated with more efficient stimulus processing when exposed to synchronous as opposed to asynchronous audiovisual events (Reynolds et al., 2014). Although important, these results do not provide any insights into the effects of short-term experience on multisensory processing. In other words, no studies to date have directly measured infant neural responses to unisensory information before and after a multisensory experience and whether such experience can induce changes in the cortical dynamics underlying sensory processing.

Thus, while there is evidence for multisensory processing in temporal and occipital regions by adulthood, data about the developmental origins of these cortical dynamics are controversial. Our work addresses this gap in the literature by examining the role of short-term synchronous and asynchronous multisensory experience in shaping cortical sensory processing in 3.5-5-month-old infants. This age range was chosen because infants only begin to integrate audiovisual information without the use of low-level cues, such as temporal synchrony, starting around 6–8 months of age (Lewkowicz and Röder, 2012). Thus, examining individual differences in younger infants, who have greater variability in multisensory processing, allows for mechanistic insight into the cortical dynamics that may support this developmental shift. We used fNIRS to measure the functional response patterns of occipital and temporal cortical regions to unisensory sound and visual stimuli. Critically, measurements were made before and after infants experienced the same stimuli as part of synchronous and asynchronous multisensory audiovisual familiarization events (Fig. 1). We predicted one of three results. First, the occipital and/or temporal regions might be unisensory, but functionally connected such that they bind or integrate the sound and visual signals in Synchronous but not in Asynchronous familiarization events. This finding would suggest early specialization of cortical sensory function but correlated activity in the two cortices in support of multisensory integration. Second, temporal cortex and/or occipital cortex might be multisensory and respond to both sound and visual signals. Finally, occipital and temporal regions might be functionally connected during Synchronous but not Asynchronous familiarization events and, as a result, might train each otherwise unisensory region to become multisensory. This last finding would suggest that multisensory integration observed in adults reflects an emergent, experience-driven developmental process.

Fig. 1.

Illustration of the block design used. Each infant was administered a Pretest, a Synchronous, and an Asynchronous block of events. Each familiarization and test event lasted 20 s. Curved arrows indicate that the presentation order of the Sound-Only and Visual-Only test events and the Synchronous and Asynchronous blocks was counterbalanced across participants.

2. Method

2.1. Participants

The final sample included N = 65 3.5-5-month-old infants (M age = 134.2 days, SD = 12.1 days; 31 females, 34 males, 48 white non-Hispanic, 4 black, 9 Hispanic, and 4 Asian). The Brown University Institutional Review Board approved the study procedure and parental consent was obtained prior to testing. Infants were recruited from department of health birth records. We prescreened for premature birth (<36 weeks), very low birth weight (<5 lb), or health problems. An additional 20 infants were tested but excluded because of fussiness (n = 14), poor optical contact with the scalp (n = 5), or parental interference (n = 1).

2.2. Design & procedure

Participants sat on a parent’s lap in a dimly-lit room approximately 60 cm away from a 22″ monitor. Stimuli were presented via E-Prime software. Visual stimuli were a red or green ball (2.3˚ visual angle) presented on a rectangular grid of white dots (17˚ by 14.3˚) on a black background (Fig. 1). The ball moved back-and-forth across the grid horizontally or vertically at a rate of 2.5 s per motion cycle (i.e., return to starting position). Auditory stimuli were two sounds (“bang” and “boing”). Associations between the visual stimulus and the sound it was paired with were counterbalanced. Using a block design, three blocks of 20-s events were presented. Each event was preceded by a 10-s white fixation cross on a black background.

Fig. 1 illustrates block order. In the first block, we administered two 20-s pretest events: a Visual-only event in which the ball was presented without sound, and a Sound-only event in which the sound was presented at 2.5-s intervals (the grid of white dots was presented on the screen but without a moving ball). These events were designed to measure baseline responses to unisensory information. Pretest event order was counterbalanced.

Next, we presented the Synchronous and Asynchronous blocks, each of which consisted of a 20-s familiarization event followed by a 20-s Visual-only test event and a 20-s Sound-only test event. During the familiarization event, the ball moved either up and down or back and forth across the screen for eight cycles or trials, and the sound was presented eight times. The sound was presented as the ball reached the edge of the grid and changed directions in the Synchronous event, whereas during the Asynchronous event the sound was presented 450 ms before the ball reached the edge. The subsequent Visual-only and Sound-only test events were identical to the Pretest block events. The order of the Synchronous and Asynchronous blocks and the order of Visual-only and Sound-only test events were counterbalanced across infants. Infants’ looking behavior was coded offline by a trained observer.

2.3. fNIRS recording

fNIRS recordings were collected at a rate of 50 Hz using a TechEn CW6 system with eight channels (two sources and eight detectors). Source optodes emitted infrared light at two frequencies, 690 nm and 830 nm, which are optimized to measure deoxygenated and oxygenated blood, respectively. The fNIRS channels were arranged in two arrays, each with one source optode and four detector optodes (Fig. 2). Source-detector separation was 3 cm. The lateral/temporal array (channels T1-4) was positioned in the cap so that the detector optodes were centered over EEG coordinate T4 (right superior/middle temporal lobe) and the posterior array (channels O1-4) was positioned so that the detector optodes were centered over EEG coordinate O2 (right middle/inferior occipital lobe) (anatomical correlates of international 10–20 system coordinates obtained from Kabdebon et al., 2014). This positioning aligns with the 10–20 coordinates used for localizing occipital and temporal activation in prior fNIRS work with infants (e.g., Emberson et al., 2015; Bortfeld et al., 2007). The cap was placed on infants’ heads such that the bottom of the cap aligned with the Fp1-Fpz-Fp2 line. The arrays were always positioned on the right hemisphere due to constraints on the number of optodes available.

Fig. 2.

The fNIRS optode array was placed over occipital (O1-O4) and temporal cortex (T2-T3). Image reconstruction (see Aasted et al., 2015) is for illustration purposes only to show the estimated cortical regions recorded during the study. This image was generated using the Colin 29 MRI atlas adjusted for the average head circumference for infants in our study (42 cm) paired with the EEG coordinates targeted for probe placement. Hotter colors indicate regions with higher measurement sensitivity, reflecting better ability to detect small signal changes relative to cooler colored regions.

2.4. Data preprocessing

2.4.1. fNIRS data processing

fNIRS data were exported and preprocessed in the HOMER2 v2.1 MATLAB toolbox. Data were digitally band-pass filtered at 0.01-0.1 Hz to remove systematic physiological and movement artifacts. The change in optical density was then calculated for each wavelength relative to the 10 s baseline prior to block onset. Changes in the concentration of oxygenated and deoxygenated hemoglobin were calculated from the changes in optical density using the modified Beer-Lambert law. A PCA filter was used to reduce motion artifacts and systemic physiological noise by removing 80% of the covariance in the data, following prior fNIRS studies (see Wilcox et al., 2005). After preprocessing, we averaged over 3-s time bins in each 20-s event, starting 5 s after event onset and ending 3 s after event offset, to remove serial autocorrelation and to eliminate the need to make assumptions about the timing of the peak of the hemodynamic response in subsequent analyses. We limited analyses to the period starting 5 s after stimulus onset based on previous studies that have seen that this is the typical delay in the hemodynamic response function (HRF) initiation in infants (e.g., Taga and Asakawa, 2007; Werchan et al., 2016).

As a quality check, correlations in the temporal change in HbO₂ and HbR were calculated for each channel and averaged across the lateral/temporal array (T1-4) and the posterior/occipital array (O1-4). fNIRS data with a high signal-to-noise ratio should have a high negative correlation between HbO₂ and HbR signals (Cui et al., 2010). Two one-sample two-tailed t-tests indicated that correlations were significantly less than 0 for both the lateral array over temporal cortex, M = −0.55, SD = 0.37, t(64) = −11.950, p < 0.001, and the posterior array over occipital cortex, M = −0.50, SD = 0.39, t(64) = −10.386, p < 0.001. Correlations did not differ between the lateral (temporal) and posterior (occipital) arrays, t(64) = .916, p = 0.363.

2.4.2. Functional localizer pretest

We only used data from channels T2 and T3 over temporal cortex to reduce the likelihood of recording occipital or frontal regions. Fig. 3 shows that measurement sensitivity is highest over auditory regions. Nonetheless, it is possible that the signal is picking up activation from regions in the ventral visual pathway. Thus, temporal cortex findings must be interpreted with caution. Using data recorded during the Pretest block, we conducted a Channel (T2, T3) x Time Course analysis of variance (ANOVA), with Age as a continuous variable. This analysis yielded no significant main effects or interactions (all ps > .105). The same analysis over right middle/inferior occipital Channels (O1, O2, O3, O4) x Time Course, with Age as a continuous variable, yielded no effects of Channel (all ps > .360). Therefore, data from T2-3 and O1-O4 were averaged for subsequent analyses.

Fig. 3.

Change in HbO₂ activation for occipital and temporal cortex during the Sound-only and Visual-only pretest events and during the multisensory AV Synchronous familiarization event.

2.4.3. Occipito-temporal functional connectivity measure

We generated an occipito-temporal (OT) functional connectivity value for each infant across the AV Synchronous and Asynchronous events to model in our analyses. Following methods in prior work examining task-based functional connectivity in infants using fNIRS (Homae et al., 2011; Keehn et al., 2013), we first calculated a Pearson’s r for each Synchronous and Asynchronous Familiarization event by temporally correlating the temporal and occipital activations across the six averaged time bins for each infant. We then converted the r values to z scores using Fischer’s z transformation.

3. Results

3.1. Behavioral looking time data

Looking times (in seconds) did not differ between AV Synchronous (M = 19.08, SD = 1.56) and Asynchronous (M = 18.6, SD = 2.96) familiarization events, t(61) = 1.247, p =  .217. However, a Test Event Type (Sound-only, Visual-only) by AV Familiarization Synchrony (Synchronous, Asynchronous) analysis resulted in a main effect of Synchrony, F(1,61) = 3.997, p = .05, with infants looking longer at Sound-only and Visual-only test events following an AV Synchronous (M = 17.54, SD = 0.34) than an AV Asynchronous (M = 16.54, SD = 0.46) familiarization. These data indicate differential effects of AV synchrony on infants’ subsequent visual attention to unisensory stimuli in isolation.

3.2. Cortical activations before multisensory experience

We first examined the inherent unisensory versus multisensory properties of occipital and temporal regions by analyzing baseline temporal and occipital activations to the Sound-only and Visual-only pretest events relative to the multisensory AV Synchronous event. If occipital and temporal cortical regions are multisensory, we expected to find statistically similar activations for Sound-only and Visual-only events combined with greater activation for the AV Synchronous multisensory event (Stein and Stanford, 2008). In contrast, greater activations for Visual-only than Sound-only events, and no difference between Visual-only and AV multisensory events, would indicate specialized visual unisensory processing. Preliminary data indicated no effects of age in our analyses and no order effects for whether infants received the synchronous or asynchronous block first.

An Event Type (Sound-only, Visual-only, AV) x Time Course (six 3-s bins) ANOVA on occipital data resulted in a main effect of Time Course, F(5, 270) = 23.474, p <  .001, and a Time Course x Event Type interaction, F(10,540) = 2.115, p =  .022. Occipital activation was greater for Visual-only relative to the Sound-only Pretest events F(5,270) = 3.229, p = .008, but Visual-only did not differ from the AV Synchronous multisensory event, F(5, 275) = 1.409, p =  .221 (Fig. 3). This pattern provides evidence for specialized visual processing in occipital cortex. In contrast, the same Event Type (Sound-only, Visual-only, AV) x Time Course ANOVA for temporal cortex resulted in only a main effect of Time Course, F(5,270) = 5.249, p < .001, showing that there was significant activation to the presented stimuli. However, there was no effect of Event Type, F(2, 108) = 1.418, p =  .239, nor an Event Type by Time Course interaction, F(10,540) = 0.971, p = .436, suggesting that temporal cortex was equally responsive to auditory and visual stimulation. It should be noted, however, that the results from the temporal cortex must be interpreted with caution and we return to this issue in the Discussion.

3.3. Cortical activations after multisensory experience

Next, we examined cortical responses to unisensory information following experience with Synchronous or Asynchronous multisensory events. Adult work shows that multisensory experience drives functional coupling between occipital and temporal regions and results in enhanced processing of unisensory information (Lewis and Noppeney, 2010; Tyll et al., 2013). Thus, to examine how individual differences in multisensory processing influences subsequent unisensory processing, we included the OT functional connectivity values from the Synchronous and Asynchronous familiarization events (see Method for more information) as continuous variables in our analyses.

We conducted an Event Type (Sound-only, Visual-only) x Region (Occipital, Temporal) x Time Course x AV Familiarization (Synchronous, Asynchronous) omnibus ANOVA, including the OT Synchronous and OT Asynchronous values as continuous variables. All significant results are reported in Table 1. Importantly, there was a 4-way interaction for Event Type, AV Familiarization, Region, and Time Course, F(5,260) = 2.399, p =  .038. We followed up on this interaction by examining activations separately by Region and by AV Familiarization Block to test our prediction that synchronous multisensory experience might train otherwise unisensory regions to become multisensory (Bonferroni-corrected alpha = .0125).

Table 1.

Omnibus Repeated Measures ANOVA Results.

	d.f.	F	sig.	ƞp2
Time Course	5	4.557	0.001	0.081
Time Course*Event Type	5	2.936	0.013	0.053
Time Course*OT Synchronous Connectivity	5	3.593	0.004	0.065
Event Type*Region	1	9.269	0.004	0.151
Region*OT Synchronous Connectivity	1	9.320	0.004	0.152
Time Course*Event Type*Region*AV Familiarization	5	2.399	0.038	0.044

Note: Only significant results (p < 0.05) are shown.

3.3.1. Occipital cortex activation after an AV synchronous familiarization

An Event Type (Sound-only, Visual-only) x Time Course analysis, including the synchronous OT functional connectivity value as a continuous variable, resulted in an Event Type by Time Course interaction, F(5,290) = 3.815, p = .002, and an Event Type by Time Course by OT functional connectivity interaction, F(5,290) = 4.206, p =  .001. Follow-up tests (Bonferroni-corrected alpha = 0.025) indicated significant activation for Visual-only events after a synchronous multisensory experience, F(5300) = 4.052, p =  .001, but not for Sound-only test events, F(5,295) = 1.297, p =  .265. Importantly, higher levels of OT functional connectivity resulted in increasingly higher occipital activation for Sound-only test events, F(5,295) = 3.741, p =  .003, but OT functional connectivity was less meaningful for Visual-only events, F(5,300) = 1.415, p =  .219. Fig. 4 shows Sound-only test event data from infants grouped on the mean OT functional connectivity value for illustration. These data suggest that occipital cortex exhibited multisensory properties following a Synchronous Familiarization in infants with high OT functional connectivity.

Fig. 4.

Change in temporal and occipital HbO₂ activation for the Sound-only test trial following an AV Synchronous multisensory experience. Data are presented for infants above and below the mean OT functional connectivity value for visualization only and are treated continuously in all analyses.

As an analytic check, we directly compared post AV Synchronous Sound-only events to Pretest Sound-only events in a split of infants with OT functional connectivity above (n = 39) and below (n = 26) the group mean. There were no significant interactions for infants with below-mean OT functional connectivity, all Fs < 1.635, all ps > .216. However, there was a significant Block by Time Course interaction for infants with above-mean OT functional connectivity, F(5,165) = 2.574, p =  .028, reflecting increased occipital activation in the Post Synchronous relative to the Pretest Sound-only event.

3.3.2. Temporal cortex activation after an AV synchronous familiarization

An Event Type (Sound-only, Visual-only) by Time Course analysis, incorporating OT functional connectivity as a continuous variable, resulted in a main effect of Event Type, F(1,58) = 6.543, p =  .013, a main effect of Time Course, F(5,290) = 2.904, p = .014, and an Event Type by Time Course by OT functional connectivity interaction, F(5,290) = 2.321, p =  .043. We followed up with analyses specific to Sound-only and Visual-only events (Bonferroni-corrected alpha = 0.025). The Visual-only test event did not interact with OT functional connectivity, F(5,300) = .315, p =  .904. In contrast, the Sound-only test event interacted with OT functional connectivity, F(5,295) = 3.261, p =  .007 (see Fig. 4). Thus, infants with higher OT functional connectivity had greater occipital activation and lower temporal activation to the Sound-only test events relative to infants with lower OT functional connectivity.

3.3.3. Occipital and temporal activations after an AV asynchronous familiarization

Asynchronous OT functional connectivity did not interact with any variables in the omnibus analysis (Table 1) and was thus not included here (it does not affect results if included). An Event Type (Sound-only, Visual-only) x Time Course analysis on occipital cortex data resulted in only a main effect of Time Course, F(5,290) = 9.184, p <  .001. Similarly, an Event Type (Sound-only, Visual-only) by Time Course analysis on temporal cortex data resulted in only a main effect of Time Course, F(5,290) = 3.192, p =  .008. These data thus indicate that there was no significant effect of Asynchronous AV experience on shaping cortical response patterns in our sample.

4. Discussion

Here we used fNIRS to investigate cortical sensory dynamics in the infant brain following multisensory experience. Infants’ neural responses during the pretest Sound-only and Visual-only events indicated that temporal cortex responded to both auditory and visual inputs but that occipital cortex responded primarily to visual input. Notably, infants who exhibited higher OT functional connectivity values during the Synchronous Familiarization also exhibited occipital activation to Sound-only test events. Critically, this was only the case following familiarization with a synchronous audiovisual event. Thus, our results indicate that the occipital cortex of young infants can exhibit multisensory properties following as little as 20 s of experience with a synchronous AV event when there is high OT functional connectivity during that experience (Fig. 4). This finding is consistent with adult work indicating that synchronous multisensory experience drives functional coupling between occipital and temporal regions, which subsequently enhances unisensory processing (Lewis and Noppeney, 2010; Tyll et al., 2013). Our findings are also consistent with prior fNIRS work with infants showing that audiovisual information increases activation in both occipital cortex and temporal cortex relative to visual information presented in isolation (Watanabe et al., 2013; Emberson et al., 2015).

The adult literature suggests that occipital cortex exhibits multisensory properties (Murray et al., 2016). Data from adults have shown that multisensory synchronous AV experience can drive sound-based activity in occipital cortex (Zangenehpour and Zatorre, 2010; Meylan and Murray, 2007; Watkins et al., 2006; Murray et al., 2004, 2005; Thelen et al., 2012, 2015; Matusz et al., 2015). However, data from adults have also shown that occipital cortex response to sound does not need to follow a multisensory experience. For example, studies have found that salient auditory stimuli not only activated occipital cortex (McDonald et al., 2003), but also support visual processing at the sound location (Feng et al., 2014). Our findings provide new insights into the developmental precursors of the cortical dynamics that occur whenever infants experience multisensory information in their environment. Furthermore, our findings show that exposure to concurrent (i.e., temporally synchronized) multisensory events in early infancy rapidly recruits responsiveness to the visual and auditory attributes that characterize everyday multisensory events. Given this, and given that our normal multisensory experiences consist of integrated audiovisual attributes, it is likely that long-term, everyday, integrated, multisensory experience tunes the occipital cortex to also respond to unrelated salient sounds by the time we become adults.

We interpret our temporal cortex multisensory activations with caution. For one, it is possible that the multisensory activations we observed in the pretest events could suggest that the sound-only event was not of comparable saliency to the visual-only event. However, this interpretation is unlikely given that infants with high OT functional connectivity had similar occipital activations to the Visual-only and Sound-only events after a synchronous AV familiarization. Additionally, we were only able to record activation over right temporal cortex due to practical limitations in the number of optodes available. It is possible that right and left temporal cortex may exhibit differences in multisensory processing. Finally, while the sensitivity map in Fig. 2 suggests that our activations are largely coming from auditory regions, we cannot be certain that the data are not additionally detecting some activation from the ventral visual pathway. The temporal response to Sound-only events after a Synchronous Familiarization event, however, offers confidence that we are largely measuring from auditory regions (Fig. 4). They show that as occipital cortex responded to sound, temporal cortex activation was reduced in infants with high OT functional connectivity. This finding is also consistent with prior fNIRS work in infants, which found that temporal response to unisensory visual input was reduced and occipital response was increased following an AV experience (Bortfeld et al., 2007).

Our data cannot directly speak to the mechanisms driving 3.5-5-month-old infants’ OT functional connectivity and subsequent shaping of cortical dynamics. Specifically, our data cannot discern whether these processes were driven by sensory encoding via local connections among temporal and occipital sensory cortices or from attentional mechanisms via top-down feedback influence from a higher-order region. Connectomics research indicates the existence of highly connected “hub” regions that organize global information integration between different regions of the brain (van den Heuvel and Sporns, 2013). In early postnatal life, the first hubs are primarily found in unisensory cortical areas, including temporal, occipital, and sensorimotor cortices (Fransson et al., 2011). This would suggest a local source for the correlated response of the auditory and visual regions and subsequent occipital response to sound. Moreover, multisensory interactions have been shown as early as 40 ms following sensory input, suggesting sufficiency of feedforward mechanisms (Foxe and Schroeder, 2005; Schroeder and Foxe, 2005). Regardless of the source, our data suggest that OT functional connectivity during an AV synchronous experience appears to prime or train occipital cortex on how to respond to the previously visually-bound sound. This possibility is consistent with adult data showing that increased functional connectivity enhances sensory processing by amplifying neural responses in unisensory regions (Lewis and Noppeney, 2010; Tyll et al., 2013).

In sum, the occipital cortex data offer the first cortical evidence, in infants, that functional coupling of occipital and temporal cortex during a synchronous multisensory experience predicts whether occipital cortex subsequently responds to sound-only information. These data invoke the possibility that adaptation to the sensory environment can result in broadening of perceptual function in addition to a narrowing of perceptual function. Early specialization may be the result of initial biases of cortical neurons to respond to particular types of physical input (Johnson, 2000). However, cross-modal plasticity or multisensory processing may emerge later as an adaptive and learned response to experiencing the ubiquity of multisensory information in the environment.

Conflict of Interest

None.