Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Oct 1.
Published in final edited form as: Prog Neurobiol. 2021 Jul 15;205:102121. doi: 10.1016/j.pneurobio.2021.102121

Topographical and Laminar Distribution of Audiovisual Processing within Human Planum Temporale

Yuhui Chai 1,*, Tina T Liu 2, Sean Marrett 3, Linqing Li 3, Arman Khojandi 1, Daniel A Handwerker 1, Arjen Alink 4, Lars Muckli 5, Peter A Bandettini 1,3
PMCID: PMC8405586  NIHMSID: NIHMS1730112  PMID: 34273456

Abstract

The brain is capable of integrating signals from multiple sensory modalities. Such multisensory integration can occur in areas that are commonly considered unisensory, such as planum temporale (PT) representing the auditory association cortex. However, the roles of different afferents (feedforward vs. feedback) to PT in multisensory processing are not well understood. Our study aims to understand that by examining laminar activity patterns in different topographical subfields of human PT under unimodal and multisensory stimuli. To this end, we adopted an advanced mesoscopic (sub-millimeter) fMRI methodology at 7T by acquiring BOLD (blood-oxygen-level-dependent contrast, which has higher sensitivity) and VAPER (integrated blood volume and perfusion contrast, which has superior laminar specificity) signal concurrently, and performed all analyses in native fMRI space benefiting from an identical acquisition between functional and anatomical images. We found a division of function between visual and auditory processing in PT and distinct feedback mechanisms in different subareas. Specifically, anterior PT was activated more by auditory inputs and received feedback modulation in superficial layers. This feedback depended on task performance and likely arose from top-down influences from higher-order multimodal areas. In contrast, posterior PT was preferentially activated by visual inputs and received visual feedback in both superficial and deep layers, which is likely projected directly from the early visual cortex. Together, these findings provide novel insights into the mechanism of multisensory interaction in human PT at the mesoscopic spatial scale.

Keywords: audiovisual, multisensory, planum temporale, layer, column, feedback

1. INTRODUCTION

Input signals from different sensory modalities are initially processed predominantly in dedicated low-level unisensory brain areas – e.g. visual and auditory signals in visual and auditory cortex respectively. Previous research indicates that multimodal signals originating from a common source are integrated into multimodal representations in multisensory brain areas, including the intraparietal sulcus and the superior temporal sulcus, via feedforward convergence from unimodal areas (Guipponi et al., 2013; Rockland and Ojima, 2003; Schroeder and Foxe, 2002). Also, recent studies suggest that multisensory interplay occurs in unisensory areas (Gau et al., 2020; Iurilli et al., 2012; Kayser et al., 2008; Meyer et al., 2010; Morrill and Hasenstaub, 2018; Musacchia and Schroeder, 2009; Vetter et al., 2014). For example, studies in both monkey and human have shown that planum temporale (PT), an early auditory brain area, responds to a variety of sensory inputs across auditory, visual and somatosensory modalities (Alink et al., 2008; Foxe et al., 2002; Kayser et al., 2007; Kayser et al., 2008; Schroeder and Foxe, 2005; Schroeder et al., 2001). At present, however, it is not clear how feedforward and feedback projections to PT are involved in multimodal processing.

The laminar profile of sensory activity can help to distinguish between feedforward and feedback projections, based on their different laminar termination patterns (feedforward terminates in middle layers while feedback termination excludes middle layers) (Hackett et al., 2014; Huber et al., 2017; Larkum et al., 2018; Lawrence et al., 2019; Petro and Muckli, 2017; Self et al., 2017). While it is well-established that initial auditory inputs reach PT via feedforward projections from auditory belt areas (Hackett, 2011; Hackett et al., 2014; Schroeder et al., 2001) and medial geniculate nucleus (MGN) (Hackett, 2011; Hackett et al., 2007; Hackett et al., 1998), the origins of the non-auditory or feedback influences to PT remain unclear. These influences could arise from top-down connections with higher-order multimodal areas in the temporal, parietal or frontal cortex (Bizley and King, 2012; Driver and Noesselt, 2008; Driver and Spence, 2000; Noesselt et al., 2007; Schroeder and Foxe, 2002; Vetter et al., 2014), or be directly projected from the visual cortex (Falchier et al., 2010), or be inherited through koniocellular or non-specific projections from thalamus (Cappe et al., 2012; Hackett et al., 2007; Musacchia and Schroeder, 2009; Schroeder and Foxe, 2005). Moreover, inputs from each source may contribute differently across sub-areas of PT (Falchier et al., 2010; Hackett et al., 2007). To date, most imaging studies on human PT were conducted at 3T with relatively low spatial resolution (2-3 mm) and reported overlapping representations across sensory modalities (Alink et al., 2008; Calvert et al., 1997; Foxe et al., 2002). Recently, ultra-high field, high-resolution functional magnetic resonance imaging (fMRI) has been applied to study multisensory processing in auditory cortex at laminar level and has revealed a functional dissociation between posterior and anterior sections in PT (Gau et al., 2020). In our study, we aim to address both the sensory-specific topography (columnar dimension) in human PT and the laminar-specific influence (laminar dimension) across subfields. First, we ask whether activation associated with the processing of different sensory inputs can be delineated in PT. Second, we examine whether the feedback mechanism varies across sensory-specific representations within PT.

We applied an advanced mesoscopic (sub-millimeter) fMRI methodology at 7T to parse multisensory influences over both columnar and laminar dimensions in human PT. Here, ‘columnar’ and ‘laminar’ refer to mesoscopic structures that are aligned tangentially and perpendicularly to the cortical depth, respectively. For functional measurement, in addition to blood oxygen level dependent (BOLD) contrast, we acquired integrated blood volume and perfusion (VAPER) weighted signals from the same scan (Figure S3A). In BOLD, the layer-specific microvasculature signal is often washed out by the dominant signal from ascending and pial veins (Kim and Ogawa, 2012). VAPER, on the other hand, is less biased towards cortical surface, and its laminar specificity is superior to BOLD (Chai et al., 2020). Due to the tradeoff between coverage and acquisition rate for fMRI, we acquired images of the auditory cortex in the right hemisphere only (Figure S2), as it tends to be the dominant hemisphere in spatial information processing (Alink et al., 2012; Baumgart et al., 1999). For anatomical reference, we incorporated magnetization transfer (MT) weighting with an echo-planar imaging (EPI) acquisition identical to functional imaging (Figure S3B), which we called anatomical EPI. This allows sufficient gray-white matter contrast to perform all fMRI analysis in the native EPI space, without the need for distortion correction and registration. During fMRI runs, participants viewed a sphere or heard a stream of sound moving sinusoidally along the horizontal midline of the screen (Figure 1C and S1) (Alink et al., 2008). We measured and compared activity along columnar and laminar dimensions to each unisensory (visual-only, audio-only) and multisensory (audiovisual) condition.

Figure 1. PT definition and task design.

Figure 1.

Open in a new tab

(A) Maps of BOLD signal changes induced by general sound (sound vs. silence) in blue and movement-specific sound (moving sound vs. stationary sound) in red. Voxel-wise activities were thresholded at p < 0.001. (B) Columnar distance and laminar depth determined in PT. G1 and G2 refer to the first and second transverse gyrus in auditory cortex, respectively. Based on G1/G2 landmarks, auditory cortex can be divided into three sub-fields: T1, T2, and T3 (from anterior to posterior). Brains in A-B are anatomical EPI images obtained using an identical acquisition with functional EPI images. (C) Stimuli and procedure of the main task. Participants viewed either visual-only, audio-only or audiovisual motion stimuli and reported the motion direction of the sphere or the sound at the end of each task block. Note that the sphere and the fixation cross are shown larger than actual size for display purposes only.

2. METHODS

2.1. fMRI participants

Ten healthy volunteers (age 22-30, 6 males) gave informed consent to participate in this study under an NIH Combined Neuroscience Institutional Review Board approved protocol (93-M-0170, ClinicalTrials.gov identifier: NCT00001360). All participants except one underwent multiple fMRI sessions (2 hours in each session), resulting in a combined total of 26 sessions. Details of each scan acquired can be found in Table S1.

2.2. Stimulation paradigm

To measure columnar and laminar patterns in unisensory and multisensory conditions, three types of tasks (Figure 1C) were generated using Psychophysics Toolbox 3 (Brainard, 1997) in MATLAB (MathWorks, USA), identical to those used in (Alink et al., 2008): 1) audiovisual motion, 2) audio-only motion, 3) visual-only motion. In the audiovisual motion task, a sphere (1.2° radius) with a black and white checkerboard texture moved sinusoidally along the horizontal midline of the screen (furthest at 13°), back and forth, while participants were asked to maintain fixation at a white cross 3° below the center throughout the run. At the same time, the participants heard a series of bass drum sounds with 100-ms beat then 50-ms gap cycle. Within this period of sound stimulation, the sphere pulsated (radius increased by 30%), resulting in a strong perceptual binding between visual and audio stimuli. The volume of auditory stimulus moved from one ear to the other in a sinusoidal fashion. The directions of auditory apparent motion and visual motion were always the same. For the unimodal tasks, either the audio-only or visual-only motion stimulus alone was presented with the same parameters as above.

Two experiments with different block designs were conducted in this study (Figure S1). In both experiments, participants were asked to indicate the movement direction at the end of each task block by pressing the left or right button.

In Experiment 1 (multi-condition runs), each run started with a 26.4-s fixation period, followed by 9-10 repeats of each task block (33 s, audio-only, visual-only or audiovisual in random order) interleaved with a fixation block (26.4 s). Three runs were acquired during each session, so that we collected 27-30 blocks for each condition. Experiment 2 (single-condition run) adopted the same block design (33 s task / 26.4 s fixation) with the same number of trials (27-30) as in Experiment 1. Each run, however, contained only one task (either all audio-only tasks, all visual-only tasks, or all audiovisual tasks). Three runs were acquired in a pseudo-randomized order in each scan session such that the audiovisual task always appeared at the end of Experiment 2 to minimize the potential imaginary or anticipation effect arising from perceptual binding between visual and audio stimuli.

A dual-task at fixation was introduced to measure the level of vigilance In Experiment 2. Subjects were asked to press any button whenever the fixation cross turned from white to green. The green cross was presented for 300 ms with a random interval between 50 s and 100 s. To minimize the dual-task interference, the onset of the green cross never appeared within the time window when motion direction was reported (i.e., less than 4 s before or after the end of each task block).

The design of multi-condition runs was statistically efficient to extract the columnar and laminar signal difference between different conditions, due to the same signal baseline within a run. On the other hand, the design of single-condition runs minimized the interaction between different conditions and allowed us to monitor the vigilance level for different task conditions.

2.3. Data acquisition

2.3.1. Scanning setup

The experiments were performed on a Siemens MAGNETOM 7T scanner with a Nova singlechannel transmit/32-channel receive head coil. To obtain high-resolution images of the auditory cortex, we positioned the slice prescription to a close-to-sagittal plane and covered the cortex in the right hemisphere only (Figure S2). A 3rd order B0-shimming with three iterations was applied to the imaging region.

2.3.2. VAPER-3D-EPI sequence method for functional imaging

To acquire the functional data, we introduced an integrated blood volume and perfusion (VAPER) contrast (Chai et al., 2020) acquired by combining the blood-suppression module of DANTE (Delay Alternating with Nutation for Tailored Excitation) pulse trains (Li et al., 2012) with 3D-EPI (Poser et al., 2010). The sequence was implemented to acquire fMRI images alternating between blood-signal-suppressed (DANTE prepared 3D-EPI) and blood-signal-augmented (original 3D-EPI as control) conditions (Figure S3A). Parameters of DANTE pulse train were as follows: pulse number in 1st/later segment = 230/36, pulse interval = 1.1 ms, pulse flip angle = 8.5°, gradient = 21 mT/m (applied along x and z directions simultaneously). Image acquisition parameters were: TE/TR = 28/3300 ms, flip angle of excitation = 26°, 28 slices (2 slices oversampling), imaging resolution = 0.8×0.8×0.9 mm3, matrix size = 168×168, partial Fourier of 7/8 and GRAPPA 3.

Through dynamically subtracting the signal in the blood-nulled condition from that in the control condition, VAPER contrast was generated to be sensitive to both cerebral-blood-volume (CBV) and cerebral-blood-flow (CBF) while BOLD weighting could be largely attenuated. To remove any remaining BOLD contamination, VAPER time series was further corrected through dynamical division by that of control to factor out the exp(TE∕T2∗) term. In addition, the signal of the control condition is mainly determined by BOLD contrast, thus can be treated as a conventional BOLD signal.

2.3.3. MT-3D-EPI sequence method for anatomical reference

In the human brain, a relatively large fraction of macromolecular hydrogen protons (MP) (f~0.2-0.3) is found in white matter (WM), while this number is smaller in gray matter (GM) (f~0.1) (van Gelderen et al., 2017). Through magnetization transfer (MT) with water hydrogen protons (WP), MPs can dramatically affect the MRI signal and thus different MP fractions in GM and WM will lead to different MRI signal intensities. Here we incorporated MT weighted imaging with the fMRI acquisition technique to generate the anatomical image (Chai et al., 2019).

Its sequence design is almost identical to the functional VAPER imaging (Figure S3B). To switch from functional VAPER contrast to anatomical MT weighting, we turned off the gradients in the preparation and maximize the RF power of the preparation pulses (FA = 10-13°, minimal RF duration allowed under the SAR limit). We also acquired images interleaved between MT-prepared and control conditions.

The MT-weighted anatomical image was generated as SCTRL−SMTSMT, where SCTRL is the image signal in the control condition, and SMT is the image signal of the MT-prepared condition. This combination approach extracts the MT-saturated signal and removes the T2* weighting associated with the EPI readout.

2.4. Data analysis

2.4.1. Motion correction

Each scan session (2 hours) consisted of 3 functional runs (24-30 min each) and 1 anatomical run (8-9 min). Motion correction was applied to the images of all functional and anatomical runs together within one session using SPM12 (Wellcome Trust Center for Neuroimaging, London, UK). Next, the time series of fixed control and blood-nulled images in the functional runs were used to compute the VAPER contrast. The mean images of fixed control and MT-prepared conditions in the anatomical run were used to compute the anatomical reference image. In the functional data, time points in each contrast were censored from further regression model whenever the Euclidean norm of the motion derivatives exceeded 0.4 mm or when at least 10% of image voxels were seen as outliers from the trend.

2.4.2. Determination of cortical laminar depth and columnar distance

The borderlines of CSF/GM and GM/WM in PT were manually drawn on the anatomical EPI images. To avoid singularities at the edges in angular voxel space, fMRI data was upsampled by a factor of 4 in the in-plane voxel dimensions, so that laminar and columnar depths could be defined on a finer grid than the original EPI resolution.

Inside the GM ribbon of PT, a coordinate system across cortical layers and columnar structures was estimated using LAYNII software suite (Huber et al., 2020) (https://github.com/layerfMRI/LAYNII). First, we estimated 20 equidistantly distributed layer using LAYNII’s LN_GROW_LAYERS program. This layer number was chosen more than the independent points across the thickness of the cortex, which can improve layer profile visualization and minimize partial voluming between neighboring voxels (Huber et al., 2018). Second, cortical ‘columnar’ structures were determined in LAYNII’s LN_COLUMNAR_DIST program with the following algorithm: Starting from the posterior end of the lateral sulcus, the columnar distance was calculated as Euclidean distance along the cortical ribbon for all midlayer voxels. For voxels in other layers, the columnar distance was taken from the nearest middle-layer voxels. The group of voxels throughout cortical depth which had a same columnar distance was considered a ‘columnar’ structure.

Note that the terminology of ‘laminar’ or ‘layer’ indicates a measurement taken along the cortical depth, from CSF/GM border to GM/WM border, as opposed to the cytoarchitectonically defined cortical layers. Similarly, the term of ‘columns’ refers to the geometric columnar shape of fMRI voxel groups, not to be confused with the historic term of “cortical columns” such as orientation, color, frequency, or ocular dominance columns.

2.4.3. Data smoothing

For functional data in the main experiment, smoothing was applied within each cortical depth with a kernel size of 1 mm using LAYNII’s program LN_LAYER_SMOOTH. As this kernel width is much smaller than the columnar distance between auditory and visual response peaks (8.8 mm as shown in Figure 3), the columnar analysis was not affected by this algorithm of smoothing. For functional data in the localizer experiment, smoothing was applied within GM with a kernel size of 1 mm using AFNI (Cox, 1996) program 3dBlurInMask.

Figure 3. Sensory-specific BOLD response across subareas of PT.

Figure 3.

Open in a new tab

(A) Group-averaged columnar profile of BOLD response along posterior-anterior PT. Response to visual-only, audio-only and audiovisual stimuli is plotted in blue, red and green, respectively. The distance between response peaks in the visual-only and audio-only conditions is 8.8 ± 3.4 mm along the cortical ribbon. (B) A significant interaction (**: p = 1.6×10−9) between subarea location (anterior PT, posterior PT) and stimulus modality (audio-only, visual-only) was revealed by a two-way repeated-measures ANOVA. Shaded areas and error bars represent ± SEM across 26 sessions. Results of all individual sessions can be found in Figure S4.

2.4.4. Behavioral data analysis

The accuracy of the color detection task (vigilance measure) in Experiment 2 was compared across audio-only, visual-only and audiovisual runs, using one-way repeated-measures analysis of variance (ANOVA). The same one-way repeated-measures ANOVA was used to examine if there is a significant difference in the accuracy and response time (RT) of the movement detection task (in both Experiments 1 and 2) across stimulus modalities (audio-only, visual-only, and audiovisual).

2.4.5. fMRI data analysis

fMRI analysis was performed separately for BOLD and VAPER data in each scan session, using AFNI with a multiple linear regression model (Cox, 1996). The regression model contained a regressor for each condition of interest, including visual-only motion, audio-only motion, and audiovisual motion, and another control condition for the rest period. These regressors were convolved with a gamma-variate hemodynamic response function. All the voxel-wise time series of BOLD and VAPER were normalized by each voxel’s mean signal across time before feeding to the regression model. As a result, beta weights associated with each covariate can be interpreted as percent signal changes.

2.4.6. Extraction of columnar and laminar profiles

To calculate activity profiles across laminar depths and columnar distances, all voxels were included in each ROI without any statistical thresholding. For columnar purpose, PT was defined based on auditory motion-specific activations (Figure 1A) and/or anatomical landmarks (Figure 1B). As all analysis of this study was performed in native EPI volume space, we determined the columnar distance in a 2-dimension slide on the slice showing maximal response across task blocks. This strategy works as image slices approximately parallel to the cortical ribbon in PT from posterior to anterior. Prior to the laminar analysis, three ROIs were defined independently: anterior PT covering auditory-specific peak columns (Figure 4), posterior PT covering visual-specific columns (Figure 5), and auditory T2 covering the audio-only and audiovisual tasks activated region in auditory T2 area (Figure S7).

Figure 4. Laminar profiles in anterior PT.

Figure 4.

Open in a new tab

(A) ROI of anterior PT was defined based on the auditory-specific peak columns. (B) Laminar profile of BOLD and VAPER response to stimuli from different sensory modalities in anterior PT. (C) Laminar profile of the differential response evoked by audio-only vs. audiovisual stimuli. The response difference mainly peaked in superficial layers. * for p < 0.05, ** for p < 0.001 (superficial vs. middle: p = 6.6×10−4 for BOLD, p = 0.002 for VAPER; superficial vs. deep: p = 3.3×10−5 for BOLD, p = 1.8×10−4 for VAPER; two-tailed paired t-test). (D) A two-way repeated-measures ANOVA with factors of audio-only response time (audio-only short RT trials vs. audiovisual, audio-only long RT trials vs. audiovisual) and cortical depth (superficial, middle and deep). Interaction effects were significant in both BOLD and VAPER responses at p < 0.05. In each cortical depth, NS, not significant; * for p < 0.05 (two-sided paired t-test, p = 0.003, 0.04, 0.3 for BOLD at superficial, middle and deep layers, p = 0.02, 0.4, 0.85 for VAPER at superficial, middle and deep layers). Shaded areas and error bars represent ± SEM across sessions.

Figure 5. Laminar profiles in posterior PT.

Figure 5.

Open in a new tab

(A) ROI of posterior PT is defined based on the visual-specific peak columns. (B) Laminar profile of the BOLD and VAPER responses to visual stimulus in posterior PT. Visually-induced response peaked at both superficial and deep layers under VAPER contrast, which are also present under BOLD contrast to a lesser extent. Shaded areas in the line plots represent ± SEM across sessions.

2.4.7. Statistical analysis of fMRI responses

To compare BOLD activity across subareas in PT, we performed a two-way repeated-measures ANOVA with factors of stimulus modality (audio-only, visual-only) and PT subarea (anterior PT, posterior PT). For each stimulus modality, we further performed paired t-tests (two-tailed) to compare the activity across anterior and posterior PT. The same analysis was applied to the VAPER measure.

To compare fMRI activity across layers, we performed a series of two-way repeated-measures ANOVA with factors of stimulus modality (audio-only and audiovisual) and layer (superficial, middle and deep). To further examine the basis of the reported interaction, a series of paired t-tests (two-tailed) were carried out.

To investigate the impact of task performance on enhanced feedback in the audio-only than the audiovisual condition, we divided all audio-only trials in each session into two groups based on a median split, half with shorter RTs and the other half with longer RTs. Next, we carried out a two-way repeated-measures ANONA with factors of RT (short RT, long RT) and layer (superficial, middle and deep) on the enhanced response (audio-only - audiovisual) under either BOLD or VAPER contrast.

3. RESULTS

In each session of each participant, we first defined the region of interest (ROI), PT, using anatomical landmarks as the auditory T3 field (Scheich et al., 1998), and/or motion-selective activations in the auditory cortex (Baumgart et al., 1999) from a functional localizer (Figure 1A, see details of the localizer experiment in Supplementary Materials). Next, we examined sensory-specific response profiles both along posterior-to-anterior PT and across cortical depths within subareas of PT.

3.1. PT area defined using anatomical landmarks and differential functional activation

Based on the anatomical EPI image, PT can be identified as the auditory T3 field (Scheich et al., 1998) in each session. The underlays in Figure 1 show the anatomical EPI images from a representative scanning session. Three auditory fields can be delineated on this anatomical image: (1) T1 extends from the depth of the first transverse sulcus to the first transverse gyrus of Heschl (G1); (2) T2 extends from the second transverse sulcus, which is also known as the Heschl’s sulcus, to G1 as well as into caudal areas; (3) T3 is located on the planum temporale. T1 and T2 are separated by Heschl gyrus (G1). The second gyrus (G2) is also present in 50% of subjects in our study (5 of 10 unique subjects), and lies posterior to the Heschl’s sulcus, which can be used as the border between T2 and T3.

PT can also be identified by motion-selective activations in the auditory cortex (Baumgart et al., 1999). For the purpose of functional localization, we acquired BOLD fMRI data while participants listened to moving and stationary sounds. Both moving and stationary sound with respect to silence led to activity throughout auditory T1, T2 and T3, as shown in the blue overlay images (Figure 1A). When comparing moving and stationary sound, we found that the motion-specific activation predominantly clustered in auditory T3, while the effect was relatively weak or almost absent in auditory T2 and T1 (voxel-wise activation thresholded by p < 0.001). This finding is in line with the movement-sensitive area in auditory cortex reported in previous fMRI studies (Alink et al., 2012; Alink et al., 2008; Baumgart et al., 1999).

3.2. Auditory vs. visual representation in anterior vs. posterior PT

Having localized PT in each participant, we performed a columnar analysis to map the sensory-specific topography along the cortical ribbon in the posterior-to-anterior direction. Starting from the sylvian fissure, a group of voxels across cortical depths with the same Euclidean distance along the cortical curvature was defined as a column using the LAYNII program (Huber et al., 2017). Note that the term ‘columns’ refers to the geometric columnar shape of fMRI voxel groups, which does not necessarily refer to groups of neurons that share some common properties (e.g., receptive field, color, orientation, ocular dominance).

In each session of fMRI data from each participant, we determined the columnar distance for voxels along the posterior-to-anterior axis in PT and extracted task-evoked signal changes across different columnal distances. Figure 2 shows representative columnar profiles of 9 sessions from 3 subjects. Only BOLD profiles were shown here as the columnar profiles were very similar across BOLD and VAPER contrasts. It is evident that visually-evoked activations (blue curves) were primarily located in posterior PT, whereas auditory-induced activations (red curves) were more pronounced in anterior PT. Moreover, both posterior and anterior PT were actively involved in processing multisensory stimuli. These sensory-specific patterns were highly consistent across participants and within participant across different days (Figure 2, right).

Figure 2. Columnar profiles of sensory representations from three participants across sessions on different scan days.

Figure 2.

Open in a new tab

The underlays show anatomical EPI images in each participant, and the overlays show their corresponding column definitions within PT. BOLD responses to visual-only, audio-only and audiovisual stimuli as a function of columnar distance are depicted in blue, red and green.

The group-averaged columnar profiles of sensory representations under BOLD contrast are shown in Figure 3A. The peaks of auditory and visual columnar profiles were first aligned before averaging across all sessions from all participants (26 sessions from 10 unique subjects). To quantitively compare activities across subareas of PT, we performed a two-way repeated-measures analyses of variance (ANOVA) on BOLD response, with factors of stimulus modality (visual-only, auditory-only) and subarea location (anterior PT, posterior PT) (Figure 3B). We found a significant interaction effect (F(1, 25) = 85.3, p = 1.6×10−9), such that posterior PT was more strongly activated than anterior PT during visual-only stimulation (t(25) = 7.8, p = 4.1×10−8, Cohen’s d = 2.2), but opposite during auditory-only stimulation (t(25) = −6.7, p = 5.7×10−7, Cohen’s d = −1.4). The group-mean columnar distance between the peaks of auditory and visual representations in PT was 8.8 ± 3.4 mm along the cortical ribbon, which is 1.3 times the straight-line distance (6.9 ± 2.3 mm).

3.3. Distinct laminar-specific multisensory processing in posterior and anterior PT

Based on the columnar profile of sensory-specific response in each individual scan session, we further divided PT into two ROIs for layer-dependent signal analysis: anterior PT, consisting of peak anterior columns under audio-only stimulation (Figure 4A), and posterior PT, consisting of peak posterior columns under visual-only stimulation (Figure 5A). In each ROI, we calculated 20 equi-volume cortical depths (Waehnert et al., 2014) between cerebrospinal fluid/gray matter (CSF/GM) and gray/white matter (GM/WM) boundaries manually determined on anatomical EPI images and extracted depth-dependent signal changes in native fMRI space. Throughout this paper, we used the term ‘laminar’ or ‘layer’ to indicate a measurement taken along the cortical depth, from CSF/GM border to GM/WM border, as opposed to the cytoarchitectonically defined cortical layers.

Anterior PT was mainly activated by audio-only and audiovisual stimuli, but was nearly absent in the visual-only condition (flat blue curves in Figure 4B, p = 0.65 for BOLD and p = 0.03 for VAPER, one-sample two-tailed t-test). To compare the response to auditory-only and audiovisual conditions across different cortical depths (red and green curves in Figure 4B), we performed a 2 stimulus modality (auditory-only, audiovisual) × 3 layer (superficial, middle, deep) repeated-measures ANOVA and found a significant interaction effect (BOLD: F(2, 50) = 21.9, p = 1.8×10−7; VAPER: F(2, 50) = 15.4, p = 6.7×10−6). The difference between audio-only and audiovisual response peaked prominently in the superficial layers as shown in Figure 4C (p < 0.003 for either superficial vs. middle or superficial vs. deep, consistent across BOLD and VAPER, paired two-tailed t-test). Despite the similar layer-dependent modulation effect, the laminar patterns of audiovisual response were distinct across BOLD and VAPER. In BOLD, audiovisual (green curve) and audio-only (red curve) responses were similarly weighted towards the pial surface (Figure 4B, left), consistent with the superficial bias often observed in BOLD (i.e., higher amplitude towards the pial surface). In contrast, VAPER response to audiovisual stimulus (Figure 4B, right) peaked in middle layers (green curve) and the shape of the response profile as a function of cortical depth was distinct from that in the audio-only condition (red curve).

The laminar profile of anterior PT under VAPER contrast, which has superior laminar specificity (Chai et al., 2020), revealed a mixture of feedforward inputs in both audio-only and audiovisual conditions, and enhanced feedback in audio-only relative to audiovisual conditions. While feedforward inputs are known to cascade from auditory belt area and MGN (Hackett, 2011; Hackett et al., 2014; Hackett et al., 2007; Hackett et al., 1998; Schroeder et al., 2001) and terminate in the middle layers of PT, the origins of feedback inputs to the superficial layers of anterior PT are less straightforward. We suspect that it might be due to task performance related top-down influence. For example, the longest response time (RT) and lowest accuracy in the motion perception task was found in the audio-only condition (two-tailed paired t-test, p < 1×10−4 for either audio-only vs. visual or audio-only vs. audiovisual, on either RT or accuracy, see Figure S5B for details). Thus, the high uncertainty associated with judging the audio-only motion may require additional top-down feedback influence from higher-order multimodal areas (Brang et al., 2013; Cappe et al., 2010; Hershenson, 1962; Vannini et al., 2004). On the contrary, such feedback was absent in the audiovisual condition, given synchronized bottom-up inputs across visual and auditory modalities suffice to support coherent motion perception.

To explore whether the enhanced feedback in the audio-only condition was a result of task performance related top-down effect, we first investigated whether this feedback signals can be modulated by task performance. To do so, we divided all audio-only trials from each session into two groups with a median split on RT (short vs. long RT) and extracted feedback signals (response difference between audio-only and audiovisual) in each RT group (Figure 4D). As predicted, a 2 task performance (short RT, long RT) × 3 layer (superficial, middle, deep) repeated-measures ANOVA revealed a significant interaction effect (BOLD: F(2, 50) = 6.4, p = 0.003; VAPER: F(2, 50) = 3.4, p = 0.04). Specifically, a significant response difference between short RT trials and long RT trials was observed in the superficial layers only (BOLD: t(25) = 3.3, p = 0.003, Cohen’s d = 0.46; VAPER: t(25) = 2.4, p = 0.02, Cohen’s d = 0.44), but nearly absent or not significant in the middle (BOLD: t(25) = 2.2, p = 0.04, Cohen’s d = 0.39; VAPER: t(25) = 0.85, p = 0.40, Cohen’s d = 0.20) or the deep layers (BOLD: t(25) = 1.0, p = 0.30, Cohen’s d = 0.21; VAPER: t(25) = 0.18, p = 0.85, Cohen’s d = 0.04). Second, we examined whether the response in high order areas, like intraparietal sulcus (IPS) and Brodmann area 6 (BA6, part of frontal cortex), also depended on the task performance. We found that the response in BA6 was similarly and significantly modulated by different RTs (Figure S6B) (t(25) = 2.8, p = 0.008, Cohen’s d = 0.24), suggesting frontal cortex as a potential top-down source related to the task performance. This is consistent with the findings that neural response in monkey premotor cortex (part of BA6) significantly correlated with RT in perceptual decision-making tasks (Chandrasekaran et al., 2017).

As a control analysis, we also examined whether a similar laminar response was present in a control region located in auditory T2 area, representing the primary auditory cortex (A1) (Figure S7). Again, given the known draining vein effect, BOLD response peaked in the superficial layers and decreased monotonically toward the white matter in the presence of auditory stimuli (Figure S7B, left, red and green curves). In VAPER, such strong bias toward the pial surface was much weaker, and the response profile peaked in the middle layers (Figure S7B, right, red and green curves). This peak response in the middle layers was expected given that the feedforward inputs from the MGN terminate in the middle layers of A1 (Hackett, 2011; Kaur et al., 2005; Kimura et al., 2003; Sakata and Harris, 2009). Importantly, we found no significant interaction effect between stimulus modality (audio-only, audiovisual) and cortical depth (superficial, middle, deep) (BOLD: F(2, 50) = 1.8, p = 0.2; VAPER: F(2, 50) = 0.2, p = 0.8; two-way repeated-measures ANOVA), suggesting that A1 is relatively immune to the layer-specific modulation of audiovisual inputs as we found in anterior PT.

In posterior PT, we observed a bilaminar activity in visual-only condition (Figure 5, blue curves). Layer-dependent signal change under VAPER contrast clearly resolved two response peaks in the superficial and deep laminae, which is less obvious under BOLD contrast. This double peak response feature was highly reproducible across participants and across sessions on different days within each participant (Figure S8). In sharp contrast, the bilaminar pattern to visual inputs was absent in the control region, A1 (Figure S7, blue curves), confirming that the present feedback was unique to posterior PT. This bilaminar activity in posterior PT could result via a direct feedback loop from the early visual cortex, whose projections originate mainly from the infragranular layer and are expected to terminate in superficial and deep layers (Falchier et al., 2010).

4. DISCUSSION

In this study, we examined topographical distribution of auditory vs. visual processing in PT and the laminar specificity of different projections to subareas of PT. We outlined two ways in which our study advance understanding of multisensory processing in PT: (1) Division of function in anterior vs. posterior PT: We found that the topography of visually evoked response was largely segregated from that of auditory induced response within PT. That is, visual processing is predominantly restricted to posterior PT, while auditory processing is mainly localized in anterior PT. (2) Distinct laminar profiles across subareas of PT: In anterior PT, enhanced feedback in the audio-only than audiovisual condition was present in the superficial layers, and this feedback was modulated by task performance in the audio-only condition. However, in posterior PT, there was a significant bilaminar response pattern to visual input peaking at superficial and deep layers. Below we discuss the mechanisms and anatomical substrates that underlie the topographical and laminar organizations of PT.

4.1. Division of function in anterior and posterior PT

Many studies have reported that visual stimulation alone can activate auditory cortex, particularly in PT. For example, in monkeys, the mere presentation of visual scenes led to fMRI response in auditory cortex, mostly at its caudal end (Kayser et al., 2007), which presumably corresponds to PT in humans. In mouse electrophysiological recording, neural responses to flashing lights were found to be more prominent in the secondary auditory cortex (Morrill and Hasenstaub, 2018), which also corresponds to PT. In contrast, the abovementioned animal studies both found that visual activity is much weaker or less commonly observed in the primary auditory cortex compared to secondary regions. Consistent with these findings, in our study, there was minimal response to visual stimulus in A1. Instead, the visual responses were primarily clustered in posterior PT, which has also been shown in previous human fMRI studies (Alink et al., 2008).

Importantly, our findings suggest that visual and auditory responses in PT do not result from overlapping multisensory representations. Instead, they can be clearly segregated along the anterior-posterior axis with an 8.8 ± 3.4 mm distance along the cortical curvature between the unimodal peak auditory and visual columns. Note that the cortical ribbon is convoluted and supposed to be longer than the straight-line distance. At regular imaging resolution (~ 2-3 mm) in 3T for example with spatial smoothing applied, the fMRI signals may easily spread across sulcal banks and adjacent gyri, thus making it appear as if voxels in this region can be tuned to different sensory modalities. Because of this, it has been common for previous multisensory fMRI studies to highlight the overlapping of activity induced by different modalities in the auditory cortex (Alink et al., 2008; Calvert et al., 1997; Foxe et al., 2002; Kayser et al., 2007). With the advent of high-resolution fMRI at high field, we started to see that the peak representations of different sensory modalities in PT can actually be separated. In a recent sub-millimeter-resolution fMRI study (Gau et al., 2020), audio-only stimulus induced response in PT was found mainly in anterior portions, which is consistent with our results. In contrast, crossmodal response enhancement was observed mainly in the caudal parts of PT in that study, which corresponds to posterior PT in our study where visual activation was found.

This sensory-specific topographical organization in PT is consistent with the evidence from anatomical tracing studies in monkeys. Among thalamic inputs to the caudal auditory areas (caudomedial belt (CM), caudolateral belt (CL) and temporal parietotemporal (Tpt) areas in monkeys, homolog of PT in humans), there appears to be a rostrocaudal gradient (corresponding to the anterior-posterior axis in our study) in the relative contribution of projections from auditory-specific and multisensory thalamic nuclei: multisensory projections favor Tpt (corresponding to posterior PT), over CL then CM, whereas auditory-specific thalamic projections favor CM (corresponding to anterior PT), followed by CL then Tpt (Hackett et al., 2007). These distributions of different thalamic projections are in line with our finding of auditory processing in anterior PT and non-auditory processing in posterior PT. Besides, there is a similar gradient in the distribution of projections from visual cortex directly to caudal auditory cortex: Tpt/caudal parabelt (corresponding to posterior PT) receives a higher density of visual inputs than CM (corresponding to anterior PT) (Falchier et al., 2010). This relative distribution of visual projections further supports the idea of visual processing favoring posterior PT over anterior PT.

4.2. Feedback modulation in superficial layers of anterior PT from higher-order multimodal areas

Besides the feedforward drive from auditory belt area and MGN (Hackett, 2011; Hackett et al., 2014; Hackett et al., 2007; Hackett et al., 1998; Schroeder et al., 2001), anterior PT also received enhanced feedback signals at superficial layers in response to audio-only stimuli compared to multisensory condition.

One potential source of such auditory specific feedback may come from attention. It is possible that the attentional demands involved in the three task conditions were not equivalent such that a higher amount of attention was required to perform the audio-only task. However, we did not observe a significant difference in performance across different task conditions in our vigilance measure (Figure S5A).

Given the absence of response to visual inputs in anterior PT and this feedback influence was present in the audio-only condition while absent in audiovisual condition, this feedback signal at superficial layers of anterior PT is unlikely to come from visual cortex directly (Falchier et al., 2010) or by being relayed through the multisensory thalamic nuclei (Cappe et al., 2012; Hackett et al., 2007; Schroeder and Foxe, 2005). This feedback seems specific to auditory processing, perhaps due to a task-performance related top-down influence in audio-only condition (see below).

In our study, participants were asked to detect the movement direction of the sound or the sphere under either unisensory (audio-only or visual-only), or multisensory conditions. Audio-only tasks are considerably more challenging because of the scanning noise and the intrinsic poor perception of auditory space (Choi et al., 2018). Accordingly, for every participant, audio-only tasks had a longer response time and resulted in significantly lower detection accuracy (Figure S5B). This high uncertainty associated with audio-only bottom-up inputs may lead to recruitment of higher-order brain areas, which send feedback projections to anterior PT to facilitate task performance (Brang et al., 2013; Cappe et al., 2010; Hershenson, 1962; Vannini et al., 2004). By contrast, the synchronized bottom-up inputs across auditory and visual modalities in multisensory condition were sufficient to support coherent motion perception, in which top-down feedback may not be necessary. In line with that, we found both the response in frontal cortex (Figure S6B) and the feedback signal in the superficial layers of anterior PT (Figure 4D) were similarly and significantly modulated by task performance in the audio-only condition.

In a recent sub-millimeter-resolution fMRI study (Gau et al., 2020), they also compared the response in PT to audio-only and audiovisual stimuli and found no difference across cortical depths in the mean response. Such discrepancy with our study could be due to two possibilities. First, averaging response across PT may wash out the distinct laminar profiles of audiovisual processing in subregions of PT. Second, difference in the stimuli (looming stimuli in Gau et al. vs. movement stimuli in ours) and the design of the two studies could lead to different laminar dependent response.

4.3. Bilaminar feedback in posterior PT likely due to direct projections from early visual cortex

Although both anterior and posterior PT receive feedback inputs, their underlying mechanisms seem to be distinct. While the feedback influence on anterior PT mostly reflects enhanced top-down effects from higher-order multimodal areas, the feedback posterior PT receives is more likely to come from direct projections from early visual cortex. Using retrograde tracers in monkeys, Falchier and colleagues (Falchier et al., 2010) demonstrated evidence of direct connections from visual areas V2 and prostriata to auditory cortex, most abundant in temporoparietal area and caudal parabelt (corresponding to posterior PT in humans) and to a lesser extent in CM (corresponding to anterior PT in humans). This anatomical connection could serve as the neural substrate for the observed visual response peaked in posterior PT in this study, which has also been consistently reported across species (Alink et al., 2008; Kayser et al., 2007; Morrill and Hasenstaub, 2018). These projections from visual areas originated mainly from infragranular layers, suggestive of a feedback connection which is expected to terminate in superficial and deep layers. This is consistent with the bilaminar visual activity pattern we observed in posterior PT.

Another, not mutually exclusive, possibility of the source of visual inputs to posterior PT comes from subcortical multisensory thalamic nuclei. Despite MGN being the main source of thalamic inputs to PT, there are separate inputs from the multisensory thalamic nuclei (Cappe et al., 2012; Hackett et al., 2007; Schroeder and Foxe, 2005). Based on results from animal tracing studies, non-specific/multisensory thalamus nuclei project to a wide cerebral cortical areas including the auditory cortex while these projections terminate mainly in superficial layers only, which cannot fully explain the bilaminar response in posterior PT (Jones, 1998).

Lastly, in principle, higher-order multimodal areas also send feedback to early auditory processing areas. In this study, a list of potential confounding factors, including attention, task engagement, and imaginary effect (Choi et al., 2018; Persichetti et al., 2020; Talsma et al., 2010; Vetter et al., 2014), could send top-down influences to early sensory cortices. Given that this bilaminar activity in posterior PT was only found in the visual-only condition, we expect that any, or a combination of abovementioned potential top-down factors have to be specifically present in the visual-only condition. However, as noted, there was no significant difference in our overall attention/vigilance measures across task conditions (Figure S5A). In addition, based on the response time and accuracy measure (Figure S5B), it is the audio-only condition rather than visual-only condition which required significantly more task engagements. With regard to the potential anticipation effect, this visual feedback in posterior PT was also found in single-condition runs (all unimodal runs present prior to multimodal runs, Figure S8). That is, we prevented any audio imaginary or anticipation effect in the visual-only condition by avoiding presenting the multisensory condition prior to any unisensory condition. Furthermore, given the distinct bilaminar visual feedback profile, it is unlikely that the visual inputs to posterior PT arise from the same source of higher-order multimodal areas to anterior PT (superficial layers only).

5. CONCLUSIONS

In sum, we report a division of function between auditory and visual processing in human anterior and posterior PT, each receives feedback inputs with distinct mechanisms. Anterior PT responds strongly and solely to auditory inputs and receives feedback modulation in superficial layers. This feedback was modulated by task performance and is likely coming from higher-order multimodal areas. In contrast, posterior PT showed high sensitivity to visual inputs. Moreover, a distinct visual feedback effect was found in posterior PT at both superficial and deep layers, which likely arise via direct anatomical projections from early visual cortex. Together, our findings revealed the functional topographical and laminar organization principles in human PT during multisensory processing at an unprecedented spatial resolution.

Supplementary Material

1
2

Highlights.

  • We uncover sensory responses along both columnar and laminar dimensions in human PT.

  • We report a division of function between auditory and visual processing within PT.

  • We find distinct laminar patterns in subareas of PT during audiovisual processing.

Acknowledgements:

This work was supported by the Intramural Research Program of the National Institute of Mental Health (annual report ZIAMH002783). We acknowledge Laurentius Huber for helpful discussion, Benedikt A. Poser for contributions to the 3D-EPI sequence used here, and the NIH Fellows Editorial Board for editorial assistance.

Footnotes

Competing interests: The authors declare no competing interests.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Data and code availability: The datasets generated during the current study are available from the corresponding author on request. The stimulation and its presentation script, data preprocessing and analysis code have been deposited to https://github.com/yuhuichai/Audiovisual_in_PT. It uses publicly available software packages, including Psychophysics Toolbox 3, AFNI, SPM, and LAYNII.

REFERENCES

  1. Alink A, Euler F, Kriegeskorte N, Singer W, Kohler A, 2012. Auditory motion direction encoding in auditory cortex and high-level visual cortex. Hum Brain Mapp 33, 969–978. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Alink A, Singer W, Muckli L, 2008. Capture of auditory motion by vision is represented by an activation shift from auditory to visual motion cortex. The Journal of neuroscience : the official journal of the Society for Neuroscience 28, 2690–2697. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baumgart F, Gaschler-Markefski B, Woldorff MG, Heinze HJ, Scheich H, 1999. A movement-sensitive area in auditory cortex. Nature 400, 724–726. [DOI] [PubMed] [Google Scholar]
  4. Bizley JK, King AJ 2012. What Can Multisensory Processing Tell Us about the Functional Organization of Auditory Cortex? In: The Neural Bases of Multisensory Processes. Eds. Murray MM, Wallace MT: Boca Raton (FL). [PubMed] [Google Scholar]
  5. Brainard DH, 1997. The Psychophysics Toolbox. Spat Vis 10, 433–436. [PubMed] [Google Scholar]
  6. Brang D, Taich ZJ, Hillyard SA, Grabowecky M, Ramachandran VS, 2013. Parietal connectivity mediates multisensory facilitation. Neuroimage 78, 396–401. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Calvert GA, Bullmore ET, Brammer MJ, Campbell R, Williams SC, McGuire PK, Woodruff PW, Iversen SD, David AS, 1997. Activation of auditory cortex during silent lipreading. Science 276, 593–596. [DOI] [PubMed] [Google Scholar]
  8. Cappe C, Murray MM, Barone P, Rouiller EM, 2010. Multisensory facilitation of behavior in monkeys: effects of stimulus intensity. J Cogn Neurosci 22, 2850–2863. [DOI] [PubMed] [Google Scholar]
  9. Cappe C, Rouiller EM, Barone P 2012. Cortical and Thalamic Pathways for Multisensory and Sensorimotor Interplay. In: The Neural Bases of Multisensory Processes. Eds. Murray MM, Wallace MT: Boca Raton (FL). [PubMed] [Google Scholar]
  10. Chai Y, Li L, Huber L, Poser BA, Bandettini PA, 2020. Integrated VASO and perfusion contrast: A new tool for laminar functional MRI. Neuroimage 207, 116358. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Chai Y, Li L, Wang Y, Huber L, Poser B, Duyn J, Bandettini PA, 2019. A magnetization transfer weighted anatomical reference allows laminar fMRI analysis in native functional image space. [Google Scholar]
  12. Chandrasekaran C, Peixoto D, Newsome WT, Shenoy KV, 2017. Laminar differences in decision-related neural activity in dorsal premotor cortex. Nat Commun 8, 614. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Choi I, Lee JY, Lee SH, 2018. Bottom-up and top-down modulation of multisensory integration. Curr Opin Neurobiol 52, 115–122. [DOI] [PubMed] [Google Scholar]
  14. Cox RW, 1996. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res 29, 162–173. [DOI] [PubMed] [Google Scholar]
  15. Driver J, Noesselt T, 2008. Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 57, 11–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Driver J, Spence C, 2000. Multisensory perception: beyond modularity and convergence. Curr Biol 10, R731–735. [DOI] [PubMed] [Google Scholar]
  17. Falchier A, Schroeder CE, Hackett TA, Lakatos P, Nascimento-Silva S, Ulbert I, Karmos G, Smiley JF, 2010. Projection from visual areas V2 and prostriata to caudal auditory cortex in the monkey. Cereb Cortex 20, 1529–1538. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Foxe JJ, Wylie GR, Martinez A, Schroeder CE, Javitt DC, Guilfoyle D, Ritter W, Murray MM, 2002. Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study. Journal of neurophysiology 88, 540–543. [DOI] [PubMed] [Google Scholar]
  19. Gau R, Bazin PL, Trampel R, Turner R, Noppeney U, 2020. Resolving multisensory and attentional influences across cortical depth in sensory cortices. Elife 9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Guipponi O, Wardak C, Ibarrola D, Comte JC, Sappey-Marinier D, Pinede S, Ben Hamed S, 2013. Multimodal Convergence within the Intraparietal Sulcus of the Macaque Monkey. Journal of Neuroscience 33, 4128. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hackett TA, 2011. Information flow in the auditory cortical network. Hear Res 271, 133–146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Hackett TA, de la Mothe LA, Camalier CR, Falchier A, Lakatos P, Kajikawa Y, Schroeder CE, 2014. Feedforward and feedback projections of caudal belt and parabelt areas of auditory cortex: refining the hierarchical model. Front Neurosci 8, 72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hackett TA, De La Mothe LA, Ulbert I, Karmos G, Smiley J, Schroeder CE, 2007. Multisensory convergence in auditory cortex, II. Thalamocortical connections of the caudal superior temporal plane. J Comp Neurol 502, 924–952. [DOI] [PubMed] [Google Scholar]
  24. Hackett TA, Stepniewska I, Kaas JH, 1998. Subdivisions of auditory cortex and ipsilateral cortical connections of the parabelt auditory cortex in macaque monkeys. J Comp Neurol 394, 475–495. [DOI] [PubMed] [Google Scholar]
  25. Hershenson M, 1962. Reaction time as a measure of intersensory facilitation. Journal of Experimental Psychology 63, 289–293. [DOI] [PubMed] [Google Scholar]
  26. Huber L, Handwerker DA, Jangraw DC, Chen G, Hall A, Stuber C, Gonzalez-Castillo J, Ivanov D, Marrett S, Guidi M, Goense J, Poser BA, Bandettini PA, 2017. High-Resolution CBV-fMRI Allows Mapping of Laminar Activity and Connectivity of Cortical Input and Output in Human M1. Neuron 96, 1253–1263 e1257. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Huber L, Poser BA, Bandettini PA, Arora K, Wagstyl K, Cho S, Goense J, Nothnagel N, Morgan AT, van den Hurk J, Reynolds RC, Glen DR, Goebel R, Gulban OF, 2020. LAYNII: A software suite for layer-fMRI. bioRxiv, 2020.2006.2012.148080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Huber L, Tse DHY, Wiggins CJ, Uludag K, Kashyap S, Jangraw DC, Bandettini PA, Poser BA, Ivanov D, 2018. Ultra-high resolution blood volume fMRI and BOLD fMRI in humans at 9.4 T: Capabilities and challenges. Neuroimage 178, 769–779. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Iurilli G, Ghezzi D, Olcese U, Lassi G, Nazzaro C, Tonini R, Tucci V, Benfenati F, Medini P, 2012. Sound-driven synaptic inhibition in primary visual cortex. Neuron 73, 814–828. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Jones EG, 1998. Viewpoint: the core and matrix of thalamic organization. Neuroscience 85, 331–345. [DOI] [PubMed] [Google Scholar]
  31. Kaur S, Rose HJ, Lazar R, Liang K, Metherate R, 2005. Spectral integration in primary auditory cortex: laminar processing of afferent input, in vivo and in vitro. Neuroscience 134, 1033–1045. [DOI] [PubMed] [Google Scholar]
  32. Kayser C, Petkov CI, Augath M, Logothetis NK, 2007. Functional imaging reveals visual modulation of specific fields in auditory cortex. The Journal of neuroscience : the official journal of the Society for Neuroscience 27, 1824–1835. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Kayser C, Petkov CI, Logothetis NK, 2008. Visual modulation of neurons in auditory cortex. Cereb Cortex 18, 1560–1574. [DOI] [PubMed] [Google Scholar]
  34. Kim SG, Ogawa S, 2012. Biophysical and physiological origins of blood oxygenation level-dependent fMRI signals. J Cereb Blood Flow Metab 32, 1188–1206. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kimura A, Donishi T, Sakoda T, Hazama M, Tamai Y, 2003. Auditory thalamic nuclei projections to the temporal cortex in the rat. Neuroscience 117, 1003–1016. [DOI] [PubMed] [Google Scholar]
  36. Larkum ME, Petro LS, Sachdev RNS, Muckli L, 2018. A Perspective on Cortical Layering and Layer-Spanning Neuronal Elements. Front Neuroanat 12, 56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Lawrence SJD, Formisano E, Muckli L, de Lange FP, 2019. Laminar fMRI: Applications for cognitive neuroscience. Neuroimage 197, 785–791. [DOI] [PubMed] [Google Scholar]
  38. Li LQ, Miller KL, Jezzard P, 2012. DANTE-prepared pulse trains: A novel approach to motion-sensitized and motion-suppressed quantitative magnetic resonance imaging. Magnetic Resonance in Medicine 68, 1423–1438. [DOI] [PubMed] [Google Scholar]
  39. Meyer K, Kaplan JT, Essex R, Webber C, Damasio H, Damasio A, 2010. Predicting visual stimuli on the basis of activity in auditory cortices. Nat Neurosci 13, 667–668. [DOI] [PubMed] [Google Scholar]
  40. Morrill RJ, Hasenstaub AR, 2018. Visual Information Present in Infragranular Layers of Mouse Auditory Cortex. The Journal of neuroscience : the official journal of the Society for Neuroscience 38, 2854–2862. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Musacchia G, Schroeder CE, 2009. Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex. Hear Res 258, 72–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Noesselt T, Rieger JW, Schoenfeld MA, Kanowski M, Hinrichs H, Heinze HJ, Driver J, 2007. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. The Journal of neuroscience : the official journal of the Society for Neuroscience 27, 11431–11441. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Persichetti AS, Avery JA, Huber L, Merriam EP, Martin A, 2020. Layer-Specific Contributions to Imagined and Executed Hand Movements in Human Primary Motor Cortex. Curr Biol 30, 1721–1725 e1723. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Petro LS, Muckli L, 2017. The laminar integration of sensory inputs with feedback signals in human cortex. Brain Cogn 112, 54–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Poser BA, Koopmans PJ, Witzel T, Wald LL, Barth M, 2010. Three dimensional echoplanar imaging at 7 Tesla. Neuroimage 51, 261–266. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Rockland KS, Ojima H, 2003. Multisensory convergence in calcarine visual areas in macaque monkey. Int J Psychophysiol 50, 19–26. [DOI] [PubMed] [Google Scholar]
  47. Sakata S, Harris KD, 2009. Laminar structure of spontaneous and sensory-evoked population activity in auditory cortex. Neuron 64, 404–418. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Scheich H, Baumgart F, Gaschler-Markefski B, Tegeler C, Tempelmann C, Heinze HJ, Schindler F, Stiller D, 1998. Functional magnetic resonance imaging of a human auditory cortex area involved in foreground-background decomposition. The European journal of neuroscience 10, 803–809. [DOI] [PubMed] [Google Scholar]
  49. Schroeder CE, Foxe J, 2005. Multisensory contributions to low-level, 'unisensory' processing. Curr Opin Neurobiol 15, 454–458. [DOI] [PubMed] [Google Scholar]
  50. Schroeder CE, Foxe JJ, 2002. The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res Cogn Brain Res 14, 187–198. [DOI] [PubMed] [Google Scholar]
  51. Schroeder CE, Lindsley RW, Specht C, Marcovici A, Smiley JF, Javitt DC, 2001. Somatosensory input to auditory association cortex in the macaque monkey. Journal of neurophysiology 85, 1322–1327. [DOI] [PubMed] [Google Scholar]
  52. Self MW, van Kerkoerle T, Goebel R, Roelfsema PR, 2017. Benchmarking laminar fMRI: Neuronal spiking and synaptic activity during top-down and bottom-up processing in the different layers of cortex. Neuroimage. [DOI] [PubMed] [Google Scholar]
  53. Talsma D, Senkowski D, Soto-Faraco S, Woldorff MG, 2010. The multifaceted interplay between attention and multisensory integration. Trends in Cognitive Sciences 14, 400–410. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. van Gelderen P, Jiang X, Duyn JH, 2017. Rapid measurement of brain macromolecular proton fraction with transient saturation transfer MRI. Magnetic resonance in medicine : official journal of the Society of Magnetic Resonance in Medicine / Society of Magnetic Resonance in Medicine 77, 2174–2185. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Vannini P, Almkvist O, Franck A, Jonsson T, Volpe U, Kristoffersen Wiberg M, Wahlund LO, Dierks T, 2004. Task demand modulations of visuospatial processing measured with functional magnetic resonance imaging. Neuroimage 21, 58–68. [DOI] [PubMed] [Google Scholar]
  56. Vetter P, Smith FW, Muckli L, 2014. Decoding sound and imagery content in early visual cortex. Curr Biol 24, 1256–1262. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Waehnert MD, Dinse J, Weiss M, Streicher MN, Waehnert P, Geyer S, Turner R, Bazin PL, 2014. Anatomically motivated modeling of cortical laminae. Neuroimage 93Pt 2, 210–220. [DOI] [PubMed] [Google Scholar]
  58. Wang L, Mruczek RE, Arcaro MJ, Kastner S, 2015. Probabilistic Maps of Visual Topography in Human Cortex. Cereb Cortex 25, 3911–3931. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1730112-supplement-1.pdf (470KB, pdf)
2
NIHMS1730112-supplement-2.docx (2MB, docx)

RESOURCES