Anatomical Evidence of Multimodal Integration in Primate Striate Cortex

Abstract

The primary visual cortex (area 17 or V1) is not thought to receive input from nonvisual extrastriate cortical areas. However, this has yet to be shown to be the case using sensitive tracers in the part of area 17 subserving the peripheral visual field. Here we show using retrograde tracers that peripheral area 17 subserving the visual field at an eccentricity of 10–20° receives projections from the core and parabelt areas of the auditory cortex as well as from the polysensory area of the temporal lobe (STP). The relative strength of these projections was calculated for each injection by computing the proportions of retrogradely labeled neurons located in the auditory and STP areas with respect to number of labeled neurons constituting the established projection from the superior temporal sulci (STS) motion complex (middle temporal area, medial superior temporal, fundus of the superior temporal area). In peripheral area V1 the projection from auditory cortex corresponds to 9.5% of that of the STS motion complex and STP to 35% of that from the STS motion complex. Compared to peripheral area 17, central and paracentral area 17 showed considerably weaker inputs from auditory cortex (0.2–0.8%) but slightly more from STP cortex (3.5–6.1%). The present results show that the connectivity of area 17 is eccentricity dependent. Direct projections from auditory and STP cortex to peripheral area 17 have important consequences for higher visual functions of area 17, including multimodal integration at early stages of the visual cortical pathway.

Multimodal integration of sensory inputs has been shown to occur at different midbrain and cortical levels (Stein and Meredith, 1993). The known anatomy of the visual cortex predicts that integration of visual and nonvisual stimuli will occur at late stages of cortical processing (Hikosaka et al., 1988; Calvert et al., 2001). Previous anatomical and single-unit recording studies point to multisensory integration in polysensory areas located in temporal, parietal, and frontal cortex (Goldman-Rakic, 1988). However there is recent electrophysiological and brain imaging evidence that visual, auditory, and somatosensory integration occurs at early stages of the visual cortical pathways (Giard and Peronnet, 1999;Macaluso et al., 2000), for example around the lingual gyrus where area 17 is located. Findings in favor of early polysensory integration are intriguing because anatomical studies in the monkey have failed to detect afferent connections to primary visual cortex from areas processing modalities other than vision.

The reported absence of multisensory input to area 17 is also surprising in view of the consequences of early deprivation. Congenital blindness leads to early visual cortical areas in the dorsal stream responding to nonvisual stimuli, including tactile (Sadato et al., 1996; Buchel et al., 1998; Cohen et al., 1999) and auditory (Weeks et al., 2000) stimuli. It is thought that these nonvisual responses are being relayed back to early visual cortical stages via back projections from the parietal cortex (Pons, 1996). Alternatively, nonvisual information may have access to visual areas through developmental changes in the centrifugal pathways (Angelucci et al., 1998).

Area V1 receives projections from ∼12 areas that have all been described as belonging to the visual cortex (Maunsell and Van Essen, 1983; Felleman and Van Essen, 1991). More recently, somewhat weak projections have been described from distant areas in the ventral (temporal occipital area, temporal area) (Distler et al., 1993) and dorsal visual pathways [medial superior temporal (MST) and lateral intraparietal area (LIP)] (Boussaoud et al., 1990; Felleman and Van Essen, 1991; Rockland et al., 1994; Barone et al., 2000). However, these distant areas contain neurons that respond principally to visual stimuli and oculomotor activity, except for LIP where auditory related activity has been reported (Linden et al., 1999).

Evidence in favor of multimodal integration at early visual stages has led us to question the accepted wisdom concerning the nature of back-projections to area V1 (Jones and Powell, 1970; Macaluso et al., 2000). The vast majority of studies on the connectivity of area V1 have been centered on the operculum, where the representation of the central 0–8° is located. Of the few studies that have included data on the connectivity of the peripheral representation of area V1, that of Shipp and Zeki (1989) revealed a quantitative difference concerning the back projection of the middle temporal area (MT). Their study showed that MT projections to the calcarine cortex gave a stronger labeling in layer 1 than did the MT projections to layer I in central area V1. In the present study we have specifically addressed the question of the effects of the representation of the visual field on connectivity of area 17 using retrograde tracers coupled with quantification of labeling.

MATERIALS AND METHODS

Injections of tracers. Thirteen retrograde tracing experiments were performed on nine cynomolgus monkeys (Macaca fascicularis; Table 1). After premedication with atropine (1.25 mg, i.m.) and dexamethasone (4 mg, i.m.), monkeys were prepared for surgery under ketamine hydrochloride (20 mg/kg, i.m.) and chlorpromazine (2 mg/kg, i.m.). Anesthesia was continued with halothane in N₂O and O₂ (70:30). Heart rate was monitored, and artificial respiration was adjusted to maintain the end-tidal CO₂ at 4.5–6%. The rectal temperature was maintained at 37°C. Single injections of retrograde fluorescent tracers fast blue (FsB) and diamidino-yellow (DY) were made by means of Hamilton syringes that were in some cases equipped with glass pipettes (40–80 μm diameter). Injections were made at a shallow angle to the cortical surface and the tracer was injected while the needle or pipette was withdrawn from the cortex to form longitudinal injections sites (2–3 mm) primarily restricted to the cortical gray matter. The smallest injection was the DY injection in M85RH (0.05 μl). In the other single injections, 0.2–0.3 μl of tracer were delivered. In one case (BB270) multiple injections were made and a total of 2 μl of DY was delivered. Side-by-side FsB and DY injections were separated by 3 mm.

Table 1.

Numbers of labeled neurons in auditory, STP, and STS complex

Case	Dye	Injection	Auditory	STP	STS complex
M85 LH	FsB	V1 C	0 (20 ½)	118 (28 ¼)	2415 (37 ¼)
M81 LH	DY	V1 C	12 (40  $\frac{2}{3}$)	63 (22 ¼)	1452 (40 ¼)
M85 RH	DY	V1 C	3 (25 ½)	17 (24 ¼)	1372 (33 ¼)
M85 RH	FsB	V1 C	18 (25 ½)	79 (24 ¼)	2279 (33 ¼)
M75 LH	FsB	V1 PC	0 (46 ½)	94 (57 ½)	1417 (36 ¼)
M75 RH	FsB	V1 PC	0 (23 ¼)	52 (7 ¼)	843 (11 ¼)
M87 RH	FsB	V1 PC	53 (21  $\frac{1}{3}$)	155 (25  $\frac{2}{5}$)	2618 (25  $\frac{2}{5}$)
BB270 RH	DY	V1 PC	51 (24 ¼)	448 (24 ¼)	6399 (24 ¼)
M81 RH	DY	V1 P	208 (35 ½)	675 (25 ½)	1917 (37 ½)
M76 LH	FsB	V1 P	143 (38  $\frac{1}{1}$)	165 (17 ¼)	636 (21 ¼)
M88 LH	FsB	V1 P	236 (34 ½)	408 (39 ½)	1300 (38 ¼)
Area 17
M89 LH	FsB	Parabelt	0 (33)
M90 RH	FsB	STP	0 (42)

The dyes [fast blue (FsB); diamidino yellow (DY)] and the visual representation injected in area V1 [central (C); paracentral (PC); peripheral (P)] are provided. For all areas the number of labeled neurons, and in brackets the number of sections examined and frequency of sampling are provided.

In the present study we explored the connectivity at four different ranges of eccentricity with reference to published maps of the representation of visual space in area 17 (Daniel and Whitteridge, 1961; Gattass et al., 1981; Van Essen et al., 1984). Injections aimed at the foveal representation of area 17 and were made laterally on the operculum of the occipital lobe, 2/5 mm from the V1–V2 border (Fig.1) These central area 17 injections were in cortex subserving 0–2° in the lower visual field. Injections aimed at the paracentral representation of area 17 were located at the medial limit of the operculum (Fig. 2). These paracentral area 17 injections were in cortex subserving 6–8° in the lower visual field. The description of the location of injection sites in peripheral area 17 uses the anatomical terminology of Daniel and Whitteridge (1961) and Van Essen et al. (1984). The calcarine cortex viewed parasagitally has a mushroom configuration with a head and a stem. Injections aimed at the peripheral representation were made in the head and stem of the calcarine sulcus by means of vertical penetrations using stereotaxic coordinates (Fig.3). Two different eccentricities can be distinguished in peripheral area 17 injections. M76LH was located in the dorsal leaf of the calcarine cortex (Daniel and Whitteridge, 1961). According to the published map of area 17, this injection (M76LH) was in cortex subserving 10–12° eccentricity (Van Essen et al., 1984). The other two peripheral injections (M81RH and M88LH) were located further ventrally near the junction of the dorsal leaf with the calcarine stem and were in cortex subserving 15–20° (Van Essen et al., 1984). The auditory cortex of the caudal part of the parabelt was injected on the superior temporal gyrus (STG) between the lateral and superior temporal sulci. FsB injection into area STP [temporal parieto-occipital associated area intermediate/temporal parieto-occipital associated area caudal (TPOi/TPOc)] did not involve STG and area MST.

Fig. 1.

Injection sites in area 17 subserving the central visual field. A, Lateral view of the monkey brain showing the level of the injection sites in horizontal sections.B, Representative horizontal section showing location of central injection sites. C, Photomicrograph of DY injection site. D, Lateral view of the brain showing the visual representation of area V1 and location of central injection sites (Gattass et al., 1981). D, Bottom, Injection sites with respect to the visual field. E, Central injection sites, two sections per injection. Uptake zone is shown inblack. Gray, Dense intrinsic labeling of area V1. Numbers refer to section number. Thin lines, White matter–gray matter boundary. Dotted lines, layer 4. Arrowheads, Area V1/V2 border. Cases M85RH FsB, DY, Only one injection site of a pair is shown. IOS, Inferior occipital sulcus;LS, lunate sulcus; CaS, calcarine sulcus;IPS, intraparietal sulcus; CeS, central sulcus; OTS, occipitotemporal sulcus;STS, superior temporal sulcus; LaS,lateral sulcus; AS, arcuate sulcus; PS,principal sulcus; POS, posterior occipital sulcus.

Fig. 2.

Injection sites in area 17 subserving the paracentral visual field. A, Dorsal and lateral view of the monkey brain showing the level of the injection sites in parasagittal and horizontal sections. B, Injection sites on dorsal part of the operculum. Inset shows representative parasagittal section showing location of injection sites in M75LH and M75RH. C, Injection site on the posterior part of the operculum. Inset shows representative horizontal section showing location of injection site in M87RH.D, Lateral view of the brain showing visual representation of area V1 and location of paracentral injection sites (taken from Gattass et al., 1981). Bottom, Injection site with respect to the visual field. Conventions and abbreviations as in Figure 1.

Fig. 3.

Injection sites in area 17 subserving the peripheral visual field. A, Lateral view of the monkey brain showing the level of the injection sites in horizontal sections.B, Photomicrograph of fast blue injection site.C, Two horizontal sections illustrating the two levels in A. D, Lateral view of the brain showing visual representation of area V1 and location of peripheral injection sites (taken from Gattass et al., 1981). Below, injection sites with respect to the visual field. E, Injection sites in the calcarine sulcus. Conventions and abbreviations as in Figure 1.

Histological procedures. After 11–13 d survival, animals were deeply anesthetized before being perfused with 200 ml of 0.9% saline, 1 ml of 4–8% paraformaldehyde–0.05% glutaraldehyde in phosphate buffer (0.1 m), pH 7.4, and 10–30% sucrose in phosphate buffer (0.1m), pH 7.4. All the procedures used follow the National and European regulations concerning animal experiments (EC guidelines 86/609/EC) and have been approved by the authorized national and veterinary agencies. Brains were removed, blocked, and 40-μm-thick sections were cut on a freezing microtome. One section in three was mounted in saline onto gelatinized slides. Sections at regular intervals were reacted for cytochrome oxidase and acetylcholinesterase (AChE) activity (Barone et al., 2000), SMI-32 (Hof et al., 1996), parvalbumin (Kosaki et al., 1997), and Cat-301 (DeYoe et al., 1990). Sections were observed in UV light with oil-immersion objectives using a Leitz fluorescent microscope equipped with a D-filter set (355–425 nm). A computer-assisted program (Biocom) coupled to the microscope stage was used to trace out sections and to record the position of labeled neurons. After observation, sections were counterstained with cresyl violet and projected on to charts of labeled neurons to relate the position of labeled neurons to established histological borders.

Location of cortical areas. Immunohistochemical and myelin staining criteria made it possible to localize labeled neurons in MT, MST, FST [superior temporal sulci (STS) complex], STP, and auditory areas (Morel et al., 1993; Cusick, 1997). MT is located in the posterior bank of STS and characterized by strong immunolabeling of layers 2/3 neurons with Cat-301 (DeYoe et al., 1990) and parvalbumin reactivity in layer 3 and 4. We characterized the limit between areas located in the medial bank of STS (MST, FST) and area STP located along the anterior bank of the superior temporal sulcus (Bruce et al., 1981) using SMI-32 immunohistochemistry and myelin staining. SMI-32 reactivity distinguished MST and STP in layers 3 and 5 (Cusick, 1997). In addition, myelin staining was darker in top layers of STP compared with MST. The border of area STP with the parabelt was distinguished using myelin staining (Morel et al., 1993). This shows that the parabelt extends across the STG on to the upper bank of the STS, as shown in Figure 4C (Hackett et al., 1998). The auditory area A1 located in the posterior bank of lateral sulcus is distinguished using AChE and cytochrome oxidase reactivity (Morel et al., 1993; Hackett et al., 2001) and by the presence of a strong parvalbumin staining in layers 2–4 (Kosaki et al., 1997).

Fig. 4.

Labeling in auditory cortex after peripheral injections. A, Lateral view of the brain and horizontal section showing levels of sections in B andC. B, AChE labeling;arrows indicate limits of core and belt of the auditory cortex in the lateral sulcus. C, Individual sections showing labeling in auditory cortex. D, Density profiles. Numbers of neurons per section counted between thearrowheads indicated in C. Conventions and abbreviations as in Figure 1.

Analysis. For each cortical area and in each animal, numbers of neurons were computed at regular intervals and using high sampling frequencies in all structures containing labeled neurons. The regional extent of the area that contains labeled neurons is the projection zone (Figs. 4, 5). Density profiles (numbers of neurons per section) provide a one-dimensional reconstruction of the projection zone. The zero on the horizontal axis of the density profiles shown in Figures 4 and 5 correspond to the last section before the projection zone. Density profiles show the number of neurons falling off from a peak in the center of the projection zone to minimal values in the periphery (Batardière et al., 1998). Visual inspection of the density profiles makes it possible to ensure that optimal sampling frequencies are used for different cortical areas. Hence, the relative proportion of neurons in the areas of interest are calculated for maximal sampling frequencies (1:1) from the actual frequency used (Table 1), which varies according to the dimension and density of labeling in individual areas.

Fig. 5.

Labeling in STP after peripheral injections.A, Individual sections showing labeling in multisensory cortex. Arrowheads show limits within which neurons were counted in STP and in the STS complex. B, SMI-32 labeling; the arrow indicates the limit between area STP and the STS complex. C, Density profiles. Numbers of neurons per section in STP counted between thearrowheads indicated in A. Conventions and abbreviations as in Figure 1.

RESULTS

Three groups of injections of FsB and DY were made in area 17 subserving the central (0–2°), paracentral (6–8°), and the peripheral (10–20°) visual field (Van Essen et al., 1984). The central and paracentral injections on the operculum were made parallel to the area 17 border (see Materials and Methods). The tracers FsB and DY have highly restricted uptake zones (Kennedy and Bullier, 1985). In all cases, except for the multiple injections in BB270, injection sites were reconstructed, and the uptake zones were determined. All uptake zones were limited to area 17.

Injections of area 17 subserving the central visual field had uptake zones, which were all restricted to the cortical gray matter and were located between 1.75 and 4 mm of the V1–V2 border (Fig. 1). The smallest injection (M85RH DY) was centered on layer 4. The uptake zones of M85RH FsB and M85LH injections involved all six cortical layers. In M81LH the uptake zone failed to involve the bottom part of layer 6 (Fig. 1).

The paracentral injections were located on the operculum adjacent to the interhemispheric fissure. The two single paracentral injections (M75LH and M75RH) were located dorsally between 5 and 8 mm of the V1–V2 border. M75LH showed minor involvement of the white matter. In M75RH however, there was extensive involvement of the underlying white matter (Fig. 2). However, there was an absence of labeled neurons in the calcarine, and there was a single locus of labeling in area V2 and the lateral geniculate nucleus. These observations show that uptake and transport of the dye was restricted to the injection of the cortical gray matter of area 17 on the operculum. The third single paracentral injection (M87RH) was posterior on the operculum, and reconstruction of the uptake zone showed that it involved all cortical layers. The fourth paracentral injection (BB270RH) was made by multiple injections.

The reconstructions of the uptake zones of injections in area 17 subserving the peripheral visual field showed that all injection sites had uptake zones limited to the cortical gray matter (Fig. 3). Cases M81RH and M88LH involved all cortical layers. The uptake zone of the M76LH injection did not go right to the bottom of layer 6 (Fig. 3).

Injections in peripheral area 17 consistently labeled neurons in the infragranular layers of auditory cortex, which could be identified by means of AChE staining (Fig. 4). Labeled neurons in the auditory cortex were primarily located on adjacent sections, showing that labeling originates from a relatively circumscribed region of cortex extending 5–8 mm dorsoventrally and which can be broadly related to known subdivisions of the auditory cortex (Hackett et al., 1998) (Fig.4B). These comprise the auditory core and belt cortex, which coincide with the AChE dense region on the posterior bank of the lateral sulcus (LaS). The auditory parabelt coincides with lighter AChE staining and occupies the STG at the lateral margin of the core and belt regions (Hackett et al., 1998). The levels of histological sections corresponding to dorsal and ventral limits of the auditory projection zone are shown on a lateral view of the brain (Fig.4). This shows that the zone of labeled cells overlaps with only the more dorsal part of the AChE-dense region. Approximately one-third of the labeled cells was located inside the AChE-dense region, in the region of the core, caudal lateral belt, and the middle lateral belt components (Hackett et al., 1998). In our material we were not able to distinguish between the core, caudal lateral belt, and middle lateral belt components. The remaining two-thirds of labeled neurons were on the superior temporal gyrus where the caudal auditory parabelt is located (Fig. 4) (Hackett et al., 1998). Hence, the main site of labeled neurons was the caudal parabelt, which is part of the dorsal auditory stream (Rauschecker and Tian, 2000). After injections in the central and paracentral representation of area 17, only occasional scattered labeled neurons are found in this region (Fig.6) (Table 1).

Fig. 6.

Quantitative analysis of labeling in auditory cortex and area STP. Histogram of the mean (±SE) proportion of labeled neurons in auditory cortex and area STP. Percentages refer to the number of neurons observed in the posterior bank of STS (STS complex) (Table 1).

Injections in peripheral area 17 labeled neurons in the STP, which is localized in the anterior bank of the superior temporal sulcus (Cusick et al., 1995). Labeled neurons were located in the infragranular layers, and density profiles revealed labeling over ∼10 mm in the dorsoventral direction with peak levels of labeling located over 3–8 mm, suggesting relatively compact projection zones (Fig. 5). Labeled neurons in STP are located in the posterior part of the anterior bank of STS, a region corresponding to TPOi/TPOc, which is interconnected to the dorsal visual stream (Cusick, 1997).

To estimate the relative strength of the projections from STP and auditory cortex to area 17, we pooled the results to form two groups for the central and paracentral injections. For the peripheral injections we have separated M76LH, which is located in cortex subserving 10–12° from M81RH and M88LH, which are in cortex subserving 15–20° (see Materials and Methods) (Table 1). For each of these four groups we have proceeded to compute the fraction of neurons located in STP and auditory cortex (Fig. 6). This was done by counting the total number of neurons in STP and auditory cortex and expressing these numbers as fractions of the total number of neurons counted in the STS motion complex. This makes it possible to compare the strength of the projection at different eccentricities and helps overcome variations in the size of the effective uptake in different injections as revealed by the density plots. This is illustrated in M85LH in central area V1 in which the density profile shows a peak count in STP of 30 neurons per section, which is comparable to the peak counts in the two peripheral injections M76LH and M88LH (Fig. 5). However, the tracer uptake and transport in the central injection has been more effective in the central injection and leads to larger numbers of labeled STS complex neurons so that it returns a weaker strength of projection from STP of 4.8% compared with 24.6 and 21% for the two peripheral injections (Table 1). It could be objected that the changes we observe with eccentricity are attributable to changes in the numbers of labeled neurons in the STS complex rather than to changes in the auditory cortex and STP. To control for this we have calculated the percentage of neurons in each structure with respect to the total number of labeled neurons in the cortex. This confirmed that eccentricity influenced numbers of neurons in STP and auditory cortex and not in the STS complex.

In the present study we distinguish the auditory parabelt on the supratemporal gyrus from a combined area A1 and the auditory belt area in the lateral sulcus. At all eccentricities the parabelt contained approximately two-thirds of the labeled auditory neurons. The relative proportions of labeled neurons in auditory cortex in both central and paracentral injections is very low (0.2–0.5%) and shows a sharp increase to reach 9.5% after peripheral injections (Fig. 6). Injections in central area 17 gave relative STP projections of 3.5%, which showed a moderate increase in the paracentral injections to reach 6.1%. Again there was a sharp increase in the relative strength of the STP projection, which reached 35.3% of the number of neurons in the STS motion complex after peripheral injections. These central or periphery differences are not caused by individual variability because in one case (M81; Table 1) a simultaneous injection in the central and peripheral representation of area 17 leads to a similar large eccentricity difference (inputs from the combined auditory and STP areas, peripheral representation: 44.9%, central representation: 3.6%).

We examined whether the projections from STP and auditory cortex are reciprocal. The two cases where the auditory cortex (STG) and area STP received injections of retrograde tracer failed to reveal labeled neurons in area 17, suggesting that the projections from auditory and STP cortex to area 17 are unidirectional.

DISCUSSION

There have been only a limited number of studies investigating how the connectivity of a cortical area is influenced by representation of the visual field, and these have mostly concentrated on the interconnections between extrastriate visual areas (Baizer et al., 1991; Stepniewska and Kaas, 1996; Galletti et al., 2001; Lyon and Kaas, 2002). Relatively few studies have addressed the differences in the connectivity of area 17 subserving the central and peripheral visual field (Martinez-Millan and Hollander, 1975; Shipp and Zeki, 1989). One striking difference between central and peripheral area 17 is evidence of a direct connection of peripheral 17 with the prostriata (Sousa et al., 1991; Rosa et al., 1993). These findings are complementary to ours, suggesting that peripheral area 17 is directly influenced by nonvisual inputs.

Technical considerations: reliability of the visual field representation of the injections sites

Area 17 has a fine grain representation of the visual field, and a number of studies have established highly reliable maps of visual space representation in the primary visual area (Daniel and Whitteridge, 1961; Gattass et al., 1981; Van Essen et al., 1984). In the present study we have been able to use these maps to locate injection sites in cortical regions representing central, paracentral, and peripheral visual fields. The three injection sites in cortex subserving the peripheral visual field would not appear to concern the same representation, and an injection at 10–12° can tentatively be distinguished from two injections at 15–20° (see Materials and Methods). However, although the distinction between central, paracentral, and peripheral is sufficiently strong to not be influenced by interanimal variability, the distinction between the injection sites at 10–12° and 15–20° must be considered as a tentative in the absence of visual field mapping.

Influence of eccentricity on connectivity of area 17 with STP and area A1

The present results show that peripheral but not central area 17 receives projections from auditory cortex. Similar results are found for STP, although STP projections at 3.5% are non-negligible. Figure 6suggests that the density of the auditory and STP projection to area 17 increases progressively with eccentricity. Although the absence of precise mapping of visual space before injection in the present study makes this an unproven point, we need to keep in mind the possibility that the density of auditory and STP projections to eccentricities >20° may be more important than that revealed in the present study.

Interspecies comparison

Although sparse projections from auditory cortex to area 18 in rodents and cat have been reported (Miller and Vogt, 1984; Innocenti et al., 1988), projections to area 17 are thought to be absent in these species (Dehay et al., 1988; Sanderson et al., 1991; Montero, 1993). Whereas we cannot exclude the possibility that some of the above studies might have missed an auditory projection to peripheral area 17, at least one specifically addressed this possibility and found no such projection (Montero, 1993). These interspecies comparisons suggest that the projection of auditory cortex to peripheral area 17 might be a primate feature. Homologies of STP in nonprimates are difficult to identify. One candidate is the anterior ectosylvian sulcus in cat which has not been found to have a projection to either area 17 or 18 (Symonds and Rosenquist, 1984).

Role of auditory–STP projections to area 17 in polysensory integration

STP may play an important role in polysensory integration (Bruce et al., 1981; Baylis et al., 1987; Hikosaka et al., 1988). First, more than half of visual neurons also respond to auditory and/or somatosensory stimuli. Second, lesion experiments aimed at area STP lead to auditory or somatosensory neglect (Luh et al., 1986; Watson et al., 1994), suggesting that area STP is involved in cross-modal attentional mechanisms. All of this argues for a nonvisual modulation of peripheral V1 through the direct projection from area STP.

The auditory system is organized in parallel streams and the caudal auditory parabelt area, which contains approximately two-thirds of the labeled neurons in the auditory cortices in the present study, is part of the dorsal auditory pathway specialized in spatial information processing, including sound source localization (Romanski et al., 1999;Kaas and Hackett, 2000; Rauschecker and Tian, 2000; Recanzone et al., 2000b; Tian et al., 2001). Auditory cortex plays an important role in sound localization (Clarey et al., 1992), and receptive fields are large and extend behind the pinna axis (Barone et al., 1996). Similarly auditory receptive fields in STP are large and expand in the peripheral visual field (Hikosaka et al., 1988). Hence, there is a possibility of congruence in the spatial properties of the auditory receptive fields of A1/STP and the visual representation of area 17 to which these areas project. However, we did not find a tight clustering of labeled neurons in the lateral sulcus, in agreement with the lack of a precise spatial map of auditory space in the auditory cortex (Clarey et al., 1994;Recanzone et al., 2000b). Nor did we find clustering of labeled neurons in STP in line with the absence of retinotopic order in this area (Bruce et al., 1981; Hikosaka et al., 1988).

Polysensory integration could serve to enhance perceptual capacities (Stein and Meredith, 1993) so that the addition of an auditory signal to a visual stimulus might lead to improved detection. This could serve to reduce behavioral orientation response time to visual stimuli (Goldring et al., 1996; Giard and Peronnet, 1999; McDonald et al., 2000). This is relevant to our findings because the short latencies to auditory stimulus in A1 and STP (Bruce et al., 1981; Recanzone et al., 2000a,b) suggest that there could be temporal congruence in activation of A1/STP and area 17. This could mean that the projections from STP and auditory cortex to area 17 could participate in a foveation mechanism toward a peripheral sound source, which would be compatible with the greater prevalence of these projections to the peripheral representation of area 17 (Heffner and Heffner, 1992).

Corticocortical feedforward projections going from lower to higher hierarchical levels are thought to elaborate receptive field response (Vanduffel et al., 1997). The projections from STP and auditory cortex to area 17 form part of the feedback pathways to area 17. Feedback projections are involved in more global levels of analysis and implicate area 17 in higher visual functions (Lee et al., 1998;Pascual-Leone and Walsh, 2001; Super et al., 2001; Tolias et al., 2001). Feedback pathways have also been implicated in visual imagery (Miyashita, 1995), and cross-modal activity could be associated with the mental image of the stimuli and has been implicated in the activation of the calcarine cortex in humans (Klein et al., 2000). Cross-modal-related activity has been found in neurons of association temporal areas (Gibson and Maunsell, 1997) and in the primary somatosensory area of the monkey (Zhou and Fuster, 2000). Similarly, in humans performing a tactile object recognition task, a specific activity has been observed in the visual cortex of the calcarine in the absence of visual information (Deibert et al., 1999).

Auditory–STP projections to area 17 and functional reorganization after early deprivation

Polysensory integration could play an important role in the functional reorganization of the cortex after sensory deprivation and may depend on cortical pathways such as the auditory–STP projection to peripheral area 17 (Pons, 1996). Patients suffering from early blindness have early visual areas, including primary visual cortex responsive to tactile (Sadato et al., 1996; Cohen et al., 1997; Buchel et al., 1998) and auditory (Weeks et al., 2000) stimuli. This could improve the capacity of the blind to localize sound in peripheral auditory space (Roder et al., 1999). A recent PET study in deaf adults suggests that cross-modal plasticity underlies recovery of language comprehension after cochlear implantation (Giraud et al., 2001). These authors suggest that the activation of visual cortex is similar to the recruitment of auditory cortex during lipreading (Calvert et al., 1997). One can make the hypothesis that cross-modal cooperation could compensate for suboptimal stimulation by processing complementary input from another sensory modality. Hence, anatomical, electrophysiological, and brain imaging studies support a model of cross-modal integration within a distributed network of cortical areas in which the primary visual area could participate in the initial integration of sensory information (Stein, 1998; Calvert et al., 2000).