Selective visuo‐haptic processing of shape and texture

Abstract

Previous functional neuroimaging studies have described shape‐selectivity for haptic stimuli in many cerebral cortical regions, of which some are also visually shape‐selective. However, the literature is equivocal on the existence of haptic or visuo‐haptic texture‐selectivity. We report here on a human functional magnetic resonance imaging (fMRI) study in which shape and texture perception were contrasted using haptic stimuli presented to the right hand, and visual stimuli presented centrally. Bilateral selectivity for shape, with overlap between modalities, was found in a dorsal set of parietal areas: the postcentral sulcus and anterior, posterior and ventral parts of the intraparietal sulcus (IPS); as well as ventrally in the lateral occipital complex. The magnitude of visually‐ and haptically‐evoked activity was significantly correlated across subjects in the left posterior IPS and right lateral occipital complex, suggesting that these areas specifically house representations of object shape. Haptic shape‐selectivity was also found in the left postcentral gyrus, the left lingual gyrus, and a number of frontal cortical sites. Haptic texture‐selectivity was found in ventral somatosensory areas: the parietal operculum and posterior insula bilaterally, as well as in the right medial occipital cortex, overlapping with a medial occipital cortical region, which was texture‐selective for visual stimuli. The present report corroborates and elaborates previous suggestions of specialized visuo‐haptic processing of texture and shape. Hum Brain Mapp 2008. © 2007 Wiley‐Liss, Inc.

INTRODUCTION

Early studies of monkeys with lesions of different parts of primary somatosensory cortex (S1) indicated that ablation of Brodmann's area 1 selectively affected haptic perception of surface roughness, a microgeometric object attribute; whereas removal of Brodmann's area 2 selectively interfered with haptic perception of object size and shape, which are macrogeometric attributes (Randolph and Semmes, 1974). In contrast to this selectivity, lesions of Brodmann's area 3b (Randolph and Semmes, 1974) and of second somatosensory cortex (S2) (Murray and Mishkin, 1984) impaired perception of surface texture as well as object form. Although the idea of functional specialization for microgeometric surface texture and macrogeometric object form has not been explicitly addressed by neurophysiological studies in monkeys, findings seem to be generally consistent with this idea, since roughness changes are encoded in neuronal responses in S1 and S2 (Jiang et al., 1997; Pruett et al., 2000; Sinclair et al., 1996) and neurons responding preferentially to particular shapes have been found in Brodmann's areas 2 and 5 (Koch and Fuster, 1989). However, a framework unifying these neurophysiological observations with the results of the lesion studies cited above is still lacking.

It remains even more unclear how these functional differences described in monkeys map onto the human brain. In humans, damage to S1 compromises both roughness and object form discrimination, while parietal opercular lesions (presumably involving S2) affect perception of object form but not surface texture (Roland, 1987). Such lesions have also been reported to result in tactile agnosia, a specific inability to recognize tactile objects in the face of relatively intact perceptual capabilities (Caselli, 1993; Reed et al., 1996). Consistent with this, a functional magnetic resonance imaging (fMRI) study (Reed et al., 2004) described prominent activity in the parietal opercular region during haptic recognition of familiar objects, compared with palpation of meaningless shapes; however, this study did not control for variations in texture or attentional set. Another functional neuroimaging study did find activity during haptic shape perception in the parietal operculum (Ledberg et al., 1995), but such activity has been more commonly reported to occur in the postcentral sulcus (PCS) (Bodegård et al., 2000, 2001; Boecker et al., 1995; Servos et al., 2001) and parts of the intraparietal sulcus (IPS): the anterior intraparietal sulcus (aIPS) (Bodegård et al., 2001; Peltier et al., 2007; Roland et al., 1998; Zhang et al., 2004) and posterior intraparietal sulcus (pIPS) (Jäncke et al., 2001; Van de Winckel et al., 2005). In addition, various posterior parietal foci have been implicated in haptic shape perception (Bodegård et al., 2001; O'Sullivanet al., 1994; Peltier et al., 2007; Zhang et al., 2004). Tactile roughness perception, which has not been studied as extensively using neuroimaging, is associated with activity in both S1 and the parietal operculum (Burton et al., 1997, 1999; Ledberg et al., 1995; Roland et al., 1998; Servos et al., 2001).

Whereas some studies have noted common regions of activation in the PCS (O'Sullivan et al., 1994; Servos et al., 2001) or parietal operculum (Ledberg et al., 1995) for judgments of both surface texture and object form, and one study found that the processing of roughness involved a subset of parietal cortical areas active during form perception (O'Sullivan et al., 1994), segregated processing of these attributes is suggested by a report of parietal opercular recruitment during microgeometric discrimination of surface roughness but activation in the aIPS during macrogeometric discrimination of object shape and size (Roland et al., 1998). Studies focussing primarily on shape perception have identified areas that are shape‐selective (relative to texture) in both haptic and visual modalities (Amedi et al., 2001, 2002; Peltier et al., 2007; Zhang et al., 2004). These areas include multiple posterior parietal foci as well as a visual cortical region known as the lateral occipital complex (LOC). Thus, it is clear that there are many cerebral cortical regions that selectively process haptic shape (HS), a macrogeometric property, compared with haptic texture (HT), a microgeometric property. However, the literature to date does not permit a clear conclusion as to whether or not there is selective processing of haptically perceived texture. The present report aims to address this issue by directly contrasting haptic shape and texture perception. It was motivated by further analysis of the results of an earlier study (Zhang et al., 2004) in which haptic shape and texture conditions were contrasted; this analysis suggested the presence of texture‐selective activity in the parietal operculum and insula (Stilla and Sathian, unpublished observations).

The present study also aimed to more fully investigate haptically shape‐selective brain regions, and the multisensory nature of both shape‐ and texture‐selective regions. Regions involved in processing a particular property across modalities would be expected to show overlap of activations selective for that property in each modality. Moreover, if modality‐independent processing of a given attribute occurs in a multisensory region, we would predict that inter‐individual variations in the strength of evoked activity would be correlated across modalities. We therefore examined overlap of and correlations between visually‐ and haptically‐driven activations. Some of the shape‐selective data presented here have appeared in a previous report focusing on connectivity analysis (Peltier et al., 2007).

METHODS

Subjects

Six subjects (3 males and 3 females, mean age 22 years, age range 19–24 years) took part after giving informed consent. All were neurologically normal and had vision that was normal or corrected to normal with contact lenses. Subjects with callused fingerpads or a history of injury to the hands or their innervation were excluded, as were those with dyslexia, which is associated with tactile impairments (Grant et al., 1999; Sathian et al., 2003). All subjects were right‐handed, as assessed by the high‐validity subset of the Edinburgh handedness inventory (Raczkowski et al., 1974). All procedures were approved by the Institutional Review Board of Emory University.

Stimuli and Tasks

Stimuli (Fig. 1) were sets of objects of varying shape or texture. Independent stimulus sets were used for visual and haptic stimulation, to obviate the possibility of inducing carry‐over of processing between modalities. The HS stimuli were three‐dimensional (3D), meaningless objects with smooth, painted surfaces, measuring approximately the same (5 × 5 × 2.5 cm³), but varying in shape and configuration. The HT stimuli consisted of 4 × 4 × 0.3 cm³ cardboard substrates onto which textured fabric or upholstery was adhered. The shapes were all of identical texture and the textures were all of identical shape and size. An experimenter placed haptic stimuli in the subject's open right hand. The stimuli were presented in a fixed orientation, and subjects were instructed not to rotate or re‐orient objects during exploration. Observation of subjects during haptic exploration indicated that the thumb and all four fingers were consistently used, with contour following being the predominant exploratory procedure in the case of shape perception, and lateral rubbing motion in the case of texture perception, in accord with the work of Lederman and Klatzky (1987). Subjects were never allowed to see the haptic stimuli, and their eyes were closed during haptic exploration in order to avoid visual stimulation and distraction, and to minimize eye movements.

Figure 1.

Examples of stimuli used, which were sets of objects varying in shape but not texture, or texture but not shape. Independent visual and haptic stimulus sets were designed to avoid spillover of processing between modalities.

During the corresponding visual conditions, subjects viewed photographs of either shapes or textures, displayed centrally on a black screen. The images used for visual presentation were scanned into a computer, gray‐scaled, and resized to 10.5 × 7.5° using Adobe Photoshop. Visual shapes (VS) were photographs of objects of the same type as the HS stimuli. Photographs of common textures (Brodatz, 1966) were used to generate the visual texture (VT) stimuli.

Pilot studies were used to determine optimal trial durations, which differed substantially for vision and touch. This is not surprising given the inherently sequential nature of haptic exploration, compared to visual perception where considerably more information can be acquired in parallel across the visual field. Based on these pilot studies, we arrived at a trial duration of 5 s for haptic trials and 1 s for visual trials, which yielded accuracy above 90% correct. Within a modality, the same duration was used for shape and texture conditions. Given these very different trial durations between modalities, we designed the imaging part of the study with separate haptic and visual runs. This allowed investigation of segregated processing within a modality in the primary analyses, using pairwise contrasts between the shape and texture conditions. In secondary analyses, multisensory processing was investigated by overlaying maps derived from the primary analyses.

MR Scanning

MR scans were performed on a 3 Tesla Siemens Trio whole body scanner (Siemens Medical Solutions, Malvern, PA), using a standard quadrature headcoil. Functional images with blood oxygenation level‐dependent (BOLD) contrast were acquired using a T2*‐weighted single‐shot gradient‐recalled echoplanar imaging (EPI) sequence comprising horizontal slices of 5 mm thickness with full‐brain coverage (25 slices in four subjects and 21 in the other two) and the following parameters: repetition time (TR) 2,000 ms, echo time (TE) 30 ms, flip angle (FA) 90°, in‐plane resolution (IPR) 3.4 × 3.4 mm², in‐plane matrix (IPM) 64 × 64. High‐resolution 3D anatomic images were also acquired, using a 3D magnetization‐prepared rapid acquisition gradient echo (MPRAGE) sequence; 176 sagittal slices of 1 mm thickness were obtained (TR 2,300 ms, TE 3.9 ms, inversion time 1,100 ms, FA 8°, IPR 1 × 1 mm², IPM 256 × 256).

The subject lay supine in the scanner with the right arm outstretched and the right hand free to haptically explore objects. Foam padding under the body and beside the right arm was used to minimize movement and transfer of vibration from the gradient coils, while ensuring the subject's comfort. A mirror positioned above the subject's eyes provided unobstructed visualization of images projected on a screen at the rear magnet aperture. Head restraint straps and foam pads were utilized to minimize head movement. Subjects used earplugs to muffle scanner noise; additional noise attenuation was provided by headphones that also served to convey verbal cues (see below).

Separate haptic and visual stimulation runs were acquired. Four subjects performed haptic runs first, the other two performed visual runs first. Each type of stimulation block was repeated six times in a run. In each haptic trial, an experimenter placed stimuli in the subject's right hand for 5 s, with a 1 s inter‐stimulus interval. The onset of baseline blocks was verbally cued with the word “rest”. Immediately preceding each stimulation block, subjects were cued with the words “shape” or “texture” to instruct which task would follow. Subjects kept their eyes closed during haptic runs. Throughout the visual runs, subjects maintained fixation on a cross in the center of a black screen, keeping the right hand still. Visual stimuli were presented for 1 s per trial with a 1 s inter‐stimulus interval. For both visual and haptic runs, baseline blocks lasted for 16–20 s (16 s in 2 subjects, 18 s in 1 subject and 20 s in 3 subjects). Visual stimulation blocks were of the same duration as the corresponding baseline blocks for each subject; haptic stimulation blocks were always 30 s long. The stimuli used within blocks were in different, pseudorandomized orders for each subject.

Each subject took part in two haptic runs. The two subjects for whom visual stimulation block duration was 16 s participated in three visual runs; the remaining four subjects took part in two visual runs. Thus, there were 60 haptic trials per condition, while the number of visual trials per condition varied somewhat (144 trials for 2 subjects, 108 trials for 1 subject, and 120 trials for 3 subjects). This variation was because the number of visual trials was progressively increased over the course of the study, in order to increase the signal‐to‐noise ratio. Apart from these minor differences, the visual runs were similar to the haptic runs. The stimulation blocks were interleaved in a predetermined pseudorandom sequence that differed for each subject, using Presentation software (Neurobehavioral Systems, Albany, CA) that also controlled visual stimulus timing and the delivery of cues to the subject and experimenter. In the inter‐stimulus intervals, the subject pressed one of two buttons on a fiberoptic response box, with the second or third digit of the left hand, to indicate whether the stimulus was identical to or different from the immediately preceding stimulus. The responses were recorded by the software for later analysis.

Image Processing and Analysis

Image processing and analysis was performed using BrainVoyager QX v1.6.1 (Brain Innovation, Maastricht, The Netherlands). Each subject's BOLD images were realigned to the first image of the series using a rigid‐body transformation procedure. Functional images were pre‐processed utilizing trilinear interpolation for motion correction, sinc interpolation for slice scan time correction, and high‐pass temporal filtering to 1 Hz to remove slow drifts in the data. Anatomic 3D images were processed, co‐registered with the functional data, and transformed into Talairach space (Talairach and Tournoux, 1988). Activations were localized with respect to 3D cortical anatomy with the help of an MRI atlas (Duvernoy, 1999).

For group analysis, the transformed data were spatially smoothed with an isotropic Gaussian kernel (full width half maximum = 4 mm). Baseline periods were z‐normalized. Statistical analysis of group data used random‐effects, general linear models (GLM) followed by pairwise contrasts. Correction for multiple comparisons within the cerebral cortical volume (corrected P < 0.05) was achieved by imposing a threshold for the volume of clusters of contiguous voxels that each passed a significance threshold of P < 0.05, using a 3D extension (implemented in BrainVoyager QX) of the 2D Monte Carlo simulation procedure described by Forman et al. (1995). Since many activations were confluent with one another, the thresholds were increased as necessary to resolve separable activations. The primary analyses identified regions that were selective for texture or shape in the case of haptic or visual stimuli.

In secondary analyses, these regions were further characterized as being bisensory if there was overlap between the corresponding selective activations for haptic and visual stimuli. Within the bisensory zones, cubical (5 mm side) regions of interest (ROIs) were created, centered on the “hot spots” of overlap on the group maps. For each of these ROIs, we extracted beta weights for each subject for the haptic and visual condition that evoked activation in the ROI, relative to baseline. Taking these beta weights as indices of the magnitude of activity, we investigated correlations between the haptically‐ and visually‐driven activity across subjects.

RESULTS

Behavioral

Acuracy (mean ± SEM) was 90 ± 2.3% correct for HS, 92 ± 1.8% correct for HT, and 95 ± 0.8% correct on each visual task. A one‐way analysis of variance showed no significant effect of task on accuracy (F = 0.69; df = 5; P = 0.57), implying that the tasks were balanced for difficulty. These data were reported previously (Peltier et al., 2007) but are included here for completeness.

Imaging

In order to facilitate comparisons of the imaging data presented in the various figures, a single color scheme has been utilized: red for HS, blue for HT, green for VS, and pink for VT. Figure 2 employs inflated representations of the hemispheres and a flattened representation of the right hemisphere to illustrate that, in the haptic domain, a number of regions were shape‐selective, while a smaller number of regions were texture‐selective.

Figure 2.

Regions showing selectivity for haptic shape (HS>HT, in red) and haptic texture (HT>HS, in blue), displayed on inflated representations of both hemispheres and flattened representation of the right hemisphere, using a pseudocolor t scale (left). The haptically shape‐selective regions shown were located bilaterally in the ventral premotor cortex (PMv), supplementary motor area (SMA; shown on inflated right hemisphere representation only), postcentral sulcus (PCS), anterior, posterior and ventral intraparietal sulcus (aIPS, pIPS, vIPS), and the lateral occipital complex (LOC); in the left frontal eye field (FEF), left precentral gyrus (preCG), the left postcentral gyrus (postCG) and the right middle frontal gyrus (MFG). The haptically texture‐selective regions shown were in the parietal operculum and posterior insula bilaterally, and in the right medial occipital cortex (MOC).

Unisensory shape‐selectivity

Shape‐selective regions are listed in Table I with their Talairach coordinates and peak t values. These regions are illustrated in Figure 2, and selected regions are shown on brain slices in Figure 3. The preference of these regions for shape is corroborated by the time‐courses of the BOLD signal changes, which are shown for selected regions in Figure 4. Selectivity for shape can be seen to be for the most part relative rather than absolute.

Table I.

Talairach coordinates (x,y,z) and peak t‐values (t _max) of shape‐selective regions in the haptic and visual modalities

Region	x	y	z	t max
Haptic (HS>HT)
L SMA	−4	−3	52	4.1
R SMA	7	−3	55	6.3
R MFG	35	28	36	8.0
R anterior CiS	8	21	40	3.0
L FEF	−25	−13	56	5.1
L PMv	−45	−4	37	8.4
R PMv	53	2	36	7.0
L preCG	−27	−22	57	13.6
L postCG	−29	−33	61	11.0
L PCS	−42	−34	51	6.8
R PCS	46	−36	52	8.3
L aIPS	−27	−47	51	6.1
R aIPS	29	−49	45	9.0
L pIPS	−10	−61	55	6.0
R pIPS	16	−62	53	8.3
L vIPS	−20	−63	38	3.9
R vIPS	34	−67	27	4.6
L LG	−7	−69	−12	5.2
L LOC	−45	−62	−3	10.1
R LOC	50	−53	−5	9.3
Visual (VS>VT)
L PCS	−50	−30	49	4.1
R PCS	46	−33	52	6.7
L aIPS	−27	−44	49	9.6
R aIPS	30	−45	44	29.1
L pIPS	−6	−69	45	13.3
R pIPS	23	−71	49	4.7
L vIPS	−30	−73	27	8.7
R vIPS	36	−76	30	8.0
L LG	−14	−70	2	11.1
R LG	17	−52	−3	6.3
L precuneus	−16	−59	58	4.3
R precuneus	13	−61	58	5.7
R POF	11	−60	21	5.7
R cuneus	13	−76	15	7.6
L LOC	−48	−65	−2	7.9
R LOC (4 foci)	49	−56	−13	8.1
	55	−55	−6	11.0
	47	−67	−1	10.3
	45	−66	10	8.5

SMA: supplementary motor area, MFG: middle frontal gyrus, CiS: cingulate sulcus, FEF: frontal eye field, PMv: ventral premotor cortex, preCG: precentral gyrus, postCG: postcentral gyrus, PCS: postcentral sulcus, aIPS: anterior intraparietal sulcus, pIPS: posterior intraparietal sulcus, vIPS: ventral intraparietal sulcus, LG: lingual gyrus, LOC: lateral occipital complex, POF: parieto‐occipital fissure.

Figure 3.

Regions showing selectivity for haptic shape (HS>HT, in red) and visual shape (VS>VT, in green) displayed on horizontal brain slices (z values displayed below slices; pseudocolor t scale on right of figure). Bisensory shape‐selective zones were found in the PCS, aIPS, pIPS, vIPS, and LOC bilaterally. POF: parieto‐occipital fissure; LG: lingual gyrus; other abbreviations as in Figure 2.

Figure 4.

Time‐courses of BOLD signal changes in selected, bisensory shape‐selective regions, illustrating their preferential activation during shape compared to texture perception in both vision and touch. Abbreviations as in Figure 2.

Haptically shape‐selective regions (HS>HT) were located bilaterally in extensive regions spanning the PCS and the IPS, within which separable activations could be distinguished in the PCS, aIPS, pIPS, and ventral intraparietal sulcus (vIPS); in the LOC bilaterally; and in the left postcentral gyrus and left lingual gyrus. In addition to these classical sensory regions, haptic shape‐selectivity was also present in areas generally regarded as motor in function, including the left precentral gyrus, the left frontal eye field (FEF) and bilaterally in the ventral premotor cortex (PMv) and supplementary motor area (SMA). Haptic shape‐selectivity was also found in the right middle frontal gyrus (MFG) and in the right anterior cingulate sulcus.

As in the haptic modality, a number of regions were bilaterally shape‐selective in the visual modality (VS>VT). Apart from the LOC which is well‐known in this regard, bilateral visual shape‐selectivity was present in the lingual gyrus, vIPS, pIPS, aIPS, PCS, and precuneus. Also, loci in the right parieto‐occipital fissure and right cuneus were visually shape‐selective.

Unisensory texture‐selectivity

Haptic texture‐selectivity (HT>HS) was found bilaterally in a belt spanning the parietal operculum and posterior insula, with distinct activations in each of these regions, and in the right medial occipital cortex (MOC). There was also texture‐selective activity for haptic stimuli in the left MOC, but this activation did not achieve significance after correction for multiple comparisons and is not considered further. The significant activations are displayed in Figure 2, and on appropriate slices in Figure 5. Texture‐selectivity for visual stimuli (VT>VS) was found bilaterally in the MOC. Table II gives the Talairach coordinates of these activations along with the peak t values for the contrast on which they were identified. Figure 6 displays the time‐courses of BOLD signal change at the texture‐selective sites. It is evident from these curves that texture‐selectivity was present in all these regions, but that it was by no means absolute.

Figure 5.

Regions showing selectivity for haptic texture (HT>HS, in blue) and visual texture (VT>VS, in pink) displayed on brain slices (coronal slices at y = −10 and y = −90; sagittal slice at x = 18; horizontal slice at z = 14) using a pseudocolor t scale (right). A bisensory texture‐selective zone was found in the right MOC.

Table II.

Talairach coordinates (x,y,z) and peak t‐values (t _max) of texture‐selective regions in the haptic and visual modalities

Region	x	y	Z	t max
Haptic (HT>HS)
L parietal operculum	−53	−17	16	3.3
R parietal operculum	52	−10	19	8.8
L posterior insula	−31	−9	11	7.3
R posterior insula	36	−10	12	6.4
R MOC	19	−90	17	6.2
Visual (VT>VS)
L MOC	−9	−95	−1	3.9
R MOC	10	−92	1	3.3

MOC: medial occipital cortex.

Figure 6.

Time‐courses of BOLD signal changes in texture‐selective regions, illustrating their preferential activation during texture compared to shape perception. The parietal opercular and posterior insular regions bilaterally were active only during haptic perception; the right MOC was active during both vision and touch.

Bisensory zones

Visual and haptic shape‐selectivity overlapped in many areas (Fig. 3, Table III), including the PCS, aIPS, pIPS, vIPS, and LOC bilaterally. The time‐course curves from these ROIs (Fig. 4) confirm their bisensory selectivity for shape. It is worth emphasizing the occurrence of bisensory shape‐selectivity in the PCS, which corresponds to Brodmann's area 2 (Grefkes et al., 2001), and is usually regarded as a primary somatosensory processing region. In this region, the level of visually‐generated activity was considerably lower than that during the haptic conditions (Fig. 4), albeit still significant. Across subjects, two of the bisensory shape‐selective areas showed significant positive correlations between the beta weights of the HS and VS conditions (relative to baseline). As Table III shows, these areas were the left pIPS and the right LOC; none of the other areas showed a significant correlation between modalities.

Table III.

Talairach coordinates (x,y,z) of bisensory shape‐selective (HS/VS overlap) and bisensory texture‐selective (HT/VT overlap) zones, along with linear correlation coefficients (r) and P‐values for correlations between beta weights of haptic and visual activations relative to baseline

Region	x	y	z	r	P
HS/VS overlap zones
L PCS	−45	−33	45	0.7	0.12
R PCS	45	−34	51	−0.6	0.21
L aIPS	−26	−44	49	−0.1	0.85
R aIPS	31	−45	43	−0.23	0.66
L pIPS	−14	−63	52	0.97	0.001
R pIPS	14	−60	59	0.32	0.53
L vIPS	−25	−66	30	0.3	0.56
R vIPS	36	−68	27	0.36	0.48
R LOC	54	−57	−5	0.89	0.02
L LOC	−47	−64	−5	−0.4	0.44
HT/VT overlap zone
R MOC	18	−89	15	0.48	0.33

Significant correlations are indicated in bold type.

Overlap between visually and haptically texture‐selective activity was found in the right MOC (Fig. 5, Table III). Although retinotopic mapping was not performed in the subjects of the present report, comparing the position of the haptically texture‐selective activation in the right MOC on the flattened brain representation (Fig. 2) to published average retinotopic maps (Burton, 2003; Orban et al., 2004) indicates its probable location to be in visual area V2, for the most part. The visually texture‐selective activation appeared to be mainly in primary visual cortex (V1), with the bisensory overlap being at the V1‐V2 border.

The time‐courses from the right MOC region display texture‐selectivity in both haptic and visual modalities. Obviously, this early visual cortical area was substantially more responsive to visual than haptic stimuli, mirroring the preference of the PCS, an early somatosensory cortical area, for haptic stimuli, but it bears emphasis that there was sustained activity in the right MOC that was significantly above baseline during the haptic conditions, and especially during the texture condition. The beta weights in the MOC, however, were not significantly correlated between the HT and VT conditions (Table III).

DISCUSSION

The present report clearly establishes the presence of specialized neural processing of macrogeometric object form and microgeometric surface texture in the human somatosensory system. Although this was suggested by a previous study (Roland et al., 1998), other work examining this issue found common regions of activation (Ledberg et al., 1995; O'Sullivan et al., 1994; Servos et al., 2001) for both kinds of properties. The lack of clear consensus in the prior literature on the existence of texture‐selective processing in the somatosensory system may stem in part from the fact that such selectivity is relative rather than absolute, as demonstrated here. Shape selectivity is also not absolute. Thus, the relevant neural processing of shape and texture is not fully segregated into distinct pathways, but rather, each aspect is emphasized more or less in particular regions. Even though haptic stimuli were presented exclusively to the right hand, a number of regions showing a preference for shape or texture in the haptic modality were bilateral. The present report also expands the list of cerebral cortical regions involved in both visual and haptic processing of shape, and establishes a correlation between the magnitude of haptically‐ and visually‐evoked activity across subjects in the left pIPS and right LOC, implicating these areas specifically in modality‐independent processing of object shape. Further, we also demonstrate here, for the first time, the existence of a region that is both haptically and visually texture‐selective.

Experimental Design

It is worth reflecting on what type of experimental design is most suited to address whether or not there is segregated processing of two stimulus properties. One approach is to use a stimulus set in which both properties vary between members of the set, and cue attention to each property in turn. This approach has the advantage of keeping the stimuli constant across conditions, and would be ideal to reveal segregated processing in higher‐order areas. However, it has the potential drawback that there might be automatic processing of the uncued attribute even when this is not of interest, which might tend to minimize the likelihood of finding segregated processing, especially in lower levels of the sensory hierarchy. The alternative approach is to use two independent stimulus sets, with one property varying, and the same property cued, on each set. This approach might be expected to favor finding selective processing of each property, but runs the risk that extraneous variables could account for such selectivity. In the case of haptic shape and texture processing, one such variable might be the different exploratory procedures that are typically employed to assess these attributes (Lederman and Klatzky, 1987); variability in the type of movement could obviously be associated with different motor processing demands and therefore could underlie spurious “sensory” selectivity.

We elected the second of the two approaches, using independent stimulus sets, with the goal of maximizing the chance of revealing segregated processing in early somatosensory cortical areas. This approach is supported by previous empirical findings: prior observations of selective texture and form processing in somatosensory cortex used the independent set design (Roland et al., 1998; Stilla and Sathian, unpublished observations; Zhang et al., 2004) whereas the common set design failed to reveal such segregation (Servos et al., 2001). In order to minimize extraneous effects due to differences in exploratory procedures, we sought to keep them as comparable as possible by constraining the type of exploration used. The stimuli were explored using the thumb and all four fingers, and subjects were not allowed to re‐orient them relative to the orientation in which they were presented. Consistent with prior work (Lederman and Klatzky, 1987), the resulting exploratory procedures both used lateral motion, with texture perception relying mainly on a rubbing motion and shape perception, on contour following. Thus, the motor actions were fairly similar. Some differences between the haptic tasks, however, must be accepted as an inevitable consequence of the different perceptual goals and their intricately linked motor behaviors. These differences, albeit subtle, should be kept in mind when evaluating the findings of the present study, especially in regard to activations in regions usually considered to have motor functions. We emphasize, though, that our design effectively controlled for variations in task difficulty and attentional set, since contrasts were between task‐pairs designed to be of comparable difficulty, unlike comparisons in other studies between perceptual discrimination and a task lacking a perceptual decision (Reed et al., 2004), or between object exploration and rest (Boecker et al., 1995). Thus, we believe that the experimental design of the present study affords reasonable confidence, a priori, in the process‐specificity of haptically‐selective activations recorded in sensory regions. Further support for the validity of our design is provided post hoc by the comparable accuracy achieved by our subjects on all tasks during scanning, indicating that the conditions were balanced for difficulty, and the finding of both shape‐selective and texture‐selective regions rather than selectivity for only one of the tasks.

Similar considerations apply to the visual conditions. A separate issue is the validity of comparison between the haptic and visual conditions, given the differences between them. These differences included the eyes being closed in the haptic conditions and open in the visual conditions, the different trial durations between modalities (5 s for touch and 1 s for vision), and the slight imbalance in the order of visual and haptic runs. Despite these differences, our analytical approach is valid and meaningful because the primary analyses of selectivity for texture and shape were based on balanced, within‐modality comparisons. The investigation of bisensory responsiveness was based on secondary analyses using overlays of the unisensory maps of selectivity.

Haptic Shape‐Selectivity

Classical sensory cortex

A number of earlier studies have implicated the human PCS in processing of shape during haptic perception (Bodegård et al., 2000, 2001; Boecker et al., 1995; Servos et al., 2001), but the true selectivity of this region for shape has not previously been clear. The present report establishes haptic shape‐selectivity in this region bilaterally. The human PCS corresponds to Brodmann's area 2, based on postmortem correlation of anatomical landmarks with cytoarchitectonics (Grefkes et al., 2001), and thus haptic shape‐selectivity in this region is consistent with the observations that some neurons in monkey area 2 respond to particular shapes (Koch and Fuster, 1989) and that lesions of this area impair haptic shape discrimination in monkeys (Randolph and Semmes, 1974). The present report further identifies haptic shape‐selectivity in the left postcentral gyrus, presumably corresponding to Brodmann's area 1 (Geyer et al., 1999, 2000).

The aIPS has also been reported by many groups to be active during haptic shape discrimination (see Introduction). Its bilateral selectivity for object shape, which was proposed on the basis of earlier studies (Bodegård et al., 2001; Roland et al., 1998), is corroborated by the present report. The correspondence of the human aIPS with cytoarchitectonic or functional subdivisions of posterior parietal cortex in the monkey has not been definitively shown, but it is thought to correspond to the monkey anterior intraparietal area (AIP) (Grefkes and Fink, 2005). The present report also shows that the pIPS, another region described previously as being active during haptic shape perception (Jäncke et al., 2001; Van de Winckel et al., 2005), is bilaterally selective for shape processing. In addition, novel haptically shape‐selective foci were revealed in the vIPS bilaterally. The present report further demonstrates haptically shape‐selective activations in the LOC bilaterally, in agreement with previous work showing that LOC activity is form‐selective during haptic perception of 3D objects (Amedi et al., 2001, Zhang et al., 2004) and tactile perception of 2D stimuli composed of line elements (Prather et al., 2004). Haptic shape‐selectivity was also found in the left lingual gyrus; although this is part of visual cortex, the haptic focus did not overlap with a visually shape‐selective focus in the left lingual gyrus.

Multisensory processing of shape

The present report identifies a number of regions that are shape‐selective for visual as well as haptic stimuli. Although the visual and HS were drawn from independent sets, the two sets were quite similar, which probably tended to maximize bisensory intersection in the case of shape. The bisensory shape‐selective regions within the left and right LOC in the present study overlapped with the previously described bisensory (visuo‐haptic) sub‐regions of the LOC (Amedi et al., 2002). Shape‐selectivity in the LOC does not appear to extend to the auditory modality, indicating that this region is concerned primarily with object geometry (Amedi et al., 2002). It has been argued that haptically‐evoked activity in the LOC is not due to visual imagery (Amedi et al., 2001) but rather, signifies a common representation of shape across both vision and touch (James et al., 2002). However, the left LOC has been reported to be active during mental imagery of shape (De Volder et al., 2001), and inter‐individual variations in the strength of shape‐selective activity in the right LOC are strongly predicted by inter‐individual variations in imagery ratings (Zhang et al., 2004). At present, therefore, the potential role of visual imagery in recruiting activity in the LOC during haptic shape perception remains an open question. It is interesting to note that connectivity analyses of a subset of the haptically shape‐selective areas of the present report suggest bidirectional information flow between the PCS and the LOC via the IPS foci, implying the existence of not only bottom‐up paths from S1 into the LOC but also the top‐down paths that would be necessary for visual imagery (Peltier et al., 2007).

Bisensory shape‐selectivity was also found bilaterally and contiguously in the PCS, aIPS, pIPS, and vIPS in the present study. Bisensory shape‐selectivity has been reported previously in the left aIPS (Grefkes et al., 2002), a site also noted for multisensory convergence during contralaterally directed attention (Macaluso et al., 2002), mental rotation (Alivisatos and Petrides, 1997; Prather et al., 2004), and motion processing (Bremmer et al., 2001). Moreover, we demonstrate, for the first time in humans, the presence of bisensory shape‐selectivity in the other parietal cortical foci: the right aIPS and the PCS, pIPS and vIPS bilaterally. Activity specific for visuo‐tactile matching of 2D shape patterns on Mah Jong tiles was found earlier in a caudal region of the IPS located between our pIPS and vIPS foci (Saito et al., 2003). Converging evidence for a multisensory role of the vIPS regions bilaterally is provided by another study from our laboratory, which showed prominent deactivation during tactile form perception in subjects who were visually deprived by blindfolding for 2 h (Weisser et al., 2005). The finding of bisensory processing in IPS foci is not surprising given the description of multisensory responses in various parts of the IPS in monkeys (Duhamel et al., 1998; Iriki et al., 1996), although the homologies with humans remain uncertain.

It is especially interesting that bisensory processing was found in the PCS, which, as reviewed above, corresponds to Brodmann's area 2 in humans (Grefkes et al., 2001) and is considered to be part of S1, although it is at a higher hierarchical level than Brodmann's areas 3b and 1. The PCS is the lowest area in the somatosensory processing hierarchy that was bisensory in the present study. Visual responsiveness has not previously been reported in the human PCS, although some of the multisensory neurons described in the IPS of monkeys were on the anterior bank of this sulcus (Iriki et al., 1996) and thus may have been in area 2 (Iwamura, 1998). S1 neurons in monkeys (sub‐fields of S1 not specified) have also been found to demonstrate visual responsiveness in a visuo‐haptic orientation matching task (Zhou and Fuster, 1997), analogous to somatosensory responsiveness in visual area V4 in a similar task (Haenny et al., 1988).

Significant, positive correlations were found across subjects between the levels of activity during the HS and VS conditions in the bisensory shape‐selective zones in the left pIPS and right LOC. These correlations indicate that processing of object shape is closely linked across modalities in these two regions specifically. We hypothesized that such correlations would exist if multisensory regions performed modality‐independent processing. Thus, we propose that this finding reflects neuronal modules in the left pIPS and right LOC that are specialized for processing object shape and that are accessible through both touch and vision. These modules would be candidate loci for multisensory integrative operations. In contrast, the bisensory activity noted in the other shape‐selective areas, including the contralateral homologs of the putatively modality‐independent areas, appears not to be as closely linked between modalities. Possibly, activity in these areas may tend to reflect close intermingling of unisensory neurons rather than multisensory neurons with the capacity for integration across modalities. Fine‐grained analysis of multisensory activity in the human superior temporal sulcus (STS) has revealed intermingling of unisensory patches with differing modality specificity, with intervening multisensory patches (Beauchamp et al., 2004). If this is a general characteristic of multisensory integrative areas, the left pIPS and right LOC would be expected to contain a substantial proportion of multisensory patches, compared with the other regions in which such patches might be smaller, less numerous, or absent. This would be a fruitful topic for future investigation.

The lateralization of these modality‐independent zones is interesting, although definitive conclusions in this respect are limited by the presentation of haptic stimuli to the right hand alone, and bear verification in future studies with stimulation of each hand. Although previous studies did not report lateralization during visuo‐tactile shape matching in the caudal IPS near our pIPS foci (Saito et al., 2003) or during visual/haptic shape perception in the LOC (Amedi et al., 2001, 2002; Zhang et al., 2004), these studies did not use the approach taken here, of correlating the magnitude of activity between conditions. Lateralization of tactile and visual processing is not without precedent: The left aIPS was found to be preferentially active for discriminating grating orientation and the right temporo‐parietal junction, for discriminating small differences in grating location, regardless of which hand was used (Van Boven et al., 2005). In the LOC, an earlier study from our laboratory reported recruitment on the right during tactile form discrimination (Prather et al., 2004), and a recent study of visual object recognition demonstrated right lateralization when the task involved object matching but left lateralization when the task required object naming (Large et al., 2007). Right occipitotemporal lateralization has also been found during visual face perception (Sergent et al., 1992; Yovel and Kanwisher, 2004), and during visual discrimination of object size and orientation (Faillenot et al., 1999).

Frontal and cingulate cortex

Multiple shape‐selective foci were also noted outside traditional sensory regions. These included areas involved in motor control (the left precentral gyrus, the left FEF, bilateral SMA, and bilateral PMv) and cognition (the right MFG and the right anterior cingulate sulcus). The reasons for finding shape‐selectivity in these areas are not clear, and we cannot exclude subtle differences in eye or hand movements, or in higher‐order cognitive factors, between experimental conditions as the sources of these activations. It is entirely possible that these areas do participate in purely sensory tasks; however, future studies controlling nonsensory variables will be necessary to definitively address the role of these areas in haptic shape perception.

Haptic Texture‐Selectivity

Analysis of the data from an earlier study from our laboratory (Zhang et al., 2004) suggested that foci in the parietal operculum and posterior insula are more active during processing of HT than haptic shape (Stilla and Sathian, unpublished observations). However, subjects in this study merely noted attributes of stimuli covertly, without making overt responses, so that their performance could not be assessed directly. The present study, which included psychophysical measures, confirms the occurrence of HT‐selectivity in the parietal operculum and the posterior insula bilaterally. Multiple somatosensory maps have been described in the parietal operculum of monkeys. This region was first identified with S2 following its original description (see review by Burton, 1986). Additional areas defined later are called the ventral somatosensory area (VS) (Cusick et al., 1989) and the parietal ventral area (PV) (Krubitzer et al., 1995); and there is evidence for three separate maps in the parietal opercular hand representation of macaques (Fitzgerald et al., 2004). The human parietal operculum appears to contains four distinct cytoarchitectonic regions termed OP1‐4 (Eickhoff et al., 2006a,b), of which OP1, OP3 and OP4 are somatosensory (Eickhoff et al., 2006c). OP1 appears homologous to S2, OP4 to PV and OP3 to VS (Eickhoff et al., 2006a,b,c). Texture‐selective regions in the present study were in contiguous parts of the parietal operculum and posterior insula. Comparison with the maps of Eickhoff et al. (2006a,b,c) indicates that these regions appear to extend over the three somatosensory fields OP1, OP4 and OP3 (posterolateral, anterolateral, and anteromedial parietal operculum, respectively).

Texture‐selectivity has previously been reported in the left parietal operculum of humans (Roland et al., 1998), albeit at a site rather distant from the foci in the present studies. Moreover, responses to textures have been found in the parietal operculum of monkeys (Jiang et al., 1997; Pruett et al., 2000). Multiple regions within the posterior insula are responsive to tactile stimuli in monkeys (Robinson and Burton, 1980), and somatosensory responsiveness has been found in various parts of human insular cortex. Foci close to the insular texture‐selective regions of the present study have been reported to be specifically involved in cross‐modal (visuo‐tactile) matching of shape; however, it is not clear whether these foci were located in the insula, the claustrum, or both (Banati et al., 2000; Hadjikhani and Roland, 1998). Further work is required to precisely delineate the contribution of various parts of the parietal operculum and insula to various aspects of somatosensory and multisensory perception.

Interestingly, we also observed HT‐selectivity in the right MOC in the present study. The location of this activity makes it likely to be chiefly in area V2, although we did not perform retinotopic mapping to conclusively localize it within the visual processing hierarchy. While a number of visual cortical areas are known to be recruited during macrogeometric tactile tasks in normally sighted people, these have typically been in higher‐order visual areas specialized for the corresponding aspects of visual processing (reviewed by Sathian, 2005). In fact, a study comparing a macrogeometric tactile task, discrimination of 2D forms, to a microgeometric gap detection task revealed LOC activity in the former but not latter task (Stoesz et al., 2003). Previously reported activity in early visual cortex, including V1, of sighted subjects during tactile perception (Burton et al., 2006; Merabet et al., 2007) was rather weak and not task‐specific. Our finding of haptically texture‐selective activation of the MOC, in putative V2, extends earlier findings of tactile task‐specific recruitment of visual cortical areas into early visual cortex and the microgeometric domain. We suggest that this was triggered by the requirement for spatially extended, fine‐grained discrimination imposed by the texture task.

Multisensory processing of texture

The haptically texture‐selective MOC region in putative V2 overlapped with a region that was visually texture‐selective and appeared to be primarily in V1, the bisensory overlap appearing to be at the border between V1 and V2. These tentative localizations bear confirmation in future studies using retinotopic mapping. The magnitudes of haptically‐ and visually‐driven activity during the HT and VT conditions were not correlated in the bisensory overlap zone, suggesting that, as for the majority of the shape‐selective foci, truly multisensory neurons or patches (Beauchamp et al., 2004) are sparse or absent in this overlap zone and the multisensory responsiveness of this region may reflect mainly intermingling of unisensory neuronal populations.

The haptic and visual textures used in the present study were not only independent sets but also somewhat different, having been chosen to be representative of the types of stimuli commonly encountered in each modality. It is possible that the bisensory intersection might have been more widespread, with correlated activity levels between modalities, if the texture stimuli had been more similar between modalities: this merits empirical testing in future work. As discussed above for LOC engagement during haptic shape perception, it is uncertain whether MOC recruitment during HT perception depends on top‐down pathways, e.g. those mediating visual imagery, or on bottom‐up somatosensory inputs. Regardless, taken together with previous studies demonstrating cross‐modal recruitment of extrastriate visual cortex during tactile perception (Sathian, 2005), this finding underscores the potentially multisensory nature of much of what is usually termed visual cortex.

Diverging Somatosensory Pathways

In the posterior bank of the central sulcus, where area 3b is located (Geyer et al., 1999, 2000), there was activation during perception of shape as well as texture (relative to baseline), but no differential activity (Fig. 7). Thus, it is likely that this region is involved in both aspects of haptic perception, which fits with the results of lesion studies in monkeys (Randolph and Semmes, 1974). There appeared to be bilateral activity in area 3b, which was considerably stronger on the left than on the right (Fig. 7), in response to stimulation of the right hand. Although the bilateral activity might seem to be at odds with the lack of callosal connectivity of the hand representation in area 3b (Killackey et al., 1983), bilateral hemodynamic responses in 3b have recently been reported following unilateral median nerve stimulation in monkeys; however, the ipsilateral hemodynamic response was shown electrophysiologically to be predominantly inhibitory, in contrast to mainly excitatory contralateral responses (Lipton et al., 2006). The 3b ROIs used to derive the data for this figure were determined on anatomical grounds; hence, the lack of selectivity within area 3b would be worth confirming based on more rigorous localization with respect to cytoarchitectonic maps (Geyer et al., 1999, 2000).

Figure 7.

Time‐courses of BOLD signal changes, during haptic perception, in the posterior bank of the central sulcus (presumptive area 3b), at −40, −20, 54 on the left (contralateral to haptic stimulation) and at 35, −26, 47 on the right (ipsilateral to haptic stimulation). Note lack of selectivity for either shape or texture.

The information streams for HT and shape seem to diverge quite early, right after area 3b, albeit not into completely segregated pathways. The pathway that is specialized for processing haptic texture appears to be directed ventrally into the parietal operculum and posterior insula, whereas the shape‐selective pathway appears to be more dorsally directed, streaming through areas 1 and 2 (postcentral gyrus and sulcus, respectively) into the IPS. Such a dorso‐ventral dichotomy of information flow was hypothesized to be analogous to the dorso‐ventral dichotomy of visual processing, with the dorsal stream involved in processing “where” information and the ventral stream, in processing “what” information (Mishkin, 1979). Relevant physiological studies of this idea in the somatosensory system have been rather limited. One study (Reed et al., 2005) found evidence for partly distinct processing streams for “what” vs. “where,” although most of the differences were outside somatosensory cortex, involving fronto‐cingulate cortex for haptic object recognition and superior parietal cortex for haptically locating objects in external space. Another study (Van Boven et al., 2005) suggested hemispheric specialization within posterior parietal cortex, rather than a dorso‐ventral dichotomy, the left aIPS being preferentially active during discrimination of grating orientation and the right temporo‐parietal junction, during discrimination of small differences in grating location. Since this study employed gratings applied to the passive fingerpad and tasks that were very different from those considered in this report, it is difficult to relate its findings to ours.

The present study suggests that, rather than “what” vs. “where,” preferential processing of shape and texture information might distinguish the divergent somatosensory streams. Alternatively, one could consider the dichotomy in terms of another view of the two visual streams, i.e., dorsal specialization for action and ventral specialization for perception (Goodale and Milner, 1992). In this framework, the dorsally directed pathway could be viewed as concerned with grasping and manipulation, which may be inextricably linked to haptically sensed shape, and the ventral pathway concerned with material object properties such as texture, which are especially the province of touch (Klatzky et al., 1987). This would fit with observations of grasp‐specific activity in the human aIPS region (Frey et al., 2005) and monkey area AIP (Murata et al., 2000), for instance. Much more work is necessary to resolve these issues.

Perception of both texture and shape in the haptic modality also recruit activity in classic visual cortical areas. This activity appears to be fairly clearly segregated, consistent with the notion that multisensory cortical areas are recruited in a highly task‐specific but modality‐independent manner (Prather et al., 2004; Sathian, 2005). Thus, HT perception is associated with activity in early visual cortex, probably in area V2, in a region overlapping with one that is texture‐selective for visual stimuli. Haptic shape perception engages the LOC, a higher‐order visual area. Substantial overlap of haptic and visual shape‐selectivity occurs in this region, and also in the aIPS, pIPS, vIPS and PCS. It bears emphasis that even early visual and somatosensory cortical areas (as early as V2 and Brodmann's area 2 of S1) show multisensory activity, in keeping with findings in early auditory cortex (Schroeder et al., 2003). Of all these multisensory areas, correlations between haptically‐ and visually‐evoked activity levels across subjects suggest the prominent existence of true multisensory processing in the left pIPS and right LOC for object shape. Such modality‐independent processing may be less pronounced, or even absent, in the other regions of multisensory convergence.

CONCLUSIONS

We conclude that there is preferential neural processing of haptic shape and texture. The present report corroborates and elaborates previous suggestions of specialized processing of HT and HS, expands the list of cerebral cortical regions involved in bisensory processing of shape, suggests that modality‐independent processing of shape occurs particularly in the left pIPS and right LOC, and demonstrates for the first time the existence of bisensory texture‐selectivity in the right MOC. The precise nature of processing occurring in the unisensory and bisensory regions activated here can only be resolved by future studies.