Analysis of haptic information in the cerebral cortex

Abstract

Haptic sensing of objects acquires information about a number of properties. This review summarizes current understanding about how these properties are processed in the cerebral cortex of macaques and humans. Nonnoxious somatosensory inputs, after initial processing in primary somatosensory cortex, are partially segregated into different pathways. A ventrally directed pathway carries information about surface texture into parietal opercular cortex and thence to medial occipital cortex. A dorsally directed pathway transmits information regarding the location of features on objects to the intraparietal sulcus and frontal eye fields. Shape processing occurs mainly in the intraparietal sulcus and lateral occipital complex, while orientation processing is distributed across primary somatosensory cortex, the parietal operculum, the anterior intraparietal sulcus, and a parieto-occipital region. For each of these properties, the respective areas outside primary somatosensory cortex also process corresponding visual information and are thus multisensory. Consistent with the distributed neural processing of haptic object properties, tactile spatial acuity depends on interaction between bottom-up tactile inputs and top-down attentional signals in a distributed neural network. Future work should clarify the roles of the various brain regions and how they interact at the network level.

primates, including humans, use the hand to grasp objects not only to act on them but also, importantly, to acquire sensory information that serves both perceptual and motor goals. Termed “haptic,” such sensing covers a variety of object features, including object shape, size, weight, surface texture, compliance, and thermal characteristics (Sathian 1989). It is well known that the visual system gathers information about a host of properties of objects and the environment and parses this information for processing in different visual cortical areas. Similarly, somatosensory information acquired haptically is also subjected to detailed analysis. Although considerably less is understood with regard to the central neural processing of haptic compared with visual sensory information, a good deal of progress has been made over the past few decades. This review focuses on the current state of knowledge about the analysis of haptic inputs in the neocortex, based on neuroimaging studies in humans and neurophysiological studies in nonhuman primates. Gaps in knowledge and opportunities for future work are highlighted. While the focus is on neocortical information processing, the discussion is framed against the background of work in psychophysics and peripheral afferent neurophysiology.

Neocortical Regions Processing Somatosensory Information

The explosion of interest in multisensory processing has led to increasing appreciation that much of sensory cortex is multisensory, even regions traditionally regarded as unisensory (Ghazanfar and Schroeder 2006). Thus, somatosensory information is handled not only in classical somatosensory areas but also outside them (Lacey and Sathian 2015). A number of neocortical regions process nonnoxious haptic inputs, including primary somatosensory cortex (S1), parietal opercular cortex, parts of posterior parietal cortex, and visual cortical areas. Figure 1 illustrates these neocortical areas in the human brain, while Fig. 2 and Fig. 3 diagram their relationship in the context of specific object properties.

Fig. 1.

Locations of major human neocortical areas processing somatosensory information, illustrated on partially inflated views of left cerebral hemisphere (top and middle) or on sagittal slices (bottom; slice position indicated by Talairach x coordinate). 3a, 3b, 1, and 2 refer to Brodmann's areas constituting primary somatosensory cortex. CS, central sulcus; PCS, postcentral sulcus; aIPS, pIPS, vIPS: anterior, posterior, and ventral foci in intraparietal sulcus; MT, human MT complex corresponding to macaque middle temporal visual area; LOC, lateral occipital complex; V6, 6th visual area; MOC, medial occipital cortex.

Fig. 2.

Diagram of haptic information processing for orientation (top) and texture (bottom). Filled arrows, paths for which evidence exists; open arrows, presumptive paths for which evidence is currently lacking; underlined italics, areas known to be multisensory, i.e., visual in addition to somatosensory responsiveness. Although only feedforward paths are shown, feedback pathways are also likely to be important.

Fig. 3.

Model of haptic object representation showing modulation of LOC activity during haptic shape perception by object familiarity and imagery type. Paths shown were demonstrated with multivariate effective connectivity analysis of inferred neuronal time series data obtained by deconvolution of fMRI time series and should not be assumed to be monosynaptic. Reproduced from Lacey et al. (2014) with permission from Elsevier.

Primary somatosensory cortex.

S1, located in the postcentral gyrus, comprises four distinct cytoarchitectonic fields: From anterior to posterior, these are Brodmann's areas (BAs) 3a, 3b, 1, and 2 (Fig. 1). BA 3a is located in the depths of the central sulcus at the transition between the postcentral and precentral gyri, while BA 3b occupies the posterior bank of the central sulcus. BA 1 forms the crown of the postcentral gyrus and continues into BA 2 more posteriorly. In a classic article, Kaas et al. (1979) reported the existence of separate representations of the body in BAs 3b, 1, and 2, and probably 3a as well, in both New World (owl, squirrel, and capuchin) and the evolutionarily more recent Old World (macaque) monkeys. The BA 3a map was later confirmed in macaques (Krubitzer et al. 2004). BAs 3b and 1 responded to cutaneous stimuli, whereas BA 3a appeared to be primarily responsive to deep stimuli, consistent with it being the target of muscle spindle afferents. BA 2 was activated by deep stimuli and was additionally responsive to cutaneous stimuli in macaque but not owl monkeys. A recent report (Kim et al. 2015) challenges the idea of strictly segregated processing of cutaneous and proprioceptive input within the macaque hand representation in BAs 3a, 3b, and 1. In this study, proprioceptive responses were found in all four subfields of S1, being most frequent in BA 3a (72%), least frequent in BA 3b (32%), and intermediate in BAs 1 and 2 (just over 50%). Furthermore, about half of S1 neurons received convergent cutaneous and proprioceptive input from the hand (Kim et al. 2015). This fits with the finding that each subdivision of S1 receives convergent thalamic input from multiple nuclei, including the main ventral posterior nucleus with its well-established lateral and medial subdivisions, the ventral posterior superior and ventral posterior inferior nuclei, the posterior division of the ventral lateral nucleus, and the anterior pulvinar nucleus, while each of these thalamic nuclei provides divergent input to multiple fields of S1 (Padberg et al. 2009). Interestingly, linear summation of cutaneous and proprioceptive inputs is observed at the time of first response in S1 neurons, whereas nonlinear integration of these inputs only emerges 80 ms later, implying involvement of further neural processing (Kim et al. 2015).

Within macaque S1, receptive fields (RFs) tend to increase in size and complexity from anterior to posterior, the direction of information flow, and some neurons in BA 2 have bilateral RFs (Iwamura 1998). In BA 3b, inhibitory responses can be evoked from sites distant from the excitatory RF (Qi et al. 2016) as well as the ipsilateral hand (Lipton et al. 2006). Nonlinear modeling indicates the existence of multiple processes in BA 3b neurons, including an inhibitory RF overlapping with but larger than the excitatory RF on the dominant digit, a temporally delayed inhibition, and both facilitatory and inhibitory modulations from adjacent digits (Thakur et al. 2012). Neuronal responses in BAs 3b and 1 are consistent with convergent input from both slowly adapting type 1 (SA1) and rapidly adapting (RA) afferents (Pei et al. 2009). From the periphery to BA 3b to BA 1, spatial isomorphism of population responses to two-dimensional (2D) stimulus patterns progressively declines (Phillips et al. 1988) and spatiotemporal RFs exhibit progressively increasing surround inhibition and decreasing response linearity (Sripati et al. 2006), indicating that neuronal responses deviate more from simple mirroring of stimuli as one ascends the somatosensory hierarchy.

Parietal opercular cortex and somatosensory cortical information flow.

Parietal opercular cortex lies on the upper bank of the lateral fissure, behind the central sulcus (Fig. 1). This region was long identified as area S2 (reviewed by Burton 1986); additional fields in monkeys were later named the ventral somatosensory area (VS) (Cusick et al. 1989) and the parietal ventral area (PV) (Krubitzer et al. 1995). Each of these three areas contains a complete representation of the body surface. RFs in parietal opercular cortex were found to be quite large and often bilateral (Robinson and Burton 1980), although the subsequent discovery of multiple parietal opercular fields makes it uncertain whether all of them show bilateral responses. The parietal opercular fields S2, VS, and PV of macaques are probably homologous, respectively, to the human parietal opercular fields known as OP1, OP3, and OP4 (Eickhoff et al. 2007).

S2 is generally regarded as a higher-order area than S1: This concept is part of Mishkin's (1979) proposal of a somatosensory processing hierarchy and is supported by anatomical (Friedman et al. 1986) and neurophysiological (Burton et al. 1990; Pons et al. 1987, 1992) data in macaques. However, neurophysiological studies in marmosets (Zhang et al. 2001) and owl monkeys (Nicolelis et al. 1998) favor parallel processing in S1 and S2, rather than serial flow from S1 to S2. In a recent report, analyses of median nerve stimulation-evoked high-frequency gamma oscillations recorded from intracranial depth electrodes in humans undergoing mapping for epilepsy surgery revealed phasic responsiveness in all four S1 subfields; phasic followed by tonic responses in OP1 and tonic responses in OP2, OP3, and OP4 (Avanzini et al. 2016). The phasic responses in S1 and OP1 occurred at the same time (to an accuracy of 10 ms), tending to favor parallel rather than serial processing, although the authors urged caution regarding this conclusion because temporal resolution was no better than 10 ms. This study also suggested serial flow of information from OP1 to the other opercular areas (Avanzini et al. 2016).

Posterior parietal cortex.

Apart from these classical somatosensory cortical areas, somatosensory responses are found in regions of macaque posterior parietal cortex that are also visually responsive and are thus regarded as multisensory, including BAs 5 (Iwamura 1998) and 7b (Dong et al. 1994), and a number of regions in and around the intraparietal sulcus (IPS) (Grefkes and Fink 2005). Cooling BAs 5 and 7b modified neuronal responses, including RF sizes, in BAs 1 and 2 (Cooke et al. 2014; Goldring et al. 2014), implying the existence of feedback signals to S1 from posterior parietal areas. Although homology between macaque and human parietal cortical regions remains presumptive, probable human counterparts of many of the macaque IPS regions have been identified (Grefkes and Fink 2005), including the anterior, medial, lateral, ventral, and caudal (posterior) intraparietal areas (AIP, MIP, LIP, VIP, and CIP, respectively). Some of these areas are shown in Fig. 1. Multisensory responses of VIP have been well characterized: In macaques, VIP neurons demonstrate multisensory (visuotactile) integration sensitive to spatial and temporal congruency effects (Avillac et al. 2007), and human VIP contains a somatosensory face map aligned with a near-face visual space map (Sereno and Huang 2006). The remaining intraparietal areas have been implicated in visuomotor responses in both monkeys and humans (Grefkes and Fink 2005).

Somatosensory pathways and visuotactile convergence.

Lesions of BA 3b in macaques result in nonspecific sensory loss, affecting perception of both surface roughness and object form (Randolph and Semmes 1974). Thus, Kaas (1983) has argued that the term “S1” should really have been restricted to BA 3b, although common usage has not heeded this plea. In contrast to the nonselective effect of lesions of BA 3b, more posterior lesions within S1 cause selective deficits: Lesions of BA 1 impair perception of surface roughness, whereas lesions of BA 2 impair perception of object form (Randolph and Semmes 1974). These early studies suggested that, as in the visual system, somatosensory information is also parsed into its constituent functional components, these different components being processed in different neocortical areas. The divergence of visual information flow into dorsally and ventrally directed pathways concerned primarily with spatial and object form processing, respectively (Haxby et al. 1994; Ungerleider and Mishkin 1982), is well known. A similar organization of pathways has been shown for auditory information (Alain et al. 2001; Arnott et al. 2004; Rauschecker and Tian 2000). Analogously, work using functional magnetic resonance imaging (fMRI) in humans has demonstrated segregation of somatosensory information flow into two pathways, one dorsally directed and the other ventrally directed. Human neuroimaging studies suggest that the ventral somatosensory pathway, which flows into parietal opercular cortex as originally identified by Mishkin (1979), seems to be mainly concerned with texture (Kitada et al. 2005; Ledberg et al. 1995; Roland et al. 1998; Sathian et al. 2011; Simões-Franklin et al. 2011; Stilla and Sathian 2008) rather than form, the focus of the ventral visual and auditory pathways, while the dorsal somatosensory pathway, specialized for spatial relations, converges with the similarly specialized dorsal visual pathway in the frontal eye fields (FEFs) and IPS (Sathian et al. 2011).

A recent fMRI study in macaques (Guipponi et al. 2015) revealed extensive visuotactile convergence in multiple cortical areas, including not only posterior parietal areas accepted as multisensory such as VIP, posterior IPS, and 7b but also regions classically considered unisensory, in both somatosensory cortex (BA 2 and parietal opercular cortex) and visual cortex (V1, V2, V3, V3A, and MST). Human MST is also responsive to vibrotactile stimuli (Beauchamp et al. 2007), and the intracranial study (Avanzini et al. 2016) cited above found evidence of somatosensory input to the right MT complex. Neuroimaging studies of a variety of tasks have revealed somatosensory responses in multiple visual cortical areas, as outlined below in this review. The subsequent sections review the current state of knowledge on processing of information about specific somatosensory attributes.

Object Form

A number of aspects of object form have been studied experimentally, including shape, size, and orientation.

Orientation.

psychophysics and peripheral neural coding.

The Nobel Prize-winning work of Hubel and Wiesel (1977) called attention to the occurrence of orientation selectivity in neurons of primary visual cortex (V1); it soon became accepted that edge orientation is an important shape “primitive” (Marr 1982). In touch, orientation discrimination thresholds at the fingertip are ∼20° compared to visual thresholds of ∼4° for similar stimuli; the poorer orientation acuity in touch relative to vision contrasts with spatial acuity thresholds that are comparable in both senses when expressed in terms of receptor spacing (Bensmaia et al. 2008b). Phillips and Johnson (1981a) showed that SAI or Merkel afferents are highly edge selective, but orientation selectivity was thought to be poor at the level of single afferents, emerging only in the spatial profile of the population response, as reviewed by Hsiao et al. (2002). However, sensitivity to edge orientation has recently been reported in the responses of both Merkel (SAI) and Meissner (RA) afferents (also known as fast-adapting type I or FAI afferents) in humans, due to the pattern of distribution of receptors innervated by branches of a single axon (Pruszynski and Johansson 2014). If and how the CNS takes advantage of such peripheral orientation sensitivity remains uncertain.

Whereas studies of tactile orientation in macaques have relied on the use of bar stimuli, tactile discrimination of stimulus orientation in humans has most frequently been studied with gratings comprising alternating ridges and grooves. Although the grating orientation task was devised to measure tactile spatial acuity (discussed below in this review), its use in human neuroimaging studies, along with suitable control tasks, has provided insight into selective cortical processing of information about tactile orientation.

primary somatosensory cortex.

Bensmaia et al. (2008a) reported on orientation selectivity in macaque S1. They found that about half the neurons studied in BA 3b and nearly two-thirds in BA 1 were orientation selective for indented bar stimuli. In contrast, orientation selectivity was found in only about one-eighth of BA 2 neurons, whose responses tended to be better described by curvature tuning models, as described below (Yau et al. 2013). Estimated “neurometric” thresholds for orientation discrimination were substantially lower than the corresponding psychometric thresholds (20°, see above), being about 8° in BA 3b and 14° in BA 1 for indented bars (Bensmaia et al. 2008a). Simple linear summation of responses to collinear RFs accounts for progressively less variance as one moves from considering peripheral afferents to BA 3b neurons to BA 1 neurons (Bensmaia et al. 2008a). In humans, S1 activity during discrimination of grating orientation, compared with control tasks, appears to be right-lateralized (Kitada et al. 2006; Zhang et al. 2005); this activity is mainly in the postcentral sulcus (PCS), which probably corresponds to BA 2 (Grefkes et al. 2001).

parietal opercular and posterior parietal cortex.

Orientation selectivity was found in neurons in three separate parietal opercular areas of the macaque, identified as central, anterior, and posterior fields (Fitzgerald et al. 2006); these fields probably correspond, respectively, to the areas called PV, S2, and VS (Eickhoff et al. 2007; Fitzgerald et al. 2004). Nearly a quarter of the neurons sampled across these three areas exhibited orientation selectivity (Fitzgerald et al. 2006), which for some neurons was position invariant within the RF, leading to the suggestion that such neurons may contribute to the perception of object shape and size (Fitzgerald et al. 2006; Thakur et al. 2006). Neuronal tuning for orientation was similar in both S1 and parietal opercular regions to that classically described in V1 (Bensmaia et al. 2008a; Fitzgerald et al. 2006). Orientation selectivity has not been reported in human parietal opercular cortex. However, selective activation during tactile discrimination of grating orientation was observed in the anterior part of the IPS in humans (Kitada et al. 2006; Van Boven et al. 2005; Zhang et al. 2005); this region was found to also respond during visual orientation discrimination (Kitada et al. 2006; Shikata et al. 2001, 2008), as was a caudal region of the IPS (Shikata et al. 2001, 2008). Neuronal responsiveness to tactile orientation does not appear to have been examined in macaque IPS.

visual cortex.

Our functional imaging studies in humans using both fMRI and positron emission tomography (PET) identified a parieto-occipital focus that was preferentially active during tactile discrimination of grating orientation compared with discrimination of grating spacing (Sathian et al. 1997; Zhang et al. 2005). This focus had previously been shown to be selectively active during visual discrimination of grating orientation (Sergent et al. 1992); its location near the parieto-occipital fissure suggests it may be the human homolog of macaque area V6 (Pitzalis et al. 2006) (Fig. 1). Notably, about two-thirds of neurons in this area of macaques are orientation selective, which is about twice the proportion in V1 (Galletti et al. 1991). Our PET study (Sathian et al. 1997) was the first demonstration of activity in a putative visual cortical area during a purely tactile task. As will become clear from other examples in this article, such recruitment of visual cortical areas during corresponding tactile conditions is now widely accepted. Furthermore, work from our laboratory showed that disrupting the function of the parieto-occipital focus with transcranial magnetic stimulation (TMS) impaired tactile discrimination of grating orientation, but not grating spacing, indicating that the observed “noncanonical” activity of visual cortex is functionally meaningful (Zangaladze et al. 1999). One possible explanation for such visual cortical recruitment is visual imagery generated by top-down input (Sathian and Zangaladze 2001). It would be interesting to test whether similar orientation selectivity for tactile stimuli occurs in macaque visual cortex, but this has not so far been done, with one notable exception: Neurons in macaque V4 were selective for the orientation of a tactile grating, but only when it served as a matching cue for a subsequent visual stimulus (Haenny et al. 1988). The absence of V4 responses when the tactile stimulus was irrelevant implicates top-down factors in the responses.

summary.

Figure 2, top, summarizes these findings, indicating that tactile orientation sensitivity is distributed across S1, parietal opercular cortex, anterior IPS, and V6, although the precise contributions of each area and the presumed flow of orientation information have not yet been established.

Shape and size.

psychophysics and peripheral neural coding.

Human psychophysical studies indicate that haptic assessment of three-dimensional (3D) object size relies on both cutaneous (SAI and RA) and proprioceptive afferents (Berryman et al. 2006); presumably, this also holds for 3D object shape, although this remains to be formally tested. A key parameter for object shape is curvature, since the distribution of curvature across an object completely specifies its shape (LaMotte and Srinivasan 1987a). Humans can discriminate curvature at the fingerpad using only cutaneous receptors, i.e., with the fingers immobilized (Goodwin and Wheat 1992; LaMotte and Srinivasan 1987a; Srinivasan and LaMotte 1987). Responses of SAI afferents encode object curvature during static contact (LaMotte and Srinivasan 1987a; Srinivasan and LaMotte 1987); RAs also contribute during dynamic phases of contact (LaMotte and Srinivasan 1987b; Srinivasan and LaMotte 1987). The SAI afferent population response profile reflects stimulus curvature, in both monkeys (Goodwin et al. 1995) and humans (Goodwin et al. 1997).

primary somatosensory cortex.

As mentioned above, ablation of BA 2 in macaques produced deficits in form perception, specifically affecting judgments of size and shape (Randolph and Semmes 1974). Consistent with these observations, shape-selective neurons (Koch and Fuster 1989) were reported in macaque BA 2, and nearly a quarter of neurons examined in BA 2 demonstrated tuning for curvature direction (Yau et al. 2013). Haptic shape perception requires integration of cutaneous and proprioceptive inputs, which, as discussed above, occurs in neurons in all four subfields of S1 (Kim et al. 2015). A human fMRI study (Stilla and Sathian 2008) also found shape-selective activation in the PCS (BA 2), for both haptic and visual stimuli. In human PET studies, the same part of S1 (the PCS) was preferentially responsive during curvature discrimination (Bodegård et al. 2000, 2001) and was also active during discrimination of the length of cuboidal objects (O'Sullivan et al. 1994).

parietal opercular cortex.

In macaques, lesions of parietal opercular cortex were reported to result in nonselective impairment of object size, shape, roughness, and hardness (Murray and Mishkin 1984). Neuronal selectivity for curvature direction in macaque parietal opercular cortex (the precise subfield is unclear) was found in about a sixth of tested neurons (Yau et al. 2009), with tuning properties and temporal dynamics similar to those in BA 2 (Yau et al. 2013). Direct comparisons of curvature direction tuning properties between parietal opercular cortex and visual area V4 revealed them to also be quite similar, although such tuning was much more frequent in V4, where it was found in two-thirds of the neurons recorded (Yau et al. 2009).

In humans, parietal opercular lesions were associated with impaired form but not roughness discrimination (Roland 1987) or with tactile agnosia (Caselli 1983; Reed et al. 1996), i.e., inability to recognize shapes haptically despite otherwise intact somatosensory function. Consistent with these results, some human neuroimaging studies have indeed reported activity associated with haptic shape perception in the parietal operculum (Ledberg et al. 1995; Reed et al. 2004), although the activity did not appear to be shape specific (Ledberg et al. 1995). However, most human neuroimaging studies have found shape-selective activity in neocortical regions other than in the parietal operculum.

posterior parietal cortex.

Although lesions restricted to BA 5 (the superior parietal lobule) in macaques did not impair discrimination of shape or size (Murray and Mishkin 1984), neurons exhibiting selectivity for object shape or size were found in BA 5 (Gardner et al. 2007; Koch and Fuster 1989) and in AIP (Murata et al. 2000). In humans, cortex around the IPS is recruited selectively during haptic shape perception at a number of sites: anteriorly (Roland et al. 1998; Stilla and Sathian 2008), posteriorly (Jäncke et al. 2001; Stilla and Sathian 2008; Van de Winckel et al. 2005), and ventrally (Stilla and Sathian 2008); all of these IPS sites (Fig. 1) were bisensory, being shape selective in both vision and touch (Stilla and Sathian 2008). The anterior IPS region was also active during object length discrimination (Bodegård et al. 2001; Roland et al. 1998). One question raised by the findings of haptic and visual shape selectivity in and around the IPS is why these regions, in the dorsal visual pathway, are active during form perception tasks, generally regarded as dependent on the ventral stream. The answer may lie in our observation that these same regions are selectively activated during both haptic and visual perception in a task that can be thought of as locating a specific feature on an object (Sathian et al. 2011). Thus, IPS activity during discrimination of shape may be associated with processes involved in assembling a global representation of shape from component parts; these processes are particularly important for unfamiliar objects and may be facilitated by spatial imagery (Lacey et al. 2009b, 2014; Lacey and Sathian 2015) (Fig. 3).

visual cortex and multisensory representation of shape.

The lateral occipital complex (LOC; Fig. 1), originally identified as a visual object-selective area presumably homologous to macaque inferotemporal cortex (Malach et al. 1995), is a region in occipitotemporal cortex that has been shown in multiple human fMRI studies to be shape selective for both visual and haptic 3D stimuli (Amedi et al. 2001, 2002; James et al. 2002; Stilla and Sathian 2008). It is also active during tactile perception of 2D shapes (Prather et al. 2004; Stoesz et al. 2003). Object-selective regions of macaque visual cortex have not so far been examined for haptic responsiveness, highlighting a topic that would lend itself to profitable study. While the LOC is not recruited by object-specific sounds used to cue auditory object recognition (Amedi et al. 2002), discrimination of object shape based on material-specific sound cues does activate this region (James et al. 2011), as does recognition by trained individuals of auditory objects produced by transformation of visual images according to a specific algorithm (Amedi et al. 2007). A nearby area, the fusiform gyrus, is sensitive to semantic congruency effects between haptic objects and their associated sounds (Kassuba et al. 2013b).

Taken together, the findings reviewed above on the LOC point to its specialization for geometric representations of shape. Although earlier studies of patients with lesions presumed to involve the LOC reported poor recognition or learning of haptic shape (Feinberg et al. 1986; James et al. 2006, 2007), implying that this region is necessary for optimal haptic object perception, this view has been challenged by a recent finding that a patient with bilateral occipitotemporal lesions including the LOC was unimpaired on haptic object recognition despite severe deficits of visual object recognition (Snow et al. 2015). This raises the question of what exactly the role of LOC is—this issue is treated next.

The basis for recruitment of the LOC during haptic shape perception has been explored in some detail. An electroencephalographic (EEG) study in humans using scalp electrodes during tactile discrimination of shape showed that activity propagates from S1 into LOC as early as 150 ms after stimulus onset (Lucan et al. 2010). Human fMRI studies from our laboratory suggest a role for visual imagery: In one study, ratings of the vividness of visual imagery correlated with the magnitude of haptic shape selectivity in the LOC (Zhang et al. 2004). Analysis of activation and effective connectivity profiles in a set of studies (Deshpande et al. 2010; Lacey et al. 2010, 2014) led to the proposal of a conceptual model (Lacey et al. 2009b, 2014; Lacey and Sathian 2015) (Fig. 3), in which LOC activity is regarded as instantiating a modality-independent representation of object shape that can be flexibly accessed either bottom-up via inputs from primary somatosensory cortex or top-down via inputs from prefrontal cortex. As revealed by effective connectivity analyses (Deshpande et al. 2010; Lacey et al. 2014), the top-down route is more important for haptic shape perception of familiar objects and shares many paths with the visual object imagery network, whereas the bottom-up pathway is relied on in the case of haptic shape perception of unfamiliar objects—this circuit shares many paths with the spatial imagery network. These studies fit with the concepts that visual object imagery aids haptic perception of the shape of familiar objects while spatial imagery processes in the IPS, as outlined above, facilitate assembly of global shape representations of unfamiliar objects from their component parts. Note that the paths in this model should not be construed as being monosynaptic; details of the connections remain to be worked out. Clarifying the precise functions and interactions of all these different brain regions in haptic shape perception is fertile ground for future work, but for now it seems safe to state that LOC is probably involved both in multisensory representation of objects and in visual imagery processes that may facilitate nonvisual object recognition.

Cross-modal visuo-haptic recognition, while being somewhat less accurate than within-modal recognition, is view independent in contrast to the unisensory view dependence for unfamiliar objects found in both vision and touch (Lacey et al. 2007). Perceptual learning leading to view independence in one modality resulted in view independence in the other modality without additional training, implying that cross-modal information transfer occurs at the stage of view-independent representations (Lacey et al. 2009a), which are thought to emerge from convergence of lower-level, view-dependent representations (Riesenhuber and Poggio 1999). The LOC, fusiform gyrus, and anterior IPS were found to be active during cross-modal shape matching when haptic targets followed visual samples, especially when the shapes were congruent between modalities; such activity was less when visual targets followed haptic samples or for within-modal matching (Kassuba et al. 2013a). Thus, the LOC and the fusiform gyrus are candidate regions for housing shape representations that are both modality- and view independent; IPS regions are less likely candidates because of their involvement in orientation signaling (discussed above).

Surface Texture

Psychophysics and peripheral neural coding.

Surface texture is an object property that can be judged through multiple senses: touch, vision, and hearing. Haptic sensing is primary in the domain of texture, which is weighted more heavily in touch than vision (Klatzky et al. 1987); furthermore, discrimination of fine textures is more acute with touch than with vision (Heller 1989), and touch tends to dominate vision when texture is to be judged (Guest and Spence 2003). Texture is a multidimensional property: The main psychophysical dimensions emerging from multidimensional scaling studies in humans include rough-smooth, hard-soft (compliance), and, at least in some individuals, sticky-slippery (Hollins et al. 1993, 2000). Most experimental work has focused on the rough-smooth dimension. Auditory cues can modulate the tactile perception of both roughness and wetness (Guest et al. 2002).

Peripheral neural coding of roughness has been studied extensively and reviewed repeatedly (e.g., Bensmaia 2009; Johnson and Hsiao 1992; Sathian 1989). Suffice it here to note that the current consensus favors dual coding mechanisms (Bensmaia 2009; Hollins and Risner 2000; Weber et al. 2013). One is a spatial coding mechanism based on the spatial profile of the population of active afferents, particularly SAI and, to a lesser extent, RA afferents, with stronger firing for given afferents when the spacing between texture elements (e.g., bars or dots) passing over their receptive fields is greater (Bensmaia 2009; Weber et al. 2013). Such a population-level spatial code is similar to that mediating sensing of tactile orientation and curvature, outlined above in this article. The second mechanism involves temporal coding, which as originally postulated by Katz, probably depends on sensing of vibrations evoked by moving patterns (Hollins and Risner 2000). This code is important for very fine textures (Gamzu and Ahissar 2001; Weber et al. 2013) and also for textures where the variability is determined not by interelement spacing but by element width (Cascio and Sathian 2001; Goodwin et al. 1989; Sathian et al. 1989). Softness judgments can be made solely on the basis of cutaneous cues (Srinivasan and LaMotte 1995) and probably depend on SAI afferents (Srinivasan and LaMotte 1996).

Neocortical processing.

Reference has already been made to the effects of lesions in macaques: Ablation of BA 1 impaired judgments of roughness but not object form, whereas ablation of BA 3b nonselectively affected perception of both roughness and form (Randolph and Semmes 1974). Parietal opercular cortical lesions led to nonselective deficits in judging object size, shape, roughness, and hardness, while BA 5 lesions caused only moderate elevations of roughness thresholds (Murray and Mishkin 1984). As reviewed by Bensmaia (2009), variations in spatial features of surfaces are reflected in neuronal responses in both S1 and parietal opercular cortex (the precise subfields are not clear) of macaques. Furthermore, the spatial and temporal response patterns implicated by peripheral neurophysiologic studies in texture coding (see above) appear to be encoded by separate populations of S1 neurons (Bensmaia 2009). Similar details are not yet available for neurons of parietal opercular cortex.

Early PET studies in humans suggested that surface roughness and object form coactivated cortical regions in S1 (O'Sullivan et al. 1994) or the parietal operculum (Ledberg et al. 1995). An fMRI study also similarly reported that roughness, hardness, and shape judgments all activated S1 (Servos et al. 2001). However, another PET study (Roland et al. 1998) found roughness-selective activity in the parietal operculum, as did multiple fMRI studies (Kitada et al. 2005; Sathian et al. 2011; Simões-Franklin et al. 2011; Stilla and Sathian 2008). All three human parietal opercular somatosensory fields (OP1, OP3, and OP4) are texture selective (Sathian et al. 2011; Stilla and Sathian 2008). In addition, haptic judgments of roughness also recruit activity in early visual cortical areas located in medial occipital cortex (Eck at al. 2013; Sathian et al. 2011; Stilla and Sathian 2008) (see Fig. 1), although the exact areas involved differed somewhat between studies, and whether visual imagery is involved, or whether these activations indicate modality-independent properties of visual cortex, has not been examined as it has for haptic shape perception (see above). One study found that haptic judgments of the texture of unfamiliar 3D objects elicited activity in medial occipitotemporal cortex in the posterior collateral sulcus, adjacent to, but not overlapping with, visual texture-responsive regions (Podrebarac et al. 2014); the textures in this study were of a relatively coarse grain compared with those in the studies reporting haptic texture-selective activity in early visual cortex (Eck at al. 2013; Sathian et al. 2011; Stilla and Sathian 2008). When roughness information matched across visual and haptic modalities, early visual cortical responses were strengthened (Eck at al. 2013). Effective connectivity analyses demonstrated information flow from nonselective origins in S1 to haptically texture-selective parietal opercular cortex, and then onto bimodally texture-selective early visual cortex (Sathian et al. 2011) (Fig. 2). As already mentioned, the flow from S1 to the parietal operculum corresponds to the ventral somatosensory pathway identified by Mishkin (1979) in monkeys. The precise contributions of each of the neocortical areas to haptic texture perception remain to be ascertained.

Interestingly, we showed that parietal opercular texture-selective somatosensory cortex is recruited when listening to sentences containing textural metaphors (e.g., “He had a rough day”) (Lacey et al. 2012), consistent with theoretical accounts arguing that metaphors and other abstract concepts are grounded in sensorimotor experience (Barsalou 2008; Lakoff and Johnson 1980). Visual cortical texture-selective activity was not recruited in this study (Lacey et al. 2012), reinforcing the primacy of touch for texture perception.

Tactile Spatial Acuity

Psychophysics and peripheral neural coding.

As mentioned above, gratings comprising alternating ridges and grooves are commonly used tactile stimuli. Owing to skin mechanics, the skin deforms more into grooves that are wider, thereby evoking more activity in the peripheral afferents, particularly of the SAI type (Phillips and Johnson 1981a, 1981b). Thus, tactile spatial acuity can be indexed by the groove width corresponding to 75% correct discrimination of grating orientation, in a two-alternative forced choice in which gratings are presented aligned either along or across the long axis of the finger (Sathian and Zangaladze 1996; Van Boven and Johnson 1994). It turns out that tactile acuity as measured with the grating orientation test is inversely related to finger size, i.e., smaller fingers are associated with better acuity (because of higher receptor density), which may explain the well-known superiority of women to men on tactile acuity (Peters et al. 2009). Blindness enhances tactile spatial acuity at the fingers but not the lips, and Braille reading experience correlates with performance on the acuity task: These findings imply that experience-dependent plasticity drives the observed improvement in spatial acuity (Wong et al. 2011).

Neocortical processing.

Human neuroimaging studies using this task, by contrasting the associated activity with that associated with a control task, e.g., involving discriminating the spacing between grating ridges, have been useful to study the neocortical processing of tactile stimulus orientation (see above). The reverse contrast showed that spacing judgments activated the angular gyrus (Sathian et al. 1997; Zhang et al. 2005); a similar task using dot patterns activated the IPS (Merabet et al. 2007). Although a TMS study demonstrated a double dissociation, with TMS over S1 disrupting roughness judgments and TMS over occipital cortex disrupting spacing judgments of dot patterns (Merabet et al. 2004), this dissociation was not found in a corresponding fMRI study (Merabet et al. 2007).

An fMRI study from our laboratory (Stilla et al. 2007) used a different acuity task in which the central dot in a three-dot linear array was displaced to the left or right, with acuity being indexed by the displacement corresponding to 75% correct left-right discrimination. In this task, activity at right posteromedial parietal cortical foci [in the posterior IPS (pIPS) and precuneus] correlated, across participants, with acuity on the right index finger. Furthermore, the weights of two paths targeting the right pIPS, one from the left PCS (BA2) and the other from the right FEF, correlated with both acuity and right pIPS activation magnitude. Our interpretation was that fine tactile spatial discrimination, of the sort used in Braille reading, involves interaction in the pIPS between a top-down attentional signal and bottom-up input from S1 (Stilla et al. 2007). In a human EEG study using the three-dot acuity task (Adhikari et al. 2014), again on the right index finger, we found distinct oscillatory networks in the beta and gamma bands. The beta-band oscillations (peaking around 15 Hz) involved a network comprising left S1, right pIPS, right LOC, and left dorsolateral prefrontal cortex (dlPFC), with predominantly feedforward flow from S1, whereas the gamma-band network (peaking around 80 Hz) formed a recurrent loop from dlPFC to pIPS to S1 and back to dlPFC. Beta-band drive from pIPS to dlPFC and gamma-band drive from dlPFC to pIPS and S1 to dlPFC correlated with accuracy on the task. As in the corresponding fMRI study, these findings suggest interaction of bottom-up sensory flow with top-down attentional control (Adhikari et al. 2014).

An fMRI study of perceptual learning using the three-dot acuity task (Sathian et al. 2013) showed that learning-specific increases in activation and effective connectivity occurred principally in subcortical and anterior neocortical regions implicated in decision processes, consistent with the idea that the relevant neuroplasticity relates to reweighting of perceptual readout by decision processes. Blindness resulted in a different kind of reorganization, with activation magnitudes in left S1 and multiple early visual cortical foci bilaterally and in the right LOC correlating with acuity, as did the weights of paths from the left FEF to the left pIPS and from the left PCS to the right LOC (Stilla et al. 2008). These findings imply that both within-modal and cross-modal plasticity are involved in the reorganization that follows visual deprivation.

Other Properties

A few other object properties have been studied experimentally in relation to cortical neurophysiology, but relatively little is still known about them. One such property is tactile motion. Neurons tuned to the direction of motion of 2D stimulus patterns, irrespective of their precise form, speed, or contact force, have been reported in macaque BA1 (Pei et al. 2010). Such neurons integrate responses to component patterns in a manner analogous to that found for visual motion signals in the visual motion-selective (middle temporal) area known as MT (Pei et al. 2011), and their responses match human psychophysical capabilities assessed with the same stimuli (Pei et al. 2010, 2011). Tactile motion aftereffects transfer between vision and touch (Konkle et al. 2009), and the direction of tactile motion can disambiguate the direction of simultaneously experienced visual motion whose direction is ambiguous when vision is employed alone (Blake et al. 2004), suggesting that there may be a shared neural representation of motion that is modality independent. Tactile motion also activates the human MT complex (Fig. 1) of visual motion-selective areas (Blake et al. 2004; Hagen et al. 2002; Ricciardi et al. 2007; Summers et al. 2009), and blindness leads to more extensive activation (Ricciardi et al. 2007). Although a recent report employing localization of visual motion-sensitive areas in individual sighted participants found minimal cross-modal activation by tactile motion stimuli in the absence of a motion task (Jiang et al. 2015) and suggested that previous studies may have produced spurious cross-modal activations, multivariate pattern analyses during motion discrimination were able to distinguish motion direction for both tactile and visual stimuli in human MT (van Kemenade et al. 2014). However, the patterns differed between modalities, suggesting involvement of distinct neuronal pools and also arguing against mediation of the tactile responses by visual imagery (van Kemenade et al. 2014), although it should be pointed out that the issue of visual imagery has not been explicitly examined in this context. Moreover, TMS over the human MT complex impaired detection of changes in the speed of tactile motion (Basso et al. 2012). Thus, the weight of evidence points to involvement of the MT complex in tactile motion perception.

Object weight is another property that has been studied to a limited extent. Some neurons in macaque BA 5 were sensitive to object weight (Gardner et al. 2007). In a human fMRI study of the size-weight illusion, primary motor cortex was found to encode actual weight while ventral premotor cortex responses reflected the perceived weight due to biasing by object size (Chouinard et al. 2009).

Finally, the property of object or stimulus location has been approached in a few different ways in human fMRI studies. A task that required locating objects in near peripersonal space recruited activity in superior parietal cortex (Reed et al. 2005), while another study reported common activity in both superior and inferior parietal cortex during location of auditory stimuli in extrapersonal space and of tactile stimuli among different fingers (Renier et al. 2009). Discrimination of grating location on the fingerpad evoked activity in the right temporoparietal junction, regardless of which hand was stimulated (Van Boven et al. 2005). Work from our laboratory (Sathian et al. 2011) used a task in which a palpable bump had to be localized on a card, analogous to finding a specific feature on an object. This task and its visual counterpart activated a bisensory set of regions in the FEFs and parts of the IPS bilaterally, and effective connectivity analyses were consistent with flow of information into these spatially selective areas from nonselective activity in S1 via a dorsally directed pathway (Sathian et al. 2011).

Conclusions

It should be clear from this review that analytic segregation of the neural processing of haptic information occurs in a manner similar to that of visual information, although there are still considerable gaps in our understanding to be filled by further research. Nonnoxious somatosensory inputs are initially processed in S1 and are then partially segregated into different pathways, with the clearest segregation between pathways for texture and location. Haptic information about surface texture, which is particularly salient to touch, flows ventrally into parietal opercular cortex and thence to early visual cortex, where it converges with visual texture information. Haptic information regarding the location of features on objects converges along with corresponding visual information in a dorsal pathway comprising the IPS and FEFs. Haptic processing of shape occurs mainly in the IPS and LOC, while that of orientation is distributed across S1, the parietal operculum, anterior IPS, and V6. Tactile spatial acuity depends on interaction between bottom-up tactile inputs and top-down attentional signals in a distributed neural network. Sorting out the roles of the various brain regions involved in the perception of stimulus properties at large and small scales, and how these regions work together in concert at a network level, remain challenges for the future.

GRANTS

The author's work in preparing this article was supported by the Department of Veterans Affairs and by grants from the National Eye Institute and the National Science Foundation.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

K.S. conception and design of research; K.S. analyzed data; K.S. interpreted results of experiments; K.S. prepared figures; K.S. drafted manuscript; K.S. edited and revised manuscript; K.S. approved final version of manuscript.