Feeling form: the neural basis of haptic shape perception

Abstract

The tactile perception of the shape of objects critically guides our ability to interact with them. In this review, we describe how shape information is processed as it ascends the somatosensory neuraxis of primates. At the somatosensory periphery, spatial form is represented in the spatial patterns of activation evoked across populations of mechanoreceptive afferents. In the cerebral cortex, neurons respond selectively to particular spatial features, like orientation and curvature. While feature selectivity of neurons in the earlier processing stages can be understood in terms of linear receptive field models, higher order somatosensory neurons exhibit nonlinear response properties that result in tuning for more complex geometrical features. In fact, tactile shape processing bears remarkable analogies to its visual counterpart and the two may rely on shared neural circuitry. Furthermore, one of the unique aspects of primate somatosensation is that it contains a deformable sensory sheet. Because the relative positions of cutaneous mechanoreceptors depend on the conformation of the hand, the haptic perception of three-dimensional objects requires the integration of cutaneous and proprioceptive signals, an integration that is observed throughout somatosensory cortex.

the hand is a remarkable organ that allows us to interact with objects with unmatched versatility and dexterity. Our ability to grasp and manipulate objects relies not only on sophisticated neural circuits that drive muscles to move the hand but also on neural signals from the hand that provide feedback about the consequences of those movements (Johansson and Flanagan 2009). Individuals deprived of this sensory information struggle to perform even the most basic activities of daily living, like picking up a coffee cup or using a spoon (Sainburg et al. 1995). More complex tasks, like tying shoes or buttoning a shirt, are out of the question.

Neural signals from the hand convey information about the shape, size, and texture of an object; if the object moves across the skin, information about its speed and direction of this motion is also conveyed. Discriminative touch signals from the hand are transmitted to the cerebral cortex along an ascending pathway that includes synapses in the brainstem and thalamus. Little is known about the coding properties of these two structures. Ultimately, however, touch signals project to primary somatosensory cortex, where behaviorally relevant features about the object—its shape, size, texture, and motion—are encoded in the responses of individual neurons.

In the present review, we discuss the neural basis of haptic perception with the hands in human and non-human primates. We track neural signals about objects from their genesis in the skin through their ascension up the somatosensory neuraxis, focusing on signals that convey information about the shape of objects, defined in a geometric sense. Shape information is available at multiple spatial scales. Signals from individual fingerpads convey information about local shape, for example, the presence and configuration of edges or bumps. Braille reading is a canonical example of our ability to recognize two-dimensional (2D) spatial patterns, in this case embossed dots in different configurations, with high speed and precision. Much of our understanding of tactile shape processing comes from studies of the neural coding of local shape information. Of all sensory modalities, however, somatosensation is unique in that it contains a deformable sensory sheet: mechanoreceptors move relative to one another as hand posture changes (Hsiao 2008). This flexible sensory sheet allows us to perceive the three-dimensional (3D) structure of objects that are large enough to contact the hand at multiple locations simultaneously. Less is known about the coding of 3D “global” object shape, but synthesis of 3D shape representations likely requires integrating proprioceptive signals about the configuration of the hand with cutaneous signals stemming from the different contact points with an object. We summarize the recent progress in identifying the neural interactions between cutaneous and proprioceptive signaling for shape processing. Not only are tactile signals about shape critical to our ability to interact with objects, they also allow us to identify objects by touch when vision is unavailable. It turns out that the neural mechanisms that mediate shape processing in touch are remarkably analogous to their counterparts in vision, a theme that will be developed throughout this review.

Shape Coding at the Somatosensory Periphery

Forces applied to the skin's surface propagate through the tissue to produce stresses and strains on mechanoreceptors embedded in the skin (Phillips and Johnson 1981b; Sripati et al. 2006a). In the glabrous skin, which covers the palmar surface of the hand, four classes of mechanoreceptors each respond to different aspects of skin deformations and send projections to the brain via large-diameter nerve fibers, each with its own designation (Bensmaia and Manfredi 2012): slowly adapting type 1 (SA1) afferents, which innervate Merkel cell receptors, respond to static indentations or slowly moving stimuli; Ruffini corpuscles, innervated by slowly adapting type 2 (SA2) fibers, respond to skin stretch; Meissner corpuscles, associated with rapidly adapting (RA) fibers, respond to low-frequency skin vibrations and movements across the skin, and Pacinian corpuscles (PC), associated with PC fibers, respond to high-frequency vibrations and finely textured surfaces. The designation slowly vs. rapidly adapting refers to the responses of these afferents to static indentations: SA1 and SA2 fibers respond throughout a skin indentation whereas RA and PC fibers only produce a transient response at the onset and offset of the indentation. Importantly, however, all receptor types respond during typical interactions with most objects (Saal and Bensmaia 2014), and information about the different features of an object—size, shape, texture, motion, etc.—is multiplexed in these responses.

To understand mechanosensory transduction and afferent responses to indented spatial patterns, one must take skin mechanics into account. The forces applied on the skin when spatial patterns are indented into it must therefore propagate through the tissue before they deform and excite mechanoreceptors in the skin (Dandekar et al. 2003; Phillips and Johnson 1981b; Sripati et al. 2006a). The resulting spatial pattern of mechanoreceptor activation is thus a somewhat distorted version of the spatial pattern at the skin's surface. This transformation is well approximated by models that first convert the displacement profile of a spatial pattern indented into the skin into a pattern of stresses and strains at the location of the receptor (Sripati et al. 2006a). The impact of a tactile stimulus on any individual mechanoreceptor thus depends both on the physical properties of the skin itself, such as its mass, stiffness, and resistance, and characteristics of the receptor, such as its location in the tissue and the spatial layout of its transduction sites. Ultimately, the skin tends to enhance certain stimulus features (for example, corners) and obscure others (for example, small internal features), a process that is reflected in the neural image conveyed by the somatosensory nerves.

A critical property of mechanoreceptive afferent fibers in determining their role in tactile shape perception is the size of their receptive fields (RFs), itself determined by the depth of the receptors they innervate and by the degree to which individual afferents branch out to innervate multiple receptors. Indeed, SA2 and PC fibers respond to mechanical stimulation of large swaths of skin, so activation of these afferents conveys little information about the precise location of object contact or the spatial configuration of the object. In contrast, SA1 and RA afferents only respond to mechanical stimulation applied to very restricted patches of skin. The smaller “pixels” of SA1 and RA responses allow these afferent populations to convey more acute spatial information than do the huge “pixels” of SA2 and PC responses.

At the somatosensory periphery, then, the shape of objects is encoded in the spatial pattern of activation evoked in mechanoreceptive fibers, much like the shape of visual objects is encoded in the spatial pattern of activation evoked in photoreceptors in the retina. For example, the spatial configuration of edges (Srinivasan and LaMotte 1987), embossed letters (Phillips et al. 1988), Braille-like dot patterns (Johnson and Lamb 1981; Phillips et al. 1990, 1992), and half spheres of varying curvature (Goodwin et al. 1995) is reflected in the spatial pattern of activation evoked in mechanoreceptive afferents (Fig. 1). Furthermore, the spatial image carried by SA1 afferents is sharper than its RA counterpart. For example, the shape of embossed letters is better defined in the spatial event plots (SEPs) reconstructed from SA1 responses than in those reconstructed from RA responses (Fig. 1). SEPs have also been used to estimate the ability of peripheral afferents to signal the curvature and orientation of objects indented into the skin (LaMotte et al. 1998; LaMotte and Srinivasan 1996). Again, SA1 afferents were found to convey the most reliable information about these stimulus features (Khalsa et al. 1998). In fact, paired psychophysical and neurophysiological experiments have shown that SA1 afferents respond to the finest tangible spatial features and set the bottleneck for spatial acuity, at least for stimuli indented into the skin (Johnson and Phillips 1981; Phillips and Johnson 1981a). Spatial acuity actually improves when RA signals are reduced (Bensmaia et al. 2006).

Fig. 1.

Spatial event plots (SEPs). SEPs reconstructed from a slowly adapting type 1 (SA1) afferent (top), a rapidly adapting (RA) afferent (middle), and a neuron in area 3b (bottom). While the spatial pattern of afferent activation reflects the spatial configuration of the stimulus, cortical neurons respond to specific stimulus features (adapted from Phillips et al. 1988).

However, RA afferents also convey shape information, as evidenced by the successful use of the optical-to-tactile converter (Optacon), a sensory substitution device that consists of an array of vibrating pins (Bliss et al. 1970; Craig 1980). The idea behind the Optacon was that blind individuals would scan printed text with a camera and the text would be converted to a spatial pattern of pin activation that reflected the scanned letters. When pins configured in the shape of letter were made to vibrate, for example, subjects were able to identify the letter. Importantly, however, the Optacon excites RA and PC but not SA1 fibers (Gardner and Palmer 1989). As PC fibers are unlikely to contribute to shape perception given their sparseness and large RFs (Johansson and Vallbo 1979; Vallbo and Johansson 1984), the ability to discern shape through the Optacon is almost certainly mediated by RA fibers.

While the shape signal carried by SA1 afferents tends to be sharper for static or slowly moving tactile stimuli, that carried by RA afferents may be more informative under some circumstances, for example, during rapid hand interactions with objects that may drive RA afferent responses more robustly, but this has not been tested.

In addition to carrying a spatial image of a tactile stimulus, afferents may also begin the process of extracting behaviorally relevant features of the stimulus. Indeed, the responses of individual SA1 and RA human tactile afferents convey information about the direction of forces experienced at the fingertip and the shape (i.e., curvature) of the surfaces contacting the skin (Birznieks et al. 1999; Jenmalm et al. 2001, 2003; Johansson and Birznieks 2004). Afferents can also signal the orientation of bars and edges by producing responses whose strength and timing are dependent on orientation (Pruszynski and Johansson 2014). This selectivity for spatial features likely results from the nonuniform distribution of highly sensitive “hot spots” across its RF (Pruszynski and Johansson 2014), each of which corresponds to a receptor that is innervated by the afferent. While afferent RFs with multiple hot spots are also found in non-human primates (Phillips and Johnson 1981a), these may be more pronounced in humans due to their larger digit pads (Johansson 1978; Phillips et al. 1992). Some evidence for spatial feature selectivity is also observed in retinal ganglion cells (Gollisch and Meister 2010), suggesting that precortical geometric feature extraction may be a general operation performed by sensory systems.

Transmission of Shape Information from the Hand to the Brain

Somatosensory shape information carried by peripheral afferent populations is primarily transmitted to the cortex via the dorsal column-medial lemniscal pathway. In this pathway, axons from nerve fibers carrying somatosensory signals from the forelimb ascend the spinal cord and synapse onto neurons in the cuneate nucleus (CN) in the medulla. Second order somatosensory neurons in the CN project to the ventroposterior lateral nuclei (VPL) in the thalamus. Little is known about the tactile coding properties of neurons in the CN and VPL of primates. However, while these structures have traditionally been considered relay stations that perform limited information processing operations, recent evidence from other organisms suggests that they implement more complex processing (Jorntell et al. 2014). The implications of this processing for the neural processing of shape in primates remain to be elucidated.

Primary somatosensory cortex (S1), comprising Brodmann areas 3a, 3b, 1, and 2, is the principal recipient of somatosensory thalamic inputs. Each S1 subdivision displays a distinct pattern of thalamic and cortico-cortical connections. Neurons in area 3a receive their inputs from the shell region surrounding VPL and respond primarily to proprioceptive stimulation (Gardner 1988). Neurons in area 3b primarily receive thalamic inputs from the core regions of VPL (Jones 1983; Jones and Friedman 1982) and respond best to cutaneous stimulation (Paul et al. 1972; Sur et al. 1980, 1985). Areas 1 and 2 receive dense projections from area 3b in addition to thalamic inputs (Burton and Fabri 1995) and contain neurons whose RFs often span multiple digits (Hyvärinen and Poranen 1978b; Iwamura et al. 1993; Vierck et al. 1988). Furthermore, neurons in area 2 respond to both cutaneous and proprioceptive stimulation (Hyvärinen and Poranen 1978b; Iwamura and Tanaka 1978). After processing in S1, tactile information is routed to association areas in posterior parietal cortex (PPC) (Kalaska et al. 1983; Mountcastle et al. 1975) and to the second somatosensory cortex (S2) (Burton et al. 1995; Friedman et al. 1980, 1986). Neurons in these higher order somatosensory areas have large RFs and complex response properties with cutaneous and proprioceptive components. S2 in humans (Eickhoff et al. 2007; Eickhoff et al. 2006) and non-human primates (Fitzgerald et al. 2004; Kaas and Collins 2003) can be divided into at least three subregions, each of which with distinct response properties (Burton et al. 2008; Fitzgerald et al. 2004). In the following section, we consider how shape information is encoded and transformed along the somatosensory processing pathway.

Shape Representations and Transformations in the Somatosensory System

As discussed above, the spatial pattern of activation across SA1 and RA afferents is largely isomorphic with the stimulus. Such isomorphic neural images can be appreciated from SEPs (Fig. 1), which are generated by registering a neuron's spiking activity with the position of a stimulus (e.g., an embossed letter) that is scanned across the neuron's RF (Phillips et al. 1988). SEPs constructed from a single neuron's responses to embossed letter stimuli (Fig. 1) approximate the neural images of the letters as they would appear distributed across a population of identically responsive neurons whose receptive fields are densely and uniformly distributed across the skin (Phillips et al. 1988). Each afferent's spiking activity simply reflects the presence or absence of any stimulus component that falls within its RF, with some spatial filtering (blurring) due to skin mechanics (Phillips and Johnson 1981b; Sripati et al. 2006a) and afferent branching (Pare et al. 2002). In contrast, cortical neurons in area 3b display a wider range of response patterns to the same stimuli, ranging from strongly isomorphic to highly structured but nonisomorphic (Fig. 1) to weakly structured (Phillips et al. 1988). The heterogeneity of 3b response types implies a shift from strict isomorphism and an emergence of feature selectivity in somatosensory cortex. Critically, strongly isomorphic cortical responses may be partially inherited from subcortical processing levels as RF size, RF complexity, and neural feature selectivity in area 3b neurons are correlated with laminar position (DiCarlo and Johnson 2000) with the most “afferent-like” responses occurring in the granular layer, which receives the bulk of the thalamocortical projections (Jones and Burton 1976).

Orientation is an elementary contour feature and many S1 neurons in areas 3b and 1 respond selectively to bars and edges at particular orientations when these impinge upon their RFs (Bensmaia et al. 2008a; Pubols and LeRoy 1977; Warren et al. 1986). An orientation-selective neuron responds vigorously to a stimulus presented at its preferred orientation and weakly as the orientation of the stimulus deviates from the neuron's preferred orientation (Fig. 2A). In many neurons, orientation preferences are consistent whether the stimuli are indented or scanned over the finger (Fig. 2B) and tuning is robust to changes in stimulus amplitude (Fig. 2C) (Bensmaia et al. 2008a). The sensitivity of orientation-tuned neurons is comparable to that of human observers so orientation signals in areas 3b and 1 can in principle account for the tactile perception of oriented edges (Bensmaia et al. 2008b). The degree to which this orientation tuning in S1 is shaped by the tuning observed in afferent signals (Pruszynski and Johansson 2014) remains to be elucidated.

Fig. 2.

Orientation tuning in primary somatosensory cortex. A: responses of a neuron in area 3b to bars indented into its receptive field at different orientations. B: orientation tuning curves for another neuron in neuron area 3b reveal consistent preferred orientation, whether the bars are indented (blue) or scanned (orange) bars. C: orientation tuning is also consistent across changes in stimulus amplitude (same neuron as in B) (adapted from Bensmaia et al. 2008).

Higher order somatosensory neurons in area 2 (Hyvärinen and Poranen 1978a) and S2 (Fitzgerald et al. 2006b) also exhibit orientation selectivity, with tuning strength that is comparable to that observed in areas 3b and 1 but over much larger skin regions. Indeed, RFs in area 2 and S2 span multiple fingers and can even encompass both hands in their entirety (Fitzgerald et al. 2006a). The orientation preference of higher-order, orientation-tuned neurons tends to be consistent across their RF (Fig. 3). Thus orientation tuning becomes invariant to position in higher order neurons (Fitzgerald et al. 2006b; Thakur et al. 2006), a phenomenon that is also observed in neurons in higher order visual cortex (Ito et al. 1995).

Fig. 3.

Orientation tuning in multidigit second somatosensory cortex (S2) receptive fields. A: responses of an S2 neuron to oriented bars presented to the distal (d), middle (m), and proximal (p) finger pads on digits 2–5 (index to little finger). Colored traces indicate significant orientation tuning. B: orientating tuning curves for each finger pad depicted in A. Orientation preferences tended to be similar across tuned pads (adapted from Fitzgerald et al. 2006).

Neurons in area 2 and S2 also exhibit selectivity for the curvature of contours indented into the skin (Yau et al. 2009, 2013a). In this context, curvature denotes the change in the contour's orientation. Thus any contour or edge with nonuniform orientation is curved with levels of curvature ranging from sharp (i.e., abrupt orientation changes) to broad (i.e., gradual orientation changes). Curvature fragments can also be characterized according to their curvature direction, defined as the direction of a vector along the fragment's axis of symmetry that points away from its interior. Somatosensory cortical neurons respond preferentially to contour fragments pointing in a specific direction (Fig. 4A). These explicit neural representations of curvature, a higher order shape feature, are thought to be generated by integrating information from local orientation detectors in areas 3b and 1. Indeed, the degree to which the activity of individual neurons differentiates between curvature directions grows gradually (Fig. 4B). This dynamic suggests that curvature representations emerge from network interactions (Yau et al. 2013a), also thought to be necessary for visual curvature synthesis (Yau et al. 2013b). Because curvature direction preferences are consistent across multiple RF locations (Yau et al. 2009, 2013a), curvature tuning in area 2 and S2 neurons, like orientation tuning, is tolerant to changes in stimulus position.

Fig. 4.

Curvature direction tuning in somatosensory cortex. A, left: responses of a neuron in area 2 to curvature fragments (2D angles and arcs) sorted by curvature direction. A, right: gray scale indicates the average response rates for angles and arcs. This neuron responds best to curvature fragments pointing leftward (135–180°). B: time courses of normalized spiking activity (black) and curvature signaling (gray) in a population of neurons in area 2. Curvature modulation is computed by taking the difference between the maximum and minimum responses and normalizing it by the sum of these. Population activity peaks relatively quickly while curvature signals build gradually and peak more slowly (adapted from Yau et al. 2013a).

In sum, the transformation of 2D shape representations in the somatosensory system is similar to that in the ventral visual pathway: isomorphic spatial representations in the retina give rise to increasingly complex feature tuning that is increasingly invariant to other properties of the stimulus, such as its size and position (Connor et al. 2007). These remarkable similarities between visual and tactile shape processing suggest that the two systems implement analogous neural computations, as has been shown to be the case for motion processing (Pack and Bensmaia 2015; Pei and Bensmaia 2014; Pei et al. 2010, 2011).

Dynamic Spatial Filtering as a Mechanism for Shape Feature Selectivity

One way to describe the neuronal response properties that lead to feature selectivity is to characterize the dynamic spatial structure of the neuron's RF. The dominant feature in the RFs of mechanoreceptive afferents tends to be a small excitatory region (Sripati et al. 2006b; Vega-Bermudez and Johnson 1999). In contrast, RFs of neurons in area 3b typically comprise a central excitatory region flanked by one or more inhibitory ones (DiCarlo et al. 1998). Furthermore, this spatial structure evolves over time, on millisecond time scales (Fig. 5A); for example, the central excitatory subfield is replaced by an inhibitory field (Gardner and Costanzo 1980; Sripati et al. 2006b). The spatiotemporal RFs (STRF) of area 3b neurons (DiCarlo and Johnson 2000) can be described with three linear components: an excitatory core, inhibitory regions that flank the excitatory core, and replacing inhibition that overlaps the excitatory core after some delay (Fig. 5B). Although linear models offer adequate characterizations of RFs, models that incorporate quadratic components provide substantially improved predictions of neuronal responses (Thakur et al. 2012). It is important to note that the RFs of SA1 afferents, mapped with random dot patterns or spatiotemporal indentations, exhibit properties that, at first glance, resemble surround inhibition. However, this so-called “surround suppression” is a product of skin mechanics: the presence of an indentation just outside of the excitatory region of the RF decreases the forces exerted by an indentation in the hot-spot, thereby decreasing its effectiveness (Sripati et al. 2006b). Additionally, the STRFs of both SA1 and RA afferents exhibit what resembles replacing inhibition (Sripati et al. 2006b) but is simply neural refractoriness. Although some of the inhibition observed in cortical STRFs is undoubtedly inherited from these inhibition-like peripheral response properties, the increased degree of inhibition with respect to excitation in cortex, the dynamics of inhibition in cortex, and the differences in the properties of inhibition in areas 3b and 1 imply that a substantial component of the inhibition in cortical STRFs is of intracortical origin (Sripati et al. 2006b).

Fig. 5.

Spatial temporal receptive fields (STRFs). A: STRF estimated for a neuron in area 1 using a modified form of linear regression similar to reverse correlation. This STRF comprises a central excitatory region, fixed inhibition, and lagged inhibition on a single finger. Color bar indicates pixel intensities in spikes per second per micrometer. B: 3-component model that includes fixed excitatory (red), fixed inhibitory (solid blue), and lagged inhibitory components (dashed blue). The sum of the elliptical regions yields the full STRF. The centers of the fixed and lagged inhibitory components are indicated by the blue dots. For RFs estimated from responses to moving dot stimuli (see DiCarlo and Johnson, 2000), the location of the lagged inhibitory component, with respect to the excitatory region, depends on the direction in which the stimulus is scanned (thick black arrow). The location of the fixed inhibitory component remains unchanged over motion directions (adapted from Sripati et al. 2006b).

These RF descriptions account for several aspects of the feature selectivity and invariance that emerge along the somatosensory pathway. First, the geometry of the excitatory and inhibitory subfields accounts, at least in part, for the orientation selectivity of S1 neurons (Bensmaia et al. 2008a; DiCarlo et al. 1998), as is the case for simple cells in area V1 (Hubel and Wiesel 1968; Ringach 2002). Second, the opponency of excitatory and inhibitory subfields accounts for the observation that S1 responses are independent of the number of simultaneous contacts within their RF (Thakur et al. 2012), which stands in contrast to those of peripheral afferents, whose response rates increase logarithmically with stimulation density (Sripati et al. 2006b). Third, the presence of a lagged inhibitory field causes neural responses to the spatial features of a stimulus to be relatively tolerant to changes in scanning velocity (DiCarlo and Johnson 2002). Fourth, the position of the delayed inhibitory field relative to the excitatory core may underlie, to some extent, S1 directional selectivity (DiCarlo and Johnson 2000; Pack and Bensmaia 2015; Pei et al. 2010).

While STRFs are similar in areas 3b and 1, such linear representations of RFs capture significantly less variance in the responses of neurons in area 1 than in area 3b (Sripati et al. 2006b), consistent with the hypothesis that neurons in area 1 encode more complex and invariant spatial features. For instance, orientation tuning strength is comparable in the most selective neurons in areas 3b and 1 (Bensmaia et al. 2008a). However, orientation tuning in area 1 may be more tolerant to position changes or to movement.

While linear models can capture some of the response properties of S1 neurons, they cannot account for the responses of many S2 neurons (Fig. 6B). Like their counterparts in S1, S2 neurons are tuned for the orientation of bars presented to the centers of finger pads (Fig. 6, orientation response plots). However, these cells differ widely in their responses to oriented bars presented at other locations within the finger pad, as demonstrated using a vector field representation (Fig. 6, vector fields). Vector fields indicate the consistency of orientation tuning across a finger pad, which can provide clues as to the underlying neural mechanisms. For example, a divergent vector field (Fig. 6A), which comprises vectors radiating from a single untuned region of the finger pad, describes the orientation preference of a neuron whose RF consists solely of a central excitatory region, i.e., a “hot spot” centered on the untuned region: a bar at any location will evoke a response if it contacts the hot spot. In contrast, vector fields with relatively uniform vector directions (Fig. 6B) reveal position-tolerant orientation tuning, which cannot be easily explained by linear mechanisms (Thakur et al. 2006).

Fig. 6.

Orientation tuning and vector fields in S2. A and B, left: response rates of two S2 neurons to bars presented to the center of a finger pad at different orientations. A and B, right: vector fields indicate estimated orientation preference (vector direction) and tuning strength (vector magnitude) across a grid of positions spanning a single finger pad. Note that the vector fields are constructed from neural responses to a large bar stimulus rotated to 1 of 8 orientations and indented into the distal fingerpad at multiple locations per orientation. Response functions and vector fields for 2 neurons are depicted. Although the neurons each exhibit clear orientation tuning for bars presented at the center of the finger pad, their vector fields differ dramatically reflecting distinct orientation tuning mechanisms. A: divergent vector field, which can be explained by a linear RF model comprising a single excitatory field. B: invariant vector field, which cannot be accounted for with a linear RF model (adapted from Thakur et al. 2006).

Interactions Between Proprioceptive and Cutaneous Signals Support 3D Shape Processing

A unique aspect of the somatosensory system is that it contains a deformable sensory sheet. That is, the relative positions of cutaneous mechanoreceptors in 3D space depend on the conformation of the hand. Thus, while the perception of 2D shape can be achieved solely from cutaneous signals, haptic perception of 3D shape requires combining cutaneous signals with proprioceptive signals that track the hand's movements and posture (Berryman et al. 2006; Goodwin and Wheat 2004; Hsiao 2008; Klatzky et al. 1993; Pont et al. 1999). Hand proprioception relies on several types of mechanoreceptors (see Proske and Gandevia 2012 for a review), including muscle spindles (Cordo et al. 1995, 2002; Winter et al. 2005; Wise et al. 1999), Golgi tendon organs (Appenteng and Prochazka 1984; Houk and Henneman 1967; Houk and Simon 1967; Matthews and Simmonds 1974; Proske and Gregory 2002; Roll et al. 1989), joint receptors (Burgess and Clark 1969), and SA2 afferents innervating the skin of the hand (Edin 1992; Edin and Abbs 1991; Edin and Johansson 1995). Proprioceptive signals carried by these different afferent populations are transmitted to somatosensory cortex directly along the dorsal column-medial lemniscal pathway (Mountcastle 1984; Poggio and Mountcastle 1960) and possibly indirectly along the cuneocerebellar tract, passing through the external cuneate nucleus (Akintunde and Eisenman 1994; Campbell et al. 1974; Cerminara et al. 2003; Davidoff 1989; Huang et al. 2013; McCurdy et al. 1998; Quy et al. 2011), via the cerebellum (Blakemore et al. 1999).

According to the traditional model of somatosensory cortex, proprioceptive signals project to area 3a and cutaneous signals to areas 3b and 1 (Iwamura et al. 1995; Kalaska 1994; Krubitzer et al. 2004; Porter and Izraeli 1993) before these are integrated in higher order areas like area 2 (Porter and Izraeli 1993) and S2 (Fitzgerald et al. 2004). However, passive limb movements and postural changes alone can activate areas 3b and 1 (Ageranioti-Belanger and Chapman 1992; Chapman and Ageranioti-Belanger 1991; Rincon-Gonzalez et al. 2011) and cutaneous stimulation alone can evoke responses in area 3a (Cohen et al. 1994; Kim et al. 2015; Krubitzer et al. 2004; Prud'homme et al. 1994). Thus all S1 areas may in principle encode both proprioceptive and cutaneous information (Fig. 7A) (Kim et al. 2015).

Fig. 7.

Cutaneous, proprioceptive, and multimodal responses in S1 (adapted from Kim et al. 2015). A: population distributions of unimodal and multimodal neurons across S1. B, left: linear multimodal neuron whose responses are modulated by hand posture before cutaneous stimulation. B, right: cutaneous stimulation caused uniform increases in response rates over all hand postures. C: multimodal nonlinear neuron whose responses are modulated by hand posture before cutaneous stimulation (left) and by cutaneous stimulation (right). The response patterns over postures differ between the baseline and cutaneous conditions.

Hand posture modulates neural activity in all S1 areas, as might be expected given that the skin is stretched or compressed during finger movements (Chapman and Ageranioti-Belanger 1991; Costanzo and Gardner 1981; Gardner and Costanzo 1981; Iwamura et al. 1993; Kalaska 1994; Kalaska et al. 1998; Krubitzer et al. 2004; Nelson et al. 1991). Not surprisingly, then, S1 neurons exhibit a wide repertoire of proprioceptive responses (Kim et al. 2015). Some neurons encode the angular position of a single joint while others encode complex hand postures. Many neurons, however, exhibit both proprioceptive and cutaneous responses (multimodal neurons in Fig. 7A). In most multimodal neurons, rate modulations related to hand conformation are superimposed on rate modulations related to cutaneous stimulation (Fig. 7B). In some multimodal neurons, cutaneous and proprioceptive signals interact in complex, nonlinear ways (Fig. 7C). For example, the response to cutaneous stimulation is modulated by hand conformation or the response to hand conformation is modulated by cutaneous stimulation (Fig. 7C).

The dynamics of multimodal responses may provide clues as to the underlying neural mechanisms. In some multimodal neurons, the integration of proprioceptive and cutaneous signals is evident immediately following cutaneous stimulation and/or movement (Kim et al. 2015), consistent with a processing scheme in which cutaneous and proprioceptive interactions build from feedforward computations. In other multimodal neurons, this integration emerges more gradually (Kim et al. 2015), suggesting the involvement of network interactions. The faster multimodal signal may provide an initial estimate of 3D shape to guide object manipulation, while the slower one may be involved in object perception, a process that requires time (Klatzky and Lederman 2011).

Tactile Shape Processing in the Human Brain

Consistent with the monkey neurophysiological evidence, tactile spatial form processing results in robust activations in human S1 and S2 (also referred to as the parietal operculum). Extensive and overlapping somatosensory brain regions respond to spatial dot patterns (Harada et al. 2004; Li Hegner et al. 2007), oriented gratings (Kitada et al. 2006; Van Boven et al. 2005), embossed letters (Burton et al. 2008), and abstract shapes (Stilla and Sathian 2008). These neuroimaging results collectively support a model of hierarchical processing within the somatosensory cortical system: in S1, area 2 exhibits substantially greater shape-selective responses, e.g., to surface curvature and 3D objects, compared with areas 3b and 1 (Bodegard et al. 2001). According to the hierarchical processing model, additional recruitment of S2 and PPC reflects the elaboration of shape information following S1 processing (Ostry and Romo 2001). In fact, distributed networks of parietal and frontal brain regions are also coactivated with somatosensory cortex while participants perform tasks involving tactile shape perception (Hegner et al. 2010; Zhang et al. 2005). These association areas may not support sensory processing per se, but rather may mediate memory and executive functions related to performing the perceptual tasks. PPC may also be well suited to mediate the bidirectional exchange of shape information between the somatosensory and visual systems (Deshpande et al. 2008).

Relationship Between Tactile and Visual Shape Processing

We perceive 2D and 3D shape by touch or vision alone. Multiple lines of evidence indicate that tactile and visual shape perception are not only similar but also related. First, visual discrimination performance drops to tactile levels when subjects experience visual stimuli through a limited field of view (Dopjans et al. 2012; Loomis et al. 1991). Second, the ways in which subjects categorize (Gaissert et al. 2010) and confuse (Brusco 2004) visual and tactile shapes is highly correlated. Third, spatial form illusions, like the Muller-Lyer illusion, are experienced in both vision and touch (Mancini et al. 2011). Not surprisingly, then, somatosensory and visual neurons encode shape information in similar ways. These shared neural codes may also be important for facilitating cross talk between the somatosensory and visual systems (Ghazanfar and Schroeder 2006).

The conventional view has been that visual and tactile processing of spatial patterns is comparable after taking into account factors like the number of activated receptors (Phillips et al. 1983) and the blurring caused by skin mechanics (Loomis 1982, 1990). However, a recent study found that visual pattern perception remained superior to touch after carefully equating the visual and tactile stimulus conditions (Cho et al. 2015). Because vision outperforms touch even when peripheral processing is precisely matched, differences in central processing must contribute to the superiority of vision in shape processing. Thus, while vision and touch may exploit similar neural mechanisms for shape processing, those that support vision may be more extensive and refined. Not surprisingly, we typically rely more on vision than touch when perceiving shape information in our environment.

Tactile processing and visual shape processing is not only based on similar neural codes, but they may be supported in part by common neural circuits. Indeed, tactile perception of 2D shapes (Prather et al. 2004) and 3D objects (Amedi et al. 2001; Pietrini et al. 2004; Tal and Amedi 2009) activates regions of inferior temporal cortex and posterior parietal cortex that are typically activated during visual shape processing. Touch-related activity in visual cortex has also been observed in single-unit recordings: under some conditions, orientation tuned neurons in area V4 respond to oriented bars presented on the skin (Haenny et al. 1988; Maunsell et al. 1991). Although tactile shape responses in visual cortex may in part reflect visual imagery (Lacey and Sathian 2014), neuropsychological evidence from stroke patients suggests that occipital and temporal lobe regions contribute critically to tactile shape perception (Feinberg et al. 1986) and haptic learning of novel objects (James et al. 2006). Similarly, noninvasive brain stimulation targeting regions in visual cortex can influence tactile shape perception (Mancini et al. 2011; Merabet et al. 2004; Yau et al. 2014; Zangaladze et al. 1999). Thus the haptic perception of shape relies on distributed brain networks that extend beyond traditionally defined somatosensory cortex (Pascual-Leone and Hamilton 2001).

Tactile and visual signals interact not only in perceiving object shape but also in guiding object manipulation. PPC is known to contain neurons with multisensory properties (Avillac et al. 2005; Murata et al. 2000; Pouget et al. 2002). Although neural responses in caudal S1 and PPC exhibit some degree of shape selectivity (Gardner et al. 2007a; Iwamura and Tanaka 1978), the activity of many neurons in areas 5 and 7 and the anterior intraparietal area appears to encode information relevant to grasping and manipulating objects (Murata et al. 1997, 2000) rather than object geometry, per se. Indeed, firing rates in PPC increase gradually as monkeys reach for objects, particularly during object approach trajectories when the fingers are preshaped for grasping (Gardner et al. 2007b). In many cells, response magnitude and dynamics are highly similar if the animals grasp at objects of different shapes using similar grasps (Chen et al. 2009). Because activity in PPC peaks at the time of object contact, PPC activity may be particularly suited to serve both sensory and motor functions aimed at providing the animal with feedback concerning the accuracy of its reach and grasp and guidance for error correction (Chen et al. 2009; Gardner et al. 2007b). At the time of contact, visual and tactile feedback concerning the hand's movements relative to the object is brought into register, providing a powerful signal to shape subsequent motor commands. The dense projections between PPC and frontal motor regions provide a clear and direct pathway to support this sensorimotor processing (Andersen and Buneo 2002).

Conclusions

The spatial features of objects that we grasp or manipulate are encoded in the spatial pattern of activation of populations of mechanoreceptive afferents. Information about behaviorally relevant features is then gradually extracted from these neural images along a hierarchical processing pathway that is very analogous to the corresponding pathway in the visual system. Neurons at early stages of processing encode simple stimulus features, e.g., edge orientation, and their feature selectivity can be explained in terms of the spatial structure of the neurons' receptive fields. At higher processing stages, neurons encode more complex stimulus features, e.g., curvature, and these feature selectivities are increasingly invariant to the position of the stimulus on the hand and cannot be explained in terms of linear receptive fields. Cutaneous signals interact with proprioceptive signals that carry information about hand conformation and these multimodal interactions are likely critical to haptic stereognosis, the ability to discern the three dimensional shape of objects. While the main function of tactile shape processing is to support interactions with objects, the underlying neural circuits also enable the identification of objects when vision is unavailable.

GRANTS

S. J. Bensmaia is supported by National Science Foundation Grant IOS-1150209 and National Institute of Neurological Disorders and Stroke Grant NS-082865.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

Author contributions: J.M.Y., S.S.K., P.H.T., and S.J.B. prepared figures; J.M.Y., S.S.K., P.H.T., and S.J.B. drafted manuscript; J.M.Y., S.S.K., P.H.T., and S.J.B. edited and revised manuscript; J.M.Y., S.S.K., P.H.T., and S.J.B. approved final version of manuscript.