Multimodal interactions between proprioceptive and cutaneous signals in primary somatosensory cortex

Abstract

The classical view of somatosensory processing holds that proprioceptive and cutaneous inputs are conveyed to cortex through segregated channels, initially synapsing in modality-specific areas 3a (proprioception) and 3b (cutaneous) of primary somatosensory cortex (SI). These areas relay their signals to areas 1 and 2 where multimodal convergence first emerges. However, proprioceptive and cutaneous maps have traditionally been characterized using unreliable stimulation tools. Here, we employed a mechanical stimulator that reliably positioned animals' hands in different postures and presented tactile stimuli with superb precision. Single-unit recordings in SI revealed that most neurons responded to cutaneous and proprioceptive stimuli, including cells in areas 3a and 3b. Multimodal responses were characterized by linear and nonlinear effects that emerged during early (∼20ms) and latter (>100ms) stages of stimulus processing, respectively. These data are incompatible with the modality specificity model in SI, and provide evidence for distinct mechanisms of multimodal processing in the somatosensory system.

Introduction

Primates are able to recognize and manipulate objects with their hands (Klatzky et al., 1993; Murray and Mishkin, 1984; Thakur et al., 2008). This ability is thought to be mediated by cortical mechanisms that combine cutaneous inputs from skin receptors contacting the object (e.g. edge orientation) with proprioceptive signals representing the spatial distribution of fingers enclosing the object (e.g. hand conformations) (Berryman et al., 2006; Goodwin and Wheat, 2004; Hsiao, 2008; Pont et al., 1999). Indeed, behavioral studies show that tactile perception can be modulated by how the hand contacts an object (Corcoran, 1977; Oldfield and Phillips, 1983; Parsons and Shimojo, 1987; Rinker and Craig, 1994a). For instance, Rinker and Craig (1994) showed that the same pattern of motion delivered to a finger produces different percepts when the hand is placed in different conformations. However, where and how these cutaneous and proprioceptive neural interactions take place is unclear, but a likely area is primary somatosensory cortex (SI) since it contains neural populations that encode these types of modality-specific tactile signals (Mountcastle, 2005).

In the periphery, cutaneous inputs are processed by skin mechanoreceptors (Hsiao and Gomez-Ramirez, 2012; Johnson, 2001; Johnson et al., 2000), which convey their signals to SI where neural representations of various tactile features emerge (Bensmaia et al., 2008; Pei et al., 2010, 2011; Saal and Bensmaia, 2014; Weber et al., 2013; Yau et al., 2013). In contrast, proprioceptive inputs are processed by joint, muscle and certain skin mechanoreceptors (e.g. Ruffini corpuscle) (Cordo et al., 2002; Edin and Abbs, 1991; Houk and Henneman, 1967; Matthews and Simmonds, 1974; Olausson et al., 2000; Proske and Gregory, 2002; Roll et al., 1989). However, unlike cutaneous sensory processing, the cortical mechanisms underlying proprioception, particularly of the hand and fingers, are poorly understood.

According to the classical model of somatosensory processing, cutaneous and proprioceptive inputs are conveyed to cortex through segregated channels and make their first cortical synapse in areas 3b and 3a of SI, respectively (Mountcastle, 2005). Neural signals from these regions project to adjacent areas 1 and 2 where multimodal integration of tactile inputs first emerges. However, some studies show that cells in area 3b respond to both skin indentation and arm displacements (Cohen et al., 1994; Krubitzer et al., 2004; Prud'homme et al., 1994), challenging the prevalent model of modality segregation in SI. While much has been learned from these studies, a significant drawback is that the receptive field (RF) of cells was characterized using unreliable tools such as hand held probes and limb movements guided by experimenters. Certainly, a quantitative and systematic approach to mapping these modality inputs is essential for characterizing the codes underlying cutaneous and proprioceptive processing, as well as their integration.

Here, we used a mechanical stimulator that reliably positioned animals' hands in selective conformations and presented tactile oriented stimuli with superb spatial and temporal precision (Lane et al., 2010). Neural recordings were made while the hand was statically positioned in the desired conformation, allowing us to quantify proprioceptive and multimodal integration effects in the absence of volitional motor commands. Indeed, some studies have examined how cutaneous responses are modulated by active large-scale limb movements (London and Miller, 2013; Shaikhouni et al., 2013; Simoes-Franklin et al., 2011; Weber et al., 2011), or how somatosensory neurons represent active limb position and hand grasping (Debowy et al., 2001; Gardner et al., 2007; Mountcastle and Powell, 1959; Ro et al., 2000). However, it is unclear whether effects are due to endogenous commands enacted by the motor system (e.g. efference copy), proprioceptive signals, or a combination of the two.

The goal of this study was to investigate interactions between proprioceptive and cutaneous signals in the digit representation of SI. We characterized the modality selectivity of cells (i.e. unimodal cutaneous, unimodal proprioceptive or multimodal) and the neural coding schemes underlying proprioception. Contrary to the traditional model of somatosensory processing, we hypothesized that cells in SI encode inputs from both cutaneous and proprioceptive modalities. Furthermore, similar to neural integration mechanisms of multisensory stimuli (Karns and Knight, 2009; Lakatos et al., 2007; Molholm et al., 2002), we expected that integration of cutaneous and proprioceptive tactile inputs occurs during the initial processing phase in SI. Finally, based on previous behavioral findings (Corcoran, 1977; Oldfield and Phillips, 1983; Parsons and Shimojo, 1987; Rinker and Craig, 1994a), we hypothesized that proprioception modifies the tuning properties of orientation selective cells by sharpening the tuning strength and modifying the preferred orientation angle.

Results

Single-unit (SU) activity was recorded from four hemispheres in areas 3a, 3b, 1 and 2 of SI cortex in three animals (Macaca mulatta). Animals sat comfortably on a custom-made chair with their hands held supinated while they received a drop of water every 3-7 seconds (Figure 1A). Cutaneous stimuli consisted of a bar oriented in four directions (0°, 45°, 90°, and 135°) that was indented (1 mm) on the distal pads of D2, D3 or D4 for 500ms (Figure 1B). Proprioception was modulated by varying the animal's hand conformation using a motorized exoskeleton (Figures 1B and 1C) (Lane et al., 2010), which displaced digit 2 (D2) and digit 4 (D4) in the horizontal plane (i.e. 0°, 11.25° and 22.5°) and flexed digit 3 (D3) in the vertical plane (-22.5°, -11.25°, 0°, 11.25° and 22.5°) (Figure 1B right panels). We implemented a paradigm composed of 45 proprioceptive (9 horizontal × 5 vertical digit displacements) and 12 cutaneous stimulation conditions (4 orientations × 3 digits). However, given the large number of experimental conditions and limited lifetime of recording from a cell, we randomly selected 20 proprioceptive conditions that were presented in combination with the full set of cutaneous stimuli. Specifically, one set of the twelve cutaneous stimulation conditions was presented in a pseudo-randomized order with the hand placed in a particular conformation. After presenting the twelve cutaneous conditions, the hand posture was varied to one of the twenty pre-selected conformations and another set of cutaneous stimuli was presented. This sequence of events was repeated until the remaining proprioceptive conditions were presented. This constituted one cycle of 240 experimental conditions (i.e. 20 hand conformations × 12 cutaneous stimuli), with each neural experimental session composed of five such cycles (i.e. five repetitions of each proprioceptive and cutaneous stimulation condition). Importantly, within each cycle the order of proprioceptive conditions was randomized. This sequence of events was designed to reduce the amount of experimental time, artifacts produced by the motors, and kinesthetic effects influencing neural responses.

Figure 1.

(A) Experimental setup: Graphical illustration of the experimental setup. Animals sat on a custom-made chair with their hands supinated while being presented with bar stimuli to D2, D3, or D4. (B) Cutaneous stimulus conditions: Cartoon illustration of the oriented bars (0°, 45°, 90°, and 135°) and the stimulated fingers (D2, D3, and D4). (C) Proprioceptive manipulation. The upper panel shows a photograph of the motorized exoskeleton used for manipulating proprioception. This device is composed of three individually controlled stepper motors that varied the positions of D2, D3, and D4. D2 and D4 were displaced in the horizontal plane (i.e. 0°, 11.25° and 22.5°) and D3 was flexed the vertical plane (-22.5°, -11.25°, 0°, 11.25° and 22.5°).

In this paper modality is defined as a collection of broad functional properties of a tactile stimulus or experience. For example, proprioception, cutaneous (touch), and temperature represent different somatosensory modalities. Within each modality there exists a set of submodalities that characterize a particular feature of an object or experience (e.g. shape in the cutaneous modality, or joint velocity in the proprioceptive modality). These terms are based on a functional categorization and are aligned with the recent proposal by (Saal and Bensmaia, 2014).

Neural representation of proprioception in SI cortex

We first examined the effects of proprioception, in the absence of cutaneous inputs, using an ANOVA with factor of Hand Conformation on the averaged neural activity between -400ms to -100ms. Figure 2A shows raster plots of an example neuron in area 3a that was modulated by Hand Conformation, with highest firing rate when D2 (left panel Figure 2A) or D4 were maximally spread apart (right panel of Figure 2A). Figure 2B illustrates proprioceptive tuning curves of example neurons in other areas of SI. In particular, the neuron in area 3b exhibited greater activity when D3 was displaced below the vertical midline (upper right panel Figure 2B), whereas the neuron in area 1 had increased activity as D3 was displaced in the upward direction (lower left panel Figure 2B). Finally, the neuron in area 2 had increased activity when D4 was displaced leftwards in the horizontal plane (lower right panel Figure 2B). The population statistics revealed that over 50% of neurons in SI were modulated by Hand Conformation (Figure 2C). Specifically, 73% of area 3a (23/32), 32% of area 3b (24/74), 52% of area 1 (41/77), and 53% (43/80) of area 2 neurons were modulated by digit displacements.

Figure 2. Sub-areas of SI encode proprioceptive information.

(A) Raster plots of an example neuron in area 3a showing modulations across digit displacements. The left and right rasters show neural activity associated with horizontal displacements of D2 and D4, respectively. The lower, middle and upper sub-raster plots on the left represent neural activity in response to displacing D2 at 0°, 11.25°, and 22.5°, respectively. The lower, middle and upper sub-raster plots on the right represent neural activity in response to displacing D4 at 22.5°, 11.25°, and 0°, respectively. Each row in a sub-raster represents an individual trial. The x-axis represents time, with 0ms indicating the onset of the steady indentation period of the tactile stimulus. These data were obtained while the hand posture was in a static position. (B) Example neurons showing proprioceptive effects in all four sub-areas of SI (area 3a upper left, area 3b upper right, area 1 lower left, area 2 lower right). The error bars in the plots represent standard error of the mean (SEM). (C) Population distribution of SI neurons sensitive to hand conformations.

We performed a regression analysis on each neuron to assess whether proprioceptive neural responses are characterized by additive or non-additive interactions within and across digit positions (see equation 1 in Experimental Procedures section). The latter would be indicative of neural tuning for specific hand postures. The regression analysis revealed that 69% of proprioceptive cells were explained by a linear summation of finger positions (79 out of 114 cells; the regression analysis detected a slightly lower number of significantly modulated neurons as compared to the ANOVA). Approximately 60% of these cells showed modulations within a single-digit (47/79), while the remaining 40% displayed effects across multiple digits (32/79). These neurons are similar to the third type of kinesthetic neurons described by (Gardner and Costanzo, 1980), so-called postural neurons. However, we refer to these neurons as position-scaled cells, as they seem to represent the position of the finger(s) on a linear scale as opposed to specific hand postures. Figure 3A shows an example of a neuron with proprioceptive effects confined to a single-digit. The response of this neuron increased linearly as D3 was displaced in the vertical direction (i.e. firing rate increased across the x-axis). Figure 3B shows the response of a multi-digit proprioceptive neuron, which exhibited a linear increase as D2 and D4 were displaced in the horizontal direction. This pattern of displacement yielded the maximum response when D2 and D4 were maximally spread apart (upper left panel Figure 3B). The remaining cells modulated by proprioception were explained by nonlinear (or non-systematic) interactions across multiple digits (31%, 35/114). These proprioceptive neurons are labeled posture-selective. Figure 3C shows an example of such neuron with maximum firing rate when D2 was placed in the intermediate horizontal position and D4 was maximally extended. The population data showed a higher incidence of position-scaled as compared to posture-selective neurons (69% vs. 31%, Pearson chi-squared test; X² = 16.98, p < 0.05; Figure 3D), but this ratio was not different across sub-areas of SI. Taken together, these results indicate that the majority of SI neurons respond to proprioception, and that the spatial configuration of digits is encoded by neural populations with single-digit and multi-digit RFs employing either additive or non-additive neural mechanisms.

Figure 3. Types of proprioceptive neurons.

(A) Single-digit position-scaled neuron that is modulated by vertical displacements of D3, with maximum response when D3 is fully flexed. The x-axis represents displacements of D3 from the lowest (-22.5°) to the highest vertical position (22.5°). The y-axis represents horizontal displacements of D4 from the lowest (0°) to the highest position (22.5°). The color in each box represents the firing rate response to the hand conformation, which is depicted inside the panel. Note that the missing conditions are due to technical limitations of our protocol (see Experimental Procedures section) (B). Example of a multi-digit position-scaled neuron. This neuron shows linear modulations in firing as D2 and D4 are stretched, with maximum firing rate when D2 and D4 are farthest apart. (C). Example of a multi-digit posture-selective neuron. This neuron shows non-systematic firing rate modulations across displacements of D2 and D4. It shows highest firing rate when D4 is maximally stretched and D2 is in the intermediate horizontal position. (D) Population distribution of the three types of proprioceptive neurons in SI.

Unimodal and multimodal representations of proprioceptive and cutaneous inputs in SI

Our next goal was to assess the modality selectivity properties of SI neurons by examining their neural response to proprioceptive and cutaneous stimuli. We performed a two-way ANOVA with factors of Hand Conformations (20 levels) and Cutaneous Stimulation (2 levels, prior vs. during bar indentation) on each cell. Data across orientation conditions were pooled. We classified neurons into four categories. Cells only displaying a main effect of Hand Conformation were classified as unimodal proprioceptive, while neurons only displaying a main effect of Cutaneous Stimulation were classified as unimodal cutaneous. Neurons showing main effects of Hand Conformation and Cutaneous Stimulation, but no interaction effects, were categorized as linear multimodal neurons, while neurons displaying an interaction effect (regardless of whether they displayed a main effect) were categorized as nonlinear multimodal. Examples of each neuron type and their population distribution are shown in Figures 4 and 5. The left and right panels of each graph represent responses before and during cutaneous stimulation, respectively. Both panels are sorted as a function of the neural response during baseline (i.e. prior to the stimulus indentation).

Figure 4. Modality selectivity in SI cortex.

(A) Neuron sensitive to cutaneous stimulation only. The response of this neuron was not modulated by hand conformation prior to cutaneous stimulation (left panels), but exhibited a significant change in response to the cutaneous stimulus (right panels). However, this neural response to the cutaneous stimulus was not significantly modulated by hand conformation. (B) Example of a neuron exhibiting response changes across hand conformations prior to the bar stimulus. The response of this neuron was not statistically modulated by the addition of the cutaneous stimulus. (C) Example of a multimodal neuron modulated by proprioception. The addition of the cutaneous stimulus further modulated the neural response but homogenously across all hand conformations (i.e. no interaction effects between cutaneous and proprioceptive inputs). In Figures A, B, and C, the left and right panels show neural responses prior to and during bar indentation, respectively. Both panels were sorted as a function of the neural response prior to bar indentation. Note that cutaneous stimulation was the same across all hand conformations. (D) Population distribution of unimodal and multimodal neurons across all areas of SI (N = 211). Nonlinear multimodal neurons are shown and described in Figure 5.

Figure 5. Nonlinear multimodal neurons.

(A) Example of a Type I nonlinear multimodal neuron. This neuron was sensitive to both cutaneous and proprioceptive inputs, with significantly correlated response patterns before and after cutaneous stimulation. (B) Type II nonlinear multimodal neuron. This cell showed tuning to a particular hand conformation prior to bar indentation. However, the addition of the cutaneous stimulus modified the neuron's proprioceptive tuning properties. (C) Type III nonlinear multimodal neuron. This neuron did not show an effect of hand conformation prior to bar indentation. However, after bar indentation, the response of this neuron varied across hand conformations. (D) Type IV nonlinear multimodal neuron. This neuron displayed was tuned for a specific hand conformation. However, this tuning was suppressed by the cutaneous input.

Figure 4A shows an example of a unimodal cutaneous neuron, which only exhibited significant responses after cutaneous stimulation in all hand conformations. Figure 4B shows an example of a unimodal proprioceptive neuron, which had significant responses to different digit displacements that were not modulated by the cutaneous stimulus. Figure 4C illustrates an example of a linear multimodal neuron, which had a significant response to different hand conformations prior to tactile stimulation, followed by a homogenous increase in activity to the cutaneous stimulus across all hand conformations. Figure 4D shows the distribution of unimodal and multimodal somatosensory neurons in all areas of SI (examples of nonlinear multimodal cells are shown in Figure 5). Approximately 80% of neurons (211/263) were modulated by Hand Conformation and/or Cutaneous Stimulation. In particular, most unimodal proprioceptive neurons were observed in area 3a, but note that these 3a neurons were also modulated by cutaneous stimulation (> 55%). Areas 3b, 1 and 2 were largely populated by unimodal cutaneous neurons, but were also modulated by proprioceptive stimulation (> 60% neurons in all areas). Indeed, we found that 52% of cells in SI responded to multimodal stimuli (110/211), with 61% and 39% of these cells explained by linear and nonlinear responses, respectively. We computed a Pearson Correlation Coefficient (CC) to assess the degree of correlation between the pattern of activity before and during tactile stimulation in both multimodal linear and nonlinear neurons. As expected, the data revealed higher correlations in multimodal linear vs. nonlinear neurons (CC = 0.4391 vs. 0.0587; Mann–Whitney U test [Z = 5.295; p < 0.001]). Taken together, these data show that the majority of cells in SI respond to both proprioceptive and cutaneous inputs, indicating that the sub-areas of SI cortex are not strictly modality specific.

We further examined the underlying properties of nonlinear multimodal neurons by classifying cells into four categories. Type I neurons were those that displayed significant correlations in hand conformations, assessed by Pearson correlation analyses (i.e. CC), before and during bar indentation. Figure 5A shows a Type I neuron that responded maximally to digits spread apart in the same plane. After cutaneous stimulation, the overall response pattern remained significantly correlated, even though the neuron exhibited a marginal decreased response when D3 was placed in the lowest vertical position (right graph middle panels). Type II cells were classified as those having significant, but uncorrelated, hand conformations effects before and during bar indentation (Figure 5B). Type III neurons were those that showed significant modulations across hand conformations during the bar indentation period only (Figure 5C). Finally, Type IV neurons were those with annulled significant effects of Hand Conformation after cutaneous stimulation (Figure 5D). The population data revealed that most nonlinear multimodal neurons were of Type III (∼40%), followed by Type II (∼27%), Type IV (∼21%) and Type I (∼12%). In addition, as expected, we only observed a high correlation in neural activity between baseline and tactile stimulation periods of Type I neurons (CC = 0.469). Type II, III and IV had a CC of 0.136, -0.023, and -0.1397, respectively. Taken together, these data show that nonlinear integration effects of cutaneous and proprioceptive inputs are diverse across SI. Specifically, we observed a subset of cells whose proprioceptive tuning properties were reshaped by the cutaneous stimulus (Types I, II and IV), while a separate set of neurons whose responses to the same cutaneous stimulus was modulated by proprioception (Type III).

Temporal dynamics of proprioceptive and cutaneous integration effects

We examined the temporal evolution of modality integration effects in linear and nonlinear multimodal cells. Each trial was discretized in bins of 40ms (±20 ms). We then performed an ANOVA with factor of Hand Conformation on each bin to identify the initial time bin at which proprioception significantly interacted with cutaneous stimuli. A statistically significant effect was determined when the ANOVA revealed a p-value <0.05 for the same Hand Conformation in at least two consecutive time bins. Figure 6A shows the instantaneous firing rate of a linear multimodal neuron across four representative Hand Conformations. As the figure shows, the response of this neuron was modulated by Hand Conformation before stimulus onset. However, when the tactile stimulus was indented, this neuron exhibited a bi-phasic response suppression that was common to all Hand Conformation conditions. Figure 6B illustrates the instantaneous response of a nonlinear multimodal neuron to four example Hand Conformations. This figure shows that hand conformation did not have an effect on the neuron's response prior to or immediately after cutaneous stimulation. However, after the bar indentation, Hand Conformation modulated the cutaneous response staring at approximately 100ms, with the maximum response when D3 and D4 were displaced in the most upward and rightward positions, respectively. We remind the reader that all proprioceptive and/or cutaneous effects were observed while the hand was held statically. In addition, we point out that ‘0 ms’ in these graphs indicate the onset of the steady indentation period of the tactile bar stimulus. Thus, because of the 100ms on/off ramp, the tactile stimulus made contact with the skin prior to the ‘0 ms’ tick mark. This caused a neural response around −30ms.

Figure 6. Temporal evolution of cutaneous and proprioceptive integration effects in multimodal neurons.

(A) Example of a linear multimodal neuron across four representative hand conformation conditions. The response of this neuron was modulated by hand conformation before stimulus onset. However, when the tactile stimulus was indented, this neuron exhibited a bi-phasic response suppression that was common to all hand conformation conditions. (B) Example of a nonlinear multimodal neuron across four representative hand conformation conditions. This neuron was not modulated by hand conformation prior to or immediately after cutaneous stimulation. The interaction effects between proprioception and cutaneous inputs transpired approximately 100ms after bar indentation. Note that the neuron had an initial response to the bar stimulus (∼20ms), but this was not modulated across hand conformation. (C) This graph shows the cumulative distribution of the onset time of integration effects in multimodal neurons. In addition, this graph plots the onset response time to bar stimuli of unimodal cutaneous and nonlinear multimodal neurons. The inset illustrates the median onset time for all four conditions. For visual purposes we do not show the trace of the onset response time to cutaneous stimuli of linear multimodal cells. The statistics show that their response time to cutaneous stimuli co-occurred with the multimodal integration effect. The error bars in all plots represent SEM.

Figure 6C shows the cumulative distribution of the onset time of integration effects in multimodal neurons. This figure also shows the onset response time to bar stimuli in unimodal cutaneous and nonlinear multimodal neurons. The inset graph in Figure 6C illustrates the median onset time for all other neuronal conditions. The data revealed significant differences in the onset time of integration effects between nonlinear multimodal neurons and all other conditions (Wilcoxon signed-rank tests across all possible pairs with Bonferroni correction, p < 0.00001). Particularly, the integration effect of nonlinear multimodal neurons was delayed 80ms relative to that of linear multimodal neurons (20ms vs. 100ms). In addition, the onset time of the integration effect in nonlinear multimodal neurons occurred later than their response to cutaneous inputs (20 ms). In contrast, the onset time of integration effects of linear multimodal neurons occurred during the same time as their response to cutaneous inputs. These results suggest that integration effects in linear multimodal neurons occur during the initial phase of sensory processing, whereas integration effects of nonlinear neurons are driven by feedback neural mechanisms.

Effects of proprioception on orientation tuning

Finally, we investigated whether the effects of proprioception on neurons' cutaneous responses were specific to their feature selectivity properties. Specifically, we examined whether hand conformation modulated the tuning properties of orientation selective cells. A two-way ANOVA with factors of Hand Conformation (20 levels) and Orientation (0°, 45°, 90°, and 135°) was performed on each neuron during the sustained indentation period (100 − 400ms). We found that Hand Conformation modulated the neural response on ∼80% of cells (211/263; same as results in Figure 4D). Moreover, we observed that 21% of neurons were tuned for oriented features (54/263). However, in contrast to our hypothesis, the data revealed that only 4% of neurons showed a significant interaction effect between Hand Conformation and Orientation (11/263), indicating that proprioception did not modulate the orientation tuning properties of cells. Figure 7 shows the response of three orientation selective neurons across a set of representative samples of hand conformation. As the figure shows, proprioception only modulated the baseline response of the tuning curve.

Figure 7. Proprioception does not modulate the tuning properties of orientation selective cells.

Examples of three orientation tuned neurons in SI across three representative hand conformation conditions. Statistics showed that proprioception failed to modulate the orientation preference of tuning strength of 96% of the recorded cells. Proprioception only seemed to modulate the baseline activity of the tuning curves. The error bars in all plots represent SEM.

Discussion

We studied the neural mechanisms underlying coding of proprioception and multimodal integration between cutaneous and proprioceptive inputs in the digit representation of SI. The data showed that proprioception was mediated by additive and non-additive interactions between digit displacements (i.e. position-scaled and posture-selective cells, respectively). Further, we found that a large fraction of cells in all areas of SI responded to both proprioceptive and cutaneous stimuli. Multimodal responses in these neurons were explained by linear and nonlinear interactions. In addition, the temporal incidence of linear integration effects occurred during the initial phase of stimulus processing, while nonlinear integration effects emerged during later stages (∼100 ms post-stimulus onset). Finally, contrary to our hypothesis, we failed to observe modulations of orientation tuning by proprioception. Taken together, these data argue against the prevalent model of modality specificity in somatosensory cortex, and provide evidence for distinct neural mechanisms of proprioception and multimodal processing in the somatosensory system.

Neural representation of proprioception in SI cortex

Our data showed that a large fraction of SI cells selectively responded to different hand conformations. Area 3a had the largest incidence of proprioceptive cells (73%), followed by areas 1 (53%), 2 (52%) and 3b (32%). These effects occurred in the absence of cutaneous stimuli, indicating that proprioception, by itself, can selectively drive the responses of many SI neurons outside the traditionally-defined unimodal proprioceptive area 3a. The data also revealed that proprioception effects were explained by additive (position-scaled) and non-additive (posture-selective) interactions within and between digit positions. Specifically, we found that neural responses of position-scaled cells gradually increased (or decreased) with corresponding digit displacements along a particular spatial axis. These neurons appear to be modulated similarly to the third type of kinesthetic neurons reported by (Costanzo and Gardner, 1980; Gardner and Costanzo, 1980) whose firing rate gradually increased during joint movements in their preferred direction. On the other hand, posture-selective neurons did not scale across a particular dimension, but rather were modulated by selective configurations of digit displacements.

The systematic and unsystematic patterns of proprioceptive effects underscore the complex neural coding scheme of proprioception in the somatosensory modality. Unlike other types of tactile sensory features (e.g. intensity or flutter frequency), our data showed that not all proprioception effects are encoded on a single and linear dimension. Rather, a large number of these somatosensory signals appear to be represented in the nonlinear combination of digit positions, a pattern that is reminiscent of the hand conformations used for grasping objects (Thakur et al., 2008). Indeed, while the human hand has about 22 degrees of freedom, studies have shown that only a small set of hand conformations are used for holding and exploring everyday objects (Klatzky and Lederman, 1995; Lederman and Klatzky, 1997; Santello et al., 2002; Santello and Soechting, 2000; Thakur et al., 2008). These different hand conformations, also known as synergies, account for over 90% of the variance during reaching, grasping and skilled hand movements (Thakur et al., 2008). Further, most of these hand synergies are characterized by a nonlinear spatial spread of the fingers enclosing the object. While we did not directly examine neural activity in response to different reaching and grasping movements, our data provides evidence that these types of hand synergies might emerge in SI cortex. An example can be observed in Figure 2B (upper right panel), which illustrates the response of a neuron with highest firing rate when D3 is displaced below the vertical meridian. This pattern is akin to holding a circular object with the palm pronated, which is similar to a set of synergies previously reported (Lederman and Klatzky, 1993; Thakur et al., 2008) (see e.g. Figure 3 in Thakur et al 2008). Another example can be observed in Figure 3B, which shows a neuron with maximum response when D2 and D4 are farthest apart. This pattern is indicative of a neuron that responds to stretching of the hand, a posture that may be useful for grasping large objects. Certainly, more studies are needed to determine the exact role of these proprioceptive cells.

We propose that proprioception is segregated into two submodality pathways that encode (1) kinesthetic inputs such as joint-angle and joint-velocity used for scanning objects, and (2) posture related inputs used for representing synergistic hand patterns during object grasping. Our working model is that these pathways operate in parallel and in concert, with kinesthetic inputs encoded by position-scaled neurons, while posture related inputs processed by posture-selective neural populations.

Proprioceptive and cutaneous integration mechanisms in SI cortex

In support of our hypothesis most SI cells responded to both cutaneous and proprioceptive stimuli. This effect was even observed in areas 3a and 3b cells, which are believed to be modality specific. Yet, while the majority of these neurons responded to multimodal inputs, most unimodal cells in 3a and 3b tended to respond to proprioceptive and cutaneous modality inputs, respectively. This is significant because it indicates that, while these areas are mostly multimodal, there seems to be a preferred modality driving their activity.

Multimodal integration effects were explained by linear and nonlinear interactions. Linear multimodal neurons showed significant responses to different hand conformations during the baseline period. After the presentation of the cutaneous stimulus their response was further modulated in a uniform manner across all proprioceptive conditions. In contrast, nonlinear multimodal neurons showed non-homogenous responses to different hand conformations during the post-stimulus period as compared to baseline. Specifically, we found that the combination of cutaneous and proprioceptive inputs either modified the preferred hand conformation during baseline (Types I, II and IV cells) or evoked a novel preferred hand conformation in neurons failing to show a proprioceptive effect prior to the onset of the cutaneous stimulus (Type III cells). The data also showed that their response to multimodal stimuli was not explained by a linear sum of the somatosensory inputs, indicating that these cells combine proprioceptive and cutaneous signals using nonlinear integration mechanisms. The same was not the case for linear multimodal cells, whose response modulations to cutaneous stimuli were independent of hand conformation. These types of linear and nonlinear integration mechanisms have been reported in other neural systems that combine inputs from multiple sensory modalities (e.g. superior colliculus) (Meredith and Stein, 1986; Stein and Meredith, 1990). This suggests that neural systems employ similar mechanisms for integrating modality signals within and between senses.

Behavioral studies in humans show that perceptual representations of cutaneous stimuli can be heavily influenced by how the hand contacts the object (Corcoran, 1977; Oldfield and Phillips, 1983; Rinker and Craig, 1994a; Sekiyama, 1991; Yoshioka et al., 2011). In particular, these studies show that perception of the same cutaneous stimulus is modulated by the spatial configuration of the hand, a pattern that is analogous to the integration effects of nonlinear multimodal integration cells. However, we found that proprioception did not modulate the tuning strength or preferred angle of orientation selective neurons. This is puzzling since previous studies show that proprioception can bias the representation of cutaneous spatial features such as motion and letters, suggesting that oriented features should also be modulated by hand conformation. However, a key difference between ours and previous behavioral studies lies in the way that proprioception was varied. Specifically, in our experiment proprioception was modulated by spreading and flexing individual digits, whereas in previous studies proprioception was modulated by varying the position and orientation of the wrist, elbow and shoulder. Thus, it is possible that neural processing of orientation and other spatial features is only sensitive to proprioceptive signals from these body parts. An alternate hypothesis is that proprioceptive and cutaneous integration mechanisms mediating these behavioral effects might take place in higher-order areas such as secondary somatosensory cortex, which contains cells that are orientation tuned and respond to proprioceptive inputs (Fitzgerald et al., 2004, 2006a, b; Thakur et al., 2006). Additional studies are needed to validate either of these hypotheses.

Temporal evolution of multimodal integration in the somatosensory system

Multimodal integration effects of linear and nonlinear neurons occurred at different phases of stimulus processing. In particular, we observed that multimodal linear cells integrated cutaneous and proprioceptive inputs approximately 20ms after bar stimulus onset. This integration effect coincided with their onset response time to bar stimuli as well as that of unimodal cutaneous neurons. These effects show that integration of cutaneous and proprioceptive inputs in linear multimodal cells takes place during the initial phase of stimulus processing in SI. Further, these data suggest that convergence of cutaneous and proprioceptive inputs in multimodal linear neurons arises from the combination of efferent activity from the rods and shell sections of modality-specific thalamic nuclei. An alternate hypothesis is that these multimodal effects take place in thalamic neurons, and that multimodal responses in these SI cells merely reflect the output activity of these sub-cortical cells.

Integration effects in nonlinear cells emerged during latter stages of stimulus processing, around 100ms after cutaneous stimulation. This latency represents a 80ms delay as compared to the integration effects in multimodal linear cells. Surprisingly however, multimodal nonlinear neurons showed an initial response to cutaneous inputs that were not modulated by proprioception and occurred during the same time as the response to the bar stimulus in unimodal cutaneous neurons. This pattern of effects suggests that their response properties to cutaneous stimuli are similar to those of unimodal cutaneous and multimodal linear cells. However, unlike the convergence effects in linear multimodal cells, integration of cutaneous and proprioceptive inputs in multimodal nonlinear neurons appears to be driven by feedback neural mechanisms that arise from cortico-cortico interactions.

The question arises, why are integration effects in multimodal nonlinear cells delayed about 80ms relative to their responses to cutaneous stimuli? While the answer is unclear, we surmise that the temporal delay might allow neurons to encode and/or relay cutaneous signals that are not modulated by proprioception. This might be an important mechanism for haptic object perception because it promotes invariant coding of oriented cutaneous features across the somatosensory system. This is a speculative interpretation that requires validation through empirical data.

Conclusion

The notion that haptic object perception involves integration of cutaneous and proprioceptive inputs has a long history dating back to Aristotle who showed that perception of edges is modulated when fingers are crossed (Benedetti, 1985). This finding is supported by recent psychophysical studies showing that perception of motion, size and shape features can be modulated by proprioception (Berryman et al., 2006; Lakatos and Marks, 1999; Pont et al., 1999; Rinker and Craig, 1994b; Voisin et al., 2002). Our data provide additional support by showing where and how these multimodal interactions begin to transpire in sensory cortex. In addition, our results provide strong evidence against the prevalent model of modality specificity in SI. Based on our findings, we propose that multimodal linear neurons are important for action by providing a continuous representation of how the hand contacts an object. In contrast, multimodal nonlinear neurons play an important role in perception by providing an integrated representation of the local and global features comprising the tactile object.

Experimental Procedures

All experimental protocols complied with the guidelines of the Johns Hopkins University Animal Care and Use Committee and the National Institutes of Health Guide for the Care and Use of Laboratory Animals. Animals sat comfortably on a custom-made chair with their hands held supinated while receiving a drop of water every 3-7 seconds to keep them in an aroused state (Figure 1A).

Animal surgery and neural mapping procedure

SU responses were recorded from four hemispheres in areas 3a, 3b, 1 and 2 of three macaque monkeys that weighed between 5-8 kg). Surgical procedures are described in detail in (DiCarlo et al., 1998). Briefly, a circular recording chamber was permanently placed over the animal's skull that targeted the finger areas of SI cortex (AP/ML 6/21). On each recording day a multichannel electrode drive (Mountcastle et al., 1991) was positioned over SI cortex using a custom-made positioner. Seven electrodes, made of glass filaments with tungsten-platinum metal cores, were spaced 400 microns apart and formed a linear array. Standard neurophysiological mapping procedures were conducted to verify that the recording sites corresponded to our regions of interest. Briefly, at the beginning of a recording session, the electrode setup was placed perpendicularly to the surface of the brain near the central sulcus (CS), where the hand region of area 1 is typically located. The electrode setup was arranged such that the most anterior electrode travelled inside the CS, several of the inner electrodes travelled inside area 1, and the most posterior electrodes travelled inside area 2. Neurons in the hand representation of area 1 tend to respond to stimulation of one or more distal finger pads. As we proceeded deeper in cortex, the center of the RF crept down to the palm in a serial succession from distal to proximal pads, and traveled back to the distal finger pads in a reversed sequence. This reversed progression provides a neurophysiological marker for the transitional boundary from area 1 to 3b. As expected, this portion of cortex was marked by neurons exhibiting strong responses to cutaneous stimulation. As the electrode was displaced deeper in the posterior bank of the CS, neurons' RF moved to the tip of the finger (sometimes, beyond the tip to fingernail), and then traveled back to distal finger pad, indicating that the electrode was located in area 3a. Contrary to the responses in area 3b, cortical neurons in this area are more responsive to deep tissue stimulation (i.e. proprioception). The RF properties of area 2 cells covered multiple finger pads and were responsive to deep tissue and cutaneous stimulation. Before each experimental session, we identified neurons with cutaneous RF over the distal finger pads of digits 2, 3 and 4 (D2, D3, and D4). Preliminary characterization of each neuron's modality selectivity and RF was done using a hand-held probe and passive movement of digits and joints by experimenters.

Due to technical limitations we were not able to properly isolate the proprioceptive RF of the recorded neurons. However, since the proprioceptive conditions comprised spreading (or contraction) of D2 and D4 as well as flexion (or extension) of the proximal joint pad of D3, we surmised that proprioceptive signals emanated from joint and skin-stretch receptors with RFs in or nearby the proximal pads of these digits.

Apparatus and Stimuli

Monkeys' hands were securely held on an exoskeleton, using molded plastic, with the fingers exposed and their nails glued to a fingernail holder to restrict movement. Cutaneous stimulation was delivered via a bar controlled by a linear motor assembly mounted on a forcer/platen system (Lane et al., 2010). This motor allowed the bar to be positioned at any ‘x/y’ location over the animal's hand. The bar was attached to the bottom of a stepper motor, which allowed control of the bar's orientation with fine resolution (<1.0°). The length of the bar was 10mm with a 45° wedge-shaped edge, and the ends were smoothed to have a 1 mm radius curvature. The bar was indented 1.0mm using a 100ms on/off ramp and a 500ms steady indentation period. Note that 0ms illustrates the onset of the steady indentation period. Proprioception was modulated by systematically varying animals' hand conformation using a motorized exoskeleton.

We implemented a paradigm composed of forty-five proprioceptive and twelve cutaneous stimulation conditions (Figure 1). Cutaneous stimuli comprised bar indentations with four different orientations (0°, 45°, 90°, and 135°) applied to the distal pads of D2, D3 and D4. Proprioceptive conditions were constructed by varying the spatial positions of D2 and D4 in the horizontal plane (i.e. horizontal positions of 0°: in position, 11.25°: intermediate position and 22.5°: out position) and flexing D3 in the vertical plane (-22.5°, -11.25°, 0°, 11.25° and 22.5°). Given the large number of experimental conditions and limited lifetime of single-cell recording, we randomly selected twenty proprioceptive conditions presented in combination with the full set of cutaneous stimuli. This resulted in 240 stimulus conditions (20 proprioceptive × 12 cutaneous conditions), which were repeated five times. Proprioception conditions were fully randomized. However, to reduce the amount of experimental time, artifacts produced by the motors, and kinesthetic effects influencing neural responses, one set of the twelve cutaneous stimulation conditions was presented in a pseudo-randomized order with the hand placed in a particular conformation. After presenting the twelve cutaneous stimuli, the hand posture was varied to one of the twenty pre-selected conformations and another set of the twelve cutaneous stimuli was presented. This sequence was repeated until the remaining twenty proprioceptive conditions were presented, which constituted one cycle of 240 experimental conditions (i.e. 20 hand conformation × 12 cutaneous stimuli). Each experimental session was composed of five cycles (i.e. five repetitions of each proprioceptive and cutaneous stimulation condition; N = 1200 trials), and within each cycle the order of proprioceptive conditions was randomized. Further, the inter-stimulus interval (ISI) between cutaneous stimulation was 1050ms for two animals and 1800ms for the remaining animal. This change in ISI had no effects on the neural responses to cutaneous or proprioceptive stimuli. Importantly, we note that because of our hand conformation randomization procedure, proprioceptive effects (and interactions between cutaneous and proprioceptive conditions) were unlikely to be driven by experimental temporal factors that modulate the firing rate statistics of a neuron (e.g. firing rate modulations produced by ‘cell loss’ or ‘inclusion of multi-unit activity’). Neurons that displayed unstable isolation properties or emitted average firing rates below 3Hz were discarded. This resulted in 263 neurons: 32 in area 3a, 74 in area 3b, 77 in area 1, and 80 in area 2.

Unless otherwise mentioned, effects of proprioception were analyzed by averaging activity between 400ms and 100ms prior to the steady indentation period of the tactile bar (i.e. -400 to -100ms). In contrast, effects of cutaneous stimulation were analyzed by averaging activity between 0ms and 400ms after the onset of steady indentation time of the tactile bar stimulus. However, because orientation signals are encoded in the sustained portion of the tactile indentation period (Bensmaia et al., 2008), effects of proprioception on orientation tuning were assessed in the averaged activity between 100ms and 400ms.

Analyses

Statistical effects were assessed by conducting independent-sample analysis of variance (ANOVA) in each cell. However, given that the experimental design was unbalanced, we further analyzed the effects of proprioception in the absence of cutaneous inputs using a model-based linear regression method combined with a bootstrapping technique. This regression analysis allowed us to assess whether hand conformation effects were a product of linear or nonlinear interactions between finger positions. Ten predictor variables were used to fit the regression model (see equation 1 below).

$$Y={\beta }_0+\sum _{i=2}^4({\beta }_i{D}_i+{\beta }_{\mathit{\text{ii}}}{D}_i^2)+\sum _{i=2}^3\sum _{j=i+1}^4{\beta }_{\mathit{\text{ij}}}{D}_i{D}_j$$ (1)

D represents finger position of D2, D3 and D4, β₀ is a constant bias term, and β_i's are the first-order regression coefficients. β_ii's and β_ij's are the second-order regression coefficients. Statistically significant predictors were selected using bootstrapping techniques by repeating the regression model 1000 times and selecting predictors with same sign more than 97.5% of all cases (<0.05, two-tailed case). Note that orientation conditions were pooled.

Temporal evolution of integration effects

For each neuron, each trial epoch was discretized in separate bins of 40 sample points (±20ms), and an ANOVA with factor of Hand Conformation was performed on each bin to assess the initial time bin at which proprioception modulated the response to the cutaneous stimulus. A statistically significant effect was established when the ANOVA yielded at least two consecutive time bins with p-values < 0.05 for the same hand conformation condition. Differences across median onset times were tested using Wilcoxon signed-rank tests across all possible pairs with Bonferroni correction.

Effects of proprioception on orientation tuning

A two-way ANOVA with factors of Hand Conformation (20 levels) and Orientation (0°, 45°, 90°, and 135°) was performed on each neuron during the sustained period of the bar indentation (100ms – 400ms). A main effect of Hand Conformation only signified that the cell was modulated by proprioception, but was not tuned for oriented features. A main effect of Orientation only indicated that the neuron was tuned for oriented features, but was not modulated by proprioception. An interaction effect, regardless of whether it displayed a main effect of Orientation or Hand Conformation, indicated that proprioception modulated the tuning properties of an orientation selective cell. Interaction effects were followed up with post-hoc tests to assess whether proprioception modulated the orientation preference or tuning strength of the neuron.