High-dimensional representation of texture in somatosensory cortex of primates

Significance

The sense of touch affords a remarkable sensitivity to the microstructure of surfaces, affording us the ability to sense elements ranging in size from tens of nanometers to tens of millimeters. The hand sends signals about texture to the brain using three classes of nerve fibers through two neural codes: coarse features in spatial patterns of activation and fine features in precise temporal spiking patterns. In this study, we show that these nerve signals culminate in a complex, high-dimensional representation of texture in somatosensory cortex, whose structure can account for the structure of texture perception. This complexity arises from the neurons that act as idiosyncratic detectors of spatial and/or temporal motifs in the afferent input.

Abstract

In the somatosensory nerves, the tactile perception of texture is driven by spatial and temporal patterns of activation distributed across three populations of afferents. These disparate streams of information must then be integrated centrally to achieve a unified percept of texture. To investigate the representation of texture in somatosensory cortex, we scanned a wide range of natural textures across the fingertips of rhesus macaques and recorded the responses evoked in Brodmann’s areas 3b, 1, and 2. We found that texture identity is reliably encoded in the idiosyncratic responses of populations of cortical neurons, giving rise to a high-dimensional representation of texture. Cortical neurons fall along a continuum in their sensitivity to fine vs. coarse texture, and neurons at the extrema of this continuum seem to receive their major input from different afferent populations. Finally, we show that cortical responses can account for several aspects of texture perception in humans.

Our sense of touch endows us with an exquisite sensitivity to surface microstructure. We can perceive surface features that range in size from tens of nanometers (1) to tens of millimeters and integrate these to form a cohesive textural percept. In the somatosensory nerves, surface features at different spatial scales are encoded in different populations of afferents and rely on different neural representations. Coarse-surface features are reflected in the spatial patterns of activation evoked in slowly adapting type-1 (SA1) and rapidly adapting (RA) afferents, whose small receptive fields give rise to a faithful neural image of surface elements measured in millimeters (2, 3). However, many tangible surface features are too small and too close together to be encoded spatially because the spatial code is limited by the innervation density of the skin (4, 5). To perceive fine textural features requires movement between skin and surface, which leads to the elicitation of texture-specific skin vibrations, which in turn evoke precisely timed texture-specific spiking patterns in RA and Pacinian corpuscle-associated (PC) afferents (6–11). These spatial and temporal representations must be combined and synthesized to achieve a unified percept of texture, a process about which little is known.

While neurons in somatosensory cortex have been shown to encode information about texture, previous studies investigating cortical texture representations used surfaces with elements in the range of millimeters, such as Braille-like dot patterns (12, 13) and gratings (14–16), which span only a small fraction of the wide range of tangible textures. We have previously shown that responses to such textures—which only engage the spatial mechanism—provide an incomplete view of the neural mechanisms that mediate the perception of texture (10, 11).

To fill this gap, we examined how textures that span the tangible range are encoded in somatosensory cortex. To this end, we scanned a wide range of textures—including fabrics, furs, and papers, in addition to the traditional embossed dots and gratings—across the fingertips of (awake) rhesus macaques and recorded the responses evoked in somatosensory cortex, including Brodmann’s areas 3b, 1, and 2. First, we found that texture identity is faithfully encoded by these neuronal populations and that texture information is distributed across neurons which each exhibit idiosyncratic texture responses. Second, we showed that the heterogeneity across somatosensory neurons is in part driven by differences in the submodality composition of their input (SA1, RA, and PC). We then discovered the downstream recipients of the spatial and temporal codes observed at the periphery: A subpopulation of cortical neurons receives strong input from SA1 fibers and preferentially encodes coarse textural features, whereas another population of neurons receives strong input from PC fibers and preferentially encodes fine surface features. Finally, we showed that the responses of somatosensory neurons account for psychophysical reports of texture obtained from human observers.

Results

We recorded the activity evoked in 141 neurons in somatosensory cortex (35 from area 3b, 81 from area 1, and 25 from area 2) from three rhesus macaques with receptive fields on the distal fingertip, as each of 59 textured surfaces (SI Appendix, Table S1) was scanned across the skin by using a rotating drum stimulator, which allowed for precise control of scanning speed and indentation depth (Fig. 1 A and B). These surfaces were chosen to vary widely in microstructure and material properties in an attempt to explore as fully as possible the range of everyday textures. The objective of the study was to determine the degree to which texture information is encoded in cortex, examine the nature of this representation, and assess the degree to which this representation can account for perception.

Fig. 1.

Experimental apparatus and sample texture responses. (A) Textures were passively presented to the distal finger pads of awake macaques. (B) The rotating drum stimulator—on which the 59 textures were mounted—allows a surface to be scanned across the fingertip at a precise and repeatable speed and depth of indentation into the skin. (C) Sample spiking responses of five neurons each in areas 3b, 1, and 2 to five repetitions of eight textured surfaces.

Neurons in Somatosensory Cortex Encode Texture.

First, we examined the degree to which the responses of individual somatosensory neurons were modulated by texture (Fig. 1C). We found that nearly every neuron responded to at least one texture (140 of 141 neurons modulated above baseline firing rate; P < 0.05, permutation test with Bonferroni correction) and that each texture significantly modulated the response of at least 20% of the neurons (0.23–0.72–92, minimum–median–maximum proportion of textures across cells, permutation test). To test whether these neurons carry texture-specific information, we built a simple linear classifier based on single-trial spike counts. Nearly all neurons yielded classification performance that was significantly above chance (mean ± SD of performance: 6.7 ± 3.7%, chance performance: 1.7%, 95% of neurons > chance), and neurons that yielded better than chance performance were approximately equally prevalent in areas 3b, 1, and 2 (97%, 96%, and 88%, respectively; SI Appendix, Fig. S1A).

Next, we examined the degree to which texture identity is encoded in the responses of populations of somatosensory neurons (Fig. 2A). To this end, we implemented the texture classifier using the responses of groups of neurons of varying size. We found that high-classification performance could be achieved with a small population of somatosensory neurons (as few as 83 neurons yielded 97% performance) and that the full population yielded nearly perfect performance (Fig. 2B). Classification performance was largely comparable across cortical modules (an average of 73%, 72%, and 62% for groups of 25 neurons in areas 3b, 1, and 2, respectively; SI Appendix, Fig. S1B) and was robust to (simulated) noise correlations (SI Appendix, Fig. S2A). In summary, small populations of somatosensory neurons convey sufficient information to support texture identification for a large and diverse texture set.

Fig. 2.

Neurons in somatosensory cortex encode texture. (A) The mean firing rate (across repetitions) evoked by each of 59 textures (columns) in somatosensory neurons, split by cortical field. Firing rates are normalized within neurons for display purposes, ranging from low (blue) to high (yellow). Textures are sorted according to the first principal component of the population response from lowest to highest. Cells are ordered first by area, then by variance of their firing rates across textures. Somatosensory neurons exhibit heterogeneous responses to textures. (B) Texture-classification performance of groups of cortical neurons vs. the size of the neuron group. As expected, classification performance improves as more cells are included. Gray dots denote the performance of individual neuronal groups, and the black trace denotes the mean as a function of group size. Groups of 66 cells, marked by the gray dashed line, yield a near-asymptotic performance of 97%.

The Cortical Representation of Texture Is High-Dimensional.

Two factors drive the ability of neural populations to classify stimuli more accurately than do individual cells. First, as more neurons are included, the trial-to-trial variation in response is averaged out. Second, increasing the variety in tuning properties in the neural population can more effectively represent the high-dimensional character of a complex stimulus, and thus increase the effective dimensionality of the resulting neural representation. That is, insofar as different neurons respond to different aspects of a surface, these idiosyncratic responses will provide information beyond that available from simply averaging responses across cells.

We examined the dimensionality of texture responses—the degree to which somatosensory neurons respond heterogeneously to texture—by performing a principal components analysis (PCA) on the population response. That is, we first characterized the correlational structure in texture responses across neurons in somatosensory cortex and then assessed the degree to which responses could be reduced to a smaller set of nonredundant signals. We found that most of the variance in neuronal responses was explained by the first principal component (Fig. 3A) (65% proportion of variance explained from the first component, essentially the mean population firing rate; R² = 0.99), a signal that was strongly preserved across all three cortical areas [with intercorrelations of first principal components across pairs of areas all >0.95 (SI Appendix, Fig. S1C)]. As discussed below, this prominent neuronal dimension has a clear perceptual correlate.

Fig. 3.

The cortical representation of texture is high-dimensional. (A) Cumulative scree plot (proportion of variance explained) for the PCA on the population response to texture. The bulk of the response variance is carried by the first few components. (B) Texture classification (as in Fig. 2) using the cortical population response with the full population response (gray line) and with the first principal component removed (black line). The red dotted line denotes chance performance. Classification performance is only slightly reduced when this first component is removed. (C) Average classification performance of each individual principal component when distinguishing between pairs of textures. Responses were above chance, even for components that explained only a small proportion of the total variance. Error bars represent the SD across texture pairs and shuffles of the training and test sets. (D) Classification performance based on firing rates projected onto a subset of principal components, built by excluding the n principal components in decreasing order of their eigenvalues (i.e., removing the largest components first). Error bars denote the SD across shuffles of the training and testing sets. Even when dozens of the high-variance principal components are removed from the response, texture classification is still above chance.

We sought to characterize whether heterogeneity in texture responses across neurons provides texture-specific information beyond that found in the mean population response. To this end, we again implemented the texture classifier, this time using only a subset of the principal components of the neural response. When the population response was collapsed onto a single dimension—the first principal component—classification performance dropped to 41%, compared with 99.4% when the entire response was used. Conversely, if we removed only the first principal component from the population response and preserved all other components, we achieved 92% classification accuracy with as few as 83 cells and 97% accuracy with the full population of 141 cells (Fig. 3B). In other words, the heterogeneity of neural responses to texture is a major contributor to the texture signal in cortex.

Given the importance of low-variance dimensions to classification performance, we sought to assess how many of these dimensions are reliably informative about texture identity. To this end, we first quantified how many dimensions identified through PCA reliably carried texture information. We found that the first 30 dimensions carried sufficient texture information to distinguish pairs of textures (Fig. 3C; 95% of trial shuffles yielded above chance performance). We then examined the degree to which the response retained information about texture when multiple principal components were cumulatively removed (Fig. 3D). We found classification performance to be well above chance, even after removing 33 principal components (95% of trial shuffles yielded above-chance performance). Because the outcome of these analyses may depend on the structure of the trial-to-trial variability in the response, we verified that the measured dimensionality was robust to (simulated) noise correlations (SI Appendix, Fig. S2 B–D).

Finally, because PCA does not necessarily identify the most informative dimensions of response, we implemented a recently developed measure of dimensionality which is not based on explained variance (like PCA) but, rather, gauges the ability of the response to reliably divide up the stimulus space [cf. Rigotti et al. (17); Materials and Methods]. Using this method, we found that the cortical response to texture can consistently classify split groups of up to 22 textures, suggesting that the texture representation in somatosensory cortex is at least 21-dimensional (SI Appendix, Fig. S2 E–I). Furthermore, this measurement of 21 response dimensions is likely an underestimate: Our classifier-based estimate of dimensionality is not only capped by the dimensionality of the neuronal representation, but also by the size of the stimulus set and of the recorded neuronal population (17). Indeed, we find that the dimensionality is still rapidly increasing as a function of neuronal group size for 141 cells (SI Appendix, Fig. S2F), so more neurons would likely yield an even higher-dimensional representation in response to our texture set. In total, these classification results suggest that the dimensionality of the neural representation is driven by a large number (dozens) of components which, while often only accounting individually for a small fraction of the overall response variance, nonetheless carry significant texture information.

Some Heterogeneity in Cortical Responses Can Be Attributed to Differences in Submodality Input.

Next, we examined the degree to which the cortical response inherits its structure from the periphery, where texture signals are carried by three classes of low-threshold tactile nerve fibers. To this end, we leveraged previously obtained recordings of afferent responses (from 17 SA1, 15 RA, and 7 PC fibers) to a subset of 24 textures also used in the present study (10). We then evaluated, using multiple regression, the extent to which the mean population firing rate of SA1, RA, and PC afferents evoked by these 24 common textures could account for the firing rates of individual cortical cells, using the resulting standardized regression coefficients as a gauge of the relative similarity of each tactile submodality to each cortical neuron.

First, we found that the different cortical neurons received their strongest input from different classes of tactile nerve fibers (44.7%, 37.6%, and 17.7% of neurons showed maximum regression coefficients from SA1, RA, and PC afferents, respectively). Second, the responses of individual somatosensory neurons implied submodality convergence, as reflected by the fact that many cortical neurons were significantly better explained by a combination of multiple afferents than they were by any single afferent (F test: 28% of cells better explained by all three coefficients than any single coefficient, P < 0.05). Because this test has low statistical power given the small number of common stimuli between the peripheral and cortical datasets, we also examined the adaptation properties of cortical neurons [that is, the dynamics of their responses to trapezoidal skin indentation (19)]. We found that many neurons (69%) showed both significant responses during the sustained portion of the indentation, indicative of SA1 input, as well as significant responses upon the removal of the probe, indicative of RA or PC input (SI Appendix, Fig. S3 A–D). Overall, 80% of neurons displayed submodality convergence by one or both of these measures. Thus, even at the single-neuron level, the texture representation in somatosensory cortex is built from signals integrated across tactile submodalities.

Next, we examined what aspects of the high-dimensional texture representation in somatosensory cortex were inherited from structure in its peripheral inputs. To this end, we recalculated our PCA on both the peripheral and cortical population responses to their shared set of 24 textures. Using canonical correlation analysis (Materials and Methods), we found that the first three dimensions of the peripheral firing rates were significantly predictive of their cortical counterparts, but dimensions beyond these three did not yield better predictions (Fig. 4A). Within this shared space, the first principal axis in the cortex was highly correlated with its peripheral counterpart (r = 0.93). The second principal axis in the cortex was also correlated with its counterpart in the periphery (r = 0.89), and this axis separated neurons with strong SA1 input (and, to a lesser extent, RA input) from those with strong PC input. Indeed, the correlation between the weight of the second principal axis in the cortex and the SA1, RA, and PC regression coefficient was −0.43, −0.16, and 0.76, respectively. Furthermore, neurons that received strong PC input tended to produce texture responses that were correlated with each other but uncorrelated with the responses of neurons driven primarily by SA1 or RA responses (Fig. 4B), reflecting the stark difference in response properties of these two sources of input. Interestingly, the most strongly PC-like cells were predominantly located in area 1 (10 of 12 of cells with normalized PC weight > 0.8; the other 2 were in area 2; SI Appendix, Fig. S1D). Thus, the second dimension of variance in the cortical response has, at one extreme, SA1-like neurons and, at the other extreme, PC-like ones. The third principal axis in the cortex also showed correlation with its peripheral counterpart (r = 0.82), but its meaning is unclear. Although the first few principal axes of the texture representation in the cortex are inherited from the periphery, much of the structure in the cortical representation beyond these axes cannot be explained straightforwardly from the relative strengths of SA1, RA, and PC input.

Fig. 4.

Some heterogeneity in cortical responses can be attributed to differences in submodality input. (A) Strength of the prediction of cortical responses from the first n principal components of the peripheral texture response (implemented by using canonical correlation analysis; Materials and Methods). Beyond the first three principal components, performance declines due to overfitting. (B) Correlation matrix of texture-elicited firing rates with each row and column corresponding to a different neuron (cells with mean texture response > 40 Hz, n = 74). Cells are ordered by their PC regression weight, from least PC-like (lower left) to most PC-like (upper right). The red line divides neurons with PC regression weights greater than or less than 0.5. The most PC-like cells in somatosensory cortex tend to cluster because their texture-evoked firing rates are distinct from those of other neurons.

Neurons in Somatosensory Cortex Encode Textural Features at Different Spatial Scales.

At the periphery, texture-specific surface features are encoded through multiple mechanisms. Coarse surface features—measured in millimeters—are primarily encoded in the spatial pattern of activation across of SA1 fibers (20) [and perhaps RA fibers as well (11)]. In contrast, fine surface features—typically measured in the tens or hundreds of micrometers—drive characteristic vibrations in the skin during texture scanning (9, 21, 22). These vibrations (and, by extension, textural features) are encoded in precisely timed, texture-specific temporal patterns in RA and PC fibers (10). Next, then, we sought to examine how these peripheral codes for texture were reflected in cortical responses.

First, we tested the hypothesis that a subpopulation of somatosensory neurons act as spatial filters, well suited to extract information about coarse textural features, as has been proposed (12, 23). We also wished to assess the spatial scale over which such a mechanism might operate. To this end, we first characterized the spatial receptive fields of somatosensory neurons using well-established techniques (SI Appendix, Fig. S4 A–C). By using this approach, neurons have been shown to encode spatial features with excitatory subfields flanked by inhibitory ones (12), analogous to simple cells in the primary visual cortex (24). Consistent with previous reports, the measured receptive fields exhibited well-defined excitatory subfields (average 12.7 mm², range 3.1–37.4 mm²) and inhibitory subfields (average 12.8 mm², range 0–42.6 mm²). Inhibitory subfields tended to lag behind excitatory subfields along the scanning direction (62 of 67, or 93%, average 2.5 mm lag) (SI Appendix, Fig. S4D). Importantly, the spatial period of the subfield—that is, the distance between the excitatory and inhibitory subfields—spanned a range from 2 to 4 mm (SI Appendix, Fig. S4E). Thus, the spatial structure of cortical receptive fields is well suited to extract information about coarse features, but not fine ones. Note that this receptive field structure is ideal for computing the spatial derivative of the neural image, which has been shown to drive perceived roughness of coarsely textured surfaces (11, 18, 20). Counterintuitively, while PC fibers have substantially larger receptive fields than do SA1 or RA fibers, this tendency was not reflected in their cortical targets. Indeed, the receptive fields of PC-like neurons were of similar size as their SA1- or RA-like counterparts (excitatory subfield size: average 12.1 mm²; inhibitory subfield size: average 13.1 mm²; average 2.3 mm lag at 80 mm/s; SI Appendix, Fig. S4 F and G).

Next, we examined the cortical manifestation of the temporal code for fine textural features carried at the periphery by RA and PC fibers. A characteristic feature of PC (and to some extent RA) responses to texture is the elicitation of high-frequency spiking patterns (>50 Hz) that are highly informative about texture identity, as these patterns reflect the succession of fine textural elements moving across their receptive fields (10). To explore the presence of such timing signals in the responses of somatosensory neurons, we designed two finely textured 3D patterns—gratings with spatial periods of 0.5 and 1 mm—to elicit skin vibrations at 160 and 80 Hz, respectively (given a scanning speed of 80 mm/s). We anticipated that these highly periodic components would be readily identifiable in the cortical responses and might encode fine textural features. We found that a subpopulation of somatosensory neurons produced phase-locked responses to these and other fine textures (SI Appendix, Fig. S3E), providing a strong analog to the temporal code observed at the periphery. As expected, phase-locked responses were stronger among somatosensory neurons with PC-like responses than among their SA1-like counterparts (Fig. 5B and SI Appendix, Fig. S3E). Indeed, while the spiking patterns of both sets of neurons consistently reflected the periodic structure of coarse features, PC-like responses much more reliably reflected the periodic structure of fine features, even if these were embedded among coarse features. Neurons with PC input are thus well suited to convey information about fine textural features.

Fig. 5.

Neurons in somatosensory cortex encode textural features at different spatial scales. (A) Spiking responses of seven SA1-like neurons (green; cells with SA1 regression coefficient > 0.5, n = 53) and seven PC-like neurons (orange; cells with PC regression coefficient > 0.5, n = 23) in response to five repeated presentations of three different textures: dots spaced 7.7 mm apart, a 1-mm-period grating, and a superposition of the dots with the grating. SA1-like responses exhibit strong entrainment to the coarse component of the texture (dot pattern). PC-like cells are more strongly driven by the fine component of the texture (grating). (B) Mean amplitude spectrum of the spiking responses of SA1-like (green) and PC-like (orange) cells to the same three textures as in A. PC-like cells exhibit high-frequency phase-locking to the temporal period of the grating (80 Hz), even when the dots are present, whereas SA1-like cells do not. (C) Discriminability (d′) of nine 3D-printed textures based on the firing rates they evoke in SA1- and PC-like neurons (green and orange, respectively). Error bars denote the bootstrapped SEMs across cells and texture pairs. While PC-like cells are sensitive to both coarse and fine features, SA1-like cells are sensitive only to coarse ones.

In light of these observations, we wished to assess the respective abilities of these two subpopulations of neurons—SA1- and PC-like—to convey information about fine and coarse features. To this end, we examined the responses of these two neuronal populations to nine 3D-printed surfaces (SI Appendix, Fig. S5) in which coarse and fine features were parametrically combined (Fig. 5 A and B). We found that SA1-like neurons responded significantly more strongly to textures with coarse features than without, exhibiting only weak firing-rate modulation to the presence of fine features (20.7 vs. 2.40 spikes per s for coarse vs. fine, respectively; P < 0.001, paired t test). Conversely, PC-like neurons responded more strongly to textures with fine features than to those without (15.9 vs. 2.0 spikes per s, for fine vs. coarse, respectively; P < 0.01), and their rates were nearly independent of the presence or absence of coarse features. As might be expected, these differences in sensitivity to coarse and fine textures led to corresponding differences in the ability of individual cortical neurons to discriminate pairs of textures (measured by using a standard sensitivity index, d′). SA1-like responses were significantly better at discriminating coarse features—independent of fine features—than were their PC-like counterparts (P < 0.05, permutation test), and PC-like neurons were significantly better at discriminating fine features—independently of the coarse features—than were SA1-like neurons (P < 10⁻⁴, permutation test) (Fig. 5C). In conclusion, then, different subpopulations of somatosensory neurons preferentially encode textural features at different spatial scales.

Neuronal Responses Account for Perceptual Judgments of Texture.

Next, we examined how these different populations of neurons might account for the perception of texture, an important step in establishing a neural code (20, 25, 26). To this end, we first investigated whether the responses of neurons in somatosensory cortex could account for judgments of surface roughness. Human subjects were presented with textured surfaces in an identical setup as the neurophysiological experiments and freely rated the roughness of each surface (n = 6 subjects, subject correlation to the mean: r = 0.87 ± 0.079, mean ± SD). We found that the firing rates of most somatosensory neurons (92%) were significantly positively correlated with roughness judgments (r = 0.59 ± 0.27, mean + SD for individual cells, 130 of 141 cells with significantly positive correlation at P < 0.05, permutation test) and that the first principal component of the population response was a good predictor of roughness (Fig. 6A; r = 0.88), a consistent effect across all three cortical fields (all r > 0.85; SI Appendix, Fig. S1 E and F).

Fig. 6.

Neuronal responses account for perceptual judgments of texture. (A) Perceived roughness vs. the first principal component of the cortical population response (r = 0.88). (B) Prediction of the perceived dissimilarity for two different groups of textures based on the firing rates of SA1-like neurons (green; SA1 coefficient > 0.5, n = 53) and PC-like neurons (orange; PC coefficient > 0.5, n = 23). SA1-like cells predict perceived differences in coarse spatial structure, whereas PC-like cells predict perceived differences in fine structure.

Although roughness is the dominant sensory dimension of texture, the perceptual space of texture also comprises other well-established sensory continua, such as hardness/softness, stickiness/slipperiness, and warmth/coolness (27). Together, these continua combine to account for some, but not all, aspects of the multidimensional sensory experience of texture (27–29). To assess the degree to which the cortical representation can account for the perceptual space, we examined the degree to which neuronal responses could account for judgments of texture dissimilarity. As with the roughness experiment, human subjects freely rated the perceived dissimilarity of pairs of textures (n = 10 subjects). For this analysis, we determined the degree to which judgments of dissimilarity mirrored differences in the evoked neuronal responses (Materials and Methods). We carried out this analysis on data obtained from two sets of texture pairs: one in which textures differed in their coarse spatial features (coarse group: three fabrics and two dot patterns, yielding 10 pairs, subject correlation to the mean, r = 0.88 ± 0.10, mean ± SD) and one that comprised textures mostly lacking coarse spatial features (fine group: 13 fabrics, 78 pairs, subject correlation to the mean, r = 0.65 ± 0.12, mean ± SD). First, we examined the ability of afferent responses to predict perceived dissimilarity. We found that SA1-afferent responses best accounted for the dissimilarity of textures with different coarse spatial features (coarse group: correlation between firing rates and perceived dissimilarity = 0.78, 0.41, and 0.17 for groups of seven SA1, RA, and PC fibers, respectively), and PC-afferent responses best accounted for the perceived dissimilarity of finely textured fabrics (fine group: r = 0.37, 0.39, and 0.56 for groups of seven SA1, RA, and PC fibers). When we carried out the same analysis based on cortical responses (Fig. 6B), we found that the responses of SA1-like neurons best accounted for the perceived dissimilarity of coarse-group pairs (r = 0.70, 0.24, and 0.26 for groups of seven SA1-, RA-, and PC-like neurons, respectively), and the responses of PC-like neurons best accounted for that of fine-group pairs (r = 0.31, 0.26, and 0.67 for groups of seven SA1-, RA-, and PC-like neurons, respectively). Results from this analysis further support the hypothesis that different subpopulations of neurons encode textures at different spatial scales: SA1-like neurons are specialists for coarse textural features, and PC-like neurons are specialists for fine ones.

Discussion

The Neural Mechanisms That Give Rise to a High-Dimensional Representation of Texture in Cortex.

While texture responses in cortex are dominated by a common signal that encodes roughness, the heterogeneity of responses across individual somatosensory neurons carries considerable information about texture identity. One identifiable way in which neurons differ is in the degree to which their responses reflect SA1-afferent input vs. PC-afferent input, a continuum rather than a dichotomy, as evidenced by the continuous distribution of regression coefficients (SI Appendix, Fig. S1D). However, cortical responses to texture are not simply a linear combination of afferent firing rates. Indeed, while we found three shared dimensions between the peripheral and cortical representations of texture, our binary classification analysis showed that the effective number of dimensions is much higher than three (Fig. 3B).

The observed increase in dimensionality from periphery to cortex is supported by an amplification of the somatosensory representation: The fingertip region of area 3b contains ∼250 cells for every corresponding peripheral afferent (5, 30, 31). This expansion does not simply involve the establishment of redundancy in the cortex, giving rise to populations of similarly tuned neurons. Rather, cortical neurons exhibit heterogeneous tuning: Each cortical neuron signals the presence of a specific spatial pattern of afferent activity within a range of spatial scales (from ∼1 to 10 mm) (12, 13) and/or a specific temporal pattern of afferent activity within a range of time scales (from ∼1 to 100 ms) (32). While these spatial and temporal filters are often estimated by using linear models, this integration of peripheral input is subject to the nonlinear motifs of neural processing, including thresholding (12, 32, 33), synaptic depression (33–35), and divisive normalization (36–39). This feature extraction and the associated nonlinear transformations result in a high-dimensional representation of texture, a process that is not unique to the somatosensory system. Indeed, high-dimensional representations have been observed in the visual (40–43) and olfactory (44) systems, cerebellum (45), hippocampus (46), and prefrontal cortex (17), to name a few.

The Perceptual Space of Texture Is also High-Dimensional.

The multidimensional nature of the texture representation in somatosensory cortex reflects the complexity of the space in which surface materials and microstructures reside and the resulting perceptual space of textures. While some aspects of this space can be captured by a small number of commonly identified sensory dimensions—roughness, hardness, stickiness, and warmth (27)—many others cannot. That is, the roughness, hardness, stickiness, and warmth of a textured surface define it only partially. Many adjectives to describe texture—fuzzy, bumpy, and silky, to name just a few—evoke additional textural features not captured in low-dimensional descriptions. Further dimensions may not be simply captured by such intuitive descriptors—one can imagine fields of repeating elements arranged in different configurations that are discriminable in a way that is difficult or impossible to articulate. Thus, in a manner broadly analogous to the space of visual shape (42, 47) and visual texture (48, 49), the complex neural space of tactile texture is reflected in a complex perceptual space.

Texture Representations in Cortex Fall on a Continuum of Spatial Scales.

As described above, somatosensory neurons fall along a continuum—captured in the second principal component of their responses—that seems to be determined by their peripheral inputs. The position of a neuron along this axis relates to the spatial scale of the textures it is best suited to encode. At one end of the continuum, SA1-like neurons encode coarse textural elements; at the other end, PC-like neurons encode fine features. This differential spatial sensitivity is reflected in the ability of neurons to convey information about texture: SA1-like responses best distinguish textures with different coarse features, while PC-like responses best distinguish textures with different fine features. These differences are also reflected in the ability of neuronal populations to predict perceptual judgments of texture: SA1-like neurons account for the perception of coarse features; PC-like neurons account for the perception of fine features.

In the peripheral nerve, coarse and fine textures are encoded through two mechanisms, a spatial code and a temporal one, respectively. Somatosensory neurons are well suited to extract coarse textural features measured in millimeters as evidenced by the spatial dimensions of their receptive fields (SI Appendix, Fig. S4 E and F) (12, 24). As discussed above, the idiosyncratic receptive field structure of individual neurons (SI Appendix, Fig. S4C) confers to them idiosyncratic preferences for coarse textural features and likely drives, in part, the heterogeneity of texture responses. Furthermore, the computation that such receptive fields imply—of spatial variation—has been shown to determine perceived roughness (10, 11, 18, 25). Thus, whereas spatial variation in afferent responses predicts roughness judgments, cortical firing rates predict roughness judgments (50–52) because they reflect the output of this differentiation computation.

At the periphery, however, the spatial mechanism cannot account for the perception of fine features due to limitations of the skin to transmit those features to the receptors (2, 53, 54) and limitations set by the cutaneous innervation density (4, 5). When the finger slides across a textured surface, vibrations are elicited in the skin. These vibrations are highly texture-specific (8, 9, 21, 22, 55) and, in turn, drive precise temporal spiking patterns in RA and particularly PC fibers (10). Somatosensory neurons implement temporal variation computations (32), which amount to extracting features in the temporal spiking patterns and converting them into rate-based signals. The nature and time scale of these temporal variation computations vary from neuron to neuron, and this heterogeneity contributes to the observed heterogeneity in texture responses. This transformation allows for the possibility, in principle, that all of the relevant texture information in spike timing has been converted to a rate code in cortex.

Is Spike Timing Relevant to Texture Coding in the Cortex?

A subpopulation of neurons in somatosensory cortex exhibits precise, phase-locked spiking responses to a wide range of textures (Fig. 5B and SI Appendix, Fig. S3E). The main inference we draw from this temporal patterning is that this subpopulation receives input from PC fibers, because these afferents are far more susceptible to produce phase-locked responses to high-frequency stimuli (>50 Hz) than are their SA1 or RA counterparts (56, 57). This inference is further corroborated by the fact that phase-locked cortical neurons are strongly driven by the surfaces that also strongly drive PC fibers.

The question remains, however, of whether this temporal patterning plays a role in texture coding. In the somatosensory nerves, the precise timing of afferent responses—at a millisecond resolution—conveys information about the frequency composition of skin vibrations (58, 59) and about fine texture (10). In somatosensory cortex, temporal patterning in the responses of a subpopulation of neurons encodes the spectral composition of skin vibrations at frequencies above ∼50 Hz, a range over which cortical firing rates are frequency-independent (60). Given these previous results, one might surmise that the temporal patterning observed in cortical responses to texture may play a role in texture coding. Unfortunately, the texture set in the present study is ill-suited to address this question. Indeed, cortical firing rates carry enough information about the textures in this set to yield near-perfect classification performance. Any additional contribution of temporal patterning is thus obscured. To establish a role of spike timing in texture coding in cortex will require measuring responses to textured surfaces that evoke similar firing rates but different firing patterns.

Conclusions

Texture representations in cortex involve the extraction of spatial and temporal features from the patterns of activation across tactile fibers. The resulting high-dimensional cortical representation of texture comprises dozens of nonredundant signals, many of which account only for a small fraction of the overall neuronal variance but are nonetheless informative about texture identity. A prominent axis in the neuronal response forms a continuum of spatial scales, with coarse-feature specialists at one extreme and fine-feature specialists at the other, each population receiving dominant input from a different class of tactile fibers. This structure in the neuronal representation is reflected in perceptual judgments of textures. While the principal neural axis predicts perceived roughness, the code for texture identity seems to be distributed along the neural continuum of spatial scales. That is, coarse-feature specialists predict coarse-texture perception, and fine-feature specialists predict fine-texture perception.

Materials and Methods

Experimental Methods.

Behavioral training.

Before the beginning of recording sessions, all animals were trained to sit in a primate chair with their heads fixed and arms restrained as they were habituated to the experimental apparatus. During the task, the arm was stabilized in a supinated position with a custom-built cast (Polycaprolactone; lined on the interior with foam padding for comfort). The animal was trained to keep its hand still for the duration of the recording protocols, and a protocol was restarted from the beginning if the finger moved. Stability was further maintained by loosely taping the nonstimulated fingers down and applying a small amount of glue to the fingernail of the stimulated finger to keep it in stable contact with the hand holder.

To maintain alertness during recording, the animals performed a simple visual brightness discrimination task (60). Briefly, the animals fixated on a small square presented in the center of a monitor located in front of the tactile stimulator. After ∼1–2 s of fixation, two circles of different luminance appeared to the left and right of the fixation point. The animal was given a liquid reward (juice or water, depending on the animal’s preference) for making a saccade to the brighter target. The task was kept challenging by adjusting the relative luminance of the targets and the fixation time. Eye movements were tracked by using a camera-based eye tracker (ViewPoint PC-60; Arrington Research), and visual stimuli were presented by using in-house software based on the OpenGL library.

Surgery.

Procedures were approved by the University of Chicago Institutional Animal Care and Use Committee. First, a custom-built head-post was secured to the skull and allowed to osseointegrate for 1.5 mo before the head was first immobilized. Once the animals were sufficiently habituated to the test apparatus and visual task, a recording chamber (22-mm internal diameter) was attached to the skull by using bone cement such that it circumscribed the hand representation in somatosensory cortex, and a craniotomy was made over the internal diameter of the chamber. All surgical procedures were performed under sterile conditions; anesthesia was induced with ketamine and dexmedatomadine and maintained with a surgical plane of isofluorane and occasional redosing of dexmedatomadine (61). Postsurgery, anesthesia was reversed with atipamezole.

Neurophysiological procedures.

Extracellular recordings were made in the postcentral gyri of three hemispheres of three macaque monkeys (male, 6–8 y old, 8–11 kg) using described techniques (60). On each recording day, a multielectrode microdrive (NAN Instruments) was loaded with three tungsten electrodes insulated with epoxylite (FHC Inc.), and electrodes were lowered normal to the cortical surface, through a custom-designed 3D-printed guide tube system that arranged the electrodes in a line 650 μm apart tip-to-tip. The electrodes were then driven into the cortex until they encountered neurons from areas 3b, 1, and 2 of somatosensory cortex with receptive fields (RFs) on the distal finger pad.

The transition from area 1 to 3b exhibits a characteristic progression of RF locations. As one descends from the cortical surface through area 1 into area 3b near the central sulcus, the RFs progress from the medial and proximal finger pads to the palmar whorls. As one enters area 3b, RFs proceed back up the finger, transitioning from proximal, to medial, and ultimately to distal pads. Because responses from the distal pad were never encountered in the more superficial regions of 3b (where the palmar whorls or proximal pad typically were most responsive), there was never any uncertainty about the anatomical area from which area 3b recordings originated. The representation of the digits in area 2 lies just caudal to, and mirrors that of area 1. Thus, as one proceeds caudally, one first encounters the proximal, then medial, then distal pads. As one enters area 2, RFs remain on the distal pads and then proceed down the finger as one further proceeds caudally. The most salient feature identifying area 2 is the presence of neurons with proprioceptive response properties; that is, neurons that respond preferentially to movements of the joints. Because the distal finger pad representations in areas 1 and 2 are adjacent, we used the presence of proprioceptive responses to inform the areal classification.

We recorded from neurons whose RFs were located on the distal pads of digits 2–5. On roughly every second day of recording, the electrode array was shifted 200 µm along the postcentral gyrus until the entire representation of digits 2–5 had been covered. At the end of the recording day, the electrodes were withdrawn, and the chamber was filled with sterile saline and sealed. Recordings were obtained from neurons in areas 3b, 1, and 2 that met the following criteria: (i) action potentials were well isolated from the background noise; (ii) the RF included at least one of the distal finger pads on digits 2–5; (iii) the finger could be positioned such that the textured surface impinged on the center of the RF; and (iv) the neuron was clearly driven by light cutaneous touch. Isolations had to be maintained for at least 30 min to complete five repetitions of the basic texture protocol. When held for longer, additional protocols were run (see below).

Stimulus presentation.

Textured surfaces were presented to the fingertips of awake macaque monkeys by using a custom-built rotating drum stimulator like those used in previous studies (10, 62), but larger and more precise. The drum was attached to a rotation motor (SmartMotor SM23165D; Animatics) via a 1:100 gearbox (Animatics), which provided precise control of rotational position (±200 µm) and velocity (±1.1 mm/s). The motor was attached to a vertical stage (IMS100V; Newport), which could control the depth of indentation into the skin with a precision of 2 μm. The vertical stage was attached to another horizontal stage (IMS400CCHA; Newport), allowing smooth displacement over 40 cm at a precision of 4 μm. Thus, we achieved precise horizontal, vertical, and rotational positioning of textures, allowing 60 different slots (12 rows, five textures per row) in which texture strips (2.5 cm wide by 16 cm long) could be mounted to the drum (25.5 cm in diameter and 30 cm in length). The interstimulus interval was at least 3 s between stimulus presentations, to allow the drum to reposition and to prevent neural adaptation.

One of the available slots was dedicated to a small load cell placed on the surface of the drum (LSB200; Futek; 2 lb, 1-axis, parallel to indentation), whose function was to ensure that a consistent level of pressure was exerted on the finger across recording sessions. Each day, after the animal’s hand and finger were stabilized in place, the drum was rotated and translated such that the load cell was pressed lightly (to a force of 15 ± 3 g) into the animal’s fingertip. This position was used as a reference point for all protocols, to ensure that stimulation was consistent across days and across fingers.

Stimuli.

Texture samples were mounted on individual strips of magnetic tape (5 × 16 cm), which were then attached to a complementary sheet of magnetic tape fixed to the surface of the drum. This allowed for simple removal and replacement when textures were damaged or worn through use. In total, 59 different textures were mounted on the drum, including a wide array of natural textures such as papers, fabrics, furs, and upholsteries with coarse periodic structure, as well as tetragonal arrays of embossed dots (25) and 3D-printed gratings and dots. Twenty-four of these textures were also used in a previous experiment on peripheral afferents (10) (SI Appendix, Table S1).

Textures were presented at a speed of 80 mm/s and at a force of 15 g. To find the displacement equivalent to this desired force, a set of calibration readings were taken offline by using a second load cell mounted at the location of the hand. First, a standard reference point was found by indenting the drum-mounted load cell into the second load cell, to a force of 15 g. Then, individual textures were repeatedly indented into the second load cell to find the displacement (relative to the calibration point) necessary to achieve the calibration force (15 g). During recording, textures were scanned over the finger according to these standard displacements relative to the reference point, measured daily. See SI Appendix, SI Materials and Methods for additional stimulus protocols.

Peripheral recordings.

We have previously reported the responses of 35 afferent fibers [15 SA1, 13 RA, and 7 PC, characterized by using standard criteria based on their response properties (63)] to 55 different texture stimuli (see ref. 10 for details). Briefly, we collected extracellular single-unit recordings from the median and ulnar nerves of six anesthetized (isoflurane) rhesus macaques as texture stimuli were presented to the distal digits of the hand at a speed of 80 mm/s. Each texture was presented to each fiber at least twice.

Magnitude estimation.

All procedures were approved by the Institutional Review Board of the University of Chicago, and all subjects provided informed consent. Subjects sat with the right arm supinated and resting on a support under the drum. Stimuli were presented to the right index finger pad of each subject.

Roughness scaling (six subjects, 5 males, 1 female, ages 18–24).

On each trial, the subject was presented with one of 59 textures (80 mm/s, 25 ± 10 g) and produced a rating proportional to its perceived roughness, where a rating of zero denoted a perfectly smooth surface. If texture B was perceived to be twice as rough as texture A, texture B was ascribed a number that was twice as large as texture A. Subjects were encouraged to use fractions and decimals if necessary. Each texture was presented once in each of six experimental blocks; ratings were normalized by the mean of each block and averaged, first within then across subjects. Roughness ratings were consistent across subjects: Each subject’s mean ratings were compared with the mean ratings averaged over the remaining subjects, yielding a subject correlation to the mean of r = 0.87 ± 0.079 (mean ± SD).

Dissimilarity scaling (10 subjects, 10 female, ages 19–24).

In these experiments, two subsets of textures were used. The first comprised 5 textures (coarse group: silk, microsuede, upholstery, 4-mm embossed dots, and 5-mm embossed dots) and the second 13 textures [fine group: suede, chiffon, nylon (200 denier), denim, hucktowel, silk, microsuede, wool, satin, metallic silk, upholstery, thin corduroy, and thick corduroy]. On each trial, the subject was presented with a pair of textures (for 1 s each, separated by a 1-s interstimulus interval, 10 of 78 unique comparisons for each group, respectively, 80 mm/s, 25 ± 10 g) and produced a rating proportional to the perceived dissimilarity of the pair, where 0 denoted (perceived) identicalness. Each pair of textures was presented three times in pseudorandom order. Texture dissimilarity ratings were correlated across subjects (subject correlation to the mean, coarse group: r = 0.88 ± 0.10; fine group: 0.65 ± 0.12; mean ± SD).

Analysis.

Basic analyses of firing rate.

Because the stimulus epoch over which a texture was moved into the skin evoked a large phasic response that lasted ∼200 ms, we excluded this response from our analysis. For each trial, the baseline firing rate was measured in the 500-ms period before the drum’s initial contact with the skin.

To test whether texture responses were significantly above baseline firing rates, we first created a distribution of baseline responses for each cell. We then measured how often each cell’s texture firing-rate response (averaged across five repetitions) was greater than the average of five random draws from the baseline distribution. We set significance at P < 0.05 for all textures, Bonferroni-corrected for 8,319 comparisons (141 × 59), yielding a very conservative measure of significance and thus a lower bound on the number of textures that drive any given neuron.

Principal components analysis (PCA).

We applied a PCA to population responses to determine their major axes of variation. Specifically, we treated each cortical neuron as a signal and each texture response as an observation in 141-dimensional space.

To compare the main axes of variation in the responses of tactile nerve fibers and cortical neurons, we identified shared axes of variation using a cross-validated canonical correlation analysis. First, we split the 24 textures that were used in both the peripheral and cortical experiments into a “training” set of 23 textures, leaving out 1 “test” texture. Second, we recalculated the PCA to determine new axes of variation for the peripheral (n = 39) and cortical (n = 141) responses. Third, we used canonical correlation analysis [using the canoncorr() function in Matlab] to calculate the optimal mapping of component scores from the peripheral to the cortical representation. Fourth, we applied this mapping to the responses to the test texture: Specifically, we obtained from the afferent population response a prediction of the cortical population response. This procedure was repeated 24 times, each time leaving a different texture out of the training set, and was repeated with different numbers of principal axes (between 1 and 22 principal components). Finally, for each set of principal axes, we computed the coefficient of determination for this cross-validated procedure, using a formula analogous to the traditional R² value for linear regression:

$$R^2=1-\frac{{{\displaystyle \sum }}_{cells}{{\displaystyle \sum }}_{textures}{\left(r_{t,c}-f_{t,c}\right)}^2}{{{\displaystyle \sum }}_{cells}{{\displaystyle \sum }}_{textures}{\left(r_{t,c}-{\overline{r}}_c\right)}^2},$$

where r_t,c is the true firing rate of a cell c in response to the test texture t, f_t,c is the predicted firing rate of cell c in response to that texture, and ${\overline{r}}_c$ is the mean firing rate of cell c averaged across the 24 textures.

Texture classification.

To quantify the information about texture identity carried in the neuronal responses, we assessed the degree to which we could classify textures based on the responses they evoke. To this end, we implemented a nearest-neighbor classifier. First, for each neuron, we averaged four of the five responses evoked by each texture, leaving one repetition out, yielding 59 vectors of mean responses and 59 vectors of single-trial responses. Next, we computed the distance between each single-trial response and each mean response. For each single-trial response, the mean response yielding the lowest distance was selected. If the selected mean responses and the single-trial responses corresponded to the same texture, classification was correct. Performance was averaged across all textures and then again across 100 shuffles, each with different repetitions left out. This procedure was repeated for neuron groups of different sizes.

We wished to examine how well responses from the cortical population could support texture classification using subsets of the full response space, as defined by the axes of variation identified via PCA. To implement our classification analysis in these subspaces, we first recomputed the PCA using the mean firing rates computed from only four repetitions (the training set), as discussed above. Next, we projected the full response space (for the training and test sets) onto the relevant subspace (either single dimensions, as in Fig. 3C and SI Appendix, Fig. S2C, or lower-dimensional subspaces, as in Fig. 3D and SI Appendix, Fig. S2D). Finally, we performed the classification, as described above, in this lower-dimensional subspace.

For one analysis, we performed a pairwise texture classification, rather than the “1 in 59” classification described above (Fig. 3C and SI Appendix, Fig. S2C). Classification in this case was performed exactly as described above, but only comparing the responses to two textures at a time, rather than comparing any one texture to the remaining full set of textures. Performance was averaged over all possible pairs of textures, and then again over all test repetitions.

Estimating submodality input.

We wished to assess the relative contributions of the three functionally defined populations of tactile fibers to the response of each neuron in somatosensory cortex, having previously shown that a majority of cortical neurons receive convergent input from multiple modalities, even in area 3b (19, 32) (recognizing that afferent signals pass through at least two intermediate synapses, one in the cuneate nucleus and one in the thalamus). While afferent input is likely not integrated linearly, we estimated the relative strength of that input using a linear model. Specifically, we used a multiple regression to predict the standardized (z-scored) mean texture responses of each cortical neuron to a set of 24 textures (see above) from the standardized (z-scored) mean responses of SA1, RA, and PC afferents to those same textures. We used these normalized regression weights as measures of the relative strength of SA1, RA, and PC afferent input into each neuron.

Frequency analysis.

Spiking responses in the nerve have been shown to phase-lock at high frequencies to texture-elicited skin vibrations (10), and spiking responses in somatosensory cortex have been shown to phase-lock to high-frequency skin vibrations imposed by a vibrating probe (60). To reveal any phase-locking in the cortical responses to texture, we first binned spike trains into 0.3-ms bins. Next, we computed the fast Fourier transform of that binned spike train and, from it, the amplitude spectrum. Finally, we computed the mean amplitude spectrum across repeated presentations for each texture and neuron.

Discriminability of 3D-printed textures.

We wished to assess whether neurons in somatosensory cortex are more sensitive to differences in the coarse structure or fine structure of a textured surface. To this end, we measured neuronal responses to nine 3D-printed textures that parametrically combined three coarse patterns (blank, 7.7-mm spaced dots, and 5-mm-period grating) and three fine patterns (blank, 1-mm-period grating, and 500-µm-period grating). We then measured the discriminability of pairs of textures from their respective distributions of responses. Specifically, we sought to determine whether two textures with identical fine structure, but different coarse structure, could be discriminated from the responses of an individual neuron. For each fine pattern, we had three different coarse patterns (and vice versa for each coarse pattern), and thus three different pairwise comparisons. To quantify the discriminability of each pair from each neuron’s responses, we measured a sensitivity index (d′):

$$d^{\text{'}}=\frac{\left|{\mu }_1-{\mu }_2\right|}{\sqrt{\frac{1}{2}\left({\sigma }_1^2+{\sigma }_2^2\right)}},$$

where μ is the mean firing rate response to a texture and σ² is the variance in firing rate across repeated presentations of that texture. These values were averaged across conditions and subpopulations of cells (Fig. 5C). To calculate the statistical significance of these differences across neural subpopulations, we repeated this averaging on permutations of the same dataset with responses shuffled (50,000 permutations). We calculated significance as the proportion of times our measured difference was greater than that computed from shuffled data.

Dissimilarity correlation.

We sought to determine whether we could account for psychophysical judgments of texture dissimilarity obtained from human observers to neuronal responses in somatosensory cortex. To this end, we computed the Euclidean distance between the two population vectors for each of the two textures, each element of which is the mean firing rate of each neuron in the set. This distance was then compared with the psychophysical judgment of texture dissimilarity.