Robust tactile sensory responses in finger area of primate motor cortex relevant to prosthetic control

Abstract

Objective

Challenges in improving the performance of dexterous upper-limb brain-machine interfaces (BMIs) have prompted renewed interest in quantifying the amount and type of sensory information naturally encoded in primary motor cortex (M1). Previous single unit studies in monkeys showed M1 is responsive to tactile stimulation, as well as passive and active movement of the limbs. However, recent work in this area has focused primarily on proprioception. Here we examined instead how tactile somatosensation of the hand and fingers is represented in M1.

Approach

We recorded multi- and single units and thresholded neural activity from macaque M1 while gently brushing individual finger pads at 2 Hz. We also recorded broadband neural activity from electrocorticogram (ECoG) grids placed on human motor cortex, while applying the same tactile stimulus.

Main results

Units displaying significant differences in firing rates between individual fingers (p<.05) represented up to 76.7% of sorted multiunits across four monkeys. After normalizing by the number of channels with significant motor finger responses, the percentage of electrodes with significant tactile responses was 74.9% ± 24.7%. No somatotopic organization of finger preference was obvious across cortex, but many units exhibited cosine-like tuning across multiple digits. Sufficient sensory information was present in M1 to correctly decode stimulus position from multiunit activity above chance levels in all monkeys, and also from ECoG gamma power in two human subjects.

Significance

These results provide some explanation for difficulties experienced by motor decoders in clinical trials of cortically controlled prosthetic hands, as well as the general problem of disentangling motor and sensory signals in primate motor cortex during dextrous tasks. Additionally, examination of unit tuning during tactile and proprioceptive inputs indicates cells are often tuned differently in different contexts, reinforcing the need for continued refinement of BMI training and decoding approaches to closed-loop BMI systems for dexterous grasping.

Introduction

Intracortical brain-machine interfaces (BMIs) hold the potential to restore natural movement to those with limb loss and paralysis by drawing prosthetic control signals directly from the brain. Multiple research groups have enabled human subjects with tetraplegia to control an external prosthetic hand to feed themselves, shake hands, and interact with objects using signals obtained with microelectrode arrays [1–6]. Recently, there has been increased focus on the development of bidirectional interfaces to provide sensory signals back to users. It is likely that such feedback is necessary to enable high performance with many degree-of-freedom systems, particularly those involving dexterous manipulation of objects [7–9]. Intracortical microstimulation (ICMS) of primary somatosensory cortex (S1) cannot perfectly mimic a natural sensory percept, but it can provide a virtual tactile signal that monkeys can to use to complete BMI tasks [10,11].

With the development of these systems, it is important to consider the effects of sensory stimuli (both endogenous stimulation of the skin and virtual stimulation via ICMS) on M1 firing patterns used for motor control. Primary motor cortex (M1) itself is responsive to many types of sensory inputs, including proprioceptive, visual, and tactile (for a review, see [12]). Many cells are tuned to both sensory and motor variables, though the tunings are not always directionally similar. M1 receives direct proprioceptive inputs from deep muscle spindles via the ventral posteriolateral nucleus [13,14]. Communication with cortical sensory areas 3a, 1, 2, and 5 is necessary to access cutaneous information, though a substantial amount of ‘deep’ information comes via this route as well: stimulation of peripheral nerves elicits responses in M1, called ‘evoked potentials,’ which are reduced 75% by ablation of S1 in the monkey [15]. Though largely unexplored in more recent literature, specific examples of M1 cells responsive to tactile stimulation have been described in a number of single unit electrophysiology studies in monkeys [16–19]. These studies found anywhere between 27–53% of units were responsive to cutaneous inputs, and the great majority of their receptive fields were on the glabrous skin of the hands and feet. Still, the extent, frequency, and tuning properties of these responses have not been fully documented, particularly in the context of multielectrode array recordings, where the population of recordable units is a more random and representative sample.

Ignoring the importance of sensory signals in normal M1 firing patterns may become more of a problem as experiments incorporate more dexterous tasks. In human clinical trials, the presence of an object in or near the hand during an attempted grasp negatively affected decoder performance such that specialized calibration was required, and it could not be entirely corrected [6]. This and other BMI studies featuring interaction with objects [20–22] require a training set featuring objects to properly train the decoder. Wodlinger et al. note that without object training, the decoder will produce movements that repel the hand away from the object, instead of moving toward and grasping it. They suggest several explanations for this, one being expectations of tactile feedback. While unproven, this explanation would agree with the single unit data indicating the hands and feet have special M1 representation. It would also indicate robust sensory responses on those same motor electrodes, if the effects are strong enough to interfere with the decoder.

Another implication of M1 tactile responses is the possibility that they may cause overestimates in decoder performance during finger-related tasks. The most well-studied data sets for predicting our capability to decode finger movements online have come from mixed motor and sensory signals, as monkeys flexed their fingers to activate microswitches within a manupulandum [23,24]. These animals had constant tactile feedback as they performed the task, which would not be present in a patient using a clinical BMI system. Hand and finger decoding have also continued to improve in the ECoG BMI literature [25–27], but there is a persistent lack of clarity about the extent to which the decodes are relying on sensory versus motor signals.

In this study, we investigated the responses of M1 units to passive tactile and proprioceptive stimulation of the fingers of four macaque monkeys using intracortical microelectrode arrays. Although array recordings provide fewer isolatable units than can be achieved with repeated single unit insertions, their sensory content is of relevance to BMI performance. We found omnipresent M1 tactile fingertip representation, though the fraction of modulated units varied between animals. The tactile modulation was robust enough to successfully decode which finger is stimulated well above chance in these four animals, as well as in two humans with ECoG grids over M1. Many multiunits exhibited orderly tuning across the fingers that differed for the two types of stimuli. The modulation in firing rates is potentially strong enough to interfere with motor decodes trained only on active motor tuning.

Methods

All procedures were carried out in accordance with protocols approved by the University Committee on Use and Care of Animals at the University of Michigan. All human procedures were carried out in accordance with protocols approved by the IRB at the University of Michigan.

Surgery and experimental structure

Four rhesus macaques were implanted with multielectrode arrays in finger area of M1, as diagrammed in Figure 1A. Monkeys P and O were implanted with 2.5mm × 1.95mm 16-channel Floating Microelectrode Arrays (FMAs, Microprobes for Life Science, Gaithersburg, MD). Monkeys L and S were implanted with 4mm × 4mm 96-channel Utah Arrays (Blackrock Microsystems, Salt Lake City, UT). Finger area was located by finding the point at which a line projecting from the genu of the arcuate sulcus would intersect central sulcus, and the arrays were placed as close to this point as possible, just anterior to the sulcus.

Figure 1.

Placement of electrodes. (A) Array placement in four monkeys. Monkeys P and O had FMAs, while monkeys L and S had Utah arrays. CS: central sulcus; Sp: spur of the arcuate sulcus; A: anterior; P: posterior. (B) Subdural grid placement in two human patients. Blue circles indicate which electrodes were used in analysis.

We trained the monkeys over the course of several weeks to sit quietly in a chair (Crist Instrument, Hagerstown, MD) using small juice rewards. Animals’ head position was fixed during all training and experiments. For the brushing stimulus, the hand contralateral to the implant was gently immobilized against an acrylic plate, and a cotton-tipped applicator was used to stroke the appropriate finger pad at 2 Hz, as timed by a metronome. For the bending stimulus, the appropriate finger was lightly grasped on each side and bent toward and away from the palm repeatedly for the duration of the trial.

All of the monkeys also performed an active motor task on different days from the sensory tasks. Monkeys S, L, and P performed a finger flexion task: each monkey sat in a shielded chamber with its hand resting on an acrylic surface, thumb pointing upward. The monkey was cued to flex and extend the four fingers to hit virtual targets with a virtual model of a monkey hand (Musculoskeletal Modeling Software; MDDF, Los Angeles, CA) displayed on a computer monitor. A resistive flex sensor (Spectra Symbol, West Valley City, UT) was attached to the index finger to measure finger position. Monkey O performed a grasping task: the monkey grasped a manipulandum instrumented with a pressure sensor (Interlink Electronics, Westlake Village, CA) located under the index finger pad and squeezed to hit virtual targets with up to 1 N of force.

Neural recording

A computer running xPC Target (Mathworks, Natick, MA) cued the experimenter and synchronized behavioral and neural data for analysis. Trials were randomized and interspersed with rest trials, each lasting 5 seconds. The stimuli were entirely passive; if the monkey moved during any trial, it was flagged as invalid by an observing experimenter and not used in subsequent analysis. For monkey L experiments, the applicator was instrumented with a triple axis analog accelerometer (SparkFun) to better align behavioral and neural data.

Neural data were recorded at 30 ksps and sorted into single and multiunits offline using Plexon Offline Sorter (Fig. 2A; Plexon, Dallas, TX). Only clusters that were completely separated in component space from the other waveforms on that channel, with blank space between, were considered to be single units. The remaining clusters, whether clearly containing multiple cells or only slightly overlapping with other clusters, were combined to form up to one multiunit per electrode.

Figure 2.

Processing of neural data. (A) Spikes were sorted in Plexon OFS into single units (SU) and multiunits (MU). (B) Human ECoG voltage data were common average referenced and filtered before being squared to obtain power. Mean power during three finger conditions were the feature set for the decoder.

Human ECoG

Broadband neural data were recorded at 30 ksps with a Neuroport signal processor (Blackrock Microsystems, Salt Lake City, UT) from two human subjects who had been implanted with clinical subdural ECoG grids (Ad-Tech Medical, Racine, WI) as part of ongoing treatment for epilepsy, as described previously [28]. Grid placement is shown in Figure 1B. The same task structure was used as described in the previous section, but only the brushing (tactile) task was performed. The data were decimated to 10 ksps, and a common average reference was implemented for each bank (Fig. 2B). The data were then bandpass filtered between 66 and 114 Hz using a 3rd order Butterworth filter in MATLAB before calculating the power in that band during each trial.

A Naive Bayes decoder with leave-one-out cross-validation was used to classify the location of the stimulus on a given trial. The inputs to the decoder were the average band power on each included electrode during 2 seconds of stimulation. Chance level was 33.3% for a 1 of 3 choice, and the decoder could not perform better by choosing the most common condition, as the number of trials per finger condition were always balanced – between 20 and 24 trials per digit for the three datasets used (two from Patient 1, one from Patient 2). The two datasets from Patient 1 were recorded on consecutive days. Electrodes were chosen by starting with the set of all electrodes that were entirely anterior to central sulcus on co-registration imaging. In Patient 1, a single row of electrodes over M1 was used, as seen in Figure 1B. In Patient 2, only a cluster over hand knob of M1 was used.

Spiking analysis and statistics

Significantly tuned units during the sensory brushing and bending tasks were determined with a one-way ANOVA, α=.05, of firing rates during trials of the different finger conditions. The total number of recorded units from each animal are the following: from monkey S, 73 multiunits and 68 single units from one recording; from monkey P, 27 multiunits and 12 single units from three recordings; from monkey L, 96 multiunits and 51 single units from two recordings; from monkey O, 15 multiunits and 12 single units from one recording.

These animals had previously participated in active motor tasks, allowing for an evaluation of the accuracy of electrode array placement in finger area. The number of significantly tuned channels during an active motor task were determined by computing correlation coefficients between the finger speed (flex/extend task) or applied force (force task) and neural firing rates. In this case, spikes were detected by thresholding at −4.5 times the root mean squared (RMS) voltage on each channel, after high-pass filtering the broadband signal at 250 Hz. Spike times were separated into 100 ms bins for each electrode. A null distribution was created by shuffling the firing rate bins and re-computing the correlation coefficients. Electrodes with coefficients at least two standard deviations above the mean of the null distribution were considered modulated by the motor task. This measure is more conservative than the one described for sensory responses, but provides a good description of the amount of finger information on the array overall, since all fingers were employed simultaneously in these tasks.

Tuning curves were fit to a Von Mises function [29] in MATLAB, defined as:

$$f=b+m\exp [k\cos (x-\mu )]$$

The parameter b represents the baseline firing rate, m the depth of modulation, κ the width, and μ the preferred ‘direction,’ or finger.

A Naive Bayes decoder with leave-one-out cross-validation was used to classify the location of the stimulus on a given trial. The inputs to the decoder were the firing rates of multiunits during the center 3 seconds of each trial. The beginning and end of each trial were excluded to avoid the periods of time when the experimenter was switching between fingers. Chance level was 33.3% for a 1 of 3 choice, and the decoder could not perform better by choosing the most common condition, as the number of trials per finger condition were always balanced. Significance was determined with a 1-sample z-test with α=.01.

Results

We found some units in each monkey that were visibly modulated by the tactile stimulus, as seen in the raster plots of multiunit spike trains sorted by 2 Hz content (Fig. 3). M1 units with significant finger brushing modulation were found in all four animals tested (Fig. 4). The fraction of units varied widely across animals, from 16.7% to 76.7% of sorted multiunits, but this tracked well with the number of electrodes engaged in motor finger activity as described below. Here, modulation indicates a significant difference in firing rates between finger conditions in the brushing task. Finger bending responses are also shown where available, for monkeys L, P and O. Passive finger bending is a proprioceptive stimulus that would be predicted to modulate some M1 neurons, though this fraction also varied widely. The difference in the number of modulated units between monkeys can be attributed to some combination of placement and inherent variance in sampling from a relatively low number of cells. To determine how well the arrays were placed in finger area, we examined the number of electrodes with significant modulation in a motor finger task that each monkey performed on a different day. Monkeys S, P, and L performed a finger flex/extend task, and monkey O performed a power grasp squeezing task. All four monkeys had some amount of motor modulation, and the relative amount appeared to track with the sensory modulation, implying that a fair amount of the variance seen in sensory modulation was due to placement of the arrays. After normalizing by the fraction of electrodes with motor tuning, the percentage of electrodes modulated by the tactile stimulus was on average 74.9%, but still had a large standard deviation of 24.7%. The highest normalized fraction was seen in monkey P, who displayed, on average, an equal number of sensory and motor modulated electrodes.

Figure 3.

Example rasters from each monkey. Each row shows the spike times of one multiunit during a single trial. To emphasize the bursting activity, they units are sorted top to bottom by the average power special density in the spike train at 2 Hz, the frequency of the tactile stimulus.

Figure 4.

Fraction of recorded multiunits (MU) and single units (SU) that were significantly modulated by brushing and bending stimuli.

No somatotopic or orderly organization of digit preferences was observed across cortex (Fig. 5), but rather an even scattering of multi and single units across the arrays. In monkey S, multiple single units were separable on certain electrodes, but only in some cases did those units share the same digit preference. This result is consistent with our understanding of M1 somatotopy: large bodily areas are segregated, but representation of smaller features, like the digits, overlap significantly [30–32].

Figure 5.

Distribution of digit preferences across electrode arrays. (A) Multiunit locations and (B) Single unit locations of modulated cells for brushing stimulus, colored by digit with maximum firing rate. Monkey S had several channels with multiple single units; preferences shown as divided squares. Grey squares represent units that were not significantly tuned. CS: central sulcus.

Investigating the receptive fields of these modulated units, we found that many of them were responsive to stimulation on multiple finger pads. Tuning curves for the single unit data (some example units shown in Fig. 6) exhibit different shapes, but curved shapes were seen far more often than linear ones. As a comparison with directional tuning of motor cortical cells (Amirikian and Georgopulos, 2000), we fit the curves with a Von Mises cosine function. This function, or indeed any cosine-shaped function, fit the data better than a linear, exponential, or Gaussian for the great majority of units. Because there are only 5 digits on the hand, it is most certainly overfitting the points. Still, the general shape of the fit is very good for the majority of units – R² was greater than 0.8 for over half of modulated units – indicating that these units generally have a receptive field that encompasses multiple adjacent finger pads with firing rates decreasing as the stimulus moves farther from the preferred digit.

Figure 6.

Tuning curves for brushing task. Example tuning curves showing the range of shapes observed in all four monkeys. Blue curves are data with linear interpolation; error bars show S.E.M. Red curves are Von Mises fits as described in Methods; R² shown for each panel. Bottom right: R² histogram for all modulated multiunits; value less than zero indicates a linear fit is better than the model.

Given the depth of tuning we found, along with the fairly even distribution of finger preferences, we were able to apply a Naïve Bayes decoder to classify the stimulated finger on a given trial based only on single or multiunit firing rates (Fig. 7). Depending on the particular neurons captured by a given electrode array, it was sometimes possible to decode all five finger stimuli well, while other times some fingers were ‘missing’ (Fig. 7A). To compare across all four animals, we chose the best-represented three fingers in each animal and performed a 1-of-3 classification using only those finger trials (Fig. 7B). In this best-case analysis, multiunit decodes were significantly above chance levels (33.3% for a 1 of 3 choice, p<.01) in all animals, despite a fairly low number of recorded units in some animals. Single unit decodes were significant in two monkeys, but were less reliable in monkeys L and P due to the very low number of modulated units available (visible in Fig. 4 and 5). Confusion matrices for Monkey O (Fig. 7A) demonstrate that digits 1, 4, and 5 were best represented, which agrees with recorded digit preferences (visible in Fig. 5).

Figure 7.

Decoding sensory stimulus from M1 firing rates. (A) Example 5 finger and 3 finger confusion matrices for monkeys S and O. (B) All 3 finger decoding performances for multiunits (MU) and single units (SU). Horizontal line indicates chance (33.3%). Asterisk (*) indicates decoder performed significantly above chance (p<.01).

Similarly, we then applied the same Naïve Bayes decoder to gamma band (66–114 Hz) power recorded from subdural ECoG arrays in two human subjects. Above chance (p<.01, z-test) decodes were achieved in both subjects (Fig. 8) using only M1 electrodes, and on two consecutive days in the first subject (P1 – 63.9% and 52.4% correct; P2 – 66.7% correct).

Figure 8.

Decoding sensory stimulus from human gamma band power over M1. Patient 1 was also tested on a second day, with fairly similar performance (52.38%).

Monkeys P, L, and O underwent finger bending trials on the same day as brushing trials. We wanted to see if units were similarly tuned to both these sensory stimuli. Of those units that were significantly tuned in both conditions (had significantly different firing rates for different fingers), only one shared the same finger preference during both (Fig. 9). Though we had a small number of units, this data dovetails with previous work (Hatsopoulos and Suminski, 2011) showing that units in M1 are tuned differently to different types of stimuli.

Figure 9.

Comparison of tuning to tactile (brush) and proprioceptive (bend) stimuli. The Venn diagram shows the number of significantly tuned multiunits from monkeys P, L, and O when both types of stimuli were given in the same recording session. Examples of tuning curves for cases when units were tuned to the same and different digits are also shown.

Discussion

We have demonstrated the existence of an M1 population that is deeply tuned to tactile sensory inputs and readily apparent in multielectrode array recordings. The tuning was deep enough to correctly decode the location of a tactile stimulus from multiunit firing rates. While lower than previously reported finger motor decodes from M1 [23,24,33,34], correct classification rates of 65–90% for a purely somatosensory stimulus were surprising, especially given the relatively low number of units recorded in monkeys P and O. This result provides evidence for the possibility that motor finger decodes in NHP are to some degree due to sensory responses.

Similarly, the ECoG result confirms in humans that sensory information is present in M1 recordings, in addition to the known motor responses in S1. In the ECoG study with the best individual finger decoding [27], there was a large amount of overlap between channels used for motor (finger tapping) and sensory (vibrotactile) tasks, with the majority of electrodes used being postcentral in both cases. The authors consider that the decoded “motor task” information may be mostly cutaneous and proprioceptive, or may also be to some extent an efference copy from motor areas.

This issue has probably not been a major factor in BMI decoding until recently because of the specificity of these responses to the hand and fingers. Tactile inputs are not as important to upper limb control as they are to fine motor control, evidenced by the specificity of representation to the hands and feet. Both rats and mice have a sensorimotor overlap zone (OL) where forelimb and hindlimb sensory and motor representations overlap [35,36], and it is thought to be utilized for dexterous digit manipulation. While such a zone is not found in primates, it appears that M1 and S1 both engage in processing of sensory and motor information, and must therefore utilize corticocortical communication for dexterous tasks. There is ample evidence that M1 generates and sends sensory predictions that not only shape, but are in fact necessary for accurate sensory perception [37–39]. Still, it is not possible to determine whether the responses recorded here represent M1 sampling or sending sensory information.

Given that the tactile M1 neurons we found also seem to be responsive to other types of stimuli (proprioceptive finger bending), and that proprioceptive-tuned cells in the literature can also encode motor outputs [40], it seems likely that we are not recording a sensory-specific subpopulation. The changes in firing rates seen here were robust, and seem very capable of reducing the quality of motor decodes based purely on rate coding, in the cases where tunings differ. The multiunits such decoders are trained on are expected to have consistent preferred directions across all phases of the reach and grasp movements, when in reality, their preferences and functions change in different contexts. The subject studied by Wodlinger et al. (2015) was able to grasp different kinds of objects, after having only trained on one kind. This was an important advance, but effective use of an arm in the real world will require many more types of grasps and movements in many contexts. It has already been demonstrated that providing kinesthetic feedback to a monkey by passively moving its arm can improve motor decodes [41]. As BMIs move toward more dexterous control of individual fingers and object manipulation, understanding tactile sensory processing will become essential to improving decoding strategies.

Conclusions

This study reveals widespread tactile sensory responses in hand area of primary motor cortex of nonhuman primates and humans. It shows sufficiently tuned single- and multiunits to correctly decode stimulus location in four monkeys, suggesting that tactile sensory signals are strong within the same population of cells used for motor decoding. These results are significant to the BMI community because they provide some explanation for the difficulties experienced in human clinical trials when participants attempt to touch and interact with objects. Additionally, they inform the design of BMI experiments involving grasping and finger movements in intact animals, since they imply that native sensory signals, if available to the animal, may contaminate motor signals. Further study is needed to determine if these signals could be harnessed to improve, rather than hinder, BMI performance. It’s encouraging that with good placement, a single array is able to capture representation of all the digits. This preliminary work suggests that a more in-depth examination of population dynamics during fine dexterous tasks will improve online control of upper-limb prosthetics.