Emergent coordination underlying learning to reach to grasp with a brain-machine interface

Abstract

The development of coordinated reach-to-grasp movement has been well studied in infants and children. However, the role of motor cortex during this development is unclear because it is difficult to study in humans. We took the approach of using a brain-machine interface (BMI) paradigm in rhesus macaques with prior therapeutic amputations to examine the emergence of novel, coordinated reach to grasp. Previous research has shown that after amputation, the cortical area previously involved in the control of the lost limb undergoes reorganization, but prior BMI work has largely relied on finding neurons that already encode specific movement-related information. In this study, we taught macaques to cortically control a robotic arm and hand through operant conditioning, using neurons that were not explicitly reach or grasp related. Over the course of training, stereotypical patterns emerged and stabilized in the cross-covariance between the reaching and grasping velocity profiles, between pairs of neurons involved in controlling reach and grasp, and to a comparable, but lesser, extent between other stable neurons in the network. In fact, we found evidence of this structured coordination between pairs composed of all combinations of neurons decoding reach or grasp and other stable neurons in the network. The degree of and participation in coordination was highly correlated across all pair types. Our approach provides a unique model for studying the development of novel, coordinated reach-to-grasp movement at the behavioral and cortical levels.

NEW & NOTEWORTHY Given that motor cortex undergoes reorganization after amputation, our work focuses on training nonhuman primates with chronic amputations to use neurons that are not reach or grasp related to control a robotic arm to reach to grasp through the use of operant conditioning, mimicking early development. We studied the development of a novel, coordinated behavior at the behavioral and cortical level, and the neural plasticity in M1 associated with learning to use a brain-machine interface.

INTRODUCTION

Around 185,000 amputations are conducted in the U.S. each year, and there are almost 2 million people in the U.S. living with amputations (Ziegler-Graham et al. 2008). Over 5,000 service members in the U.S. Armed Forces had traumatic amputations in the timespan of 2000 to 2011 (Armed Forces Health Surveillance Center, 2012). A number of studies have shown that after amputation or peripheral nerve damage, the primary motor cortical area previously involved in the control of the lost limb gets remapped to areas that are somatotopically close by (Qi et al. 2000; Sanes et al. 1988, 1990; Schieber and Deuel 1997; Wu and Kaas 1999). It follows that these patients might not have the same degree of cortical representation of the lost limb in reorganized primary motor cortex as they had before amputation.

Cortically controlled brain-machine interfaces (BMIs) have been largely focused on either intact nonhuman primates or short-term nerve blocks of the arm to model severe upper-limb motor disabilities. In such cases, neural decoders for limb or cursor control are initialized with the use of a biomimetic technique where paired measurements of neural signals and intact limb kinematics are available (Ganguly and Carmena 2009; Li et al. 2011; Taylor et al. 2002). However, in amputees, a biomimetic approach that maps neural modulation to intact limb movements is not possible because of the lack of an intact limb. Indeed, recent work has shifted away from the traditional, biomimetic paradigm either by introducing adaptive decoders during closed-loop BMI control or by relying on learning to more effectively control the BMI (Ganguly and Carmena 2009; Ganguly et al. 2011; Li et al. 2011; Mussa-Ivaldi and Miller 2003; Orsborn et al. 2012; Taylor et al. 2002; Willett et al. 2013).

Learning to control a BMI provides a unique window into studying plasticity involved in developing a novel motor behavior at the level of primary motor cortex (MI). By establishing a direct, causal connection between MI and the robotic plant that bypasses both spinal cord and neuromuscular dynamics, we can infer that any learning at the behavioral level is directly due to plastic changes in primary motor cortical activity, although the actual site of synaptic plasticity cannot be isolated to MI. BMI learning may also provide insights into the development of a novel, coordinated motor behavior at the cortical level. Extensive psychophysical work has shown that during reach-to-grasp behavior, reaching of the proximal arm (reaching or transport component) is temporally and spatially coordinated with the preshaping of the hand (grasp component) (Haggard and Wing 1995; Jeannerod 1984). The development of this coordination has been studied in infants and children (Berthier and Carrico 2010; Kuhtz-Buschbeck et al. 1998b; von Hofsten 1984); they tend to have larger relative apertures, start prehension earlier, and rely more on visual feedback than adults. As children age, their kinematic profiles and interjoint coordination become more stereotyped. The role of cortex in coordinating and learning to coordinate reaching to grasp is still not well understood.

We studied the emergence of a coordinated reach-to-grasp behavior in rhesus macaques that had been the recipients of therapeutic amputations. The macaques were taught to control a robotic arm and hand via a cortically controlled BMI through the use of operant conditioning. Previous authors have examined learning in a BMI context, but because of instabilities in neural recordings, their studies were limited in scope to less than 3 wk (Ganguly and Carmena 2009; Ganguly et al. 2011). Furthermore, these studies have largely focused on changes in single neurons (Fetz 1969; Ganguly and Carmena 2009; Ganguly et al. 2011), although prior work has examined population-level measures of adaptation during a single recording session (Song and Giszter 2011). We extended this approach by examining plasticity in coordinated activity between pairs of neurons over longer time periods of BMI exposure and related these changes to the emergence of coordinated motor behavior. We focused our efforts on three primary questions: Can macaques learn to coordinate independent reach and grasp control dimensions in a BMI task using neurons that are not necessarily “reach related” or “grasp related”? What changes do we see in the structure of neural coordination among the ensembles controlling reaching and grasping that are associated with learning this coordinated behavior? Finally, are there similar correlates that arise among neurons that are not directly involved in BMI control?

METHODS

All surgical and behavioral procedures involved in this study were approved by the University of Chicago Institutional Animal Care and Use Committee and conform to the principles outlined in the Guide for the Care and Use of Laboratory Animals.

Neural recordings.

Data used for this analysis were collected from two female rhesus macaque monkeys (Macaca mulatta) that had undergone therapeutic amputations ~10 yr before the study: an elbow disarticulation amputation in the case of monkey K and a transradial amputation in the case of monkey Z. Utah 100-microelectrode arrays (1-mm-long electrodes; Blackrock Microsystems, Salt Lake City, UT) were surgically implanted in the forelimb region of primary motor cortex (MI) ipsilateral to the amputated limb in monkey K and in MI contralateral to the amputated limb in monkey Z. For monkey K, we stimulated the surface of the motor cortex ipsilateral to the amputation and implanted in a region that elicited movement of the contralateral arm and hand. Previous studies have shown that after amputation or peripheral nerve damage, the primary motor cortical area previously involved in the control of the lost limb gets remapped to areas that are somatotopically close by (Qi et al. 2000; Sanes et al. 1988, 1990; Schieber and Deuel 1997; Wu and Kaas 1999). Informed by this work, for monkey Z we stimulated the surface of motor cortex contralateral to the amputation and implanted in a region that elicited movement of the shoulder and stump, bearing in mind the idea that this region could have previously been part of the forelimb region before amputation. During a recording session, spike waveforms of units from up to 96 electrodes were amplified, filtered, and recorded digitally (14 bit) at 30 kHz per channel using a Cerebus acquisition system (Blackrock Microsystems). Single units were sorted online with a hoop-sorting method. Potential spikes were first identified when the filtered voltage dropped below a user-defined threshold. These spikes were sorted by placing lower and upper voltage thresholds (the “hoops”) at specific times relative to the initial threshold crossing. All units were then tracked online through visual inspection for the duration of the study; any units that were not present on a given day were deemed unstable and were not tracked for the rest of the study.

Experimental setup.

The macaques were trained on a BMI task using an operant conditioning paradigm (Fig. 1A). In this task, the macaque had to learn how to drive two control dimensions of a robotic arm to perform a reach-to-grasp task. The robot was composed of a 7-degrees of freedom (DOF) WAM arm attached to a 4-DOF BarrettHand (Barrett Technology, Newton, MA). The macaque simultaneously learned to control the reaching motion (i.e., the reach control dimension), toward and away from the base of the robot, and the grasping motion (i.e., the grasp control dimension), opening or closing all three digits of the hand concurrently. A successful trial involved reaching to grasp a sphere attached to a spring placed on a board in front of the robot, pulling it back, and then releasing it. Only the reach-to-grasp portion of completed trials was analyzed in this study. We attempted to online sort single units from each electrode and then identified distinct clusters of functionally correlated groups of neurons across the array (Eldawlatly et al. 2009). One cluster was assigned to the reach control dimension and another to the grasp control dimension with the idea that it might be easier for macaques to comodulate neurons that were already functionally connected to control a single dimension. Clusters only contained neurons that were stable across seven prior sessions of recording spontaneous activity before online decoding, based on previous work showing that if neurons were stable across this timescale, they were highly likely to remain stable (Dickey et al. 2009) (Fig. 1B). For monkeys K and Z, each of the clusters was 15 and 10 neurons, respectively. Twenty-tap Wiener filter decoders (using 50-ms bins) were initialized in an unsupervised manner by using spontaneous data from each ensemble of neurons (Badreldin et al. 2013) and remained static over the course of the entire study (Fig. 1C). The macaques were concurrently exposed to the decoders for both control dimensions and learned both mappings such that they could coordinate them while performing successful trials. No initial training sessions on these decoders preceded simultaneous exposure. It should be noted, however, that both animals had limited previous exposure to robotic control through earlier variants of the approach that we ultimately took (i.e., different types of decoders and controllers). The training paradigm used was operant conditioning: the macaques were rewarded with juice for completed trials (reach to grasp, pull, and release). Early in the study, to encourage parameter space exploration, the macaques were rewarded for making portions of the complete movement (i.e., opening the hand for the first time). No constraints were placed on the behavior; the macaques were rewarded for each completed trial, regardless of what or whether any coordination strategy was observed.

Fig. 1.

Experimental setup. A: the macaques were taught to use a neural cluster to reach (red) and grasp (green) with a robotic arm. Twenty-tap Wiener filter decoders were initialized using spontaneous data from each ensemble and remained static over the course of the study. The macaques learned this mapping such that they could coordinate both control dimensions to perform successful trials. Instruction was provided through operant conditioning (reward feedback). B: average daily waveforms of example neurons from a reach decoding cluster (red), grasp decoding cluster (green), and the pool of stable indirect neurons (black). C: filter coefficients of the 20-tap Wiener filter reach (red) and grasp (green) decoders for monkey K (top row) and monkey Z (bottom row).

The study progressed for each macaque until neural instabilities were detected in either decoder. It should be noted that these neural instabilities were the ones that we could detect using daily online sorting. Unfortunately, there is no way to ensure perfect stability using current, available methods; theoretically, units could have been lost and replaced by other units with the same or similar features on the same channel. Additionally, throughout the experiment, we recorded from stable, “indirect” neurons that were not used for decoding (72 neurons for monkey K and 70 for monkey Z). The data were subdivided into shorter training experiments based on frequency and length of daily training sessions to avoid long gaps during which the behavior and associated neural control might be forgotten: there were no more than 3 contiguous days within each training experiment when the macaques had been trained for less than 30 min, and we required at least 2 days of training in an experiment. There were two training experiments for monkey K with lengths of 679 and 777 trials, respectively, and three training experiments for monkey Z with lengths of 193, 238, and 298 trials, respectively. To examine any learning effects within a single training experiment, each experiment was subdivided into three intervals composed of equal numbers of trials. In our analysis, we examined the emergence of coordination between the first and the last intervals in each training experiment.

Analysis: emergence of reach-to-grasp behavior.

For this study, we started by examining the successful reach-to-grasp portions of each full trial; that is, from the beginning of every trial to the time of first successful grasp. The use of such a free-form paradigm allowed the animal to explore its neural and behavioral parameter space to learn the mapping between the two, analogous to our own development of reach-to-grasp behaviors: infants and children continuously explore their own environments while learning to coordinate transport and prehension to reach to grasp objects (Kuhtz-Buschbeck et al. 1998a, 1998b; Schneiberg et al. 2002; von Hofsten 1980, 1983, 1984; von Hofsten and Fazel-Zandy 1984; Wimmers et al. 1998). Another consequence of using this paradigm is that the macaques may not necessarily have been uniformly engaged throughout the course of a single training session. For these two reasons, some of the trials were on the order of minutes, whereas others were measured in seconds. Thus we focused on the very end of a successful reach-to-grasp movement: for the remainder of the analysis, we defined a behavioral trial as the last 500 ms of the reach-to-grasp behavior before the time of first successful grasp. Similarly, because both static neural decoders for the reaching and grasping control dimensions had twenty 50-ms taps, or 1 s of neural history, we examined the last 1,500 ms of concurrent neural activity of both ensembles for each trial to account for neural activity associated with the last 500 ms of behavioral data.

We used principal components analysis (PCA) to reduce the dimensionality of the reaching and grasping velocity profiles concurrently in the early and late intervals for each training experiment. For each trial, the 500-ms reaching trajectory was concatenated with the 500-ms grasping trajectory to generate a 1-s joint reach-grasp velocity profile. To examine whether these joint reach-grasp velocity profiles became more structured, we first grouped together trials by interval and then used PCA to reduce the dimensionality. Each trial was treated as an observation, whereas each time point in the joint reach-grasp velocity profile was considered a variable. We then compared how much of the variance was explained by one or two principal components in the early interval vs. the late interval in each experiment to examine whether the joint reach-grasp velocity profiles were becoming more stereotyped with learning.

Analysis: emergence of behavioral coordination.

We probed for evidence of behavioral coordination by examining the cross-covariance patterns that arose between the reaching and grasping robotic velocity trajectories. The cross-covariance patterns presented in this study were all normalized (i.e., correlation coefficient) such that the autocovariances at zero lag were 1, and as such, are unitless quantities. The cross-covariance between two signals is equivalent to the cross-correlation between the signals except that the means have been computed and subtracted beforehand. Cross-covariance patterns were computed on a single-trial basis and then averaged across trials. The averaging was done both 1) across all of the trials on each day of training and 2) across all of the trials in every interval in an experiment.

To examine the emergence of a stable coordination pattern, we calculated the dissimilarity index, defined as the Euclidean distance between the pattern of cross-covariance on a given day and the average pattern over the 3 prior days of training in each learning experiment. A decrease in dissimilarity index would indicate that cross-covariance patterns are becoming more stable across days.

Analysis: emergence of neural coordination.

We probed for similar dynamics between pairs of neurons in the reaching and grasping clusters to investigate the neural correlates of this emerging coordinated behavior. For each pair, composed of one reach neuron and one grasp neuron, we calculated the trial-averaged cross-covariance between their firing rates using the last 1,500 ms of every trial. As with the behavioral cross-covariance patterns, the averaging was done both 1) across all of the trials on each day of training and 2) across all of the trials in every interval in an experiment.

We determined statistical significance of correlated firing activity by shuffling the trial order 50 times for each pair of neurons and then conducting the same analysis on the shuffled data. A cross-covariance pattern between a pair of neurons was considered significant if its peak value was greater than the peak values of 95% of the cross-covariance patterns generated from the shuffled data or if its trough value was less than the trough values of 95% of the cross-covariance patterns generated from the shuffled data. Significant pairs would indicate that the macaques tended to covary the modulation of that particular neural pair to a greater extent, either positively or negatively, for the purpose of coordinating the reach-to-grasp behavior on a trial-by-trial basis. Essentially, this would indicate that the increase in cross-covariance is not solely due to an increase in the firing rates as the monkey learns to reach to grasp with the robotic arm, but due to the development of coordination strategies.

To examine the emergence of this stabilization, we again calculated the dissimilarity index in the same manner as with the behavior, using the Euclidean distance between the population average pattern of cross-covariance (calculated by averaging across trials for each pair and then across pairs) on a given day and the population average pattern over the 3 prior days of training in each learning experiment.

In addition, we examined the dynamics between pairs of neurons that were not included in either decoder (i.e., indirect neurons as defined by Ganguly et al. 2011) to probe larger network-level effects associated with the emergence of this coordinated reach-to-grasp behavior. For this analysis, we used all of the neurons on the array that were not used for either decoder and were stable for the duration of the study, as identified by online sorting on every training session: 72 neurons for monkey K and 70 neurons for monkey Z. As with the reach-grasp neural pairs, for each pair of indirect neurons we calculated the trial-averaged cross-covariance between their firing activities in the last 1,500 ms of every trial for each day of training. Significant pairs were identified through identical shuffling analysis.

Analysis: coordination comparison.

We directly compared the cross-covariance patterns that emerge after learning between reach-grasp and indirect pairs of neurons by considering the average cross-covariance patterns among all significant reach-grasp pairs and all indirect pairs in the last interval of each experiment. We examined both the magnitude and timing of the peak cross-covariance for both pair types.

In addition, we compared the degree to which reach-grasp pairs and indirect pairs showed significant coordinated patterning on each day of training by trial-averaging across each day instead of each interval. On each day of training, we compared both the degree of coordination, by examining the peak magnitude of covariation for each pair type, as well as the degree of participation in coordination, by looking at the percentage of all pairs that shows significant coordinated patterning for each pair type.

We extended our analysis to include the remaining two pair types: intracluster pairs, i.e., pairs of neurons that were both in the reach cluster or the grasp cluster, and indirect-cluster pairs, composed of one neuron from either the reach or grasp cluster and one indirect neuron. As with the other pair types, we looked for the emergence and stability of cross-covariance patterns and compared the peak covariation magnitudes and the degree of participation on each day of training.

RESULTS

Emergence of reach-to-grasp behavior.

Both macaques learned how to reach to grasp with the robotic arm: toward the end of the study, they were able to complete successful trials at much faster rates (monkey K: 9.60 ± 16.63 s; monkey Z: 16.50 ± 15.96 s; means ± SD) than at the beginning of the study (monkey K: 46.97 ± 33.09 s; monkey Z: 69.36 ± 95.16 s), although the daily trial rates tended to also fluctuate based on the motivational state of the animal (Fig. 2A). Because no constraints were placed on the behavior during the trial, there was substantial trial-to-trial variation in the reach-to-grasp trajectories, both within and across training sessions. The latter was especially evident when the very first trial that each animal successfully completed was compared with trials completed on the last day of training (Fig. 2B).

Fig. 2.

BMI reach-to-grasp behavior. A: success rates are indicated as the number of trials normalized by training time (trials/min) over the course of the study. The data were subdivided into shorter training experiments based on frequency and length of daily training sessions: there were no more than 3 contiguous days within each training experiment where the macaques had been trained for less than 30 min, and we required at least 2 days of training in an experiment. K1 and K2, monkey K’s first and second training experiments; Z1–Z3, monkey Z’s first, second, and third training experiments. B: example trials. Top row, robotic trajectories of the very first successful trials completed by each animal in this study plotted as a function of time (displayed as color gradient). Total trial times were 99.15 s for monkey K (left) and 93.45 s for monkey Z (right). Bottom row, robotic trajectories of the trials completed on the last day of training. Color gradient denotes percentage of total trial time from the beginning (blue) to the end (red) of the trial. Total trial times were 2.5 s for monkey K (left) and 3.95 s for monkey Z (right). C: joint reach-grasp velocity profiles of the reach dimension (left) and grasp dimension (right) in the last 500 ms of an example trial. D: percentage of variance accounted by the first (top) and first two (bottom) principal components of the reach-to-grasp velocity trajectories in the early (light shading) and late (dark shading) interval of each training experiment. *P < 0.05 indicates significance after bootstrapping.

We used PCA to examine the emergence of coordinated structure between the reaching and grasping robotic trajectories (Fig. 2C) from the early to late intervals for each training experiment (see methods). For all of the data sets, we found that the first one or two principal components explained more of the variance in the last interval compared with the first interval of each experiment, indicating that the reach-to-grasp velocity trajectories became more structured over the course of each experiment (Fig. 2D).

Coordination: reach-to-grasp behavior.

We found that consistent cross-covariance patterns emerged between the reaching and grasping velocity trajectories of the robot from the first to last day of training in a training experiment (Fig. 3A). Over both learning experiments, monkey K exhibited a stronger negative covariation at zero time lag between reach and grasp velocities (Fig. 3B, left). This implies that the robotic hand was opening up (i.e., negative grasp velocity) while simultaneously moving the hand toward the object (i.e., positive reach velocity) or, conversely, the robot hand was closing (i.e., positive grasp velocity) while moving the hand in a direction away from the object (i.e., negative reach velocity). This coordination pattern was made possible by the fact that the object was held in place by a spring: the macaque could move the arm forward while opening up the hand, overshoot and thereby push the object forward, and then close around the object while moving the arm backward. (Pulling back after the object was grasped was not considered part of the reach-to-grasp trial.) Monkey K also exhibited positive covariation at lead/lag times, which is consistent with a behavioral strategy where the arm moves forward (i.e., positive reach velocity) while the hand closes around the object (i.e., positive grasp velocity) either leading or at a lagged time.

Fig. 3.

Cross-covariance patterns: velocity profiles. A: cross-covariance patterns between the reaching and grasping velocity profiles on the first day (thin, light gray dotted line) through the last day (thick black solid line) of a learning experiment for monkey K (left, purple gradient) and monkey Z (right, blue gradient). Color gradient from light gray through black along with line width indicates progression from first to last day of training experiment. B: cross-covariance patterns between the reaching and grasping velocity profiles in the learning experiments of monkey K (left) and monkey Z (right); cross covariance patterns between reaching and grasping velocity profiles of 2 other animals, monkey O (right, inset, brown) and monkey A (right, inset, black), making reach-to-grasp movements with intact limbs. C: dissimilarity index. Data are Euclidean distances between the pattern of cross-covariance on every day and the average pattern over the 3 prior days of training.

In contrast, monkey Z developed a very different but consistent cross-covariance pattern with a positive covariation between the reach and grasp velocities, where the reach velocity leads the grasp velocity, coupled with a negative covariation, where the grasp velocity leads the reach velocity (Fig. 3B, right). The former implies a behavioral strategy where the hand moves toward the object (i.e., positive reach velocity) followed by the hand closing in on the object (i.e., positive grasp velocity), whereas the latter implies a behavioral strategy where the hand starts to open before the arm starts to move forward or the hand starts to close before the arm moves back (made possible by the spring). This coordination pattern closely matches the strategy employed during natural reach-to-grasp movement with intact limbs (Fig. 3B, right, inset). In a prior study, monkeys O and A were trained to make reach-to-grasp movements to five different objects placed at seven different locations (Saleh et al. 2012). Monkeys O and A both tended to reach forward toward the object, followed by closing in on the object to be grasped.

We then examined the dissimilarity index to appraise whether the cross-covariance patterns between the reaching and grasping trajectories were becoming more similar as training progressed. The dissimilarity index decreased from the beginning to the end of every training experiment for each macaque (Fig. 3C). Such decreases indicated that the cross-covariance patterns became more stable with learning.

Coordination: reaching and grasping neurons.

Stereotyped patterns of cross-covariance emerged between pairs of neurons from the first to last day of training in a training experiment (Fig. 4A). For example, the animals could learn to modulate a pair of reach and grasp neurons synchronously to drive behavior (Fig. 4A, monkey Z) or to comodulate them with a characteristic delay of ~50 ms (Fig. 4A, monkey K). Among all reach-grasp pairs of neurons with significant cross-covariation peaks (see methods), we found a significant increase in peak covariation magnitude in the last interval of training compared with the first (Wilcoxon rank sum test) for both training experiments in monkey K (K1: P = 9.0514 × 10⁻⁵; K2: P = 1.0781 × 10⁻¹²) and the second (P = 5.2105 × 10⁻²⁰) and third training experiments (P = 6.6585 × 10⁻⁵) in monkey Z (Fig. 4B). This increase in covariation magnitude is also evident by comparing the average cross-covariation profile among all significant neuron pairs in the first and last intervals of training (Fig. 4C). We found that 61.33% and 64.44% of all potential reach-grasp pairs showed significant cross-covariance in the last interval of monkey K’s first and second training experiments, respectively, and that 45%, 58%, and 53% of all potential reach-grasp pairs showed significant cross-covariance in the last interval of monkey Z’s first, second, and third training experiments, respectively.

Fig. 4.

Cross-covariance patterns: reach-grasp neuron pairs. A: patterns of cross-covariance were observed to arise between pairs of reach and grasp neurons. Data are cross-covariance on the first day of a training experiment (thin, light gray dotted line) and cross covariance on the last day of a training experiment (thick black solid line) of an example reach-grasp pair for monkey K (left, purple gradient) and monkey Z (right, blue gradient). Color gradient from light gray through black along with line width indicates progression from first to last day of training experiment. B: positive cross-covariance extrema between the first and third intervals of a training experiment, plotted for pairs of reach and grasp neurons. C: average cross-covariance of significant reach-grasp pairs in the first (gray) and third (colored) intervals of an experiment for each monkey. Shaded areas represent SE. D: dissimilarity index. Data are Euclidean distances between the pattern of cross-covariance on every day and the average pattern over the 3 prior days of training.

Similar to the behavior, we found that, with training, the cross-covariance patterns between the firing activities of pairs of reach and grasp neurons became stereotyped for each animal. We found that within training experiments, the dissimilarity index decreased, indicating that the cross-covariance patterns between pairs of reach and grasp neurons became more stable (Fig. 4D). However, during instances where we observed instability in behavioral cross-covariance patterns across training experiments, we still observed the stabilization of patterning between reach-grasp neuron pairs. This would indicate that patterns in coordinated neural activity remained stable through periods of time when the animal was not being trained and the coordination structure in the behavior was unstable.

Coordination: indirect neurons.

Stereotyped patterns of cross-covariance were observed to arise between pairs of indirect neurons (Fig. 5A). As with the reach-grasp neuron pairs, among all indirect pairs of neurons with significant cross-covariation peaks (see methods), we found a significant increase in covariation magnitude in the last interval of training compared with the first (Wilcoxon rank sum test: K1: P = 1.9789 × 10⁻⁴⁶; K2: P = 4.3247 × 10⁻⁶¹; Z1: P = 3.4657 × 10⁻¹⁸; Z2: 6.8673 × 10⁻²²⁸; Z3: 1.0746 × 10⁻²⁶⁴) for all training experiments in both monkeys (Fig. 5B), which was evident in the average cross-covariation profiles between the first and last intervals of training (Fig. 5C). We found that 57.2% and 64.44% of all potential indirect pairs showed significant cross-covariance in the last interval of monkey K’s first and second training experiments, respectively, and that 32.34%, 64.93%, and 42.82% of all potential indirect pairs showed significant cross-covariance in the last interval of monkey Z’s first, second, and third training experiments, respectively. Likewise, these cross-covariation patterns became more stable with learning within each learning experiment (except for the third training experiment in monkey Z).

Fig. 5.

Cross-covariance patterns: indirect neural pairs. A: patterns of cross-covariance were observed to arise between pairs of indirect neurons. Data are cross-covariance on the first day of a training experiment (thin, light gray dotted line) and cross covariance on the last day of training experiment (thick black solid line) of an example indirect pair for monkey K (left, purple gradient) and monkey Z (right, blue gradient). Color gradient from light gray through black along with line width indicates progression from first to last day of training experiment. B: positive cross-covariance extrema between the first and third intervals of an experiment, plotted for pairs of indirect neurons. C: average cross-covariance of significant indirect neuron pairs in the first (gray) and third (colored) intervals of an experiment for each monkey. Shaded areas represent SE. D: dissimilarity index. Data are Euclidean distances between the pattern of cross-covariance on every day and the average pattern across the 3 prior days of training.

Coordination comparison.

We directly compared the coordinated patterning that emerges after learning between reach-grasp and indirect pairs of neurons. By considering the average cross-covariation profiles among all significant reach-grasp pairs and all significant indirect pairs, we found that the peak covariation magnitude tended to be larger among the reach-grasp pairs (Figs. 6, A and B, and 7A). In monkey K, the average cross-covariation profile exhibited a range of time shifts (including zero time lags) among reach-grasp and indirect pairs (Fig. 6C, left). In contrast, in monkey Z, both sets of pairs exhibited synchronous peaks in covariation (Fig. 6C, right).

Fig. 6.

Ensemble comparison. A: average cross-covariance of reach-grasp neuron pairs (pink) and indirect neuron pairs (cyan) in the third interval of an experiment for monkey K (left) and monkey Z (right). B: distributions of the peak cross-covariance of the reach-grasp (pink) and indirect neuron pairs (cyan) in the third interval of an experiment for each monkey. C: distributions of peak cross-covariance times between reach-grasp (pink) and indirect (cyan) pairs in the third interval of an experiment for each monkey.

Fig. 7.

Broader network comparison. A: the median maximum of the cross-covariance patterns between the reach-grasp neurons plotted against the median maximum of the cross-covariance patterns between indirect neuron pairs for each day of training for monkey K (left) and monkey Z (right). Each point corresponds to a training day. Colored points refer to the last training days in each experiment. B: the percentage of all potential reach-grasp neuron pairs that showed significant extrema plotted against the percentage of all potential indirect neuron pairs that showed significant extrema for each day of training. Colored points refer to the last training days in each experiment.

In addition, we compared the degree to which reach-grasp pairs and indirect pairs showed significant coordinated patterning (see methods) on each day of training. We found that, for both monkeys, the peak covariation magnitudes of the reach-grasp pairs and the indirect pairs were highly correlated on a daily basis (Fig. 7A; monkey K: correlation coefficient = 0.78, P = 3.2801 × 10⁻⁶; monkey Z: correlation coefficient = 0.95, P = 2.1068 × 10⁻⁷). The peak correlation magnitudes tended to be greater for the reach-grasp pairs than for the indirect pairs; this trend was significant for monkey K (Fig. 7A; one-tailed sign test, P = 4.3988 × 10⁻⁵) and marginally significant for monkey Z (Fig. 7A; one-tailed sign test, P = 0.0898). In addition, the percentage of all potential reach-grasp pairs and the percentage of all potential indirect pairs that showed coordinated patterning were highly correlated on a daily basis (Fig. 7B; monkey K: correlation coefficient = 0.91, P = 9.1797 × 10⁻¹¹; monkey Z: correlation coefficient = 0.89, P = 1.8296 × 10⁻⁵). A greater percentage of reach-grasp pairs tended to participate in this coordination than indirect pairs; this trend was significant for monkey K (Fig. 7B; one-tailed sign test, P = 4.3988 × 10⁻⁵) and marginally significant for monkey Z (Fig. 7B; one-tailed sign test, P = 0.0898).

We extended our analysis to include the remaining two pair types: intracluster pairs, i.e., pairs of neurons that were both in the reach cluster or the grasp cluster, and indirect-cluster pairs, composed of one neuron from either the reach or grasp cluster and one indirect neuron (see methods). We found coordinated patterning among these pair types as well. The cross-covariance patterns between pair types became more stable with learning within a subset of training experiments (Fig. 8A). We found that the peak covariation magnitudes and the percentage of pairs participating in coordination were highly correlated across all pair types (Fig. 8, B and C). In addition, we found that intracluster pairs tended to have larger peak covariation magnitudes than indirect-cluster pairs (except in the case of reach-reach vs. reach-indirect pairs for monkey Z) and that a greater percentage of intracluster pairs tended to show significant coordinated patterning than indirect-cluster pairs (Fig. 8, B and C).

Fig. 8.

Broader network comparison. A: dissimilarity index. Data are Euclidean distances between the pattern of cross-covariance on every day and the average pattern over the 3 prior days of training for intracluster (reach-reach, grasp-grasp) and indirect-cluster (reach-indirect, grasp-indirect) neuron pairs for monkey K (left) and monkey Z (right). B: the median maxima of the cross-covariance patterns for each day of training for different types of neuron pairs plotted against one another. Colored points refer to the last training days in each experiment. C: the percentage of all potential pairs that showed significant extrema for each day of training for different types of neuron pairs plotted against one another. Colored points refer to the last training days in each experiment.

DISCUSSION

This is the first demonstration of chronically amputated monkeys learning to cortically control a BMI to perform a coordinated prehension task. Both macaques learned to perform this task using neurons that were arbitrarily assigned to either control reaching or grasping components of the task, and not based on any inherent tuning properties. Several studies have shown that after amputation or peripheral nerve lesions, the cortical area previously involved in the control of the lost limb undergoes reorganization (Qi et al. 2000; Sanes et al. 1988, 1990; Schieber and Deuel 1997; Wu and Kaas 1999). Therefore, it was not self-evident that neurons in reorganized motor cortex could be recruited to learn a complex motor task. Our positive results suggest that a similar learning approach could be used in a clinically viable BMI for human amputees.

Our approach provides us with a unique window into studying plasticity in cortical activity. We have documented the development of specific neural coordination patterns that take place when learning to use a fixed decoder for a novel, coordinated, ethologically relevant motor behavior. We have also observed the emergence of coordination patterns among neurons not directly involved in controlling the BMI. Furthermore, these neural coordination patterns stabilized with training, much like the behavioral coordination patterns between the reach and grasp components of the task. A key difference emerged between neural and behavioral stabilization: neural stabilization could occur across training experiments (see Figs. 4 and 5), even in the case of behavioral stabilization only occurring within individual training experiments (see Fig. 3), indicating that neural coordination was stable even when the behavioral coordination was not. The asymmetric stability observed in our study with regard to the coordinated neural activity and behavior may provide insights into the role of motor cortical neurons in natural motor adaptation and, more generally, the rules governing the plasticity of neural ensembles during learning.

Preservation of structure at the cortical level could prove beneficial during development. The stability of neural covariance structure could be emblematic of connectivity that emerges during development. This connectivity could emerge early on (for example, humans develop the ability to reach to grasp in infancy) and remain stable while these movements are refined through early adolescence (Kuhtz-Buschbeck et al. 1998b; von Hofsten 1984; von Hofsten and Fazel-Zandy 1984). A neural manifold, defined as a subspace of the neural state space where most of the neural variability resides (Sadtler et al. 2014), associated with a motor behavior or set of behaviors could develop early on when a behavior is first learned. Refinement could involve exploring this neural manifold and developing a set of states within that manifold. Neural stability in the face of behavioral instabilities would allow for the learning of new coordination strategies to not come at the expense of unlearning old ones. Given that there are marked differences in the strategies that children and adults use in reaching to grasp (children have wider relative grips, longer movement durations, and longer deceleration times; Kuhtz-Buschbeck et al. 1999; Zoia et al. 2006), this particular capacity could be crucial for learning a coordinated motor behavior or set of behaviors over longer timescales. In this study, we have demonstrated stabilization of neural covariance structure and behavioral coordination structure for both monkeys. Future BMI learning experiments could attempt to decouple neural and behavioral stabilization by exploring how the stability of neural covariance structure is preserved when individual or populations of neurons and/or behavior are perturbed at varying timescales.

Perhaps there are distinct subpopulations of motor cortical neurons that play a role in preservation of stable neural covariance structure. Previous work has suggested evidence for “memory” neurons in primary motor cortex (Li et al. 2001). In a force-field adaption task, one class of memory neurons was observed to change their preferred direction and then to retain their learned preferred direction even after the force field was removed during the washout period. The other class of memory neurons tended to shift their preferred direction not in the force-field phase of the experiment but in the washout phase, and in the opposite direction, as if to compensate for any persisting neural adaption in the first group.

Previous work has shown that learning a decoder involves modification in the modulation of both neurons involved in BMI control as well as indirect neurons. In particular, a similar fraction of both populations undergo a change in preferred direction during brain control, although neurons directly involved in decoding tended to modulate to a greater degree than indirect neurons (Ganguly et al. 2011). Similarly, we found that both reach-grasp pairs and indirect pairs exhibited stronger coordinated activity as learning took place and that the fractions of both pair types involved in coordination were highly correlated throughout learning experiments. However, we also observed that peak correlation values were stronger and the fraction of significant pairs were higher among reach-grasp pairs than indirect pairs. Our findings are compatible with prior work: neurons used for decoding and indirect neurons can individually modulate to varying extents but can still covary to similar extents. In fact, we found evidence of this structured coordination between pairs composed of all combinations of reach, grasp, or indirect neurons (see Fig. 8). Future theoretical and experimental work could attempt to decouple these two phenomena to see if either result could hold in isolation.

Harnessing existing network structure in a neural population has been shown to influence learning to use a BMI (Sadtler et al. 2014). Animals were able to more proficiently learn a brain-computer interface task using neural activity in primary motor cortex that was within the intrinsic manifold (a low-dimensional subspace of natural neural activity) than if the neural activity was outside the manifold, even when trained on the timescale of hours. The emergence of novel, coordinated structure in our experiment could illustrate the potential to expand the intrinsic manifold during learning on the timescale of days or weeks. This capacity could be critical for understanding motor learning during periods of development, when the intrinsic manifold could still be forming or unstable, and for designing BMIs for chronic amputees who have already undergone cortical reorganization and possibly experienced a shift or a shrinkage of the intrinsic manifold.

Although both monkeys exhibited the emergence and stabilization of coordination at the behavioral and cortical level during all of the learning experiments, there were a few notable differences in the findings for monkey K, whose neural recordings were ipsilateral to the amputation in “normal” cortex, and for monkey Z, whose neural recordings were contralateral to the amputation in reorganized cortex. Given that both animals were trained using the exact same paradigm, differences in coordination and control could be due to use of normal vs. reorganized primary motor cortex. Monkey K was using the same cortex to control the robotic arm and her intact arm. Thus her neural activity was constrained by any intact network structure (similar to the idea of the intrinsic manifold; Sadtler et al. 2014). Monkey K was able to covary neurons at a variety of lags to drive coordinated reach to grasp. Monkey Z was not operating under the same constraints and was able to use synchronous activity to drive coordinated reach to grasp that more closely resembled natural reach to grasp. Future work could compare the differences in the intrinsic manifold present in normal and reorganized cortex to better understand principles underlying functional cortical reorganization and to design decoders that are easier for patients with amputations to learn.

It is well established that there is representation of the ipsilateral upper limb in motor cortex (Donchin et al. 1998, 2002; Ganguly et al. 2009; Lecas et al. 1986; Steinberg et al. 2002). The extent to which bilateral neural encoding of the ipsilateral limb is represented in the intrinsic manifold of reorganized cortex, along with the extent to which vestiges of representation of the amputated limb remain in the intrinsic manifold of normal cortex, have yet to be uncovered. Harnessing any limb representation still present in the reorganized cortex of amputees or stroke patients could not only assist in the development of targeted BMIs but also in the rehabilitation or repurposing of existing cortical circuitry using brain-computer interfaces (Soekadar et al. 2015).

Along a similar vein, the feedback paradigm used in our approach could be further expanded to facilitate learning. We only made use of two channels of feedback in the control of the robotic arm: visual feedback about the state of the arm and reward feedback. By adding feedback channels to provide somatosensory (touch or proprioceptive) information, we could investigate the role of different types of sensory feedback in driving plasticity in motor cortex. This work could have potential bearing on how best to augment prosthetics with sensory feedback, building on previous work (Chatterjee et al. 2007; Collinger et al. 2013; Murphy and Miller 2009; Suminski et al. 2010; Tabot et al. 2013; Weber et al. 2011). In addition, future work could augment this paradigm by examining the effect of certain choices in filter design, such as number of neurons used for decoding, distributions of weights, and filter shape, perhaps shaping the coordination structure such that control is more naturalistic and easier to learn.

Our approach provides a unique model for both studying the development of novel, coordinated reach-to-grasp movements at the behavioral level and cortical level, and for examining the plasticity afforded to us by primary motor cortex through the lens of learning to use a BMI. In addition, the possibility of training subjects to perform a naturalistic motor task by modulating cortical neurons using artificial mapping and of potentially influencing network activity in a lasting manner could have clinical implications not only for patients with amputations but also for the rehabilitation of those who have strokes or traumatic brain injuries (Soekadar et al. 2015).

GRANTS

This research was supported by Defense Advanced Research Projects Agency Grant N66 001-1211-4023; Integrative Graduate Education and Research Traineeship: Integrative Research in Motor Control and Movement, National Science Foundation Grant DGE-0903637; National Institute of Neurological Disorders and Stroke Grant R01NS045853; National Science Foundation IOS 145704; and National Eye Institute EY023371.

DISCLOSURES

N. G. Hatsopoulos serves as a consultant for BlackRock Microsystems, Inc., the company that sells the multielectrode arrays and acquisition system used in this study. The remaining authors declare no competing financial interests.

AUTHOR CONTRIBUTIONS

M.V., K.B., J.S., I.B., A.E., M.S., A.F., K.O., and N.G.H. conceived and designed research; M.V., K.B., J.S., K.S., and S.G. performed experiments; M.V. analyzed data; M.V., L.O., and N.G.H. interpreted results of experiments; M.V. prepared figures; M.V. drafted manuscript; M.V., K.B., I.B., A.F., K.O., and N.G.H. edited and revised manuscript; M.V., K.B., J.S., I.B., A.E., K.S., S.G., M.S., L.O., A.F., K.O., and N.G.H. approved final version of manuscript.