Abstract
A fundamental organizational principle of the primate motor system is cortical control of contralateral limb movements. Motor areas also appear to play a role in the control of ipsilateral limb movements. Several studies in monkeys have shown that individual neurons in primary motor cortex (M1) may represent, on average, the direction of movements of the ipsilateral arm. Given the increasing body of evidence demonstrating that neural ensembles can reliably represent information with a high temporal resolution, here we characterize the distributed neural representation of ipsilateral upper limb kinematics in both monkey and man. In two macaque monkeys trained to perform center-out reaching movements, we found that the ensemble spiking activity in M1 could continuously represent ipsilateral limb position. Interestingly, this representation was more correlated with joint angles than hand position. Using bilateral electromyography recordings, we excluded the possibility that postural or mirror movements could exclusively account for these findings. In addition, linear methods could decode limb position from cortical field potentials in both monkeys. We also found that M1 spiking activity could control a biomimetic brain–machine interface reflecting ipsilateral kinematics. Finally, we recorded cortical field potentials from three human subjects and also consistently found evidence of a neural representation for ipsilateral movement parameters. Together, our results demonstrate the presence of a high-fidelity neural representation for ipsilateral movement and illustrates that it can be successfully incorporated into a brain–machine interface.
Introduction
Although the main organizational principle of primate motor systems is cortical control of contralateral limb movements, motor areas also appear to play a role in ipsilateral limb movements (Matsumani and Hamada, 1981; Tanji et al., 1988; Rao et al., 1993; Donchin et al., 1998; Cisek et al., 2003; Verstynen et al. 2005; Wisneski et al., 2008; Brus-Ramer et al., 2009). Several studies in monkeys have shown that individual M1 neurons, on average, are modulated by ipsilateral arm movements (Donchin et al., 1998; Cisek et al., 2003). Numerous studies have also presented evidence that, after unilateral damage, the “contralesional” intact hemisphere plays an increased role in ipsilateral movements (Brinkman and Kuypers, 1973; Dancause, 2006; Hummel and Cohen, 2006). Indeed, studies have demonstrated increased activity in homologous regions of the intact hemisphere in stroke patients (Blasi et al., 2002). However, the intact contralesional hemisphere may also play a maladaptive role under certain conditions (Dancause, 2006; Hummel and Cohen, 2006).
To better understand the bihemispheric control of movements, it remains important to understand the distributed neurophysiological representation of ipsilateral limb control. Recent advances in recording technology and computational processing have led to greater characterization of information encoded by simultaneously recorded neural ensembles (Wessberg et al., 2000; Carmena et al., 2003; Mulliken et al., 2008). These efforts have increasingly highlighted differences in the encoding of information at the ensemble level relative to that for single neurons (Wessberg et al., 2000; Averbeck et al., 2006; Mulliken et al., 2008). Here we characterize the distributed ensemble representation of ipsilateral kinematics in both monkey and man using linear regression methods.
We further tested the generality of such a finding by decoding ipsilateral kinematics from cortical field potentials [i.e., local field potential (LFP) in monkeys and subdural electrocorticogram (ECoG) in human subjects]. Past work has demonstrated that both LFP (in monkey) and ECoG (in man) can be used to decode direction of contralateral limb movements (Mehring et al., 2003; Schalk et al., 2007). Less is known about continuous decoding of ipsilateral movement parameters from cortical field potentials. Although two recent studies demonstrated that ipsilateral limb movements can result in specific patterns of activity (Rickert et al., 2005; Wisneski et al., 2008), it remains unclear if cortical potentials can continuously represent ipsilateral kinematics.
Reliable, continuous decoding of movement parameters represents an important step toward the creation of fully functional biomimetic Brain–Machine Interfaces (BMIs) (Wessberg et al., 2000; Serruya et al., 2002; Taylor et al., 2002; Carmena et al., 2003; Schalk et al., 2007; Schalk et al., 2008; Mulliken et al., 2008). Although the majority of studies supporting the development of BMIs have incorporated the contralateral neural representation of movements, there is increasing interest in designing BMIs compatible with extensive hemispheric injury (Buch et al., 2007; Wisneski et, 2008). Ipsilateral control would allow a large cadre of patients with motor cortex damage and contralateral weakness to eventually benefit from BMIs. In this study, we also demonstrate that the ipsilateral neural representation can be used in a closed-loop BMI.
Materials and Methods
Monkeys
Surgery.
Two adult male rhesus monkeys (Macaca mulatta) were chronically implanted in the brain with arrays of 64 Teflon-coated tungsten microelectrodes (35 μm in diameter, 500 μm separation between microwires) in an 8 × 8 array configuration (CD Neural Engineering). Monkey P was implanted in the arm area of primary motor cortex (M1) and the arm area of dorsal premotor cortex (PMd), both in the left hemisphere, and the arm area of M1 of the right hemisphere, with a total number of 192 microwires across three implants. Monkey R was implanted bilaterally in the arm area of M1 and PMd (256 microwires across four implants). Localization of target areas was performed using stereotactic coordinates from a neuroanatomical atlas of the rhesus brain (Paxinos et al., 2000). All procedures were conducted in compliance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the University of California at Berkeley Institutional Animal Care and Use Committee.
Electrophysiology.
Unit activity was recorded using the MAP (Multichannel Acquisition Processor) system (Plexon). For this study, only units from each primary motor cortex were used. Only units that had a clearly identified waveform with a signal-to-noise ratio of at least 4:1 were used. Activity was sorted using an on-line sorting application (Plexon) before recording sessions. Isolation of units was then verified off-line. Large populations of well-isolated units were recorded during each daily session in both monkeys.
Electromyography.
Surface gold disc electrodes (Grass Technologies) were mounted on medical adhesive tape and placed on the skin overlying muscle groups at the beginning of select sessions. Bilateral muscle groups tested included pectoralis major, biceps, deltoid, triceps, trapezius, latissimus dorsi, neck muscles, and forearm muscles. Electromyography (EMG) signals were amplified by a 10,000 factor with a multichannel differential amplifier (Grass Technologies) and stored (Plexon). Signals were then high-pass filtered, rectified, and smoothed by convolution with a 25 ms triangular kernel and normalized. Directional activation of each EMG signal was estimated by measuring the activity in a 300 ms window after the onset of movement to each target. EMG signals were collected over ∼10 trials in each direction. The significance of this effect was assessed using ANOVA.
Experimental setup and behavioral training.
Monkeys were trained to perform a center-out delayed reaching task using a Kinarm (BKIN Technologies) exoskeleton (see Fig. 1A). During training and recording, animals sat in a primate chair that permitted limb movements and postural adjustments. Head restraint consisted of the animal's head post fixated to the chair. Kinematic variables (position, velocity, and acceleration) were continuously monitored and recorded.
The behavioral task consisted of hand movements from a center target to one of eight peripheral targets (i.e., “center-out” task) distributed over an ∼8 cm diameter circle. The workspace was created to minimize any requirement for postural changes during task performance. Target radius was typically 0.75 cm. Trials were initiated by entering the center target and holding for a variable time period of 500–1000 ms. The GO cue (center changed color) was provided after the hold period. A liquid reward was provided after a successful reach to each target and a peripheral hold period (200–500 ms). Visual feedback of hand position was provided by a cursor precisely colocated with the center of the hand (radius, 0.5 cm). During the task, the nontask arm was immobilized in a padded splint.
Decoding motor parameters from neural ensembles.
A linear regression model was used to predict limb position and velocity (both joint position and end point position). In this model (Equation 1), the inputs, X(t), were a matrix with each column corresponding to the discharges of individual neurons, and each row representing one time bin. The output Y(t), was a matrix with one column per motor parameter. The linear relationship between neuronal discharges in X(t), and behavior in Y(t) was expressed as follows:
in which a and b are constants, calculated to fit the model optimally. First, a(u) are the impulse response functions required for fitting X(t) to Y(t)as a function of time lag u between inputs and the outputs. Ten time lags were used during these experiments. Second, b represents the Y-intercept in the regression. The final term in the equation, ε(t), represents residual errors.
Brain–machine interface.
We used the linear filter described in the previous section to predict shoulder and elbow joint angles from the recorded neural activity (only M1-ipsi activity was included). The model was trained on 10 min of activity and then used to predict position from subsequent neural activity (Wessberg et al., 2004). Neural activity was streamed over a local intranet via the PLEXNET client–server application (Plexon) and converted into 100 ms bins of spiking activity. Each binned value was used to generate real-time predictions of the shoulder and elbow joint angles that were streamed to the Kinarm interface as control signals. The cursor position was updated on the Kinarm projection screen at 10 Hz.
Filter parameters were not changed during each daily brain control (BC) experiments (usually 2–3 h per day). For the multiple experiments reported in Figure 5, the need for daily retraining of the filter (i.e., at the start of a BC session) was determined by the stability of the units. The stability of a recorded unit was solely determined by visually comparing the waveform shape with the previous day's stored template. When all units were putatively stable, no retraining of the filter was performed. If there were any changes in the waveform (e.g., a single waveform change), then the filter was retrained during a manual control session. The animals were then allowed a period of time to relearn the decoder properties (typically ∼1 h). Task performance in BC was determined after this period of learning. After this defined period, all subsequent trials and attempts were included in the analysis of task performance (also see below).
Data analysis
Task performance analysis.
A correct trial was defined as successful movement of the cursor to the target. We minimized the number of false-positive self-initiations (i.e., the number of trial attempts by adjusting the required hold period). This threshold was determined by measurements of false triggers when the BMI was engaged but the screen was turned off (i.e., in the absence of volitional control of the cursor). The time-to-target measurement reflected the movement time from the center to each peripheral target. An error trial consisted of inability to reach the target in 10 s.
Predictive power of the decoder.
The predictive power of each decoder was determined by comparison (i.e., correlation) of neural predictions of shoulder and elbow angular position with that of measured values. Estimation of predictive power was performed using 2 min of movements outside of the 10 min training window.
Preferred direction.
The significance of the directional modulation of a unit's firing rate was determined using an ANOVA test. Directional tuning was estimated by comparing the mean firing rate as a function of target angle during execution of the movement. The tuning curve was estimated by fitting the firing rate with a sine and a cosine as follows:
in which θ corresponds to reach angle and f corresponds to the firing rate across the different angles (Georgopoulos et al., 1986). Linear regression was used to estimate the B coefficients. The preferred direction (PD) was calculated using the following: PD = tan−1 (B2/B3), resolved to the correct quadrant.
LFP analysis.
Performed similarly to that outlined below for the electrocorticographic analysis.
Human subjects
Three subjects (age range, 18–35 years) with refractory epilepsy were recruited from a pool of patients undergoing intracranial monitoring for the localization of an epileptogenic focus. Each patient had undergone a craniotomy for chronic (1–2 weeks) implantation of a subdural electrode array and/or depth electrodes. Electrode placement was solely determined on clinical grounds and varied between subjects (see Figs. 5 and 6). Subject 1 was right-handed with a left hemispheric grid, subject 2 was left-handed with bilateral strips, and subject 3 was left-handed with right hemispheric grid. None of the subjects had overt cognitive deficits, and antiepileptic drug therapy was discontinued during ECoG recordings. Consenting patients participated in the research study during the week of ECoG monitoring. The study protocol, approved by the University of California San Francisco and University of California Berkeley Committees on Human Research, did not interfere with the ECoG recording made for clinical purposes, and presented minimal risk to the participating subjects.
Recordings.
The electrode grids used to record ECoG signals for this study were either 64-channel 8 × 8 (patients 1, 2, 4, 5) or four strips of 8 × 1 (patient 3) platinum–iridium electrodes. Electrode diameter was 4 mm (2.3 mm exposed), with 10 mm center-to-center spacing. Signals from the ECoG grids were split and sent to both the clinical system and a custom recording system. A broadband (∼50 kHz), 256 channels preamplifiers (PZ2-256 256-Channel PreAmp; Tucker-Davis Technologies) was used to amplify the ECoG signals with the electrode furthest from the motor cortex used as a reference for all other grid electrodes. The amplified data were then sent to an ultrahigh performance data acquisition processor over a fast fiber optic connection (RZ2 Z-Series Base Station; Tucker-Davis Technologies) that digitized the signal at 3052 Hz with 16-bit resolution.
Subjects used a stylus to perform arm movements on the touch-screen connected to the designated laptop. The stylus position was registered as a mouse position and was sampled using custom-made MATLAB software. A PC-based, bus-powered USB device (Measurement Computing's USB-1208FS) was used to convert the mouse position to an analog voltage (1–4 V), and these voltages were sent to the analog input of the data acquisition processor (RZ2 Z-Series Base Station; Tucker-Davis Technologies) to be sampled and stored together with the ECoG signals. During the performance of the task additional event markers (e.g., beginning of a trial, appearance of a target, acquiring of a target, etc.) were sent to other analog inputs of the data acquisition processor from the digital ports of the PC-based bus-powered USB analog to digital convertor (Measurement Computing's USB-1208FS).
Behavioral task.
During the recording, subjects were seated in a hospital bed with a touch-screen (Keytec) placed in front of them in the horizontal plane. They were asked to use a stylus to perform arm movements on the touch screen using their shoulder and elbow rather than their wrist. To evaluate the coupling between the ipsi and contra ECoG activity to arm movements, we asked each subject to perform the task once with their right hand and one with their left. A trial began with the appearance of a rectangular target (1 cm side) at the center of the reach field. This cue indicated to subjects to move their hand while holding a stylus toward the target; once the center target was obtained, one of several (six or eight) randomly chosen peripheral targets (1 cm radius) appeared on the touch screen. After the 400 ± 200 ms delay, the center target disappeared. This was the “GO” signal indicating that the subject should perform a reach toward the lit target. Once the target was hit, a new trial began by the appearance of a rectangular target at the center of the reach field. Each subject made 30 reaches to each target (total of 180 or 240 reaches).
Movement reconstruction.
The first step in our analysis included filtering, re-referencing, and down-sampling of the ECoG and movement signals. Line noise (60 Hz and its harmonics) was removed using a notch filter and then re-referenced by subtracting the common average reference (CAR) from the data of each electrode. CAR was calculated by averaging the raw signal of all the electrodes, omitting the ones that visual inspection suggested poor signal quality. After the re-referencing, the data were band-passed between 1 and 250 Hz and down-sampled to 500 Hz.
To reconstruct a subject's movements (X and Y position) from the ECoG data, we used the ECoG activity as an input to the Wiener filter. ECoG signals were band-passed into nine frequency bands (1–8, 9–15, 16–30, 31–50, 51–70, 71–90, 91–110, 111–131, and 131–150), followed by calculating the analytic amplitude of each frequency band using the Hilbert transform. The resulting nine time series were appended to the original time series of the ECoG signal (i.e., a total of 64 × 10 = 640 ECoG “channels” in subjects 1 and 2). The time series were down-sampled to 15 Hz, and 1 s of ECoG data (15 bins) preceding a given point in time was used to train the model and generate predictions. First, we tested the contribution of each individual new time series to the prediction of the hand movements and then selected the ones that produced the best prediction to be used as the inputs to the Wiener filter. Estimation of predictive power was always performed using 60 s of movements outside of the training window, and the predictive power of each decoder was determined by comparison of neural predictions of X and Y position with that of measured values.
Statistical analysis.
To test the statistical significance of our results, we compared correlation-coefficient (CC) distributions for actual and a “randomly shifted” version of the same data. To obtain CC distributions for the actual runs, we circularly shifted (MATLAB function circshift) both the ECoG and movement data with a random shift (300 times). We found the best Wiener filter weights for the first 4 min of the shifted data and then applied them to the remaining portion. This process resulted in a distribution of CCs for each movement parameter (the means and SDs for the ipsilateral trials are depicted in Table 1). For the “random-shift” method, we created a distribution by picking a random lag between the movement and the ECoG data (300 times). This time, however, while we shifted the ECoG data, the movement data was held constant. The result of this procedure was another distribution of CCs.
Table 1.
Neural signal | Elbow | Shoulder | Hand (X) | Hand (Y) |
---|---|---|---|---|
Spikes (n = 10) | 0.81 ± 0.03 | 0.78 ± 0.05 | 0.61 ± 0.04 | 0.75 ± 0.04 |
LFP (n = 8) | 0.47 ± 0.02 | 0.42 ± 0.02 | 0.29 ± 0.05 | 0.45 ± 0.02 |
ECoG (n = 3) | 0.60 ± 0.03 | 0.61 ± 0.03 |
aValues indicate the correlation coefficient R (mean ± SEM). The value of n is the number of sessions used in the analysis.
Results
Monkeys
We trained two macaque monkeys to perform a center-out reaching task with the right upper limb using the Kinarm Exoskeleton system. Reaching movements with the proximal arm and hand were limited to 2 degrees of freedom (flexion/extension of the elbow and shoulder) in the horizontal plane. A cursor (r = 0.5 cm) on the horizontal screen was collocated with hand movements (Fig. 1A). Following chronic implantation of microelectrode arrays into bilateral M1, we recorded the neural activity (both spike and LFP) during the performance of a center-out reaching task (Fig. 1B). We first estimated the percentage of units that were significantly modulated by the direction of arm movements. Figure 1C illustrates a single unit from ipsilateral M1 (M1-ipsi) whose firing rate was directionally modulated. For both animals, the respective fraction of modulated neurons for M1-ipsi were 43 ± 5% and 54 ± 6%, whereas those for contralateral M1 (M1-contra) were 67 ± 6% and 73 ± 7% (Fig. 1D). These estimates (from our chronic recordings) are in-line with past reports using acute recording methods (Donchin et al., 1998; Cisek et al., 2003).
We next used linear regression techniques to characterize the bihemispheric ensemble representation of movement parameters (Humphrey et al., 1970; Ashe and Georgopoulos, 1994; Wessberg et al., 2000; Carmena et al., 2003). In general, although regression techniques have found that multiple parameters (e.g., target direction, position and velocity) are correlated with activity at the level of single neurons, correlations with velocity appear to be the most prominent (Ashe and Georgopoulos, 1994; Reina et al., 2001; Paninski et al., 2004). This analysis used neural activity that was closely temporally linked to external movements (i.e., temporal lag of <100 ms).
We first performed a similar analysis for units recorded from both M1-ipsi and M1-contra. Consistent with past results, we found that individual unit activity in M1-contra was more closely associated with velocity than position (data not shown). For M1-ispi, we also found that individual unit activity was significantly more correlated with velocity than position (position: 0.05 ± 0.03; velocity: 0.16 ± 0.02 mean ± SEM; p < 0.001 t test). However, when the same analysis was performed with neural ensembles from each hemisphere (i.e., single bin of 100 ms with at least 50 units per hemisphere), both parameters could be decoded equally well (M1-ipsi: 0.50 ± 0.05 and 0.49 ± 0.06 for position and velocity respectively, p > 0.3 t test). Identical results were obtained regardless of whether angular joint or hand-based coordinates were used for comparison of position and velocity predictions. Together, this further indicates that information not readily apparent at the single neuron resolution (i.e., velocity more represented than position) can be reliably decoded from neural ensembles (i.e., velocity and position are equally represented).
We next performed an additional set of analysis to directly compare with methods typically used for real-time continuous prediction of movement parameters (Wessberg et al., 2000; Serruya et al., 2002; Carmena et al., 2003). One key difference is the simultaneous inclusion of multiple temporally lagged bins into the regression model (e.g., 10 lags are typically used). While the animals performed center-out reaching movements with the right upper limb, the recorded M1 spike activity (the respective ipsilateral and contralateral spike activity were grouped separately) was correlated with limb kinematics to generate decoders for each variable (Fig. 2A). Hence, we will use the term “decoder” to refer to the combined transforms. Figure 2B illustrates the predictive ability of either the ipsilateral or the contralateral neural ensemble activity during a single session. For multiple sessions in both monkeys (n = 5 sessions each, 10 lags with at least 50 units/hemisphere), ipsilateral ensemble activity could reliably and continuously predict angular joint positions (Table 1). We subsequently generated a “neuron-dropping curve” (Wessberg et al., 2000; Carmena et al., 2003) for each movement parameter to estimate the relationship between ensemble size and the representation of a given parameter. For both subjects, the fidelity of the representation improved as a function of the size of the neural ensemble (Fig. 2C).
With the simultaneous inclusion of temporally lagged bins from M1-ipsi, limb position could be better decoded than velocity (with 10 lags, r = 0.8 ± 0.02 and 0.69 ± 0.04 mean ± SEM for position and velocity respectively, p < 0.0001 t test). Consistent with this notion was the observation of a relatively sharper decline in velocity-related information for increasing temporally lagged bins (supplemental Fig. 1, available at www.jneurosci.org as supplemental material). It is possible that the inherent autocorrelation of changes in limb position or velocity during this task underlies this result (Paninski et al., 2004). We also assessed the relation between M1-ipsi activity and the coordinate system for estimating limb position (i.e., joint vs hand position). In our experimental system, the robotic exoskeleton allows accurate monitoring of joint angles as well as hand position. In both animals, we consistently found that both the contralateral and ipsilateral ensemble activities were more correlated with angular joint kinematics than end-point hand coordinates (p < 0.001; ANOVA; Table 1).
What is the temporal evolution of the predictions from each hemisphere relative to limb movements? We performed high resolution (time bins of 10 ms) analysis of the predictive ability of neural ensembles from each hemisphere. Consistent with past reports, we observed a delayed peak in the relationship between M1-contra activity and limb position (∼50 ms) (Fig. 3). In contrast, the value of this relationship was delayed for M1-ipsi (∼110 ms). Thus, it appeared that at least a portion of the M1-ipsi neural representation is delayed relative to that from M1-contra.
To exclude the possibility that M1-ipsi activity simply reflected spurious activation of the opposite limb (e.g., postural adjustments or mirror movements with the left hemibody during reaches with the right arm), we measured bilateral EMG activity during select sessions (Cisek et al., 2003). Figure 4 illustrates the directional modulation of the EMG activity for the right biceps and left pectoralis during reaches with the right arm. For Monkey P, there was no evidence of significant activation of the left hemibody during reaching movements. For Monkey R, only one left hemibody muscle (left trapezius) demonstrated significant activation during the task. Together, these results confirmed that M1-Ipsi activity largely did not reflect spurious activation of the opposite hemibody.
We next quantified the ability of cortical field potentials to continuously represent ipsilateral kinematic parameters. We used spectral decomposition of the LFP signal as an input to the Wiener filter (Schalk et al., 2007). The M1-ipsi LFP was also found to be significantly correlated with ipsilateral limb kinematics (Table 1).
Human subjects
We assessed the generalizability of our results to primate motor systems by testing this relationship in three human subjects. ECoG recordings from patients with epilepsy offer a means to evaluate the ability of cortical field potentials to predict ipsilateral motor parameters (Schalk et al., 2007; Schalk et al., 2008). ECoG signals (from either the left or right hemisphere) were recorded from three subjects during the performance of center-out reaches with each hand. Traces of hand movements are depicted in Figure 5B. Shown in Figure 5C are representative velocity profiles of the movement from the center to each of the targets.
We next evaluated whether linear regression methods could continuously decode ipsilateral upper-arm position. A reconstruction of hand trajectories from the recorded neural signals is illustrated in Figure 6A. For three such subjects, cortical field potentials were found to be significantly correlated with ipsilateral limb kinematics (Table 1). In addition, bilateral surface EMG measurements during the performance of this task did not reveal evidence of the opposite hemibody activation.
We also assessed the anatomical distribution of such predictive information. Although this was observed to be relatively distributed, activation appeared to be most prominent in sensorimotor regions (Fig. 6B and supplemental Fig. 2, available at www.jneurosci.org as supplemental material). Shown in Figure 6C are the bands which contributed the most to the prediction of ipsilateral limb movements (also see supplemental Fig. 3, available at www.jneurosci.org as supplemental material, for mean of all subjects). Moreover, we attempted to compare the temporal evolution of predictions for both hemispheres. For cortical field potentials at the level of ECoG, no significant differences could be detected between the two hemispheres.
Closed-loop BMI in monkeys using the ipsilateral neural representation of movement
The results above indicate that ipsilateral movement parameters can be reliably represented in both M1 spike activity as well as cortical field potentials in both monkey and man. We subsequently asked whether a decoder trained under such conditions can be successfully used in a closed loop BMI in monkeys (Fig. 7A). Figure 7B illustrates the typical cursor trajectories under control BC (in which the neural activity exclusively controlled the position of the computer cursor). After a period of training, each monkey could perform the center-out task in BC. For multiple sessions, both animals could accurately perform the center-out task in BC (Monkey P: 83 ± 6% accuracy with a mean time to reach the target of 2.4 ± 0.5 s; Monkey R: 76 ± 9% accuracy with a mean time to reach the target of 2.9 ± 0.3 s; all reported as mean ± SEM).
Discussion
Our results demonstrate that the distributed activity in primate motor areas can reliably and continuously represent ipsilateral upper limb kinematics. We found that such information could be decoded by applying linear methods to neural signals at a variety of temporal and spatial scales (ensemble spike activity as well as the aggregate cortical field potential at two different resolutions). We further demonstrate that the spike activity from M1 can be used in a biomimetic closed-loop BMI designed to control ipsilateral limb kinematics.
Role of motor cortex in ipsilateral movements
Several studies have demonstrated that the activity of single M1 neurons can be modulated by ipsilateral arm, hand and finger movements (Tanji et al., 1988; Donchin et al., 1998; Cisek et al., 2003). Studies in monkeys have further shown that while subsets of M1 neurons are exclusively tuned to the direction of ipsilateral arm movements, another fraction of neurons are active during bimanual movements (Donchin et al., 1998). Lesion and stimulation studies in both monkey and man provide additional support for a role of motor regions in ipsilateral limb control (Brinkman and Kuypers, 1973; Rao et al., 1993; Chen et al., 1997; Verstynen et al., 2005; Dancause, 2006; Brus-Ramer et al., 2009).
We demonstrate that ipsilateral limb kinematics can be reliably decoded, in real-time, from the population activity at multiple scales in motor areas. However, the exact role of motor cortex in the control of ipsilateral proximal and distal limb movements remains unclear. The anatomical substrate for the control of ipsilateral movements has been hypothesized to be mediated through either descending uncrossed fibers or trancallosal pathways (Dancause, 2006). Our finding that the peak of ipsilateral movement prediction is delayed relative to that for contralateral movements lends some support to the notion of interhemispheric transfer of this representation.
One focus of this study was to characterize the representation of arm and hand movement parameters in ipsilateral neural ensembles. Our finding that both the ensemble spiking activity and the population cortical field potentials obtained at two temporal and spatial resolutions can continuously predict ipsilateral limb kinematics further demonstrates that distributed neural ensembles can reliably encode information. Coordinated and dexterous bimanual movements likely require a high-fidelity representation of ipsilateral kinematics. One possibility is that it is an efference copy of contralateral motor commands. Another possibility is that it mediates bimanual coupling (Donchin et al., 1999). Thus, it could provide a neural substrate for the observed phenomenon of spatial coupling during bimanual tasks (Oliveira et al., 2001).
An interesting observation was that M1 activity was more indicative of joint position than hand kinematics (Table 1). Past analysis of the contralateral representation of limb movements have shown that it is correlated with multiple movement parameters (Ashe and Georgopoulos, 1994; Reina et al., 2001; Paninski et al., 2004). However, because hand position covaries with joint angles and other aspects of arm movement, it remains difficult to conclude what is truly encoded in these neurons (Reina et al., 2001). Moreover, it remains possible that a generative model could provide a more parsimonious explanation for the apparent encoding of multiple parameters (Todorov, 2000). It is less clear how to formulate such a model (e.g., direct control of musculature) for the ipsilateral neural representation.
It is also important to note that an alternate explanation for the apparent modulation of neural activity by ipsilateral movements is that the opposite nontask arm and the axial musculature may be active (Cisek et al., 2003). There are at least two lines of evidence suggesting that this does not exclusively account for all neural activity putatively related to ipsilateral limb movements. Classical studies demonstrated that proximal arm movements can be exclusively controlled by ipsilateral motor cortex (Brinkman and Kuypers, 1973). Moreover, distal limb movements made in the absence of proximal movements resulted in ipsilateral neural activity (Tanji et al., 1988). In this study, we minimized the need for postural adjustments. The workspace was optimized such that reciprocal postural movements were not obviously required. Although one monkey was found to have limited spurious nontask arm EMG activity, the other did not. Moreover, the task performed by the human subjects did not appear to result in such spurious activity. Although it remains difficult to completely exclude that a component of neural activity was related to nontask arm activity, it seems unlikely to account for our observations.
Possible role of the ipsilateral representation in neurorehabilitation
The exact role of ipsilateral motor regions in the recovery of arm function after brain injury remains unclear. There is a significant fraction of patients who do not recover function after a stroke (Dancause, 2006; Hummel and Cohen, 2006). Large subcortical strokes (i.e., with loss of descending contralateral corticospinal pathways) are associated with such a lack of functional recovery (Shelton and Reding, 2001). This further suggests that ipsilateral motor areas and its associated descending connections are not sufficient by themselves to support functional recovery. In contrast, in patients who experience spontaneous recovery of limb function, the ipsilateral hemisphere may play a greater role (Gerloff et al., 2006; Lotze et al., 2006). Trancallosal pathways have been implicated in this process (Dancause, 2006; Gerloff et al., 2006). Thus, a likely possibility is that in the presence of the appropriate anatomical substrate, ipsilateral motor areas may assist the process of functional recovery.
Importantly, there is also evidence that the contralesional hemisphere (i.e., ipsilateral to the affected limb) can play a maladaptive role in stroke patients (Dancause, 2006; Hummel and Cohen, 2006). For instance, inhibition of the contralesional hemisphere through noninvasive methods can transiently improve motor function (Hummel and Cohen, 2006). Thus, the ipsilateral hemisphere appears to dampen the excitability of the damaged hemisphere and impede the process of recovery. These studies indicate that the balance of interhemispheric inhibition is important. In this context, another interpretation of our findings is that it represents the transcallosal shaping of ipsilateral motor areas. In the damaged brain, this may impede recovery. Future research may help to uncover how the ipsilateral hemisphere can either facilitate or impede recovery.
A BMI using the neural representation of the ipsilateral arm
Our results demonstrate that the ensemble representation of ipsilateral arm movements can be used in a closed-loop biomimetic BMI. Past work has not shown that such a representation can be exclusively used to create a closed-loop biomimetic BMI. The decoder used in these experiments predicted the natural relationship between M1 spike activity and ipsilateral movements. It is likely that neurons more sensitive to ipsilateral arm movements than to contralateral arm movements (Donchin et al., 1998) needed to be actively modulated during BC. As in past studies, feedback and learning during closed-loop control were important for improvements in task performance over the course of a session (Taylor et al., 2002; Carmena et al., 2003; Hochberg et al., 2006; Mulliken et al., 2008).
Past work has suggested that the accuracy of movement parameter decoding (Wessberg et al., 2000) is important for BMI function (Taylor et al., 2002; Carmena et al., 2003; Schalk et al., 2007). Linear algorithms have been demonstrated to be reliable in extracting parameters from simultaneously recorded neural activity (Wessberg et al., 2000; Carmena et al., 2003; Mulliken et al., 2008). There is mixed evidence regarding the magnitude of improvements with more sophisticated decoding techniques (Kim et al., 2006; Wu et al., 2006; Mulliken et al., 2008). Most importantly, however, our results indicate that even while the ipsilateral predictions are less than that for the contralateral limb, they can be successfully incorporated in a BMI.
An important question for future research is to fully understand the role of biomimetic decoders (Radhakrishnan et al., 2008). Recent work has suggested that a combination of volitional control of neural activity, visual feedback, and plasticity mechanisms can allow direct neural control of neuroprosthetic devices independent of any relationship to natural movements (Mortiz et al., 2008; Ganguly and Carmena, 2009). However, given that cortical networks in M1 are likely optimized for dexterous bimanual limb control, a seemingly likely possibility is that biomimetic decoders can best capitalize upon existing cortical architecture. For instance, comparison of both unimanual and bimanual movements has revealed that both overlapping and nonoverlapping patterns of activity are present (Donchin et al., 1998; Wisneski et, 2008). Accordingly, it remains possible that integration of a biomimetic decoder for ipsilateral movements will not interfere with existing cortical networks for contralateral movements. Such interference could occur if neurons were incorporated into a BMI without regard for their actual relationship to natural movements.
A BMI for patients with chronic stroke
A recent study demonstrated that volitional control of noninvasively recorded neural signals (with magnetoencephalography) is possible even in chronic stroke patients with limb paralysis (Buch et al., 2007). Although arm function did not improve outside of training sessions, subjects could control a prosthetic device through modulation of the μ-rhythm. Interestingly, they could use μ-rhythms from either ipsilateral or contralateral brain regions.
In general, the resolution of recorded neural signals (e.g., noninvasive vs invasive intracortical recordings) required for a long-term, reliable BMI remains unclear. Thus, continued basic and translational research using a variety of neural signals is important. Our study explores the eventual creation of an invasive biomimetic BMI based on the ipsilateral neural representation of arm movements. We characterized the ipsilateral ensemble representation of continuous limb movements in nonparalyzed subjects. Methodological limitations (i.e., recording technique and size of cortical area monitored) prevent detailed comparison of the ipsilateral representation in nonhuman and human subjects. However, it is reassuring that qualitatively similar results were obtained in both healthy nonhuman primates as well as in patients with chronic focal epilepsy with unclear long-term effects on cortical organization. It will be important to demonstrate that this representation is intact after damage to the opposite brain hemisphere. In support of this concept are imaging studies demonstrating that even after extensive damage to contralateral motor areas, contralesional motor areas are active in a similar manner (Cramer et al., 1999; Dancause, 2006; Buch et al., 2007).
Conclusion
In summary, our results provide evidence that motor areas encode ipsilateral limb kinematics with high precision. Moreover, this representation can be used in a closed-loop BMI. These findings suggest the possibility of eventually creating fully functional BMIs for patients suffering from extensive unilateral hemisphere brain injury.
Footnotes
This work was supported by the American Heart Association, the Christopher and Dana Reeve Foundation, National Institute of Neurological Disorders and Stroke (NINDS) Grant NS21135, National Institute on Drug Abuse Grant R01DA19028, and NINDS Grant P01NS040813.
References
- Ashe J, Georgopoulos AP. Movement parameters and neural activity in motor cortex and area 5. Cereb Cortex. 1994;4:590–600. doi: 10.1093/cercor/4.6.590. [DOI] [PubMed] [Google Scholar]
- Averbeck BB, Latham PE, Pouget A. Neural correlations, population coding and computation. Nat Rev Neurosci. 2006;7:358–366. doi: 10.1038/nrn1888. [DOI] [PubMed] [Google Scholar]
- Birbaumer N, Cohen LG. Brain–computer interfaces: communication and restoration of movement in paralysis. J Physiol. 2007;579:621–636. doi: 10.1113/jphysiol.2006.125633. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blasi V, Young AC, Tansy AP, Petersen SE, Snyder AZ, Corbetta M. Word retrieval learning modulates right frontal cortex in patients with left frontal damage. Neuron. 2002;36:159–170. doi: 10.1016/s0896-6273(02)00936-4. [DOI] [PubMed] [Google Scholar]
- Brinkman J, Kuypers HGJM. Cerebral control of contralateral and ipsilateral arm, hand and finger movements in the split brain rhesus monkey. Brain. 1973;96:653–774. doi: 10.1093/brain/96.4.653. [DOI] [PubMed] [Google Scholar]
- Brus-Ramer M, Carmel JB, Martin JH. Mortex cortex bilateral motor representation depends on subcortical and interhemipheric interactions. J Neurosci. 2009;29:6196–6206. doi: 10.1523/JNEUROSCI.5852-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Buch E, Weber C, Cohen LG, Braun C, Dimyan MA, Ard T, Mellinger J, Caria A, Soekadar S, Fourkas A, Birbaumer N. Think to move: a neuromagnetic brain–computer interface (BCI) system for chronic stroke. Stroke. 2008;39:910–917. doi: 10.1161/STROKEAHA.107.505313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cardoso de Oliveira S, Gribova A, Donchin O, Bergman H, Vaadia E. Neural interactions between motor cortical hemispheres during bimanual and unimanual arm movements. Eur J Neurosci. 2001;14:1881–1896. doi: 10.1046/j.0953-816x.2001.01801.x. [DOI] [PubMed] [Google Scholar]
- Carmena JM, Lebedev MA, Crist RE, O'Doherty JE, Santucci DM, Dimitrov D, Patil PG, Henriquez CS, Nicolelis MAL. Learning to control brain–machine interface for reaching and grasping by primates. PloS Biol. 2003;1:192–208. doi: 10.1371/journal.pbio.0000042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chen R, Gerloff C, Hallett M, Cohen LG. Involvement of the ipsilateral motor cortex in fine finger movements of different complexities. Ann Neurol. 1997;41:247–254. doi: 10.1002/ana.410410216. [DOI] [PubMed] [Google Scholar]
- Cisek P, Crammond DJ, Kalaska JF. Neural activity in primary motor and dorsal premotor cortex in reaching tasks with the contralateral versus ipsilateral arm. J Neurophys. 2003;89:922–942. doi: 10.1152/jn.00607.2002. [DOI] [PubMed] [Google Scholar]
- Dancause N. Vicarious function of remote cortex following stroke: recent evidence from human and animal studies. Neuroscientist. 2006;12:489–499. doi: 10.1177/1073858406292782. [DOI] [PubMed] [Google Scholar]
- Donchin O, Gribova A, Steinberg O, Bergman H, Vaadia E. Primary motor cortex is involved in bimanual coordination. Nature. 1998;395:274–278. doi: 10.1038/26220. [DOI] [PubMed] [Google Scholar]
- Donchin O, de Oliveira SC, Vaadia E. Who tells one hand what the other is doing: the neurophysiology of bimanual movements. Neuron. 1999;23:15–18. doi: 10.1016/s0896-6273(00)80748-5. [DOI] [PubMed] [Google Scholar]
- Fetz EE. Volitional control of neural activity: implications for brain–computer interfaces. J Physiol. 2007;579:571–579. doi: 10.1113/jphysiol.2006.127142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ganguly K, Carmena JM. Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol. 2009;7:e1000153. doi: 10.1371/journal.pbio.1000153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Georgopoulos AP. Higher order motor control. Annu Rev Neurosci. 1991;14:361–377. doi: 10.1146/annurev.ne.14.030191.002045. [DOI] [PubMed] [Google Scholar]
- Georgopoulos AP, Schwartz AB, Kettner RE. Neuronal population coding of movement direction. Science. 1986;233:1416–1419. doi: 10.1126/science.3749885. [DOI] [PubMed] [Google Scholar]
- Gerloff C, Bushara K, Sailer A, Wassermann EM, Chen R, Matsuoka T, Waldvogel D, Wittenberg GF, Ishii K, Cohen LG, Hallett M. Multimodal imaging of brain reorganization in motor areas of the contralesional hemisphere of well recovered patients after capsular stroke. Brain. 2006;129:791–808. doi: 10.1093/brain/awh713. [DOI] [PubMed] [Google Scholar]
- Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner A, Chen D, Penn RD, Donoghue JP. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature. 2006;442:164–171. doi: 10.1038/nature04970. [DOI] [PubMed] [Google Scholar]
- Hummel FC, Cohen LG. Non-invasive brain stimulation: a new strategy to improve neurorehabilitation after stroke? Lancet Neurol. 2006;5:708–712. doi: 10.1016/S1474-4422(06)70525-7. [DOI] [PubMed] [Google Scholar]
- Humphrey DR, Schmidt EM, Thompson WD. Predicting measures of motor performance from multiple cortical spike trains. Science. 1970;170:758–762. doi: 10.1126/science.170.3959.758. [DOI] [PubMed] [Google Scholar]
- Kim SP, Sanchez JC, Rao YN, Erdogmus D, Carmena JM, Lebedev MA, Nicolelis MA, Principe JC. A comparison of optimal MIMO linear and nonlinear models for brain–machine interfaces. J Neural Eng. 2006;3:145–161. doi: 10.1088/1741-2560/3/2/009. [DOI] [PubMed] [Google Scholar]
- Lotze M, Markert J, Sauseng P, Hoppe J, Plewnia C, Gerloff C. The role of multiple contralesional motor areas for complex hand movements after internal capsular lesion. J Neurosci. 2006;26:6096–6102. doi: 10.1523/JNEUROSCI.4564-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Matsunami K, Hamada I. Characteristics of the ipsilateral movement-related neuron in the motor cortex of monkey. Brain Res. 1981;204:29–42. doi: 10.1016/0006-8993(81)90649-1. [DOI] [PubMed] [Google Scholar]
- Mehring C, Rickert J, Vaadia E, Cardosa de Oliveira S, Aertsen A, Rotter S. Inference of hand movements from local field potentials in monkey motor cortex. Nat Neurosci. 2003;6:125:1253–1254. doi: 10.1038/nn1158. [DOI] [PubMed] [Google Scholar]
- Moritz CT, Perlmutter SI, Fetz EE. Direct control of paralysed muscles by cortical neurons. Nature. 2008;456:639–642. doi: 10.1038/nature07418. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mulliken GH, Musallam S, Andersen RA. Decoding trajectories from posterior parietal cortex ensembles. J Neurosci. 2008;28:12913–12926. doi: 10.1523/JNEUROSCI.1463-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Paninski L, Fellows MR, Hatsopoulos NG, Donoghue JP. Spatiotemporal tuning of motor cortical neurons for hand position and velocity. J Neurophysiol. 2004;91:515–532. doi: 10.1152/jn.00587.2002. [DOI] [PubMed] [Google Scholar]
- Paxinos G, Huang X, Toga AW. San Diego: Academic; 2000. The rhesus monkey brain in stereotaxic coordinates. [Google Scholar]
- Radhakrishnan SM, Baker SN, Jackson A. Learning a novel myoelectric-controlled interface task. J Neurophys. 2008;100:2397–2408. doi: 10.1152/jn.90614.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rao SM, Binder JR, Bandettini PA, Hammeke TA, Yetkin FZ, Jesmanowicz A, Lisk LM, Morris GL, Mueller WM, Estkowski LD, Wong EC, Haughton VM, Hyde JS. Functional magnetic resonance imaging of complex human movements. Neurology. 1993;43:2311–2318. doi: 10.1212/wnl.43.11.2311. [DOI] [PubMed] [Google Scholar]
- Reina GA, Moran DW, Schwartz AB. On the relationship between joint angular velocity and motor cortical discharge during reaching. J Neurophys. 2001;85:2576–2589. doi: 10.1152/jn.2001.85.6.2576. [DOI] [PubMed] [Google Scholar]
- Rickert J, Oliveira SC, Vaadia E, Aertsen A, Rotter S, Mehring C. Encoding of movement direction in different frequency ranges of motor cortical local field potentials. J Neurosci. 2005;25:8815–8824. doi: 10.1523/JNEUROSCI.0816-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schalk G, Kubánek J, Miller KJ, Anderson NR, Leuthardt EC, Ojemann JG, Limbrick D, Moran D, Gerhardt LA, Wolpaw JR. Decoding two-dimensional movement trajectories using electrocorticographic signals in humans. J Neural Eng. 2007;4:264–275. doi: 10.1088/1741-2560/4/3/012. [DOI] [PubMed] [Google Scholar]
- Schalk G, Miller KJ, Anderson NR, Wilson JA, Smyth MD, Ojemann JG, Moran DW, Wolpaw JR, Leuthardt EC. Two-dimensional movement control using electrocorticographic signals in humans. J Neural Eng. 2008;5:75–84. doi: 10.1088/1741-2560/5/1/008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Serruya MD, Hatsopoulos NG, Paninski L, Fellows MR, Donoghue JP. Instant neural control of a movement signal. Nature. 2002;416:141–142. doi: 10.1038/416141a. [DOI] [PubMed] [Google Scholar]
- Shelton FN, Reding MJ. Effect of lesion location on upper limb motor recovery after stroke. Stroke. 2001;32:107–112. doi: 10.1161/01.str.32.1.107. [DOI] [PubMed] [Google Scholar]
- Tanji J, Okano K, Sato KC. Neuronal activity in cortical motor areas related to ipsilateral, contralateral, and bilateral digit movements of the monkey. J Neurophysiol. 1988;60:325–343. doi: 10.1152/jn.1988.60.1.325. [DOI] [PubMed] [Google Scholar]
- Taylor DM, Tillery SI, Schwartz AB. Direct cortical control of 3D neuroprosthetic devices. Science. 2002;296:1829–1832. doi: 10.1126/science.1070291. [DOI] [PubMed] [Google Scholar]
- Verstynen T, Diedrichsen J, Albert N, Aparicio P, Ivry RB. Ipsilateral motor cortex activity during unimanual hand movements relates to task complexity. J Neurophysiol. 2005;93:1209–1222. doi: 10.1152/jn.00720.2004. [DOI] [PubMed] [Google Scholar]
- Wessberg J, Nicolelis MAL. Optimizing a linear algorithm for real-Time robotic control using chronic cortical ensemble recordings in monkeys. J Cogn Neurosci. 2004;16:1022–1035. doi: 10.1162/0898929041502652. [DOI] [PubMed] [Google Scholar]
- Wessberg J, Stambaugh CR, Kralik JD, Beck PD, Laubach M, Chapin JK, Kim J, Biggs SJ, Srinivasan MA, Nicolelis MAL. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature. 2000;408:361–365. doi: 10.1038/35042582. [DOI] [PubMed] [Google Scholar]
- Wisneski KJ, Anderson N, Schalk G, Smyth M, Moran D, Leuthardt EC. Unique cortical physiology associated with ipsilateral hand movements and neuroprosthetic implications. Stroke. 2008;39:3351–3359. doi: 10.1161/STROKEAHA.108.518175. [DOI] [PubMed] [Google Scholar]
- Wu W, Gao Y, Bienenstock E, Donoghue JP, Black MJ. Bayesian population decoding of motor cortical activity using a kalman filter. Neural Computation. 2006;18:80–118. doi: 10.1162/089976606774841585. [DOI] [PubMed] [Google Scholar]