Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2009 Nov 11;29(45):14363–14370. doi: 10.1523/JNEUROSCI.2081-09.2009

Brain Control of Movement Execution Onset Using Local Field Potentials in Posterior Parietal Cortex

Eun Jung Hwang 1,✉, Richard A Andersen 1
PMCID: PMC2805702  NIHMSID: NIHMS163960  PMID: 19906983

Abstract

The precise control of movement execution onset is essential for safe and autonomous cortical motor prosthetics. A recent study from the parietal reach region (PRR) suggested that the local field potentials (LFPs) in this area might be useful for decoding execution time information because of the striking difference in the LFP spectrum between the plan and execution states (Scherberger et al., 2005). More specifically, the LFP power in the 0–10 Hz band sharply rises while the power in the 20–40 Hz band falls as the state transitions from plan to execution. However, a change of visual stimulus immediately preceded reach onset, raising the possibility that the observed spectral change reflected the visual event instead of the reach onset. Here, we tested this possibility and found that the LFP spectrum change was still time locked to the movement onset in the absence of a visual event in self-paced reaches. Furthermore, we successfully trained the macaque subjects to use the LFP spectrum change as a “go” signal in a closed-loop brain-control task in which the animals only modulated the LFP and did not execute a reach. The execution onset was signaled by the change in the LFP spectrum while the target position of the cursor was controlled by the spike firing rates recorded from the same site. The results corroborate that the LFP spectrum change in PRR is a robust indicator for the movement onset and can be used for control of execution onset in a cortical prosthesis.

Introduction

A series of recent studies show that local field potentials (LFPs) encode movement-related variables, such as reach target location, grasp type, and hand velocity, in the brain areas in which the spiking activity of single units was previously known to encode those variables (Rickert et al., 2005; Heldman et al., 2006; O'Leary and Hatsopoulos, 2006; Asher et al., 2007; Spinks et al., 2008). A unique variable that LFPs encode more efficiently than single units is the movement-related behavioral state which may be classified as baseline, movement planning, and movement execution states (Pesaran et al., 2002; Scherberger et al., 2005).

Information on the behavioral state is essential for a cortical motor prosthesis (Pesaran et al., 2006). The selectivity of neural activity to movement-related variables, such as reach target and speed, varies with the behavioral state (Crammond and Kalaska, 2000; Churchland et al., 2006). It is even sensitive to the time elapsed within the same state (Musallam et al., 2004). Consequently the decoding accuracy suffers if the decoder is applied at a different time or in a different behavioral state (Bokil et al., 2006). A traditional approach to avoiding this issue is to limit the application to a specific task and assume that the behavioral state changes in a fixed temporal pattern as the task sequence advances (Hochberg et al., 2006). Thus, the decoder would turn on at a specific task epoch only. This approach works only for an artificial test environment. A more universal approach would be to decode the subject's desired behavioral state and use this information to autonomously gate prosthetic operation.

For such autonomous operation, a few studies attempted to find and decode the neural signal that reflects the subject's behavioral state. Mason and Birch proposed to use the low-frequency component (1–4 Hz) of electroencephalogram (EEG) to distinguish the active control state from the attentive idle state (Mason and Birch, 2000; Borisoff et al., 2006). Bokil et al. (2006) proposed the cepstral analysis on LFPs or spike trains to detect and identify a specific behavioral event. More recently, two different approaches using the large ensemble activity of single or multiple units were proposed: a state-machine-based state estimator (Achtman et al., 2007; Kemere et al., 2008) and a linear-filter-based temporal duration estimator (Lebedev et al., 2008). These studies reported high performance accuracy if a large number of neurons are used.

Here, we sought an alternative way to achieve autonomous operation using LFPs in posterior parietal cortex (PPC), since earlier studies found LFPs to be more efficient to discriminate behavioral states than spikes in this region (Pesaran et al., 2002; Scherberger et al., 2005). First, we verified that the LFP spectrum change indeed reflects transitions of behavioral state by using a self-paced reach task in which the movement onset, and thus the state transition, was self-determined in the absence of any visual stimulus change. Second, we conducted a brain-control experiment in which the change of LFP spectrum successfully controlled cursor movement onset time to demonstrate the feasibility of an autonomous cortical prosthesis using LFPs in PPC.

Materials and Methods

Animal preparation and neural recording

Two male rhesus monkeys (Macaca mulatta; subject E: 7.5 kg; subject Y: 8.3 kg) participated in this study. All procedures were in accordance with National Institutes of Health guidelines and were approved by the California Institute of Technology Animal Care and Use Committee. Both subjects were implanted with a head holder and recording chamber over the intraparietal sulcus following the procedures described by Scherberger et al. (2003).

Extracellular recordings were made using a single-channel microdrive (FHC) in subject E and a 16-channel chronic microdrive (Neuralynx) in subject Y. Although a 16-channel drive was used for subject Y, we used neural activity from a single channel (the one with strongest tuning to the target location) in each brain-control session. The LFP and spike signals were separated by hardware bandpass filters in a preamplifier with the frequency bands for each signal preset by the manufacturer (LFP: 3.3–88 Hz and spike: 154 Hz to 8.8 kHz; Plexon). We recorded neural signals from PRR in which Scherberger et al. (2005) found a behavioral state-dependent LFP spectrum (7–12 mm posterior and 3.5–7.5 mm lateral in Horsley-Clarke coordinates, at 1–7 mm subdural surface).

Behavioral experiment setup

The subject sat in a primate chair and viewed visual stimuli presented on the vertical LCD monitor placed in the frontoparallel plane, ∼40 cm away from the eyes. The subject's eye position was recorded with an infrared CCD camera (240 Hz; ISCAN) and hand position was recorded with a 19 inch translucent touch-sensitive screen (IntelliTouch; ELO Systems) placed against the LCD monitor. The visual stimulus presentation, online monitoring of eye and hand positions, and reward control were handled by a real-time LabView program (National Instruments, LabView7.1) running on a real-time operating system (National Instruments, LabView Real-Time).

Experiment 1: instructed versus self-paced delayed reach tasks

The purpose of experiment 1 was to rule out the possibility that the previously observed LFP spectrum change reflected the change of visual stimuli rather than the behavioral state. To do so, we examined LFPs in the self-paced reach task in which reaching initiation was self-determined in the absence of a change of visual stimuli and compared them with LFPs in the instructed reach task.

Instructed delayed reach task.

A trial began as the subject acquired the ocular and manual fixation targets in the center of the screen (Fig. 1A). After a 0.5 s fixation period, a single reach target was flashed at a peripheral location (∼10.3 ° eccentricity) for 0.3 s. The subject was required to maintain his central fixation for a variable delay (randomly sampled between 1.2 s and 1.45 s) until the manual fixation target extinguished. Once the manual fixation target disappeared, the subject was allowed to reach to the remembered target without moving his gaze from the fixation point. After the subject held his hand on the reach target and eyes on the fixation target for 0.3 s, he received a juice reward.

Figure 1.

Figure 1.

Open in a new tab

A, Instructed delayed reach. The subject reaches to the remembered target location once the central manual fixation target disappears. B, Self-paced delayed reach I. The subject reaches to the remembered target location at his own pace. C, Variable reward. In the self-paced tasks, the reward size was increased with the delay length. The rewarded delay range (t1 < t < t3) was fixed while the minimum reward (r_min), the maximum reward (r_max), and the slope (r_max) − (r_min)/(t2 − t1) were adjusted daily. D, Self-paced delayed reach II (parameter calibration set). The subject reaches to the green target at his own pace. E, Brain-control task (online decoding set). The execution time was signaled by the change in the LFP spectrum while the target position was determined by the spike firing rate. In all tasks, the subject fixated the gaze on the central eye fixation target (red square) throughout a trial. The hand icon indicates the subject's hand position on the touch screen during each event and it is absent in the movement period during which the hand is off the screen. The yellow flares in the target acquisition epoch indicate the reappearance or reflashing of the reach target once the target is acquired.

Self-paced delayed reach task.

This task was identical to the instructed delayed task except that the manual fixation target in the center changed its shape from a triangle to a circle concomitantly with the target onset and did not extinguish (Fig. 1B). The circular fixation target indicated that the subject should decide when to initiate a reach without any external cue. While self-paced, the subject was encouraged to delay his reaching initiation for at least 1.2 s after the target offset to match the mean delay length to the instructed delayed reach task. This was achieved by increasing the reward size with the delay length as shown in Figure 1C. The rewarded delay range was fixed, spanning from 1.2 to 3 s in all sessions, while the slope and minimum and maximum reward size were adjusted daily. Both subjects became proficient in this task (performance accuracy, subject E: 94 ± 4.3%; subject Y: 95 ± 3.5%). While the delay lengths were more variable in the self-paced task, the mean values were similar (instructed vs self-paced delay length, subject E: 1.6 ± 0.11 s vs 1.7 ± 0.26 s, Wilcoxon rank sum test p < 1.4 × 10−17; subject Y: 1.5 ± 0.10 s versus 1.5 ± 0.12 s, p < 0.84). One might wonder whether the variable reward size in the self-paced reaches influenced the LFP power because a previous study found that tuning quality for spiking activity changed depending on the expected reward size (Musallam et al., 2004). However, we did not find a systemic influence of the reward size on the LFP power before movement onset.

For each task condition, subject E performed 56 trials (7 trials per each of 8 target locations) and subject Y performed 72 trials (12 trials per each of 6 target locations).

Experiment 2: brain control of execution time and target position

The purpose of experiment 2 was to examine whether the LFP activity could be used to control the execution onset time and spiking activity to control the target position of a cursor in a closed-loop, brain-control task. Note that the LFP and spiking activity used in this experiment were recorded from the same recording site using a single electrode. Hereafter, execution time refers to execution onset time. To decode the target position, we applied a simple linear discriminant method to the spike firing rate. To decode the execution time, we detected the LFP spectrum change which was characterized by the rapid decrease of a single measure, the execution signal. The execution signal was the difference in the changing rate of the average power in the 0–10 Hz and 20–40 Hz bands and computed in the following way. First, the mean LFP power in the 0–10 Hz and 20–40 Hz bands were computed for 500 ms bins, sliding in steps of 50 ms. Second, the first-order time derivatives of these power functions were computed. Last, the time derivative of the 0–10 Hz power was subtracted from that of the 20–40 Hz power. Consequently, the execution signal becomes a large negative value before the movement onset as the 20–40 Hz power decreases while the 0–10 Hz power increases (see Fig. 4D). The execution time for a cursor movement was determined as the moment when the execution signal falls below a threshold.

Figure 4.

Figure 4.

Open in a new tab

The brain-control task. A, The raster plot for a single neuron during the calibration reach and brain-control trials (10 trials are shown per target and per condition). Thick vertical ticks represent the fixation acquisition, cue onset, and movement onset (go detection) from left to right. B, Firing rate over the shaded interval versus the target locations in the calibration reach (solid) and brain-control trials (dotted). The error bars are SD. C, The LFP trace and execution signal for seven consecutive calibration reach and brain-control trials. Gray vertical lines mark the movement onset for the reach trials and the go detection for the brain-control trials. Dashed horizontal lines mark the threshold to detect the “go” signal. D, The mean execution signal in the calibration reach (solid) and brain-control trials (dotted). E, The mean time course of the power change in the 0–10 and 20–40 Hz bands. At the time of go detection, the power in the 0–10 Hz band increases at the maximal rate while the power in the 20–40 Hz band decreases at the maximal rate. The bands are SEM. Note that in this figure, reach trials are from the parameter calibration set, while the brain-control trials are from the online decoding set.

Each brain-control session consisted of a parameter calibration set and an online decoding set. The LFP power spectrum was computed using a conventional fast Fourier transform (Matlab, MathWorks).

Parameter calibration set.

The subject performed ∼20 self-paced delayed reaches to each of the two selected targets based on the spatial tuning of a single neuron (20 trials in 27 sessions, and 10 trials in 3 sessions). The self-paced delayed reach task in the parameter calibration set was slightly modified from that in experiment 1 (Fig. 1D). The difference was that after 0.5 s fixation maintenance, two reach targets were simultaneously presented and stayed on throughout the delay period. The subject was required to reach for the green target to receive a reward. Two targets instead of a single target were presented to ensure that we decoded the location of an intended reach target and not a visual stimulus. In the first trial both targets were green and, in the second trial, the target chosen in the first trial was blue so that the subject had to reach to the different target from the first trial. Therefore, in each pair of trials, the first trial was always free-choice while the second trial was instructed. The reward size varied with the delay length (time between target onset and movement onset) in the same way as experiment 1 (Fig. 1C). The rewarded delay range was fixed, spanning from 1.2 to 3 s.

After the required number of reaches was made, two parameters were computed; the decision boundary for the linear discriminant and the threshold for the execution signal. First, the mean firing rate for each target during 0.5 s before the movement onset was measured and the midpoint between the two firing rates was set as the decision boundary (see Fig. 4B). That is, if the firing rate is below the decision boundary, the cursor position would be decoded to be target 1 and vice versa. Second, the mean execution signal and its maximum negative deflection within 0.1 s before the movement onset were computed (see Fig. 4D). We set 0.3 to 20 times this maximum value as the execution signal threshold. The multiplication gain was adjusted so that the threshold produced a false detection ratio of <3% during the actual reach trials.

Online decoding set.

This task proceeded in a similar way to the reach task in the calibration set (Fig. 1D). From 0.3 s after the reach target onset, the online execution signal was computed using the latest 500 ms LFP segment stepped every 50 ms. The 500-ms-long window was used to reliably estimate the low-frequency components (<10 Hz) of the LFP signal. One drawback of such a large window is that it could lead to a substantial delay between the initiation of the execution signal and its actual detection. Despite this inherent delay, the execution signal computed using this window size began to show a dramatic change ∼100 ms before the actual movement onset (see Fig. 4C,D), which still allowed us to predict the movement onset.

At the first instant when the execution signal reached the threshold (go detection), the delay period ended and the cursor was flashed at the target location decoded using the mean firing rate during the latest 500 ms (target acquisition). If the go detection occurred within the predetermined execution time window (1.3–3 s after the target onset in 28 sessions, 1.3–4 s in 2 sessions) and the decoded location was the green target, the subject received a reward. The reward size increased with the delay length between the target onset and go detection (Fig. 1C). The rewarded delay range was fixed, coinciding with the predetermined execution time window. Subject E was required to maintain the ocular and manual fixations only until the target acquisition while subject Y was required to continuously maintain the fixations for another 0.3 s after the target acquisition. Each brain-control session consists of 152 trials on average (range: 56–281).

Performance evaluation in the online decoding set.

To evaluate learning within a session, we computed the success rate as a function of the trial number. More specifically, the success rate for the latest 40 consecutive trials was computed after each trial once the subject completed the first 40 brain-control trials. Then, a linear regression was performed to this success rate curve. If the slope of the regressed line was significantly positive (p < 0.01), we considered the subject's performance to have improved within a session. Altering the moving average bin size for the success rate computation produced qualitatively similar results except for the slightly lower or higher mean values.

To rule out that the observed performance is a mere reflection of random fluctuations of the execution signal, we first estimated the false-detection probability (p) for a given threshold using reach trials from the calibration set. Then, the chance performance was computed using the following equation: (1 − p)n1 · [1 − (1 − p)n2].

Here, n1 and n2 are the number of discrete time samples before and within the execution time window, respectively. The first and second terms represent the chance probability for the execution signal to stay below the threshold before the execution time window and to surpass the threshold within the execution time window, respectively.

Results

The LFP spectrum change in PRR is a robust indicator for movement onset

Figure 2, A and C, shows typical LFP signals for seven consecutive trials in instructed and self-paced tasks, aligned to the movement onset. In both tasks fast oscillations are prominent during the planning period while slow waves are dominant near movement onset. This temporal structure is clearer in the spectral decomposition of LFPs (Fig. 2B,D). The 0–10 Hz power starts to sharply increase while the 20–40 Hz power starts to decrease as time nears the movement onset. This LFP spectrum change, time locked to the movement onset, was robustly observed in the population of 122 PRR recording sites (subject E: 17, subject Y: 105) (Fig. 2E,F). The consistent LFP change associated with the movement onset can be seen in the temporal domain as well. Figure 2G shows the average LFP trace across all recording sites aligned to the movement onset. This movement-related LFP starts with a negative deflection before the movement onset followed by a positive deflection.

Figure 2.

Figure 2.

Open in a new tab

The LFP spectrum change in PRR is a robust indicator for the movement onset. A, LFP traces aligned at the movement onset from seven consecutive trials during the instructed reach task. The vertical dashed lines indicate the times for fixation acquisition, reach target onset, target offset and movement onset. B, The time-resolved spectral decomposition of the corresponding LFP traces in A. The black line represents the power in the 20–40 Hz band and the gray line represents the power in the 0–10 Hz band. C, D, The same analysis as in A and B, but for the LFPs from the same recording site during the self-paced reach task. E, F, The time course of the LFP power averaged across 122 PRR recording sites for the instructed and self-paced reaches respectively. The vertical dashed lines indicate the mean times for fixation acquisition (instructed versus self-paced, −2.3 ± 0.11 s vs −2.4 ± 0.18 s), reach target onset (−1.8 ± 0.11 s vs −1.8 ± 0.18 s), target offset (−1.5 ± 0.11 s vs −1.5 ± 0.18 s) and movement onset. G, The average movement-related LFP across 122 PRR sites. The LFPs were aligned to the movement onset. The first vertical line indicates the time of movement onset and the second line indicates the mean time when the movement ends (0.2 ± 0.03 s). The bands in E, F, and G are SEM.

In addition, when we aligned the LFP signal to the “go” cue, i.e., extinction of the fixation target instead of the movement onset in instructed reaches, the timing of spectrum change was more widely spread. These results verified that the LFP spectrum change indeed reflected the behavioral state transition from the plan to execution state instead of the visual stimulus change.

So far we limited our analysis to the center-out reach movements since we were unable to trace the hand position if a movement did not involve an interaction with the touch screen. However, we could estimate the occurrence of two additional types of arm movement inherent to performing this task: reach to the central fixation target (rest-to-screen) and release from the touch screen after target acquisition (screen-to-rest). Note that the precise onset time of rest-to-screen reaches was not available as this type of reach did not originate from the screen. Instead, we used the end of the reach, which was precisely measured as the time when the hand touched the screen, to detect the occurrence of the rest-to-screen reach. To further test the robustness of the LFP spectrum change, we examined these other types of movement as well. Figure 3 shows the LFP trace and execution signal over a 30 s period containing a mixture of center-out reaches and these two other movement types. For all three types of reach, the movement onset was associated with a movement-related potential similar to that depicted in Figure 2G. Recall that the onset of rest-to-screen reaches must have occurred a few hundred milliseconds earlier than the marked reach end (for more details, see supplemental Fig. S3, available at www.jneurosci.org as supplemental material).

Figure 3.

Figure 3.

Open in a new tab

The execution signal. Rows show an LFP trace recorded for 30 s, its power in the 0–10 Hz band, its power in the 20–40 Hz band, and the corresponding execution signal. Squares represent the movement onset for center-out reaches, pentagons for the movement onset for screen-to-rest reaches, and circles for the movement end for rest-to-screen reaches. The shaded zones are intertrial intervals (rest periods). The open icons represent a specific movement sequence which is referred to in the manuscript. Notice that the execution signal reaches the threshold (dashed horizontal line) much earlier than the event markers for the rest-to-screen reaches as the markers indicated the time when the movement ended instead of started. Meanwhile, the execution signal reaches the threshold closer to the event markers for the other two types of events.

In the frequency domain, the movement onset was associated with an increase of the 0–10 Hz power and a decrease of the 20–40 Hz power, although the decrease was less consistent than the increase. This spectrum change is well reflected in the execution signal as a sharp negative deflection time locked to each movement onset.

Figure 3 shows an example where the three movements were initiated within short intervals (open icons in Fig. 3, rest-to-screen, screen-to-rest, rest-to-screen) as the monkey adjusted the touch to the screen. The execution signal discerned these three movement onsets that were separated by only ∼200 ms. These results suggest that our behavioral state decoder might be practical even in more natural situations in which the subject continuously initiates new movements.

The LFP spectrum change can be used to control the movement execution time

We implemented a brain-control task in which the cursor target position was decoded from the spiking activity and the cursor movement onset time was decoded from the LFP activity recorded from the same electrode. This implementation is based on the previous finding that the target location is better represented by the spiking activity while the behavioral state is better represented by the LFP activity in PRR (Scherberger et al., 2005). Reach trials in Figure 4, A and B, show an example neuron whose delay period activity clearly discriminated the two reach targets during the calibration set. The midpoint between the two mean firing rates was measured from the tuning curve to use as the decision boundary for target position decoding. Displayed on the left panel of Figure 4C are LFPs and their corresponding execution signals for seven consecutive trials during the calibration set. Note that these LFPs were recorded from the same site as the neuron shown in Figure 4A, as in all our brain-control sessions. The rapid spectrum change is well reflected in the execution signals as the sharp negative deflection time locked to the movement onset. To determine the threshold for the execution time detection, we computed the mean execution signal across all calibration trials and the maximum negative deflection within 0.1 s before the movement onset was measured (Fig. 4D, solid line). With this threshold, the rate of false detection (e.g., the third reach trial in Fig. 4C) is <3% and the estimated chance performance is 19%.

The online decoding set proceeded in a similar way to the self-paced delayed reach task in the calibration set except the delay period ended at the first instant when the online execution signal reached the threshold (go detection). Upon go detection, the target position was decoded using the spike firing rate during the latest 0.5 s and the decoded location was flashed on the screen whether or not the decoded target location was correct. However, the subject was rewarded only if the decoded location was correct and the go signal was detected during the predetermined execution time window (e.g., 1.3–3 s after the target onset).

The spatial tuning of the firing rate for the successful brain-control trials are shown in Figure 4, A and B. The firing rates for the two targets were as well separated as for actual reaches. Note that the spike rasters for the brain-control trials were aligned to go detection. Figure 4C displays execution signals that were computed offline and the time when our online algorithm detected go signals. This shows that the online algorithm detected go signals with a short lag (mean = 28 ms), caused by the online computation of the execution signal. Due to the lag, the execution signal crossed the threshold in brain-control trials before go detection as in reach trials (Fig. 4D). Unlike execution signals, however, LFP traces in the time domain looked different between reach and brain-control trials (Fig. 4C). The distinct movement-related potential is not present in the brain-control trials near go detection, most likely due to the absence of an actual movement. This implies that some spectral features of LFP used in this study consistently change to indicate when to initiate a movement, even in the absence of the actual movement and corresponding movement-related potential.

Because the execution signal was a simple difference in the changing rates of the power in the 0–10 Hz and 20–40 Hz bands, the lower-frequency band contribution to the execution signal was ∼10 times larger than the higher-frequency band by the virtue of the 1/f2 roll-off. It is possible that the execution signal could have been controlled by the 0–10 Hz band signal alone without the opposing change in the 20–40 Hz band. However, our data showed the maximal changing rates were in the opposite directions for the two frequency bands at the time of go detection, indicating that power from both frequency bands contributed to driving the execution signal across the threshold (Fig. 4E).

Next, we examined the subjects' performance. In the brain-control session shown in Figure 4, the success rate was higher than chance for both target and execution time controls and the subjects' success rate in the execution time control gradually increased as the training progressed (Fig. 5A). The improvement was accompanied by the tighter distribution of execution times (Fig. 5B). The variance during the last 30 trials was smaller than the variance during the first 30 trials (F(29,29) = 2.6, p < 0.05).

Figure 5.

Figure 5.

Open in a new tab

Performance improvement. A, The behavioral performance over the latest 40 trials during the brain-control session shown in Figure 4. The gray dotted line is the overall performance, black for go detection, and the light gray for target control. The horizontal dashed line indicates the chance level for go detection. B, The distribution of the delay lengths in the brain-control task. The gray and white boxes correspond to the first and last 30 trials in the session. Variance of the two distributions are significantly different (F(29,29) = 2.6, p < 0.05). C, The success rate for the execution state control as a function of the experimental session number. The open circle represents the performance in the session shown in Figure 4. The linear regression to the distribution (gray line) has the significantly positive slope (slope = 0.005, p < 0.05).

We conducted 30 brain-control sessions in total (subject Y: 22, E: 8) and the subjects' performance was higher than chance in all sessions. The peak success rate over 40 consecutive trials within each session were 79 ± 7.1, 85 ± 6.4, and 94 ± 4.2% for the combined performance, execution time and target position control for subject Y and 81 ± 7.2, 94 ± 7.5, and 91 ± 8.1% for subject E. Among 30 brain-control sessions, we found significant within-session improvement in 15 sessions (12 in subject Y and 3 in subject E). Performance improvement was also manifested in the long-term as shown in Figure 5C. The slope of the linear regression line between peak performance and session number was significantly positive for subject Y (Fig. 5C) (slope = 0.5%/session, r2 = 0.28, p < 0.05), indicating a gradual increase of performance. Long-term performance change was not assessed for subject E because subject E had already perfected this task from the previous training in a similar brain-control task. Throughout the sessions the predetermined execution time window was fixed while the recording site, and thus threshold to detect movement onset, changed from session to session. Therefore, it appears that the short-term learning reflects the adaptation to a new threshold within each session while the long-term learning reflects the generalization of acquired skills or strategies from previous sessions. Despite the observed learning in terms of success rate in the task, this study does not suggest that the monkeys were actually aware of the causality between the LFPs and a cursor movement or other reward contingencies in the task.

Discussion

Relation to human EEG studies

Similar to our intracortical LFP of the nonhuman primates, the scalp EEG in humans show a behavioral state-dependent spectrum change in the sensorimotor cortex (Pfurtscheller, 1981; Stancák et al., 2000). That is, the EEG also shows the disruption of beta-band oscillations and the emergence of slow movement-related potentials as the state transits from plan to execution. A series of studies found that the slow movement-related potentials consist of at least three phases in self-paced voluntary movements (Barrett et al., 1986; Deecke, 1987; Hallett, 1994). The earliest phase is termed the “readiness signal” or “bereitschafts potential,” while the last phase is termed the “motor potential.” Considering its temporal features, the sharp increase of the low-frequency power in LFPs seems mainly related to the motor potential that begins just before movement onset.

These EEG spectral features have been used to decode behavioral states. Mason and Birch (2000) showed that the low-frequency component of the EEG could distinguish the attentive idle state from the active control state. Bai et al. (2008) reported that human subjects learned to control the binary state of a cursor using the disruption or continuation of beta-band oscillation. While the decoding method in the current study apparently relies on the same phenomenon as those EEG studies, it is different in two ways. First, the execution signal in the current study combines the features from both the low-frequency and beta-band powers, albeit dominated by the low-frequency component. Second, it uses the intracortical LFPs which have better spatial resolution than EEG (Schwartz et al., 2006). These differences are expected to provide higher accuracy and selectivity than the previous approaches using EEG.

Applicability to natural conditions

A practical issue of concern is the feasibility of our decoding method in natural conditions in which the subject continuously initiates new movements involving multiple effectors. This issue can be divided into two parts: decoding sequential movements and effector selectivity. Although it was not a primary part of our experiment, we examined the LFP execution signals over a longer period of time during which multiple types of reaches were sequentially initiated (Fig. 3). The execution signal clearly indicated the onset of each reach. It could even parse out two movements that occurred with temporal separation as short as 200 ms. Furthermore, the execution signal was selective to arm movements. During rest periods, the subject made multiple saccades and the execution signal was insensitive to these saccade initiations. This agrees with the previous finding that the LFP spectrum in PRR changes selectively to reaches (Scherberger et al., 2005). These results imply that the LFP execution signal in PRR can be used to control more complex types of movement in an effector-specific manner.

Concerns over potential mechanical or electrical artifacts in LFP signals

It is important to ensure that the origin of spectrum change in LFPs is biological rather than electrical or mechanical. For example, could the mechanical contact to the touch screen during the planning period have led to an electrical noise oscillating at 20–40 Hz? This is unlikely for multiple reasons. First, the LFP power at 20–40 Hz during the screen press is modulated by the upcoming target location. Typically, the power is higher for the contra-lateral side target. If the source of LFP power at 20–40 Hz is electrical noise induced by the mechanical action of maintaining a manual contract to the screen, it is hard to explain why the same mechanical action induces different power modulation depending on the future movement target location. Second, LFP power increase in this frequency band during the movement planning period has been reported for several brain areas (Murthy and Fetz, 1992; Sanes and Donoghue, 1993; MacKay and Mendoncca, 1995; O'Leary and Hatsopoulos, 2006). In some of those studies, a robotic manipulandum or push buttons instead of a touch screen was used, yet oscillations in this frequency band still appeared during planning period. Last, the oscillations above 10 Hz occurred only when the electrodes were placed inside the brain and the 20–40 Hz oscillations occurred selectively in PRR sites among the multiple sites simultaneously recorded (for more details, see supplemental Fig. S1, available at www.jneurosci.org as supplemental material).

It is also unlikely that increase of LFP power in the low-frequency band originates from a movement artifact. First of all, the emergence of a slow movement-related potential (<10 Hz) is found in various other studies including the aforementioned studies of EEG. Second, the increase of the low-frequency component starts before movement onset as shown in the individual trial traces (Figs. 2–4). If the mechanical artifact is the source, we expect the increase would lag instead of lead the movement onset. Third, the slow movement-related potential was not seen in the electrode located outside the brain. Fourth, the slow movement-related potential showed spatial selectivity, i.e., it varied with the direction of reach (for more details, see supplemental Figs. S2, S3, available at www.jneurosci.org as supplemental material). Last, the increase of the low-frequency component was present in the brain-control trials in the absence of movements.

Conclusions

We report that the LFP signal in PRR changes the spectrum depending on the behavioral state and can provide the information on the behavioral state for an autonomous motor prosthesis. In our study, the LFP activity from a single electrode was sufficient to achieve >90% accuracy in execution time control. In addition to the ease of recording and long-term stability, the efficiency and the simplicity of our detection algorithm promote the LFP as a compelling control signal for a motor prosthesis (Andersen et al., 2004). However, we do not claim that the proposed decoding method is the ideal solution to achieve the autonomous prosthesis or that its parameters, such as the threshold for the execution signal and the temporal window length, are optimal. For example, it may be possible to implement similar or more sophisticated algorithms using spikes instead of LFPs, given that spike trains can also show state-dependent changes, albeit less consistently across movement directions and neurons. A comparison with other available methods or the optimization of parameters may be further investigated in future studies.

Footnotes

This work was supported by the National Institute of Health, the James G. Boswell Foundation, and the Defense Advanced Research Projects Agency. We thank H. Cui, A. Graf, and M. Hauschild for comments on this manuscript, K. Pejsa and N. Sammons for animal care, and G. Mulliken, V. Shcherbatyuk, and T. Yao for technical and administrative assistance.

References

  1. Achtman N, Afshar A, Santhanam G, Yu BM, Ryu SI, Shenoy KV. Free-paced high-performance brain-computer interfaces. J Neural Eng. 2007;4:336–347. doi: 10.1088/1741-2560/4/3/018. [DOI] [PubMed] [Google Scholar]
  2. Andersen RA, Musallam S, Pesaran B. Selecting the signals for a brain-machine interface. Curr Opin Neurobiol. 2004;14:720–726. doi: 10.1016/j.conb.2004.10.005. [DOI] [PubMed] [Google Scholar]
  3. Asher I, Stark E, Abeles M, Prut Y. Comparison of direction and object selectivity of local field potentials and single units in macaque posterior parietal cortex during prehension. J Neurophysiol. 2007;97:3684–3695. doi: 10.1152/jn.00886.2006. [DOI] [PubMed] [Google Scholar]
  4. Bai O, Lin P, Vorbach S, Floeter MK, Hattori N, Hallett M. A high performance sensorimotor beta rhythm-based brain-computer interface associated with human natural motor behavior. J Neural Eng. 2008;5:24–35. doi: 10.1088/1741-2560/5/1/003. [DOI] [PubMed] [Google Scholar]
  5. Barrett G, Shibasaki H, Neshige R. Cortical potentials preceding voluntary movement: evidence for three periods of preparation in man. Electroencephalogr Clin Neurophysiol. 1986;63:327–339. doi: 10.1016/0013-4694(86)90017-9. [DOI] [PubMed] [Google Scholar]
  6. Bokil HS, Pesaran B, Andersen RA, Mitra PP. A method for detection and classification of events in neural activity. IEEE Trans Biomed Eng. 2006;53:1678–1687. doi: 10.1109/TBME.2006.877802. [DOI] [PubMed] [Google Scholar]
  7. Borisoff JF, Mason SG, Birch GE. Brain interface research for asynchronous control applications. IEEE Trans Neural Syst Rehabil Eng. 2006;14:160–164. doi: 10.1109/TNSRE.2006.875526. [DOI] [PubMed] [Google Scholar]
  8. Churchland MM, Santhanam G, Shenoy KV. Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach. J Neurophysiol. 2006;96:3130–3146. doi: 10.1152/jn.00307.2006. [DOI] [PubMed] [Google Scholar]
  9. Crammond DJ, Kalaska JF. Prior information in motor and premotor cortex: Activity during the delay period and effect on pre-movement activity. J Neurophysiol. 2000;84:986–1005. doi: 10.1152/jn.2000.84.2.986. [DOI] [PubMed] [Google Scholar]
  10. Deecke L. Bereitschaftspotential as an indicator of movement preparation in supplementary motor area and motor cortex. Ciba Found Symp. 1987;132:231–250. doi: 10.1002/9780470513545.ch14. [DOI] [PubMed] [Google Scholar]
  11. Hallett M. Movement-related cortical potentials. Electromyogr Clin Neurophysiol. 1994;34:5–13. [PubMed] [Google Scholar]
  12. Heldman DA, Wang W, Chan SS, Moran DW. Local field potential spectral tuning in motor cortex during reaching. IEEE Trans Neural Syst Rehabil Eng. 2006;14:180–183. doi: 10.1109/TNSRE.2006.875549. [DOI] [PubMed] [Google Scholar]
  13. Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner A, Chen D, Penn RD, Donoghue JP. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature. 2006;442:164–171. doi: 10.1038/nature04970. [DOI] [PubMed] [Google Scholar]
  14. Kemere C, Santhanam G, Yu BM, Afshar A, Ryu SI, Meng TH, Shenoy KV. Detecting neural-state transitions using hidden Markov models for motor cortical prostheses. J Neurophysiol. 2008;100:2441–2452. doi: 10.1152/jn.00924.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Lebedev MA, O'Doherty JE, Nicolelis MA. Decoding of temporal intervals from cortical ensemble activity. J Neurophysiol. 2008;99:166–186. doi: 10.1152/jn.00734.2007. [DOI] [PubMed] [Google Scholar]
  16. MacKay WA, Mendonça AJ. Field potential oscillatory bursts in parietal cortex before and during reach. Brain Res. 1995;704:167–174. doi: 10.1016/0006-8993(95)01109-9. [DOI] [PubMed] [Google Scholar]
  17. Mason SG, Birch GE. A brain-controlled switch for asynchronous control applications. IEEE Trans Biomed Eng. 2000;47:1297–1307. doi: 10.1109/10.871402. [DOI] [PubMed] [Google Scholar]
  18. Murthy VN, Fetz EE. Coherent 25- to 35-Hz oscillations in the sensorimotor cortex of awake behaving monkeys. Proc Natl Acad Sci U S A. 1992;89:5670–5674. doi: 10.1073/pnas.89.12.5670. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Musallam S, Corneil BD, Greger B, Scherberger H, Andersen RA. Cognitive control signals for neural prosthetics. Science. 2004;305:258–262. doi: 10.1126/science.1097938. [DOI] [PubMed] [Google Scholar]
  20. O'Leary JG, Hatsopoulos NG. Early visuomotor representations revealed from evoked local field potentials in motor and premotor cortical areas. J Neurophysiol. 2006;96:1492–1506. doi: 10.1152/jn.00106.2006. [DOI] [PubMed] [Google Scholar]
  21. Pesaran B, Pezaris JS, Sahani M, Mitra PP, Andersen RA. Temporal structure in neuronal activity during working memory in macaque parietal cortex. Nat Neurosci. 2002;5:805–811. doi: 10.1038/nn890. [DOI] [PubMed] [Google Scholar]
  22. Pesaran B, Musallam S, Andersen RA. Cognitive neural prosthetics. Curr Biol. 2006;16:R77–R80. doi: 10.1016/j.cub.2006.01.043. [DOI] [PubMed] [Google Scholar]
  23. Pfurtscheller G. Central beta rhythm during sensorimotor activities in man. Electroencephalogr Clin Neurophysiol. 1981;51:253–264. doi: 10.1016/0013-4694(81)90139-5. [DOI] [PubMed] [Google Scholar]
  24. Rickert J, Oliveira SC, Vaadia E, Aertsen A, Rotter S, Mehring C. Encoding of movement direction in different frequency ranges of motor cortical local field potentials. J Neurosci. 2005;25:8815–8824. doi: 10.1523/JNEUROSCI.0816-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Sanes JN, Donoghue JP. Oscillations in local field potentials of the primate motor cortex during voluntary movement. Proc Natl Acad Sci U S A. 1993;90:4470–4474. doi: 10.1073/pnas.90.10.4470. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Scherberger H, Goodale MA, Andersen RA. Target selection for reaching and saccades share a similar behavioral reference frame in the macaque. J Neurophysiol. 2003;89:1456–1466. doi: 10.1152/jn.00883.2002. [DOI] [PubMed] [Google Scholar]
  27. Scherberger H, Jarvis MR, Andersen RA. Cortical local field potential encodes movement intentions in the posterior parietal cortex. Neuron. 2005;46:347–354. doi: 10.1016/j.neuron.2005.03.004. [DOI] [PubMed] [Google Scholar]
  28. Schwartz AB, Cui XT, Weber DJ, Moran DW. Brain-controlled interfaces: movement restoration with neural prosthetics. Neuron. 2006;52:205–220. doi: 10.1016/j.neuron.2006.09.019. [DOI] [PubMed] [Google Scholar]
  29. Spinks RL, Kraskov A, Brochier T, Umilta MA, Lemon RN. Selectivity for grasp in local field potential and single neuron activity recorded simultaneously from M1 and F5 in the awake macaque monkey. J Neurosci. 2008;28:10961–10971. doi: 10.1523/JNEUROSCI.1956-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Stancák A, Jr, Feige B, Lücking CH, Kristeva-Feige R. Oscillatory cortical activity and movement-related potentials in proximal and distal movements. Clin Neurophysiol. 2000;111:636–650. doi: 10.1016/s1388-2457(99)00310-7. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES