Abstract
In a brain-computer interface (BCI) utilizing a process control strategy, the signal from the cortex is used to control the fine motor details normally handled by other parts of the brain. In a BCI utilizing a goal selection strategy, the signal from the cortex is used to determine the overall end goal of the user, and the BCI controls the fine motor details. A BCI based on goal selection may be an easier and more natural system than one based on process control. Although goal selection in theory may surpass process control the two have never been directly compared, as we are reporting here. Eight young healthy human subjects participated in the present study, three trained and five naïve in BCI usage. Scalp recorded electroencephalograms (EEG) were used to control a computer cursor during five different paradigms. The paradigms were similar in their underlying signal processing and used the same control signal. However, three were based on goal selection, and two on process control. For both the trained and naïve populations, goal selection had more hits per run, was faster, more accurate (for 7/8 subjects), and had a higher information transfer rate than process control. Goal selection outperformed process control in every measure studied in the present investigation.
1. Introduction
There are several diseases and conditions, which lead to loss of muscular control. Those include amyotrophic lateral sclerosis (ALS), brainstem stroke, spinal cord injury, muscular dystrophies, and cerebral palsy. Although people living with these conditions suffer major muscular loss, their cognitive abilities are left intact, thus they still want to communicate and manipulate their environment (Kunst 2004). To do this, these patients require an output pathway from their brain that does not rely on muscular output, or a brain-computer interface (BCI) (Wolpaw et al 2002; Vallabhaneni et al 2005).
A brain-computer interface can be either invasive or non-invasive. One of the most promising types of non-invasive BCI utilizes scalp-recorded electroencephalograms (EEG) to monitor sensorimotor rhythms (SMRs) (Wolpaw et al 2002; Vallabhaneni et al 2005). Sensorimotor rhythms are produced by the primary sensory and motor cortices. SMR based BCI’s utilize two distinct states: event related synchronization (ERS) and event related desynchronization (ERD). When an awake person is not processing sensory data or producing motor output, the primary sensory and motor cortices are in an idling state, which creates a rhythmic EEG pattern known as ERS. The mu rhythm from 8 to 12Hz and the beta rhythm from 13Hz to 26Hz have been particularly useful as BCI control signals. The sensorimotor rhythms decrease in amplitude, an effect that is known as ERD, when processing sensory data or planning or executing movement. ERD occurs during both actual and imagined movement (Pfurtscheller and Lopes da Silva 1999).
The process of planning and executing movement in a healthy individual is a multistep process that involves many parts of the central nervous system. The primary motor cortex initiates this process, but other structures such as the basal ganglia, cerebellum, thalamus, brainstem nuclei, spinal interneurons, and spinal motor neurons are vital to the proper functioning of the process. Since BCIs gather their control signal from the cortex, a BCI bypasses all the complicated, trained interactions that produce normal motion (Wolpaw 2007).
The best BCI systems, both invasive and non-invasive, produce motion that would be classified as ataxic by neuromuscular control specialists (Wolpaw 2007). A possible reason for this ataxia may be that most BCIs in existence today call for the primary motor cortex to control all the fine motor details normally handled by other parts of the brain. These BCIs use the signal obtained primarily from the cortex to determine the position, velocity, and/or acceleration of the controlled device, here a cursor. The user must ensure that they are properly encoding position, velocity, and/or acceleration to hit the desired target. This is known as process control. Process control is not the only control strategy used in BCIs. An alternative control strategy is goal selection. In goal selection, the BCI uses the signal it obtains primarily from the cortex to determine the overall end goal of the user, here the selection of the desired target. The BCI execution unit then determines the necessary position, velocity, and/or acceleration parameters of the cursor to hit the desired target. The user must only encode the desired action, not the details necessary to achieve that action (Wolpaw 2007).
To date, the majority of BCIs employ process control. Some of the few exceptions which employ goal selection include the non-invasive P300 based BCIs (e.g. Farwell and Donchin 1988) and a few invasive studies (e.g. Musallam et al 2004). Lately, goal selection has been applied to SMR based BCIs. Two studies (McFarland et al 2008, Friedrich et al 2008), implemented goal selection in a sensorimotor rhythm based BCI. They met with modest success.
Since goal selection more closely resembles the normal process for motor execution, it follows that a BCI based on goal selection would be an easier and more natural system than one based on process control. An easier and more natural system would be more accurate, faster in use, and easier to learn. This system would have a higher information transfer rate, with a decreased training period. Although several goal selection based BCIs exist, we present the first study directly comparing goal selection and process control. This study will test the first two ideas: that a goal selection BCI should be more accurate and faster to use, which together lead to a higher information transfer rate.
2. Methods
2.1 Data acquisition and cursor control
The human study was conducted according to a human protocol approved by the Institutional Review Board (IRB) of the University of Minnesota. Eight healthy, young volunteers, 1 female and 7 male, participated in the 1-dimensional BCI study. They were seated facing a computer monitor while wearing a 64 channel EEG cap set up according to the 10–20 international system. The scalp-recorded EEG signal passed to a Neuroscan amplifier, and was sampled at 1000Hz. The BCI2000 system (Schalk et al 2004) was used to conduct the online experiments with visual feedback. The control signal was the difference between the autoregressive (AR) spectral amplitudes from 7.5 to 13.5Hz of electrodes C4 and C3 (Wolpaw and McFarland 2004). The magnitude of cursor movement was determined by the normalized AR amplitude difference. At the end of each trial, the control signal was normalized so that it had a zero mean and unit variance across a multiple trial buffer. The parameters used in the normalization, namely the normalizer offset and normalizer gain, were recorded at the end of each session for use at the beginning of the subject’s next session. This is the adaptation built into the BCI2000 system, version 2.0 (Schalk et al 2004). Since the BCI used sensorimotor rhythm as the control signal, the subjects were encouraged to use motor imagination, such as imagining squeezing their right hand to move the cursor right and imagining squeezing their left hand to move the cursor left (Wang & He, 2004; Qin et al, 2005; Pfurtscheller et al 2006; Kamousi et al 2007). Imaginations were not dictated to the subjects. They were free to imagine whatever worked best for them. Several props, such as squeeze balls and dumb bells, were available during breaks to aid in the imagination.
2.2 Study design
The 8 subjects fell into one of two groups. The first group received BCI training, and consisted of three subjects. The second group was naïve to BCI usage, and included five subjects. The trained subjects used a BCI approximately once per week for six to eight weeks. The naïve group, who had never used a BCI prior to this study, completed 2 sessions on different days. The trained subjects completed either one or two sessions. During each session, subjects completed 3 runs of five different paradigms. Each run, regardless of paradigm, was four minutes long, and consisted of as many trials as the subject could complete in four minutes. Although the exact details of a trial varied based on paradigm, each trial presented the targets to the user, and the user attempted to select the yellow target. Between each trial, the subject had three seconds of rest. Each paradigm utilized the same control signal as the other paradigms, so acquired skill transferred easily from one paradigm to the next. The order of the paradigms was reversed between the first and second session.
2.3 Experimental paradigms
Each session consisted of five similar, yet distinct, paradigms. Figure 1 outlines the flow of each paradigm. Each started with the targets appearing on the screen at time 0. The subject was instructed to hit the yellow target. One second later, the cursor appeared. For all the paradigms except goal selection (GS), the cursor then moved under cortical control.
Figure 1.
Experimental paradigms illustrated for a hit. The yellow target is the intended target. Times and explanations apply to paradigms with an image in that row. If the end result of a trial is a miss, the target and cursor will turn red instead of green. (A) Goal selection (GS) and (B) goal selection with feedback limited by time (GSFT). Once a target is selected (blue), it must be reconfirmed (purple) before the cursor will automatically move to that target. Starting at second 3 there are two possible paths. Either the target that was originally selected is reconfirmed (left path), or a new target is selected (right path). If a new target is selected, the cursor will move to the target that was selected two out of three times. Target selection (blue) at seconds 2 and 3, and target reconfirmation (purple) at seconds 3 and 4 may occur slightly earlier if the cortical control signal is especially strong. The times given are the typical, and maximal, times. (C) Goal selection with feedback limited by distance (GSFD) and (D) process control with no aborts (PCNA). No limit* had a limit of 60s. Only one subject ever reached this limit to experience an abort. (E) Process Control (PC). If the subject did not hit a target in under 6s (left picture), the trial timed out and aborted at 6s (right picture).
In GS (figure 1A), the subject did not have the visual feedback of the cursor movement. Instead, the cursor moved invisibly according to the same rules as the other paradigms, but a fixation point identical to the cursor stayed in the middle of the screen. At t = 2s, whichever target the invisible cursor was closest to became selected and turned blue. After selection, the invisible cursor returned to the middle of the screen and began moving again under cortical control. At t=3s, one of two things happened. The first option was that the invisible cursor was closest to the selected target so that the target was reconfirmed, turned purple, and the visible cursor automatically went to that target. The other option was that the invisible cursor was closer to the other target, which became selected and turned blue. In this case, a third round of selection began, and final target selection was determined by being selected two out of three times.
The GS paradigm involved minimal feedback. Both Hochberg et al (2006) and Hinterberger et al (2005) posited that feedback improves BCI performance. Data to support the role of online feedback in improving performance comes from numerous sources (e.g. Neuper et al 1999, Brunner et al 2006). Without sufficient feedback, goal selection may not live up to its potential. In order to increase the amount of feedback in GS, two other variations of goal selection paradigms were developed. Goal selection with feedback limited by time (GSFT) was almost identical to GS, except that it displayed the movement of the cursor. One can see the similarities in figure 1B. One difference between GS and GSFT was that the cursor did not return to the centre after selection in GSFT.
The other variation of a goal selection paradigm with feedback is illustrated in figure 1C. Goal selection with feedback limited by distance (GSFD) was more similar to the remaining 2 paradigms than it was to GS or GSFT. In GSFD, there was a grey circle in the centre of the screen. This circle was visible whenever the targets were visible. The radius of the circle was set at 20% of the screen. GSFD showed the movement of the cursor. When the cursor crossed the circle, it automatically moved to the closest target. In order to allow the subjects to feel like there was no time limit on the trial, and to eliminate trials that timed out and aborted, the maximum trial time was set to 60s. All trials of all subjects were completed in the 60s.
The next experimental paradigm was process control with no aborts (PCNA), illustrated in figure 1D. PCNA was a typical cursor task used in BCI studies (Wolpaw and McFarland 2004, Shenoy et al 2006, Krusienski et al 2007, Yuan et al 2007, Yuan et al 2008, Blankertz et al 2008), where the user controlled the movement of the cursor to hit a target. Similar to GSFD, PCNA was intended to not allow the subjects to abort a trial through timing out. Therefore, the maximum trial time was also set to 60s. However, one subject did have two trials that extended to 60s and thus aborted. Those were exceptional trials, and all the other subjects were able to complete all trials within 60s.
The last experimental paradigm was process control (PC), illustrated in figure 1E. As in PCNA, the user controlled the movement of the cursor until the cursor hit a target. The only difference between PC and PCNA was that a PC trial only allowed the subject 6s to hit a target. If no target had been hit after 6s, the trial aborted.
The five experimental paradigms, GS, GSFT, GSFD, PCNA, and PC, can be divided into two groups. The first group consists of the paradigms based on goal selection: GS, GSFT, and GSFD. The second group consists of the paradigms based on process control: PCNA and PC. To limit confusion, the paradigms will henceforth be referred to by their acronym, whereas the spelled out words goal selection and process control will be reserved for the groups of paradigms and the concepts the words represent.
In order to allow a valid comparison of subject performance across all five experimental paradigms, the inner workings and programming of each paradigm were made consistent. Key paradigm program parameters are summarized in table 1. The parameter FeedbackDuration in PC, PCNA, and GSFD was used to set the cursor movement speed. Note that it is the sum of the GoalSelectionDuration and GoalReconfirmDuration for GS and GSFT. Cursor speed was carefully set to be the same across all paradigms. PostFeedbackDuration in PC, PCNA and GSFD was split into MovementDuration and ResultDuration for GS and GSFT. The BufferLength is the amount of time that was used to normalize the control signal. The SampleBlockSize indicates how often the cursor position was updated.
Table 1.
Paradigm program parameters
| GS and GSFT | Value (s) | PC, PCNA, and GSFD | Value (s) |
|---|---|---|---|
| PreSelectionDuration | 1 | PreFeedbackDuration | 1 |
| GoalSelectionDuration | 1 | FeedbackDuration | 2 |
| ReactionTime (GS) | 0.25 | MaxFeedbackDuration (PC) | 6 |
| ReactionTime (GSFT) | 0 | MaxFeedbackDuration (PCNA & GSFD) | 60 |
| GoalReconfirmDuration | 1 | ||
| MovementDuration | 0.5 | PostFeedbackDuration | 1 |
| ResultDuration | 0.5 | ||
| BufferLength | 30 | Buffer Length | 30 |
| SampleBlockSize | 0.04 | SampleBlockSize | 0.04 |
| ITI Duration | 3 | ITI Duration | 3 |
| MinRunLength | 240 | MinRunLength | 240 |
The naïve subjects started with PC followed by PCNA, GSFD, GSFT, and GS. The order was reversed for their second session. The trained subjects used the same order as the naïve subjects and reversed the order each session, resulting in alternating sessions having the same order.
2.4 Data analysis
Several aspects of subject performance were analyzed in order to compare the different paradigms. The features analyzed included the average number of hits per run, time distribution to a hit, overall accuracy, and information transfer rate in bits per minute. The paradigms were compared both within each subject and across all the data pooled from each group.
The time required for a subject to hit the target was determined from the time during which the cursor was under cortical control. For all paradigms, the time started when the cursor appeared. In GS and GSFT, time ended when a target was reconfirmed and turned purple. In GSFD, time ended when the cursor crossed the circle and turned green. In all three of those cases, the additional time required to actually hit the target was a user settable programmed parameter that was constant for each trial. Therefore, that time was not held against the user. In PCNA and PC, time ended when a target was hit and both the target and cursor turned green. As discussed above, the cursor moved under the same control signal and at the same speed for all five paradigms. Any remaining differences in time to task completion were due to the facets of the paradigms we wished to compare. Because of this, the time required for a subject to hit the target as determined from the time during which the cursor was under cortical control was a valid comparison and measure of the different paradigms.
For the purpose of data analysis, accuracy was defined as the number of hits in a run divided by the total number of trials in a run. For PC runs, aborted trials were counted in the total number of trials. Accuracy could also be viewed as the percentage of trials that ended in a hit. This form of calculating accuracy effectively normalized the data for the different number of trials in each run, which allowed a fair comparison of the different paradigms.
A useful way to compare different BCIs is via their information transfer rate, either in bits/trial or bits/min. As given by Wolpaw et al. (2002), bits/trial can be calculated from the following equation:
| (1) |
In the above equation, N is the number of targets, and P is the probability of a hit, or the accuracy. The number of trials per minute can then be multiplied by (1) to obtain the information transfer rate in bits/min. One useful feature of this measure is that it incorporates both speed and accuracy into one number. The information transfer rate in bits per minute was calculated for each trial. The average of those trials was then used for plotting.
Significance testing utilized one of two statistical tests. A pair wise t-test was used for measures that had an underlying normal distribution, such as the number of hits per run and the information transfer rate in bits per minute. The standard deviation was pooled between paradigms, and no p-value correction was applied. The chi-squared pair wise proportion test was used for significance testing the accuracies between paradigms since that measure had an underlying binomial distribution. No p-value correction was applied to the chi-squared test. Asterisks on the plots indicate pair-wise significance to the previous paradigms. Tables providing more information on the significant differences follow any figure with asterisks. Error bars on the plots indicate standard error.
The box plots show the distribution of the data. The lower whisker extends from the minimum value to the 25th percentile. The lower, darker box extends from the 25th percentile to the median. The upper, lighter box extends from the median to the 75th percentile. The upper whisker extends from the 75th percentile to the maximum value.
3. Results
3.1 Number of hits in an average run
The average run for each subject and each paradigm is shown in figure 2. Significant differences in the number of hits are shown in table 2. Across the board, the goal selection paradigms have more hits in an average run than the process control paradigms. Figure 2A shows the data for the trained subjects. It is interesting to note that the trained subjects still experienced a good number of aborted trials in PC even though the number of hits stayed relatively consistent with or without aborts. Both GSFT and GSFD had significantly more hits than PCNA for all three subjects. Additionally, GS had significantly more hits than PCNA for two out of the three subjects. Significance could not be calculated for subject T2’s process control paradigm. Both of the remaining two subjects showed that PC had significantly fewer hits than GSFT and GSFD. In the pooled data, GSFT and GSFD had more hits than GS, with a significant difference between GS and GSFT. Importantly, all of the goal selection paradigms had significantly more hits than either of the process control paradigms. The increase in hits from process control to goal selection paradigms ranged from 25% to 59%.
Figure 2.
Average run breakdown for both trained (A) and naïve (B) subjects. Overall, the goal selection paradigms have significantly more hits than the process control paradigms. Asterisks indicate pair-wise significance to the previous paradigms for the number of hits.
Table 2.
Significant differences for number of hits
| GS | GSFT | GSFD | PCNA | PC | |
|---|---|---|---|---|---|
| GS** | GS*** | ||||
| GSFT** | PCNA*** | PCNA*** | GSFT*** | GSFT* | |
| T1 | GSFD** | PC* | PC** | GSFD*** | GSFD** |
|
| |||||
| GS*** | |||||
| GSFT*** | |||||
| T2 | PCNA*** | PCNA*** | PCNA*** | GSFD*** | NA |
|
| |||||
| GSFT* | GS* | GS* | GS** | ||
| PCNA* | PCNA*** | PCNA*** | GSFT*** | GSFT*** | |
| T3 | PC* | PC*** | PC*** | GSFD*** | GSFD*** |
|
| |||||
| GSFT** | GS** | GS** | GS* | ||
| PCNA** | PCNA*** | PCNA*** | GSFT*** | GSFT*** | |
| Pooled | PC* | PC*** | PC*** | GSFD*** | GSFD*** |
|
| |||||
| GS*** | GS*** | ||||
| PCNA*** | PCNA*** | PCNA*** | GSFT*** | GSFT*** | |
| N1 | PC*** | PC*** | PC*** | GSFD*** | GSFD*** |
|
| |||||
| GS*** | GS*** | ||||
| GSFD* | GS* | GSFT*** | GSFT*** | ||
| PCNA*** | PCNA*** | PCNA** | GSFD** | GSFD*** | |
| N2 | PC*** | PC*** | PC*** | PC*** | PCNA*** |
|
| |||||
| GS*** | GS*** | ||||
| GSFT*** | GSFT*** | ||||
| PCNA*** | PCNA*** | PCNA*** | GSFD*** | GSFD*** | |
| N3 | PC*** | PC*** | PC*** | PC* | PCNA* |
|
| |||||
| GSFD* | GS** | ||||
| PCNA** | GSFT* | GSFT*** | |||
| N4 | PC** | PC*** | PC* | GSFT** | GSFD* |
|
| |||||
| GS*** | GS** | ||||
| GSFT** | GSFT*** | ||||
| GSFD*** | GSFD** | PCNA*** | GSFD*** | GSFD*** | |
| N5 | PC** | PC*** | PC*** | PC** | PCNA** |
|
| |||||
| GS*** | GS*** | ||||
| GSFT*** | GSFT*** | ||||
| PCNA*** | PCNA*** | PCNA*** | GSFD*** | GSFD*** | |
| Pooled | PC*** | PC*** | PC*** | PC*** | PCNA*** |
p<0.05,
p<0.01,
p<0.001
The data for the naïve subjects is displayed in figure 2B. As in the trained data, the goal selection paradigms resulted in more hits than the process control paradigms. The naïve subjects experienced many aborts during PC, with individual subjects aborting 39% to 61% of the trials. The pooled data shows 49% of the PC trials timed out. Not surprisingly then, PC always had significantly fewer hits than any form of goal selection. PCNA had significantly fewer hits than any form of goal selection for three of the five subjects, whereas PCNA had significantly fewer hits than GSFT or GSFD for the other two subjects. In the pooled data, all of the goal selection paradigms have significantly more hits than the process control paradigms. The increase in hits from PCNA to the goal selection paradigms ranged from 48% to 65%. Astoundingly, the increase in hits from PC to the goal selection paradigms ranged from 124% to 151%. The goal selection paradigms had more than twice the number of hits than PC.
3.2 Time to a hit
The distribution of time the cursor was under cortical control leading to a hit is displayed in figure 3. The median times are also summarized in table 2. The trained and the naïve subjects showed similar results. For all subjects, the goal selection paradigms were faster than the process control paradigms. Another commonality is that a GSFT hit occurred at approximately the same time, with very little spread both within and between subjects. This implies that a hit occurred when a target was first selected and then reconfirmed. A hit hardly ever resulted from a best two-out-of-three condition. GS, which had the same selection and reconfirmation timing as GSFT, had a larger spread. This means that more hits resulted from a best two-out-of three condition. Although GS and GSFT were inherently time constrained, even the non-time constrained GSFD was typically faster than both of the process control paradigms, as shown by a smaller 25th to 75th percentile distribution in the pooled data.
Figure 3.
Time required for a hit across the five paradigms for trained ((A) through (D)) and naive ((E) through (J)) subjects. The goal selection paradigms are faster than the process control paradigms. Axes labels and graph legends on the bottom row of each column ((D) and (J)) apply to all figures in that column. The plots to the right of each box plot are the same data, shown as a cumulative distribution.
Looking at the pooled data from each group provides further insight. The pooled trained data (figure 3A) showed that the median of all three goal selection paradigms was approximately twice as fast as the median hit time for PCNA and 30% to 60% faster than the median time for PC. The data from the naïve subjects (figure 3E) is qualitatively very similar to the data from the trained subjects. It is not surprising that the non-time constrained paradigms of GSFD, PCNA, and PC had slightly longer medians with a larger spread in times in the untrained subjects. This led to all three of the goal selection paradigms being at least twice as fast as PCNA, and 48% to 72% faster than PC in the pooled data.
3.3 Accuracy
The accuracy of each of the subjects using each paradigm is given in figure 4, with the significant differences shown in table 4. For the trained subjects (figure 4A), GSFD had the highest accuracy. For subjects T1 and T3, GSFD had significantly higher accuracy than both GS and PC. T1 preferred feedback limited by distance over feedback limited by time, but T3’s performance did not distinguish between the different forms of feedback. Both were significantly more accurate than GS and PC. In the pooled data, GSFT and GSFD were more accurate than GS or PC. GSFD was significantly more accurate than both GS and PC, whereas GSFT was significantly more accurate than PC.
Figure 4.
Accuracy of the trained (A) and naïve (B) subjects across the five paradigms. GSFT or GSFD commonly had the highest accuracy, with PC having the worst accuracy.
Table 4.
Significant differences for accuracy
| GS | GSFT | GSFD | PCNA | PC | |
|---|---|---|---|---|---|
| GS*** | |||||
| GSFD*** | GSFT* | ||||
| T1 | PCNA** | GSFD* | PC* | GS** | GSFD* |
|
| |||||
| T2 | |||||
|
| |||||
| GS*** | |||||
| GSFT** | GSFT*** | ||||
| GSFD** | GS** | GS** | GSFD*** | ||
| T3 | PC*** | PC*** | PC*** | PC*** | PCNA*** |
|
| |||||
| GSFT** | |||||
| GS* | GSFD** | ||||
| Pooled | GSFD* | PC** | PC** | PC* | PCNA* |
|
| |||||
| GS*** | |||||
| GSFT*** | |||||
| GSFD*** | |||||
| N1 | PC*** | PC*** | PC*** | PC** | PCNA** |
|
| |||||
| GS*** | |||||
| GSFT*** | |||||
| GSFD*** | |||||
| N2 | PC*** | PC*** | PC*** | PC*** | PCNA*** |
|
| |||||
| GS*** | |||||
| GSFT*** | |||||
| GSFD*** | |||||
| N3 | PC*** | PC*** | PC*** | PC*** | PCNA*** |
|
| |||||
| GS*** | |||||
| GSFT*** | |||||
| GSFD*** | |||||
| N4 | PC*** | PC*** | PC*** | PC*** | PCNA*** |
|
| |||||
| GS*** | GS* | GS** | |||
| GSFD*** | GSFD*** | GSFT*** | GSFT* | GSFT** | |
| PCNA* | PCNA* | PCNA** | GSFD** | GSFD*** | |
| N5 | PC** | PC** | PC*** | PC*** | PCNA*** |
|
| |||||
| GS*** | |||||
| GSFT*** | |||||
| GSFD*** | |||||
| Pooled | PC*** | PC*** | PC*** | PC*** | PCNA*** |
p<0.05,
p<0.01,
p<0.001
In the naïve subjects (figure 4B), there was considerable variation from individual to individual. No one paradigm was consistently the most accurate. However, goal selection of some form was the most favoured. Four out of the five subjects had the highest accuracy with a form of goal selection. Although one paradigm did not stand out as the best, one did establish itself as the worst. For all subjects, PC was significantly less accurate than all other paradigms.
3.4 Information transfer rate
The information transfer rate as measured in bits per minute is a convenient way to compare different BCI systems. This one number incorporates both accuracy and speed. Figure 5 shows the information transfer rate for this experiment, and table 5 shows the significant differences. In general, GSFT and/or GSFD transferred more bits per minute than the process control paradigms. Looking at the trained subjects (figure 5A), for T1 and T3 either GSFT or GSFD had a significantly higher bit rate than GS, PCNA, and PC. In the pooled data, GSFT and GSFD had a significantly increased rate of information transfer over GS. GSFT and GSFD had a significantly higher information transfer rate than one or both forms of process control. Numerically, GS had a bit rate 25% higher than PC. GSFT and GSFD increased the information transfer to rates more than twice as high as PC. PCNA did have a higher bit rate than PC, but GSFT and GSFD still had a 47% to 60% higher information transfer rate than PCNA.
Figure 5.
Information transfer in bits/min for both the trained (A) and naïve (B) subjects across the five paradigms. GSFT or GSFD consistently had the highest information transfer rate.
Table 5.
Significant differences for information transfer rate
| GS | GSFT | GSFD | PCNA | PC | |
|---|---|---|---|---|---|
| GS*** | |||||
| GSFT* | GSFT* | ||||
| GSFD*** | GS* | PCNA* | GS** | ||
| T1 | PCNA** | GSFD* | PC** | GSFD* | GSFD** |
|
| |||||
| T2 | |||||
|
| |||||
| GS* | |||||
| PCNA* | GSFT** | ||||
| T3 | GSFT* | PC** | PC** | GSFT* | GSFD** |
|
| |||||
| GS** | |||||
| GSFT* | GS* | PCNA* | GSFT** | ||
| Pooled | GSFD** | PC** | PC** | GSFD* | GSFD** |
|
| |||||
| N1 | PC* | GSFD* | |||
|
| |||||
| N2 | |||||
|
| |||||
| GS** | |||||
| PCNA* | GSFT*** | ||||
| N3 | PC** | PC*** | PC** | GSFT* | GSFD** |
|
| |||||
| N4 | GSFD* | GSFT* | |||
|
| |||||
| GS*** | |||||
| GSFT** | |||||
| PCNA* | |||||
| N5 | GSFD*** | GSFD** | PC** | GSFD* | GSFD** |
|
| |||||
| GSFT* | |||||
| Pooled | PC* | PC* | GSFD* | ||
p<0.05,
p<0.01,
p<0.001
The naïve subjects showed much variability in their information transfer rates (figure 5B). For four of the five subjects, GSFT or GSFD had the highest bit rate. For some of these subjects, the difference was significant. N1 transferred significantly more information using GSFD than PC. N3’s information transfer rate for GSFT was significantly higher than both process controls, and PC transferred significantly less information than any of the goal selection paradigms. N5’s GSFD had a significantly higher information transfer rate than all other paradigms. In the pooled data, PC was significantly worse than both GSFT and GSFD. Numerically, GS transferred 85% more information than PC. GSFT and GSFD more than doubled PC’s information transfer rate. GSFT and GSFD had a higher bit rate than PCNA by 47% and 58%.
4. Discussion
Although several goal selection based BCIs exist, we presented the first study directly comparing goal selection and process control. We found the following to be true in both trained and naïve populations studied: (1) Goal selection had more hits than process control. (2) Goal selection was faster than process control. (3) Goal selection was more accurate than process control for most subjects, and (4) goal selection had a higher information transfer rate than process control.
Goal selection outperformed process control in every measure studied here. This is summarized well by the information transfer rate data presented in figure 5. However, the goal selection paradigms were not optimized, and further increases in performance could be reasonably expected. For instance, in GS and GSFT, the selection and reconfirm times were set somewhat arbitrarily at 1s. As demonstrated by Santhanam et al (2006), optimizing selection and reconfirm times can significantly increase the information transfer rate. In fact, a version of GS or GSFT could be implemented where the time was not a set time, but was instead determined by a statistical confidence threshold. This could lead to an asynchronous, user paced, BCI. Information transfer using GSFD could likewise be optimized by determining the optimal circle radius for each user. This study produced a 50% to 1600% improvement from a process control based paradigm to an un-optimized goal selection based paradigm. It is exciting to think of the magnitude of improvement that could be possible with an optimized goal selection based paradigm.
Two other sensorimotor rhythm based BCIs have adopted goal selection to some extent. One study (McFarland et al 2008) emulated computer mouse control where the subject used 2-dimensional control to move a computer cursor around a screen. When the cursor was above the target of interest, the subject could then select that target, like a mouse click. Adding the requirement that the target not only be hit by the cursor, but also selected, increased accuracy. This study combined process control, i.e. moving the cursor, with goal selection, i.e. selecting the target, to increase accuracy.
Another study (Friedrich et al 2008) used a scanning protocol similar in some respects to a P300 system (Donchin et al 2000). When the target of interest was highlighted, the subject selected the target through modulation of sensorimotor rhythms. Although it is a novel application of sensorimotor rhythms, it shares a common disadvantage of the P300 systems that the scanning protocol is relatively slow. The best mean information transfer rate was approximately 2.5 bits per minute for the trained subjects in Freidrich et al’s study. In the work we presented here, the pooled data from the trained subjects had a higher information transfer rate for all three forms of goal selection. In fact, goal selection with feedback had twice the information transfer rate as the Friedrich et al study. We were able to achieve this despite having double the resting period between trials.
One trend apparent throughout all the results presented here is the importance of feedback. Goal selection with a form of feedback, GSFT and GSFD, tended to outperform goal selection alone, GS, in every measure presented. This agrees with previous literature (Neuper et al 1999, Hinterberger et al 2005, Brunner et al 2006, Hochberg et al 2006). An important distinction between GS and GSFT or GSFD is that GS offered discrete feedback, whereas the other two paradigms offered continuous feedback. In the Neuper et al (1999) study, continuous feedback was demonstrated to be more effective than discrete feedback.
In the present study, continuous feedback was only temporarily removed when the subjects performed the GS runs. When the subjects moved on to the other paradigms, continuous feedback was once again available. This is similar to another study that temporarily removed feedback in a similar fashion (McFarland et al 1998). McFarland et al found that there was no significant effect when feedback was removed. Their trained subjects could maintain performance for short periods of time. Although the group analysis found no significant effect, the removal of feedback either increased or decreased individual accuracy. Both of the above points were corroborated by the comments of the current subjects while they were performing this experiment. On the first point, some subjects thought the GS paradigm was easier when it was last because they had already experienced feedback that day. On the second point, some subjects found the movement of the cursor to be distracting, whereas others liked seeing it.
As McFarland et al showed (2008) in their computer mouse emulation, combining goal selection and process control can improve a system. This is consistent with our findings. Our subjects were moving the cursor in a similar manner across all five paradigms in what could be described as a process control strategy. However, in the goal selection paradigms, how the cursor moved under cortical control was only a small part of the overall process necessary to hit the target. GS did not show the motion of the cursor to the user. Instead, target selection was indicated by the colour of the target. This is similar to the scanning protocol study (Friedrich et al 2008). Once a target was reconfirmed, the cursor moved under automated control to the target. Our other two goal selection paradigms, GSFT and GSFD, combined goal selection with process control in a much more obvious fashion, yet kept the final control of cursor movement under the automated movement of the BCI. GSFT simply allowed the visualization of the underlying motion of the cursor in GS. GSFD allowed the user to point the BCI in the right direction. The BCI would then complete what the user began. In none of the goal selection paradigms did the user have to complete the task under cortical control. The final execution of the task was completed by the execution unit of the BCI.
In the present study we utilized two different process control paradigms. In the first paradigm, PCNA, the trial did not end until a subject hit a target. In the second paradigm, PC, the trial timed out and aborted at 6s if a target had not been hit. PC was chosen purposefully to facilitate comparison to the previously published work that allowed the trial to time out (e.g. Wolpaw and McFarland 2004, Yuan et al 2007, Yuan et al 2008). PCNA is more relevant to non-computer applications, such as controlling a wheelchair or a neuroprosthetic, where an abort might entail the wheelchair or the prosthetic returning to its starting position. That style of abort could be quite frustrating to the user. For a wheelchair, a prosthetic, or some similar application, a user will try until they succeed at their task.
Some would argue that, for trained subjects, there is no difference between a protocol that allows aborts and one that does not. Our data showed that PCNA and PC were different in every measure presented. The trained subjects were faster than the untrained subjects, but they still experienced a good number of aborts. Individually, the percentage of aborted trials ranged from 9% to 35%, and the pooled data showed 22% of the trials ended by timing out. This shows that trained subjects could not always hit a target within 6s. When timing out was not an option, more than 29% of the trained subjects’ hits occurred after 6s, with individual subjects having 17%, 34%, and 36% of their hits after the 6s point. Because of the number of aborts, PC had lower accuracy than PCNA. When both speed and accuracy are combined in the information transfer rate, allowing aborts reduced the information transfer rate.
Despite the above statements, allowing aborts in process control does have some redeeming value. If the aborted trials were thrown out during data analysis, as if the trial had never occurred and the subject had an extended rest, the accuracy and the information transfer rate would be approximately equal to or greater than the values for PCNA. This makes process control with aborts, PC, the preferred paradigm when the consequences of an abort are acceptable, such as when interacting with a computer. In conclusion, process control with aborts, PC, and process control with no aborts, PCNA, are not the same paradigm. The proper strategy should be chosen based on the final application.
The present study compared five different BCI paradigms, three based on goal selection and two based on process control. Since goal selection uses the cortical signal in a manner similar to normal motor control, it follows that a BCI based on goal selection would be an easier and more natural system than one based on process control. It was hypothesized that the goal selection paradigms would be more accurate, faster in use, and easier to learn. All of those things would lead to a higher information transfer rate with decreased training time. This study tested only the first two points and showed that the goal selection paradigms were more accurate and faster in use, which together led to a higher information transfer rate. The third point, that goal selection should be easier to learn, was not tested. The design of the study did not and could not address learning. That question will be addressed in future work.
Table 3.
Median time to a hit (s)
| GS | GSFT | GSFD | PCNA | PC | |
|---|---|---|---|---|---|
| T1 | 2.25 | 2.05 | 1.83 | 3.19 | 2.28 |
| T2 | 2.25 | 2.05 | 2.58 | 5.01 | 3.74 |
| T3 | 2.25 | 2.05 | 1.48 | 4.46 | 3.28 |
| Pooled | 2.25 | 2.05 | 1.82 | 4.05 | 2.92 |
| N1 | 2.25 | 2.05 | 1.82 | 6.23 | 3.65 |
| N2 | 2.26 | 2.05 | 2.48 | 6.75 | 4.45 |
| N3 | 2.25 | 2.05 | 1.59 | 3.81 | 3.06 |
| N4 | 2.25 | 2.05 | 1.89 | 3.79 | 2.82 |
| N5 | 2.25 | 2.05 | 2.12 | 3.58 | 3.33 |
| Pooled | 2.25 | 2.05 | 1.94 | 4.46 | 3.33 |
Acknowledgments
The authors are grateful to Andrew McCullough and Cristina Rios for assistance in data collection. This work was supported by NIH Grants RO1EB007920-01, 5 T90 DK70106, and 1 T32 EB008389-01.
References
- Blankertz B, Losch F, Krauledat M, Dornhege G, Curio G, Mueller KR. The Berlin brain–computer interface: accurate performance from first-session in BCI naive subjects. IEEE Trans Biomed Eng. 2008;55:2452–62. doi: 10.1109/TBME.2008.923152. [DOI] [PubMed] [Google Scholar]
- Brunner C, Scherer R, Graimann B, Supp G, Pfurtscheller G. Online control of a brain-computer interface using phase synchronization. IEEE Trans Biomed Eng. 2006;53:2501–6. doi: 10.1109/TBME.2006.881775. [DOI] [PubMed] [Google Scholar]
- Donchin E, Spencer KV, Wijesinghe R. The mental prosthesis: Assessing the speed of a P300-based brain–computer interface. IEEE Trans Rehab Eng. 2000;8:174–9. doi: 10.1109/86.847808. [DOI] [PubMed] [Google Scholar]
- Farwell LA, Donchin E. Talking off the top of your head: toward a mental prothesis utilizing event-related brain potentials. Electroenceph Clin Neurophysiol. 1988;70:510–23. doi: 10.1016/0013-4694(88)90149-6. [DOI] [PubMed] [Google Scholar]
- Friedrich EVC, McFarland DJ, Neuper C, Vaughan TM, Brunner P, Wolpaw JR. A scanning protocol for a sensorimotor rhythm-based brain–computer interface. Biological Psychology. 2008 doi: 10.1016/j.biopsycho.2008.08.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hinterberger T, Veit R, Wilhelm B, Weiskopf N, Vatine J, Birbaumer N. Neuronal mechanisms underlying control of a brain–computer interface. European Journal of Neuroscience. 2005;21:3169–81. doi: 10.1111/j.1460-9568.2005.04092.x. [DOI] [PubMed] [Google Scholar]
- Hochberg LR, Serruya MD, Friehs GM, Mukand J, Saleh M, Caplan AH, Branner A, Chen D, Penn RD, Donoghue JP. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature. 2006;442:164–71. doi: 10.1038/nature04970. [DOI] [PubMed] [Google Scholar]
- Kamousi B, Amini AN, He B. Classification of motor imagery by means of cortical current density estimation and Von Neumann entropy for brain-computer interface applications. Journal of Neural Engineering. 2007;4:17–25. doi: 10.1088/1741-2560/4/2/002. [DOI] [PubMed] [Google Scholar]
- Krusienski DJ, Schalk G, McFarland DJ, Wolpaw JR. A mu-rhythm matched filter for continuous control of a brain-computer interface. IEEE Trans Biomed Eng. 2007;54:273–80. doi: 10.1109/TBME.2006.886661. [DOI] [PubMed] [Google Scholar]
- Kunst CB. Complex genetics of amyotrophic lateral sclerosis. Am J Hum Genet. 2004;75:933–47. doi: 10.1086/426001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McFarland DJ, Dean J, Krusienski DJ, Sarnacki WA, Wolpaw JR. Emulation of computer mouse control with a noninvasive brain–computer interface. J Neural Eng. 2008;5:101–10. doi: 10.1088/1741-2560/5/2/001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McFarland DJ, McCane LM, Wolpaw JR. EEG-based communication and control: short-term role of feedback. IEEE Trans Rehabil Eng. 1998;6:7–11. doi: 10.1109/86.662615. [DOI] [PubMed] [Google Scholar]
- Musallam S, Corneil BD, Greger B, Scherberger H, Andersen RA. Cognitive control signals for neural prosthetics. Science. 2004;305:258–62. doi: 10.1126/science.1097938. [DOI] [PubMed] [Google Scholar]
- Neuper C, Schlögl A, Pfurtscheller G. Enhancement of left-right sensorimotor EEG differences during feedback-regulated motor imagery. J Clin Neurophysiol. 1999;16:373–82. doi: 10.1097/00004691-199907000-00010. [DOI] [PubMed] [Google Scholar]
- Pfurtscheller G, Lopes da Silva FH. Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical Neurophysiology. 1999;110:1842–57. doi: 10.1016/s1388-2457(99)00141-8. [DOI] [PubMed] [Google Scholar]
- Pfurtscheller G, Brunner C, Schloegl A, Lopes da Silva FH. Mu rhythm (de)synchronization and EEG single-trial classi cation of different motor imagery tasks. NeuroImage. 2006;33:153–9. doi: 10.1016/j.neuroimage.2005.12.003. [DOI] [PubMed] [Google Scholar]
- Qin L, He B. A wavelet-based time-frequency analysis approach for classification of motor imagery for brain-computer interface applications. Journal of Neural Engineering. 2005;2:65–72. doi: 10.1088/1741-2560/2/4/001. [DOI] [PubMed] [Google Scholar]
- Santhanam G, Ryu SI, Yu BM, Afshar A, Shenoy KV. A high-performance brain-computer interface. Nature. 2006;442:195–8. doi: 10.1038/nature04968. [DOI] [PubMed] [Google Scholar]
- Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR. BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng. 2004;51:1034–43. doi: 10.1109/TBME.2004.827072. [DOI] [PubMed] [Google Scholar]
- Shenoy P, Krauledat M, Blankertz B, Rao RPN, Mueller KR. Towards adaptive classification for BCI J. Neural Eng. 2006;3:R13–R23. doi: 10.1088/1741-2560/3/1/R02. [DOI] [PubMed] [Google Scholar]
- Vallabhaneni A, Wang T, He B. Brain computer interface. In: He B, editor. Neural Engineering. New York: Kluwer Academic/Plenum; 2005. pp. 85–122. [Google Scholar]
- Wolpaw JR. Brain–computer interfaces as new brain output pathways. J Physiol. 2007;579:613–9. doi: 10.1113/jphysiol.2006.125948. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wolpaw JR, McFarland DJ. Control of a two-dimensional movement signal by a noninvasive brain–computer interface in humans. Proc Natl Acad Sci U S A. 2004;101:17849–54. doi: 10.1073/pnas.0403504101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain computer interfaces for communication and control. Clinical Neurophysiology. 2002;113:767–91. doi: 10.1016/s1388-2457(02)00057-3. [DOI] [PubMed] [Google Scholar]
- Yuan H, Doud AJ, Gururajan A, He B. Cortical imaging of sensorimotor rhythm during on-line control of brain-computer interface. Proc of NFSI&ICFBI Conf. 2007:327–9. [Google Scholar]
- Yuan H, Doud AJ, Gururajan A, He B. Cortical imaging of event-related (de)synchronization during online control of brain-computer interface using minimum-norm estimates in the frequency domain. IEEE Trans Neural Sys & Rehab Eng. 2008;16:425–31. doi: 10.1109/TNSRE.2008.2003384. [DOI] [PMC free article] [PubMed] [Google Scholar]