Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Feb 1.
Published in final edited form as: Clin Neurophysiol. 2008 Feb;119(2):399–408. doi: 10.1016/j.clinph.2007.09.121

Towards an Independent Brain - Computer Interface Using Steady State Visual Evoked Potentials

Brendan Z Allison a,*, Dennis J McFarland b, Gerwin Schalk b, Shi Dong Zheng a, Melody Moore Jackson a, Jonathan R Wolpaw b
PMCID: PMC2274834  NIHMSID: NIHMS38485  PMID: 18077208

Abstract

Objective

Brain - computer interface (BCI) systems using steady state visual evoked potentials (SSVEPs) have allowed healthy subjects to communicate. However, these systems may not work in severely disabled users because they may depend on gaze shifting. This study evaluates the hypothesis that overlapping stimuli can evoke changes in SSVEP activity sufficient to control a BCI. This would provide evidence that SSVEP BCIs could be used without shifting gaze.

Methods

Subjects viewed a display containing two images that each oscillated at a different frequency. Different conditions used overlapping or non-overlapping images to explore dependence on gaze function. Subjects were asked to direct attention to one or the other of these images during each of twelve one-minute runs.

Results

Half of the subjects produced differences in SSVEP activity elicited by overlapping stimuli that could support BCI control. In all remaining users, differences did exist at corresponding frequencies but were not strong enough to allow effective control.

Conclusions

The data demonstrate that SSVEP differences sufficient for BCI control may be elicited by selective attention to one of two overlapping stimuli. Thus, some SSVEP-based BCI approaches may not depend on gaze control. The nature and extent of any BCI's dependence on muscle activity is a function of many factors, including the display, task, environment, and user.

Significance

SSVEP BCIs might function in severely disabled users unable to reliably control gaze. Further research with these users is necessary to explore the optimal parameters of such a system and validate online performance in a home environment.

1. Introduction

Many people with motor disabilities cannot use conventional interfaces such as mice or keyboards. Although some of these users can use other interfaces such as eye trackers or EMG switches (Cook and Hussey, 2002), some severely disabled users require a means of communication that does not rely on motor control at all. Brain computer interface (BCI) systems translate direct measures of brain activity into messages or commands. A variety of BCI systems have been described in the literature and typically are categorized according to the cognitive and neural activity needed for control (for review, see Kübler et al., 2001; Wolpaw et al., 2002; Allison, 2003; Kübler and Neumann, 2005; Jackson et al., 2006; Allison et al., 2007).

One type of BCI utilizes changes in steady state visual evoked potentials (SSVEPs). In this approach, a subject views one or more stimuli that each oscillate at a different constant frequency. When the subject focuses attention on one such stimulus, EEG activity may be detected over occipital areas at corresponding frequencies. Hence, an SSVEP BCI can infer user intent by measuring EEG activity at a specific frequency or frequencies over occipital areas. Although SSVEP BCIs work with healthy subjects (e. g.., Middendorf et al., 2000; Cheng et al., 2002; Lalor et al., 2005) and subjects with moderate disabilities (Sutter et al., 19921; Wang et al., 2004), they have not been validated with subjects unable to control gaze.

The prevailing view in the BCI literature is that SSVEP BCIs would not work in such subjects. SSVEP BCI articles typically note that subjects were told to shift gaze (Sutter et al., 1992; Middendorf et al., 2000; Cheng et al., 2002; Gao et al., 2003). Two BCI reviews (Kübler et al., 2001; Wolpaw et al., 2002) define SSVEP BCIs as “dependent” BCIs, meaning that they use EEG features that depend on muscle activity and thus would not work in patients without control over that activity. SSVEP BCI development would then be less important, as other assistive technologies based on gaze direction might be more effective (Cook and Hussey, 2002).

However, strong evidence from the visual attention literature suggests that people can shift attention among visual stimuli without shifting gaze. This phenomenon, called covert attention, has been verified in many human studies in which gaze shifting was carefully measured (e.g., van Voorhis and Hillyard, 1977; Regan, 1989; Mangun and Buck, 1998; Golla et al., 2005). It has also been shown in SSVEP studies in which covert attention to an oscillating region or regions resulted in increased SSVEP activity at corresponding frequencies (Müller et al., 1998; Müller and Hillyard, 2000; Müller et al., 2003). These SSVEP studies were designed to rule out the possibility that results could be explained by shifting gaze. MEG work also shows that humans can produce changes in brain activity by attending to one of two overlapping images (Chen et al., 2003). Thus, an independent BCI based on covert attention may be a viable communication system even for users without gaze control.

The main goal of the study was to determine whether selective attention to one of two overlapping images would produce enough change in SSVEP activity to control an online BCI. This study compares an SSVEP display using non-overlapping checkerboxes to displays using overlapping stimuli. To determine whether color would help distinguish overlapping stimuli, two types of overlapping stimuli were used: colored and black/white (BW).

2. Methods and Materials

2.1 Subjects

Subjects were 14 healthy adults (8 women, 6 men; age range 18−29 years, mean = 19.7, SD = 2.9), 11 of whom were undergraduate students at Georgia State University. All subjects were free of neurological or psychiatric disorders or medications known to adversely affect EEG recording. None had prior experience with EEG recording or BCIs. All subjects signed a consent form and earned credit in a psychology course or $10/hour for their participation. The nature and purpose of the study was explained to each subject before preparation for EEG recording. No subjects were excluded from the study nor chose not to participate. Everyone who asked to be a subject was a subject, and all data collected from these subjects are reported below. The study was reviewed and approved by the Georgia State University IRB.

2.2 Data collection

Subjects wore a 64-channel electrode cap (Electro-Cap International) using the International 10−20 system of electrode placement (Scharbrough et al., 1990). EEG channels were referenced to an electrode attached to the right earlobe, and a ground electrode was placed behind the right mastoid. All impedances were kept below 10 kOhms. Data were sampled at 160 Hz, band-pass filtered between 0.1−50 Hz, and amplified 20,000× on an SA Instruments biosignal amplifier. The BCI2000 software package (Schalk et al., 2004) was used for all data acquisition. Stimuli were presented using Presentation (Neurobehavioral Systems) and analyzed using BCI2ASCII (Wadsworth Center) and Matlab Release 12 (Mathworks). Data were collected in a busy office area with occasional uncontrolled distractions, rather than a shielded room, as this represents a more realistic environment for BCI use.

2.3 Display and procedure

After being prepared for EEG recording, subjects were seated in a comfortable leather chair about 3 feet from a 21” ViewSonic CRT monitor with a 60 Hz refresh rate. In all conditions, subjects viewed two images that each oscillated at a different frequency (see below). All subjects participated in twelve one-minute runs that were separated by breaks of 30−60 seconds (see Table 1). Subjects completed questionnaires after the last run.

Table 1.

The protocol used in this study. Subjects viewed two images that oscillated at different frequencies. Before each run began, subjects were asked to attend to the image in the “Target” column.

Run Image 1 and frequency Image 2 and frequency Target Color?
1 Horizontal linebox 10 Hz Vertical linebox 12 Hz Horizontal NO
2 Horizontal linebox 10 Hz Vertical linebox 12 Hz Vertical NO
3 Horizontal linebox 10 Hz Vertical linebox 12 Hz Horizontal YES
4 Horizontal linebox 10 Hz Vertical linebox 12 Hz Vertical YES
5 Horizontal linebox 10 Hz Vertical linebox 12 Hz Horizontal NO
6 Horizontal linebox 10 Hz Vertical linebox 12 Hz Vertical NO
7 Horizontal linebox 10 Hz Vertical linebox 12 Hz Horizontal YES
8 Horizontal linebox 10 Hz Vertical linebox 12 Hz Vertical YES
9 Left checkerbox 6 Hz Right checkerbox 15 Hz Left NO
10 Left checkerbox 6 Hz Right checkerbox 15 Hz Right NO
11 Left checkerbox 6 Hz Right checkerbox 15 Hz Left NO
12 Left checkerbox 6 Hz Right checkerbox 15 Hz Right NO
Open in a new tab

Figure 1 illustrates the images used in the three conditions. For half the subjects, the first eight runs involved spatially overlapping images called “line boxes” that each consisted of parallel vertical or horizontal lines against a black background (Chen et al., 2003). During these runs, the two images appeared at the same location in the center of the monitor. All line boxes were about 8.5 inches tall by 8 inches wide and subtended about 10 degrees of user - centered space. The image containing horizontal lines oscillated at 10 Hz, and the image containing vertical lines oscillated at 12 Hz. This was achieved by presenting each image for two frames followed by either three or four frames without that image. During frames in which both images appeared, an image that represented the superposition of both images was presented. During runs 1, 2, 5, and 6, the line boxes used alternating dark gray and white lines (top row of Fig. 1). During runs 3, 4, 7, and 8, the line boxes used alternating colored (red or green) and dark gray lines (second row of Fig. 1). The dark gray lines used in these images appeared slightly different from the black background of the monitor to eliminate the possibility that subjects could ignore one of the line boxes by fixating on a particular region of the monitor. That is, there was no region of the monitor that displayed only one of the images.

Figure 1.

Figure 1

Open in a new tab

The top row presents the three images used in the BW linebox condition, and the middle row contains three images used in the color linebox condition. The four images used in the BW checkerbox condition are on the bottom row.

The bottom row of Fig. 1 shows the display used in runs 9−12. The left and right sides of the monitor contained a tall rectangular black and white checkerbox that oscillated between two reversed images at 6 Hz and 15 Hz respectively. The checkerboxes were separated by about 7 inches. Each checkerbox was about 2 inches wide by 8.5 inches tall and consisted of a 4×18 matrix of squares each measuring about .5 inches long. Thus, the entire display used in runs 9−12 was about 11 inches (or 12.9 degrees) wide by 8.5 inches (or 10 degrees) tall.

Seven subjects (four women) were placed in group one and used the protocol shown in Table 1. The remaining seven subjects, placed in group two, used an identical protocol except that the four runs using checkerboxes occurred first. Before each run began, subjects were asked to focus on one of the two images, called the “target image,” and maintain this focus throughout the run. Subjects were given no instructions regarding eye fixation.

2.4 Pilot testing

Six subjects participated in a pilot version of this study to determine optimal stimulus frequencies. Subjects participated in several runs comparing combinations of 6, 10, 12, 15, 20, and 30 Hz with both overlapping and nonoverlapping stimuli. These frequencies were chosen to match reports of successful experiments in the SSVEP literature (e.g., Regan, 1989; Cheng et al., 20022; Beverina et al., 2003; Gao et al., 2003; Pastor et al., 2003) within the limited set of frequencies available a monitor with a 60 Hz refresh rate. Although there were substantial differences between subjects, SSVEP differences were most apparent with checkerboxes at 6 and 15 Hz and lineboxes at 10 and 12 Hz. Pilot subjects initially reported that it was easier to ignore the red line boxes than the green ones, and hence the colors used in that display were adjusted until the pilot subjects reported that bias was eliminated.

2.5 Data analysis

In the pilot and full studies, the two one-minute runs that used the same display and target image were grouped and divided into 80 1.5-second epochs. For example, both runs in which the subject attended to horizontal colored lines – runs 3 and 7 for group one, or runs 7 and 11 for group two – were grouped and epoched. The power at each integer frequency between 1 and 65 Hz was computed for each 1.5 second epoch as the average of 5 spectra within that epoch that were each calculated using an autoregressive spectral analysis with model order 25, a window size of 0.5 seconds, and 50% overlap. This analysis produced 80 spectral estimates at each frequency and each site. The autoregressive approach was chosen because the AR approach was found more effective than an FFT with a similar SSVEP BCI approach (Lalor et al., 2005).

Data from these two runs were then compared to data from the runs that used the same display and different target image. Hence, the two runs in which horizontal colored lines were designated as the target image were compared to the two runs in which the vertical colored lines were attended. R2, the proportion of the signal variance that was accounted for by the task of attending to the horizontal and vertical lines, was computed for each electrode site and frequency. These analyses led to six topographic images, one for each of the two stimulation frequencies and the second and third harmonics of each, in which color represented R2 values at that location. These images were visually reviewed to ensure there was no excess artifact and to determine whether the R2 activity appeared consistent with SSVEP activity. R2 spectra for sites O1 and O2 were also computed.

Statistical analysis began with converting R squared values into F values using the formula: F= R2 / ((1.0 - R2) / (N-1)). N was equal to 80 since each comparison utilized one group of 80 epochs compared to another group of 80 epochs. To limit the total number of comparisons, this was only done for sites O1 and O2 for the two stimulation frequencies and the second and third harmonics of each. F values were also used to calculate p values.

Each subject's highest R2 value for each of the three conditions at the six frequencies and two sites studied was used to categorize SSVEP differences as low, moderate, high, or very high (see Table 2). Subjects rated as Low did not attain an R2 value greater than .08. Moderate subjects had a peak R2 value between .08 and .15. High subjects had a peak R2 value of .15 to .3. Subjects with a peak R2 value of greater than .3 were considered Very High. These values were chosen based on our prior work, which established that offline R2 analyses of EEG power spectra could effectively predict online control (Sheikh et al., 2003).

Table 2.

SSVEP differences across the three conditions. Subjects' SSVEP differences were grouped into one of four categories based on the maximum R2 difference produced by selective attention to one of two images. These groups correspond to a maximum R2 difference of less than .08 (low), .08−.15 (medium), .15−.30 (high), and above .30 (very high). Numbers reflect how many subjects were in each category for each display type.

Condition Low (R2 below .08) Moderate (R2 .08−.15) High (R2 .15−.30) Very High (R2 above .30)
BW Checkerbox 1 1 5 7
BW Linebox 6 4 4 0
Color Linebox 7 4 2 1
Open in a new tab

Online control was estimated using 100,000 simulations with R2 values of .08, .15, and .3. These simulations assumed Gaussian noise and continuous control in a two target task with a trial length of 1.5 seconds. R2 values of .08, .15, and .3 corresponded to accuracies of 61, 66, and 74 percent, which would yield information throughput of .04, .07, and .17 bits per trial, respectively. If an online version of this system included a 1 second delay between each trial, and hence allowed one selection every 2.5 seconds, subjects could complete 24 trials per minute. These three values would correspond to about .96, 1.68, or 4.18 bits per minute.

3. Results

3.1: Statistical analysis

Table 2 summarizes results for all subjects for each condition.

Averaged across all subjects, the three conditions (checkerbox, BW linebox, and color linebox) produced maximum R2 values of .34, .10, and .12, respectively. Subjects in group two produced slightly greater differences to the checkerbox and color linebox display, and slightly smaller differences in the BW linebox condition, than subjects in group one. Female subjects produced greater differences than male subjects in all conditions. The effects of color, group, and gender were not significant due largely to the high variance between subjects.

3.2: Examples of individual analyses for each condition

Figure 2 shows spectral power and R2 activity for one subject from the BW checkerbox condition. Like most subjects, this subject showed a stronger difference at 15 Hz than any other frequency. In this subject, differences are significant to p < .0001 over sites O1 and O2 at 6, 12, and 30 Hz. Selective attention produced broad bilateral occipital differences in the topographies, which was seen in some other subjects for this condition.

Figure 2.

Figure 2

Open in a new tab

Spectral power and R2 values for subject R. Panels A and C show the power spectra over sites O1 and O2 from 0−65 Hz when the subject focused on the 6 Hz checkerbox (solid line) or the 15 Hz checkerbox (dotted line). Panels B and D show the R2 between the two lines in panels A and C. Panel E shows a topographic map of R2 differences at all sites for that subject over six frequencies: the two stimulation frequencies (6 and 15 Hz) and the second and third harmonics of each. Please note that this figure presents data derived with different stimulation frequencies than the subsequent figures.

Figure 3 shows spectral power and R2 for one subject from the BW linebox comparison. In this subject, differences are most apparent at 10 Hz at site O2 (p < .004) and at 12 Hz at sites O1 (p < .006) and O2 (p < .014). This subject's topographies showed relatively narrow differences focused more over the central occipital area.

Figure 3.

Figure 3

Open in a new tab

Spectral power and R2 values for subject J. Panels A and C show the power spectra over sites O1 and O2 from 0−65 Hz when the subject focused on the black and white 10 Hz linebox (solid line) or the 12 Hz linebox (dotted line). Panels B and D show the R2 between the two lines in panels A and C. Panel E shows a topographic map of R2 differences at all sites for that subject over six frequencies: the two stimulation frequencies (10 and 12 Hz) and the second and third harmonics of each.

Figure 4 shows an example of spectral power and R2 activity for one subject from the color linebox comparison. In this subject, differences are most apparent at 12 and 24 Hz, both of which were significant to p < .0001 at sites O1 and O2. This subject's topographies showed fairly broad occipital differences as well as some activity from the right temporal region.

Figure 4.

Figure 4

Open in a new tab

Spectral power and R2 values for subject A. Panels A and B show the power spectra over sites O1 and O2 from 0−65 Hz when the subject focused on the colored 10 Hz linebox (solid line) or the 12 Hz linebox (dotted line). Panels C and D show the R2 between the two lines in panels A and C. Panel E shows a topographic map of R2 differences at all sites for that subject over six frequencies: the two stimulation frequencies (10 and 12 Hz) and the second and third harmonics of each.

This right temporal activity is probably EMG noise because of its spatial distribution and the fact that power spectra over that region revealed strong broadband high frequency activity consistent with EMG (Allison, 2003; Goncharova et al., 2003; McFarland et al., 2005). It is clear from both the topographies and spectra that this noise did not affect SSVEP results. The topographies show that the EMG noise does not spatially overlap the occipital activity characteristic of SSVEP, and spectra show that the peaks at 12 and 24 Hz are independent of the broadband high frequency activity.

3.3: Atypical spectra

Figure 5 shows an example of spectral power and R2 activity for a different subject from the color linebox comparison. In this figure, differences are not significant at 10 Hz or 12 Hz. However, this subject produced a high difference at 24 Hz (p < .01 at site O1; p < .0001 at site O2) This subject's topographies at 24 Hz are consistent with SSVEP activity, while almost no activity is apparent at either stimulation frequency.

Figure 5.

Figure 5

Open in a new tab

Spectral power and R2 values for subject K. Panels A and C show the power spectra over sites O1 and O2 from 0−65 Hz when the subject focused on the colored 10 Hz linebox (solid line) or the 12 Hz linebox (dotted line). Panels B and D show the R2 between the two lines in panels A and C. Panel E shows a topographic map of R2 differences at all sites for that subject over six frequencies: the two stimulation frequencies (10 and 12 Hz) and the second and third harmonics of each.

This phenomenon is not unusual among subjects in this study. Two subjects showed a greater difference at the second or third harmonic than the stimulation frequency in all three conditions. Four other subjects showed a greater difference at one of these harmonics than the stimulation frequency in one or two conditions. Three subjects showed differences at frequencies that were not related to the stimulation frequency.

Although SSVEP BCI papers typically use site O1 or O2 for control, other work (Beverina et al., 2003) suggests that sites PO7 or PO8 may be best for an SSVEP BCI. Hence, the analyses were repeated with sites PO7 and PO8 to determine whether selective attention produced greater differences there. In five subjects, the R squared differences resulting from selective attention were larger over PO7 or PO8 than at O1 or O2 for at least one of the frequencies studied in one of the conditions. However, the site that produced the greatest difference was O1 or O2 for all subjects except one.

3.4: Questionnaires

All subjects drank at least one caffeinated beverage per day (mean = 2.1) and reported little or no use of alcohol, tobacco, or other recreational drugs. Several questions could be answered on a 1−5 scale, with a 1 meaning a strong “no” and a 5 meaning a strong “yes.” When asked if they felt tired after the study, subjects generally said no (mean = 1.83). Subjects said they could perform additional runs (mean = 3.83).

There was no significant correlation between EEG measures and gender, age, substance use, or any other questions asked in the questionnaire except one. All subjects reported that they did not play video games, with three exceptions. These three subjects each reported playing games for either one or two hours a day. Subject K had a high difference in both linebox conditions and a very high difference in the checkerbox condition. Subject J had a high difference in the color linebox condition, low difference in the BW linebox condition, and moderate difference in the BW checkerbox condition. Subject A had a high difference in the BW linebox and checkerbox conditions, and a very high difference in the color linebox condition.

4. Discussion

This study explored SSVEP activity elicited by attention to one of two images. In about half the subjects, selective attention to one of two overlapping images produced SSVEP differences robust enough to allow effective communication in an online BCI (Sheikh et al., 2003). Further research is warranted to validate an online adaptation of this BCI approach in a real world environment, ideally in typical users' homes.

4.1: Display type and gaze shifting

Subjects' SSVEPs resulting from selective attention to colored lines were similar to those resulting from selective attention to black and white lines. Therefore, this form of stimulus should work with subjects who have impaired color vision. However, color may affect results with different SSVEP displays (Mullen, 1985; Regan, 1989; Arakawa et al., 1999).

Subjects produced a much stronger response to the checkerboxes than the lineboxes, probably because the checkerboxes did not overlap, and thus subjects could shift gaze between images. Gaze shifting would likely improve performance with an SSVEP BCI, but is not required, at least for some users (Kelly et al., 2005a, 2005b; Lalor et al., 2005). The importance of gaze shifting depends heavily on the display and task. For example, if targets are numerous or located outside the fovea, gaze shifting may be essential. The need for gaze shifting may also depend on other factors such as the user's head position and attentional abilities, training, lighting, equipment, analysis parameters, fatigue, medication, and motivation. The latter comments probably apply to P300 BCIs as well, since they also rely on selective attention. Although P300 BCIs are considered independent3, this assumption has been questioned (Allison, 2003; Kaper et al., 2004).

4.2: Inter–subject differences and implications for BCIs

The considerable variety in SSVEP activity across subjects suggests that SSVEP BCIs should customize parameters used to translate SSVEP activity into control for each subject based on initial screening (e.g., Middendorf et al., 2000; Beverina et al., 2003). Figure 5 shows that the same frequencies that other subjects might use for control, 10 or 12 Hz, would not have allowed this subject to control a BCI. While other BCI studies have also reported that subjects may show stronger activity over the first, second, or even third harmonic (Gao et al., 2003; Müller-Putz et al., 2005), it is unclear whether this difference occurs due to natural variation among subjects, display and task parameters, different attentional strategies, or other factors.

The stimulation frequencies used in this study were based on pilot studies, which did not yield universal results across all subjects. Some subjects performed best with checkerboxes at 10 and 15 Hz, as in Beverina et al. (2003). Further research should identify optimal stimulation frequencies and groups of frequencies.

The observation that subjects who play video games every day perform better on a visual attention task is consistent with other reports (Green and Bavelier, 2003; Allison and Pineda, 2006), and has two implications for BCIs that utilize selective attention. First, subjects who have a background playing video games or performing similar activities might be better suited to certain types of BCIs. Second, subjects can be trained to perform better on tasks of visual attention. Thus, subjects who did not produce SSVEP differences robust enough for effective communication might be trained to perform better. These implications should be explored further with more gamers and more rigorous evaluation of their gaming backgrounds.

The questionnaires also suggested that subjects interpreted instructions differently. Although the request to focus attention on a stimulus may seem straightforward, it is not. Subjects were asked how they focused on one image and ignored the other. Some subjects said they focused on a specific part of the image, some looked for imaginary movement in the image, some looked at the whole image, and two (who did not produce strong differences) said they “just zoned out.” Thus, the attentional strategy that subjects use may affect their SSVEP activity within and perhaps across recording sessions. Allison et al. (2006) reported that one reason that SSVEP performance improved with practice was development of new attentional strategies. Similarly, subjects who utilized first-person movement imagery learned mu BCI control better than subjects who were told to adopt third-person movement imagery (Neuper et al., 2005).

The inter–subject differences in numerous measures relevant to BCI control suggest that parameters should be customized to each subject like most other types of BCIs. Initial runs(s) may characterize the best site(s), stimulus presentation frequency or frequencies, frequency or frequencies used to detect differences, power threshold necessary for control, and other parameters. If EEG activity is not consistent within or across sessions, adaptive mechanisms will need to be developed and incorporated. Without subject customization, performance will be excellent in some subjects but poor in others.

4.3: Implications for online control

The principal aim of this study was to explore the hypothesis that some displays, which do not allow gaze shifting and might be adapted for online BCI systems, could elicit SSVEP activity sufficient for effective communication. Online control might be more or less effective than the estimations provided above. Subjects in this study did not receive feedback reflecting their SSVEP activity nor how this activity might effect control, which might improve performance. Training with SSVEP BCIs (Middendorf et al., 2000; Allison et al., 2006) or other demanding visual attention tasks such as FPS games (Allison and Pineda, 2003, 2006; Green and Bavelier, 2003) should also help. However, subjects who use a BCI online would probably need to switch attention between targets more rapidly than subjects in this study, who had 30−60 seconds to do so. The additional mental activity needed for an online system, such as processing a moving cursor or identifying errors, could distract subjects and impair performance. Despite these concerns, R2 values derived from data collected offline have been used to infer online BCI control in prior work (Sheikh et al., 2003), and extensive and ongoing research in the Wadsworth research laboratory has shown that such offline estimates are reasonably accurate predictors of online control.

Both the trial length and inter-trial delay used to estimate control are realistic values for a 2 target task based on continuous spectral analysis of EEG data, with both mu (McFarland et al., 1997) and SSVEP (Allison et al., 2006) BCI systems. Allison et al. (2006) demonstrated average accuracy above 80% in an online SSVEP BCI using a 2 target task, with trial times shorter than 2.5 seconds and more than 24 selections per minute, despite extreme environmental noise and extensive distractions. However, gaze shifting was allowed in that study. Other work (Kelly et al., 2005a, 2005b) further supports the argument that covert attention can be used for online SSVEP BCI control, although these studies used substantially different displays (which did not overlap) and a different paradigm.

The information throughput estimated in this article compares poorly with some modern BCIs. Other articles have reported information throughput on the order of 30−40 bits per minute (for mu BCIs; Wolpaw and McFarland, 2004) or 68 bits per minute (for SSVEP BCIs; Gao et al. 2003). However, these articles both allowed gaze shifting and used an elite subset of subjects. Subjects run at the Wadsworth lab are first prescreened to evaluate their mu activity, and fewer than 20% are selected for further training (Vaughan, personal communication). Of these, only the best subjects were selected for the 2D training reported in Wolpaw and McFarland (2004). The performance reported in Gao et al. (2003) was from the best subject found during that group's prior work (Cheng et al., 2002), which reported information throughput as low as .76 bits per minute. Indeed, very many BCI articles prescreen subjects, exclude subjects who perform poorly, or fail to report these or other selection factors that might result in a much lower mean information throughput (Jackson et al., 2006).

A remaining question is whether information throughput as low as .96 bits per minute does in fact constitute effective communication. This might allow a patient to answer one yes/no question per minute or spell one letter every 5 minutes. Articles by groups with patient experience have described even slower systems that patients chose to continue using (e. g., Kübler et al., 2001; Kübler et al., 2005). However, a brief unpublished anecdote might provide a more direct response to this question.

Patient A1 was a retinal surgeon before developing ALS. Between 2002 and 2006, the first and fifth authors each made several visits to his home to try to develop an effective BCI for him. These efforts were not consistently successful, nor were attempts by at least two other BCI research groups (e. g., Birbaumer et al., 2003). The fifth author later developed a galvanic skin response (GSR) communication system that did work, although it was very slow, ranged from 60−80% correct, and allowed effective information throughput more than an order of magnitude slower than .96 bits per minute (Moore and Dua, 2004). A1 rarely communicated, leading to concern about his remaining cognitive abilities. One day, A1 spelled “EYE.” On subsequent days, he spelled “R EYE,” then his night nurse's name with an X before and after it, then the name of a medication. His mother installed a hidden camera and summoned his doctor. It was found that A1's night nurse neglected to apply eyedrops. A1's doctor found an infection in his right eye, prescribed the same medication that A1 had suggested, and stated that he had been in extreme pain and would have lost vision in his right eye without this medication.

4.4: Future directions with SSVEP BCIs

Many other important questions involving practical long-term use of SSVEP BCIs have not been addressed. Training with SSVEP BCIs can improve performance (Middendorf et al., 2000; Allison et al., 2006), but the best feedback type, training schedule, and other parameters are not known. Training with other tasks requiring selective attention, such as playing certain types of computer games, may also improve performance with SSVEP or other BCIs (Green and Bavelier, 2003; Allison and Pineda, 2006). Subjects might learn to use an SSVEP or other BCI effectively while using another interface (perhaps a second type of BCI) or otherwise multitasking.

Cautious optimism about the future of SSVEP BCIs is warranted. They can exhibit good information throughput relative to other BCIs (Sutter, 1992; Cheng et al., 2002; Gao et al., 2003; Müller-Putz et al., 2005). They can operate in challenging environments with uncontrolled distraction and electrical noise (Cheng et al., 2002; Wang et al., 2004; Lalor et al., 2005; Allison et al., 2006; Trejo et al., 2006) and hence could work well in homes or hospital settings. The approach used here does not produce significant fatigue. People can voluntarily modulate SSVEP activity without shifting gaze using a display and task that could be adapted to a BCI. Many avenues toward improvement that have been successful with other BCIs, such as improved referencing, filtering, subject and classifier training, task and display optimization, noise rejection, incorporation of additional signal features, spectral analysis parameters, and other parameters have not yet been fully explored (McFarland et al., 1997, 1998, 2005, 2006; Wolpaw et al., 2002; Allison, 2003; Birbaumer et al., 2003; Kübler and Neumann, 2005; Neuper et al., 2005; Allison and Pineda, 2006). Initial efforts have suggested that some of these avenues may be useful in SSVEP BCI systems (Wang et al., 2004; Kelly et al., 2005a, 2005b; Lalor et al., 2005; Müller-Putz et al., 2005; Trejo et al., 2006).

Dependence on muscle control is best regarded as a facet of individual BCIs, rather than a whole category of them. User and environmental factors are also important. Hence, the labels “dependent” and “independent” might be best regarded not as absolutes, but endpoints of a continuum. Some SSVEP BCIs should work with severely disabled users unable to control gaze.

Acknowledgements

This work was supported in part by National Institutes of Health Grant EB00856 (National Institute of Biomedical Imaging and Bioengineering and National Institute of Neurological Disorders and Stroke). The authors wish to thank Christopher Agocs, Dan Ratanasit, Luke McCampbell, and Steve Hillyard for technical advice, and Theresa Vaughan for comments on design and writing.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1

Sutter's approach uses m – sequence encoding, which is not a steady state stimulus and does not produce a classic steady state response. However, his 1992 article is typically grouped with SSVEP BCIs since this approach is somewhat similar.

2

Cheng et al. (2002) does not describe the stimulation frequencies used; they were in the range of 7−20 Hz (S. Gao, personal communication, 2005).

3

Donchin et al., (2000) states that a P300 BCI needs an intertrial interval to allow for “shifting gaze between characters.” This suggests that Donchin considers his P300 BCI dependent. However, Donchin meant to say shifting attention, not shifting gaze, and considers P300 BCIs independent (Donchin, personal communication, 2002).

References

  1. Allison BZ. P3 or not P3: Toward a Better P300 BCI. La Jolla; UC San Diego: 2003. [Google Scholar]
  2. Allison BZ, Pineda JA. ERPs evoked by different matrix sizes: Implications for a brain computer interface (BCI) system. IEEE Trans Neural Syst Rehabil Eng. 2003 Jun;11(2):110–113. doi: 10.1109/TNSRE.2003.814448. [DOI] [PubMed] [Google Scholar]
  3. Allison BZ, Pineda JA. Effects of SOA and flash pattern manipulations on ERPs, performance, and preference: Implications for a BCI system. International Journal of Psychophysiology. 2006 Feb;59(2):127–140. doi: 10.1016/j.ijpsycho.2005.02.007. [DOI] [PubMed] [Google Scholar]
  4. Allison BZ, Boccanfuso JB, Agocs C, McCampbell LA, Leland DS, Gosch C, Moore Jackson M. Sustained use of an SSVEP BCI under adverse conditions. Journal of Cognitive Neuroscience Supplement. 2006:129. [Google Scholar]
  5. Allison BZ, Wolpaw EW, Wolpaw JR. Poll E, editor. Brain computer interface systems: Progress and prospects. British review of medical devices. Jul;4(4):463–474. doi: 10.1586/17434440.4.4.463. [DOI] [PubMed] [Google Scholar]
  6. Arakawa K, Tobimatsu S, Tomoda H, Kira J, Kato M. The effect of spatial frequency on chromatic and achromatic steady-state visual evoked potentials. Clinical Neurophysiology. 1999 Nov;110(11):1959–1964. doi: 10.1016/s1388-2457(99)00139-x. [DOI] [PubMed] [Google Scholar]
  7. Beverina F, Palmas G, Silvoni S, Piccione F, Giove S. User adaptive BCIs: SSVEP and P300 based interfaces. PsychNology. 2003;1(4):331–354. [Google Scholar]
  8. Birbaumer N, Hinterberger T, Kübler A, Neumann N. The thought-translation device (TTD): neurobehavioral mechanisms and clinical outcome. IEEE Trans Neural Syst Rehabil Eng. 2003 Jun;11(2):120–123. doi: 10.1109/TNSRE.2003.814439. [DOI] [PubMed] [Google Scholar]
  9. Chen YQ, Seth AK, Gally JA, Edelman GM. The power of human brain magnetoencephalographic signals can be modulated up or down by changes in an attentive visual task. Proceedings of the National Academy of Sciences of the United States of America. 2003 Mar;100(6):3501–3506. doi: 10.1073/pnas.0337630100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cheng M, Gao X, Gao S, Xu D. Design and implementation of a brain-computer interface with high transfer rates. IEEE Trans Biomed Eng. 2002 Oct;49(10):1181–1186. doi: 10.1109/tbme.2002.803536. [DOI] [PubMed] [Google Scholar]
  11. Cook A, Hussey S. Assistive technologies: Principles and practice. Second ed. Elsevier; New York: 2002. [Google Scholar]
  12. Gao X, Xu D, Cheng M, Gao S. A BCI-based environmental controller for the motion-disabled. IEEE Trans Neural Syst Rehabil Eng. 2003 Jun;11(2):137–140. doi: 10.1109/TNSRE.2003.814449. [DOI] [PubMed] [Google Scholar]
  13. Golla H, Thier P, Haarmeier T. Disturbed overt but normal covert shifts of attention in adult cerebellar patients. Brain. 2005 Jul;128:1525–1535. doi: 10.1093/brain/awh523. [DOI] [PubMed] [Google Scholar]
  14. Goncharova I, McFarland DJ, Vaughan TM, Wolpaw JR. EMG contamination of EEG: spectral and topographical characteristics. Clinical Neurophysiology. 2003 Sep;114(9):1580–1593. doi: 10.1016/s1388-2457(03)00093-2. [DOI] [PubMed] [Google Scholar]
  15. Green CS, Bavelier D. Action video game modifies visual selective attention. Nature. 2003 May;423(6939):534–537. doi: 10.1038/nature01647. [DOI] [PubMed] [Google Scholar]
  16. Jackson MM, Mason SG, Birch GE. Analyzing trends in brain interface technology: a method to compare studies. Ann Biomed Eng. 2006 May;34(5):859–78. doi: 10.1007/s10439-005-9055-7. [DOI] [PubMed] [Google Scholar]
  17. Kaper M, Meinicke P, Grossekathoefer U, Lingner T, Ritter H. BCI Competition 2003--Data set IIb: support vector machines for the P300 speller paradigm. IEEE Trans Biomed Eng. 2004 Jun;51(6):1073–1076. doi: 10.1109/TBME.2004.826698. [DOI] [PubMed] [Google Scholar]
  18. Kelly SP, Lalor EC, Finucane C, McDarby G, Reilly RB. Visual spatial attention control in an independent brain-computer interface. IEEE Trans Biomed Eng. 2005 Sep;52(9):1588–1596. doi: 10.1109/TBME.2005.851510. [DOI] [PubMed] [Google Scholar]
  19. Kelly SP, Lalor EC, Reilly RB, Foxe JJ. Visual spatial attention tracking using high-density SSVEP data for independent brain-computer communication. IEEE Trans Neural Syst Rehabil Eng. 2005 Jun;13(2):172–178. doi: 10.1109/TNSRE.2005.847369. [DOI] [PubMed] [Google Scholar]
  20. Kübler A, Kotchoubey B, Kaiser J, Wolpaw JR, Birbaumer N. Brain-computer communication: unlocking the locked in. Psychol Bull. 2001 May;127(3):358–375. doi: 10.1037/0033-2909.127.3.358. [DOI] [PubMed] [Google Scholar]
  21. Kübler A, Neumann N. Brain-computer interfaces - the key for the conscious brain locked into a paralyzed body. Boundaries of Consciousness: Neurobiology and Neuropathology. 2005:513–525. doi: 10.1016/S0079-6123(05)50035-9. [DOI] [PubMed] [Google Scholar]
  22. Lalor EC, Kelly SP, Finucane C, Burke R, Smith R, Reilly RB, et al. Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment. Eurasip Journal on Applied Signal Processing. 2005;(19):3156–3164. [Google Scholar]
  23. Mangun GR, Buck LA. Sustained visual-spatial attention produces costs and benefits in response time and evoked neural activity. Neuropsychologia. 1998 Mar;36(3):189–200. doi: 10.1016/s0028-3932(97)00123-1. [DOI] [PubMed] [Google Scholar]
  24. McFarland D, McCane LM, David SV, Wolpaw JR. Spatial filter selection for EEG-based communication. Electroencephalography and Clinical Neurophysiology. 1997;103(3):186–194. doi: 10.1016/s0013-4694(97)00022-2. [DOI] [PubMed] [Google Scholar]
  25. McFarland D, McCane LM, Wolpaw JR. EEG-based communication and control: short-term role of feedback. IEEE Trans Rehabil Eng. 1998;6(1):7–11. doi: 10.1109/86.662615. [DOI] [PubMed] [Google Scholar]
  26. McFarland DJ, Sarnacki WA, Vaughan TM, Wolpaw JR. Brain-computer interface (BCI) operation: signal and noise during early training sessions. Clin Neurophysiol. 2005 Jan;116(1):56–62. doi: 10.1016/j.clinph.2004.07.004. [DOI] [PubMed] [Google Scholar]
  27. McFarland DJ, Krusienski DJ, Wolpaw JR. Brain-computer interface signal processing at the Wadsworth Center: mu and sensorimotor rhythms. In: Neuper C, Klimesch, editors. Progress in brain research: Event-related Dynamics of brain oscillations. Elsevier; The Netherlands: pp. 411–419. [DOI] [PubMed] [Google Scholar]
  28. Moore M, Dua U. A galvanic skin response interface for people with severe motor disabilities; Proceedings of the ACM conference on assistive technology (ASSETS 2004); Atlanta. 2004. [Google Scholar]
  29. Middendorf M, McMillan G, Calhoun G, Jones KS. Brain-computer interfaces based on the steady-state visual-evoked response. IEEE Trans Rehabil Eng. 2000 Jun;8(2):211–214. doi: 10.1109/86.847819. [DOI] [PubMed] [Google Scholar]
  30. Mullen KJ. The contrast sensitivity of human color vision to red-green and blue-yellow chromatic gratings. Journal of Physiology. 1985;359:381–400. doi: 10.1113/jphysiol.1985.sp015591. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Müller MM, Hillyard S. Concurrent recording of steady-state and transient event-related potentials as indices of visual-spatial selective attention. Clinical Neurophysiology. 2000 Sep;111(9):1544–1552. doi: 10.1016/s1388-2457(00)00371-0. [DOI] [PubMed] [Google Scholar]
  32. Müller MM, Malinowski P, Gruber T, Hillyard SA. Sustained division of the attentional spotlight. Nature. 2003 Jul;424(6946):309–312. doi: 10.1038/nature01812. [DOI] [PubMed] [Google Scholar]
  33. Müller MM, Picton TW, Valdes-Sosa P, Riera J, Teder-Salejarvi WA, Hillyard SA. Effects of spatial selective attention on the steady-state visual evoked potential in the 20−28 Hz range. Cognitive Brain Research. 1998 Apr;6(4):249–261. doi: 10.1016/s0926-6410(97)00036-0. [DOI] [PubMed] [Google Scholar]
  34. Müller-Putz GR, Scherer R, Brauneis C, Pfurtscheller G. Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components. J Neural Eng. 2005;2(3):123–130. doi: 10.1088/1741-2560/2/4/008. [DOI] [PubMed] [Google Scholar]
  35. Neuper C, Scherer R, Reiner M, Pfurtscheller G. Imagery of motor actions: Differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Brain Res Cogn Brain Res. 2005 Dec;25(3):668–677. doi: 10.1016/j.cogbrainres.2005.08.014. [DOI] [PubMed] [Google Scholar]
  36. Pastor MA, Artieda J, Arbizu J, Valencia M, Masdeu JC. Human cerebral activation during steady-state visual-evoked responses. Journal of Neuroscience. 2003 Dec;23(37):11621–11627. doi: 10.1523/JNEUROSCI.23-37-11621.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Regan D. Human Brain Electrophysiology. Elsevier; New York: 1989. [Google Scholar]
  38. Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR. BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Trans Biomed Eng. 2004 Jun;51(6):1034–1043. doi: 10.1109/TBME.2004.827072. [DOI] [PubMed] [Google Scholar]
  39. Scharbrough F, Chatrian G-E, Lesser R, Luders H, Nuwer M, Picton T. Guidelines for standard electrode position nomenclature. Am. EEG Soc.; Bloomfield, CT: 1990. [Google Scholar]
  40. Sheikh H, McFarland DJ, Sarnacki WA, Wolpaw JR. Electroencephalographic (EEG) - based communication: EEG control versus system performance in humans. Neurosci Lett. 2003 Jul 17;345(2):89–92. doi: 10.1016/s0304-3940(03)00470-1. [DOI] [PubMed] [Google Scholar]
  41. Sutter E. The brain response interface: Communication through visually - induced electrical brain responses. Journal of Microcomputer Applications. 1992;15:31–45. [Google Scholar]
  42. Trejo LJ, Rosipal R, Matthews B. Brain-computer interfaces for 1-D and 2-D cursor control: Designs using volitional control of the EEG spectrum or steady-state visual evoked potentials. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2006 Jun;14(2):225–229. doi: 10.1109/TNSRE.2006.875578. [DOI] [PubMed] [Google Scholar]
  43. Van Voorhis S, Hillyard SA. Visual evoked potentials and selective attention to points in space. Percept Psychophysics. 1977;22:54–62. [Google Scholar]
  44. Wang Y, Zhang Z, Gao X, Gao S. Lead selection for SSVEP-based brain-computer interface; Proceedings of the 26th annual international conference of the IEEE EMBS; San Francisco. 2004; [DOI] [PubMed] [Google Scholar]
  45. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces for communication and control. Clin Neurophysiol. 2002 Jun;113(6):767–791. doi: 10.1016/s1388-2457(02)00057-3. [DOI] [PubMed] [Google Scholar]

RESOURCES