A Tactile P300 Brain-Computer Interface

Abstract

In this study, we investigated a Brain-Computer Interface (BCI) based on EEG responses to vibro-tactile stimuli around the waist. P300 BCIs based on tactile stimuli have the advantage of not taxing the visual or auditory system and of being potentially unnoticeable to other people. A tactile BCI could be especially suitable for patients whose vision or eye movements are impaired. In Experiment 1, we investigated its feasibility and the effect of the number of equally spaced tactors. Whereas a large number of tactors is expected to enhance the P300 amplitude since the target will be less frequent, it could also negatively affect the P300 since it will be difficult to identify the target when tactor density increases. Participants were asked to attend to the vibrations of a target tactor, embedded within a stream of distracters. The number of tactors was two, four or six. We demonstrated the feasibility of a tactile P300 BCI. We did not find a difference in SWLDA classification performance between the different numbers of tactors. In a second set of experiments we reduced the stimulus onset asynchrony (SOA) by shortening the on- and/or off-time of the tactors. The SOA for an optimum performance as measured in our experiments turned out to be close to conventional SOAs of visual P300 BCIs.

Introduction

A brain-computer interface (BCI), the first of which was developed by Vidal (1973, 1977), enables a user to communicate with the world through her or his brain signals alone. One of the most well-known BCIs was based on the P300 event-related potential (Farwell and Donchin, 1988). The P300 is a positive deflection in EEG that occurs around 300 ms after a target stimulus has been presented. In the P300 speller by Farwell and Donchin (1988), rows and columns of a matrix consisting of letters were sequentially flashed in random order. Every time the row or column was flashed that contained the target symbol that the user wanted to spell, a P300 occurred. In this way, users could spell words. A P300-speller could be especially helpful for, and has been tested on paralyzed users (Hoffmann et al., 2008; Nijboer et al., 2008). More than two decades later, P300s are still popular to drive BCIs since they are relatively robust and easy to detect (Donchin et al., 2000).

The vast majority of P300 based BCIs use visual stimuli to elicit P300s. Recently (off-line) auditory P300 BCIs have also been developed (Sellers and Donchin, 2006; Furdea et al., 2009; Schreuder et al., 2010). The fundamental research on P300s focuses on visual and auditory stimulation. For BCI purposes tactile stimuli can be an interesting alternative. Tactile stimuli can be delivered by tactors that can be hidden under the user's clothes, making the device go unnoticed to others. Using tactile stimuli will also keep the eyes and ears of the user free. Another important advantage is that a tactile P300 BCI can be used by patients with impaired vision or eye movements. This is especially relevant because one of the main potential groups of BCI users, patients with ALS, experience problems with their eyes as the disease progresses. A tap on your shoulder seems to automatically draw attention into the direction of the person tapping. Consistent with this, tactors applied around the waist have proven to be successful as navigation display, both for pedestrians (van Erp et al., 2005) and vehicle drivers (van Erp and van Veen, 2004) and can even result in better navigation performance than comparable visual displays. Tactile stimuli applied around the waist may therefore be a good choice for navigation BCIs (van Erp, 2005b, see also Thurlings et al., in press). Users can focus their attention on the tactile stimulus that corresponds to the direction in which they want to move, possibly eliciting a useful P300 signal. Although tactile P300 BCIs have not been studied before, the feasibility of a BCI based on steady-state somatosensory evoked potentials has been demonstrated (Müller-Putz et al., 2006) and the tactile modality has been used as a BCI feedback channel (Chatterjee et al., 2007; Cincotti et al., 2007).

Previous research has shown that tactile stimuli can elicit P300s (Satomi et al., 1995; Ito and Takamatsu, 1997; Nakajima and Imamura, 2000). In these studies, electrical and mechanical stimuli were delivered to the hands and wrists using relatively long inter stimulus intervals (around 2 s in the study by Nakajima and Imamura). Obviously, for BCI purposes, short stimulus durations and inter stimulus intervals are required. The finding that P300s can be elicited by tactile stimulation of the hands does not guarantee that tactile stimulation of the torso easily elicits P300s as well because the sensitivity threshold and the spatial resolution of the torso is lower than that of the hands (Wilska, 1954).

Recently, we investigated whether quickly presented vibro-tactile stimuli at different locations around the waist can elicit robust P300s (Brouwer et al., submitted). We presented participants bursts of vibration delivered by one of eight tactors around the participants’ waist. Vibration duration was 188 ms and vibrations were interleaved by 438 ms breaks. In other conditions, we presented analogous visual stimuli consisting of flashed circles on a monitor in a schematic drawing of the tactor layout. Participants attended to the vibrations and/or flashes of the target tactor presented in a rapid stream of stimuli that also contained the seven distracters. The target was always the front tactor. In addition, this tactor physically stood out from the distracter: the distance to the distracters was larger than the distance between the distracters themselves, and the stimulus intensity of the target was stronger than of the distracters (in the tactile condition, a tactor above and below the target tactor vibrated simultaneously with the target, whereas in the visual condition the target circle was enlarged when flashed). By standing out from the distracters, the target probably drew attention by itself. The results demonstrated that the amplitudes of P300s elicited by tactile and visual stimuli depended on electrode site, but on the whole, they were equally high. Classification accuracy (how well an algorithm can identify the target stimulus) as determined off-line was also similar for visual and tactile stimuli. Whereas one or the other modality could probably have been favored by choosing different experimental parameters, we concluded that it is possible to elicit reliable P300s with quickly presented vibro-tactile stimuli around the waist.

The present study aimed at extending these proof-of-concept findings to a more realistic BCI setting. Whereas the oddball task as used in the previous experiment more or less guaranteed the attention of the participant to the target, it could not be generalized to a BCI where the user can choose the target. Furthermore, reliable online classification of tactile P300s remained to be demonstrated.

Besides tackling the issues mentioned above, we also wanted to explore the effect of varying the number of stimuli (i.e. the number of tactors around the waist) and the effect of varying stimulus timing on classification accuracy and rate of information transfer.

With a large number of tactors, it will be difficult to identify the individual targets because they will be close to the distracters. It has been shown that a low discriminability reduces the P300 amplitude (Comercho and Polich, 1999). On the other hand, with a large number of tactors, the target tactor is activated relatively infrequent, which positively affects the P300 (Polich et al., 1991), possibly via a longer target-to-target interval (Gonsalvez and Polich, 2002; Croft et al., 2003; Martens et al., 2009). In case of few tactors, the discriminability is high but the probability of target presentation is also high. In order to guide the development of a tactile BCI for navigation, we are interested in the relative importance or tradeoff of these potential effects on type classification accuracy and rate of information transfer within the context of our vibro-tactile stimuli around the waist.

Presenting stimuli quickly after each other shortens the time needed to go through the several different options. This means that the BCI could reach its decision as to which option the user wants to select sooner. However, increasing presentation speed will also likely decrease the quality of the P300 and decrease classification performance. Firstly, refractory effects could result in a lower P300 (Gonsalvez and Polich, 2002; Martens et al., 2009). Secondly, the quicker stimuli are presented, the more stimuli will be reflected in the EEG of the epochs analyzed by the classification algorithm, probably making successful classification harder (Martens et al., 2009). Finally, there will be a limit to presentation speed due to human limits in perception and attention. Several studies investigated the effect of different stimulus timing parameters on the P300. Generally, more time between stimuli seem to result in larger P300 amplitudes. However, this is not always found (Sellers et al., 2006) and the exact timing variable(s) underlying possible P300 effects are still not clear. In most studies about timing, the stimulus onset asynchrony (SOA) was varied, i.e. the time interval in between the start of one (non) target stimulus and the next (Farwell and Donchin, 1988; Polich et al., 1991; Allison and Pineda, 2006). Increasing the SOA while holding everything else constant will also increase the time interval between target stimuli (target-to-target interval), which may be the key variable affecting the P300 amplitude (Gonsalvez and Polich, 2002; Croft et al., 2003; Martens et al., 2009). Studies that show an increasing P300 with decreasing relative frequency of target presentation amongst non-targets (Polich et al., 1991; Cahill and Polich, 1992; Allison and Pineda, 2003; Sellers et al., 2006) also affect the target-to-target interval. Regardless of the specific underlying cause, decreasing the SOA (which decreases the target-to-target interval if the target probability stays the same) could cause a decrease in the P300 amplitude. Nonetheless, with the resulting shorter required time to go through the different user options, the rate of information transfer may increase. Many P300 BCI studies successfully used visual stimuli with SOAs ranging from as little as 125–500 ms (Farwell and Donchin, 1988; Serby et al., 2005; Sellers et al., 2006; Mell et al., 2008; Martens et al., 2009). We are interested in whether similar SOAs work for tactile stimuli as well.

In sum, the present study further evaluated (the feasibility of) a tactile P300 BCI. The tactile stimuli were delivered by several tactors placed around the waist. Classification for all experiments in this study was performed online, and the target stimulus did not stand out from the distracters. In Experiment 1, we varied the number of tactors. In Experiment 2 we reduced the SOA with respect to Experiment 1, both by reducing the vibration duration and the breaks in between vibration. Since it turned out that this was possible without sacrificing classification accuracy, we further reduced the SOA in Experiment 3.

Materials and Methods

Experiment 1

Participants

For all experiments described, participants were recruited from a pool of potential participants maintained by TNO Human Factors. Most of them study at a nearby university. In addition, some TNO interns (also students) participated. For each experiment, participants gave informed consent before the experiment started. Approval of the experiments was acquired from the local ethical committee.

Eleven healthy participants (six female) between 20 and 27 years old volunteered to participate in Experiment 1. None of them had participated in any other BCI experiment before.

Stimuli

Participants wore an adjustable vest over their clothes lined with 62 vibrating elements called tactors. For this study, we used one row of 12 equally spaced tactors around the participant's waist, approximately at the height of the navel. Different combinations of tactors were used in different experimental conditions. Tactors vibrated successively with a vibration time (on-time) of 188 ms and breaks in between vibrations (off-time) of 438 ms. The tactors were custom built. They consisted of plastic cases with a contact area of 1 × 2 cm, containing 160 Hz vibrating motors. (TNO, The Netherlands, model JHJ-3; see van Erp et al. (2007) for comparable equipment and tactor layout). During the experiment, participants viewed a dimmed LCD (Dell 20 inch flatpanel, refresh rate 75 Hz) displaying instructions and a fixation cross. To prevent participants from using auditory cues, we played pink noise from two speakers, one placed left and the other placed right on the desk in front of the participant.

EEG Recording

EEG activity was recorded at the Fz, Cz, Pz, Oz, P3, P4, PO7 and PO8 electrode sites of the 10-20 system (Jasper, 1958) using electrodes mounted in an EEG cap (g.tec medical engineering GmbH). A ground electrode was attached to the forehead. The EEG electrodes were referenced to linked mastoid electrodes. The impedance of each electrode was below 5 kΩ. Data were sampled with a frequency of 256 Hz and filtered before storage by a 0.1-Hz high pass-, a 60-Hz low pass- and a 50-Hz notch filter (USB Biosignal Amplifier, g.tec medical engineering GmbH). The experiment (stimulus presentation and data recording) was controlled by a combination of custom built software and BCI2000 (Schalk et al., 2004).

Conditions

We used four experimental conditions called Two, Four, Six and FourPlus. Conditions Two, Four and Six refer to the number of equally distributed tactors used (see Figure 1 for a schematic indication of their location). In these conditions, the number of distracters was respectively one, three and five. The probability of target presentation decreased over these conditions (expected to enhance the P300), whereas the distance between the tactors and therewith the spatial discriminability between target and distracters decreased (expected to reduce the P300). The FourPlus condition served as a comparison for condition Four and Six. In this condition, the same tactors were used as in condition Four (see Figure 1), that is, the discriminability in Four and FourPlus was the same. However, by having the front and back tactor vibrate twice as often as the other tactors and by only designating the left and right tactors as targets, we made the target probability equal to that in condition Six. This will allow us to directly evaluate the relative importance of the factors discriminability and target probability: when discriminability plays a more important role, condition Four and FourPlus will produce similar results; when target probability is more important, condition Six and FourPlus will produce similar results.

Figure 1.

Schematic overview of the tactor layout (top view) in the different conditions of Experiment 1. The tactors used are colored gray whereas the inactive tactors are white. In the FourPlus condition, only the gray tactors could be targets. The black tactors vibrated twice as often as the gray tactors.

Design

The order of conditions was randomized for each participant. For every condition, there were three blocks: two training blocks followed by one test block. After the training blocks, a classification algorithm was applied to the collected data. The resulting model was used in the test block to give participants feedback about the tactor that the algorithm classified as the target. Aside from this feedback, the training and test blocks were the same (please note that training refers to training the classifier, not training the participant).

A block consisted of six sequences of stimuli. Each sequence started with one of the tactors being designated as the target. In condition Six, each of the tactors served as a target once in one block, starting with the front-left tactor and every time going to the next tactor counter-clockwise. In condition Four, the front target and the one counter clockwise served as targets again after the others had been a target. In condition Two and FourPlus, the left and right tactor were designated as a target three times each, alternating and starting with the tactor at the left.

Each of the six sequences consisted of activating each tactor in that condition (the target and the one, three or five distracters) 10 times, in random order. With a stimulus on-time of 188 ms and a stimulus off-time of 438 ms this resulted in mean target-to-target intervals of 1252 ms (condition Two), 2504 ms (condition Four) and 3756 ms (conditions Six and FourPlus).

Task and procedure

Before the experiment started, the complete procedure was explained to the participants, with their task being to concentrate on the target by counting the number of times it occurred, and to ignore the distracters. They were further asked to fixate the fixation cross displayed on the screen, to blink as little as possible and to limit any other movements during tactile stimulation.

Participants were seated comfortably in front of a monitor in a dimly lit, shielded room, wearing the tactile vest and an EEG electrode cap. During the recording, an analog noise generator produced pink noise in order to mask the sound of the tactors. The monitor always displayed a fixation cross. A sequence of stimuli started by the appearance of the word ‘focus’ on the monitor. Simultaneously, one of the tactors vibrated for 750 ms. This indicated the target for the upcoming sequence. Then, each of the tactors used in that condition vibrated 10 times in random order. The appearance of a dashed line (in the case of a training block) or the word ‘result’ (in the case of a test block) indicated the end of a sequence. In the latter case, one of the tactors vibrated for 750 ms simultaneously with the word ‘result’, to indicate to the participant which tactor the algorithm designated as being the target. Participants took 1- to 5-min breaks in between blocks.

Classification algorithm and analysis

Classification models were built during the experiment for each participant and each of the four conditions after two training blocks. Using step-wise linear discriminate analysis (SWLDA; Krusienski et al., 2006) a maximum of 60 features were extracted from the EEG data. The epochs used in the analysis started at stimulus onset and ended 797 ms afterwards. The data was down sampled with a factor of four. Each of the two training blocks yielded 60 target epochs (6 sequences × 10 repetitions) for the algorithm to be trained on. Since the number of distracters differed between conditions, the number of distracter epochs per block was 60 in condition Two, 180 in condition Four and 300 in condition Six and Four Plus.

For each participant and each condition, we determined classification accuracy as the proportion of targets that was correctly identified by the algorithm. However, the conditions differed with respect to chance level: 50% or three targets correct for each participant in condition Two, 25% or 1.5 target correct in condition Four, and 17% or one target correct in the other conditions. Therefore, we also present the results after correcting for chance by subtracting chance performance from the observed classification accuracy and normalizing the results.

Besides classification accuracy, BCI performance can also be expressed in terms of bitrate (Wolpaw et al., 2002; Serby et al., 2005). This is a measure of performance that represents the amount of information being communicated, taking into account the number of possible options, the classification accuracy and the time required to communicate the chosen option. The bitrate is calculated as follows (Serby et al., 2005):

Bitrate = BM

where M is the mean number of decisions per minute and B is the amount of bits communicated per decision:

B = log₂N + Plog₂P + (1 − P)log₂[(1 − P)/(N − 1)].

N is the number of possible decisions and P is the accuracy probability. In this study P is the classification performance, without the correction for chance. N is set to 2, 4, 6 and 6 in conditions Two, Four, Six and FourPlus condition respectively. We computed bitrate for each subject and each condition, where we set bitrate to the theoretically maximal achievable bitrate of log₂(N) × M if the target was correctly identified after each of the six sequences, and to 0 if performance was below chance level.

We used one-sample t-tests, repeated measures ANOVA and post-hoc Tukey tests to evaluate classification accuracy and bitrate.

Results

Figure 2A shows the classification accuracy for each condition, together with an indication of chance performance. Figure 2B presents the classification accuracy corrected for chance. The one-sample t-tests against zero on classification accuracy corrected for chance showed that classification accuracy was well above chance for all conditions (all p-values < 0.01). A repeated measures ANOVA showed no effect of condition [F_(3,30) = 0.19, p = 0.90]. Figure 2C shows the achieved bitrate for each condition. Again, a repeated measures ANOVA indicated no effect of condition [F_(3,30) = 0.11, p = 0.95]. Table 1 displays timing characteristics of the conditions in the different experiments, the maximal achievable bitrate, the mean achieved bitrate, the number of participants that achieved the maximal achievable bitrate and finally the mean bitrate aligned to Experiment 2 (see General Discussion). In every condition of Experiment 1, at least one participant achieved the maximal achievable bitrate.

Figure 2.

Mean performance of the classification model in the four conditions of Experiment 1 as expressed by classification accuracy, or proportion of targets correct (A), classification accuracy corrected for chance (B) and bitrate (C). The stars in (A) indicate chance performance. Error bars represent standard errors of the mean.

Table 1.

Timing characteristics and bitrate results for the different conditions in the different experiments. The maximal bitrate is the bitrate that would be achieved when classification would always be correct. The mean bitrate is the bitrate achieved in the corresponding condition, averaged across participants. Asterisks indicate a significant difference within an Experiment as indicated by Tukey post-hoc tests. In the next column, the number of participants that achieved the maximal achievable bitrate is indicated. The last column presents the mean bitrate aligned to Experiment 2 (see General Discussion). Equal conditions in different experiments that serve to compare results between experiments are highlighted in bold and italic font.

Exp	Condition	On-time (ms)	Off-time (ms)	SOA (ms)	Max bitrate (b/m)	Mean bitrate (b/m)	Number of participants with max bitrate (out of 11)	Mean bitrate aligned to Experiment 2
1	Two	188	438	626	4.79	1.72	3	2.37
	Four	188	438	626	4.79	1.81	2	2.49
	Six	188	438	626	4.13	1.56	2	2.15
	FourPlus	188	438	626	4.13	1.43	1	1.97
2	Baseline	188	438	626	4.13	2.15	5	2.15
	Short on	63	313	376	6.87	2.18	2	2.18
	Medium on	188	188	376	6.87	3.71*	4	3.71
	Long on	376	0	376	6.87	0.75*	0	0.75
3	Long on–Long off	188	188	376	6.87	2.80	3	3.71
	Long on–Short off	188	63	251	10.30	2.82	1	3.74
	Medium on–Medium off	125	125	250	10.34	1.87	1	2.48
	Medium on–Short off	125	63	188	13.75	2.16	0	2.86

Figure 3 depicts the mean Pz EEG samples from 200 ms before stimulus onset until 800 ms after, for each condition, and separately for target and standard presentations (solid and dashed curves respectively). Pz is a location where the P300 is usually strongly displayed (e.g. Ravden and Polich, 1999; Srinivasan, 2007). Clearly, a P300 is present in all conditions, even in condition Two, although the amplitude seems to be smaller. The P300 occured later than 300 ms after the start of the stimulus. This corresponds to many P300 studies that report this as well (e.g. Ravden and Polich, 1999; Bledowski et al., 2004; Gerson et al., 2006).

Figure 3.

EEG averaged across participants and presented separately for targets (solid lines) and distracters (dashed lines) in the four conditions of Experiment 1. Only data from electrode Pz is presented.

Discussion

In a proof-of-concept study (Brouwer et al., submitted) we showed that an oddball tactile stimulus presented within a stream of tactile distracters around the waist elicited P300s that could be classified offline. Experiment 1 demonstrated that a BCI in which a tactile target is chosen among similar non-targets in an array around the waist is also feasible. Using different tactors as targets, online classification performance was well above chance for all conditions; ranging from an average of 58% correct for six alternatives to an average of 73% correct for two alternatives.

We did not find significant differences in classification accuracy corrected for chance between the conditions that varied in target-distracter discriminability and target probability. We expected at least to find a difference between the FourPlus condition and either the Four or the Six condition, or both. The FourPlus condition was expected to produce clearer P300s than the Four condition because of a more favorable target probability, and than the Six condition because of a more favorable discriminability. An explanation could be that in all conditions, target probability was low enough (or target-to-target interval long enough) and that the distinction between target and distracter was clear enough (van Erp, 2005a), even though some participants remarked that in the Six condition, they sometimes experienced difficulties in distinguishing the target from adjacent distracters.

Since at least under these experimental conditions, different numbers of tactors do not affect classification accuracy and bitrate, the number of choices in a tactile BCI (two, four or six) can be directly tailored to the user application. For instance, it makes no sense to use a six tactor BCI for a yes–no choice.

The mean EEG signals suggested a slightly different picture than the classification results. The amplitude in condition Two seemed smaller than in the other conditions. The fact that this was not reflected in the classification accuracy could be because even though the amplitude was smaller, it was sufficiently different from that of the distracters. When fewer trials would have been used to train the classification algorithm, or fewer repetitions of the stimuli before the BCI decided on the target, there may have been a difference. Another possible reason for discrepancies between classification results and P300 amplitudes is that the classification model used different features than P300 amplitude only. From classification results obtained in experiments that are supposed to manipulate the P300, one cannot directly conclude anything about P300s without further looking at the signals and the features that a model uses. So-called P300 BCIs (including our own) probably often rely on non-P300 components in differentiating between targets and standards (Allison and Pineda, 2006; Brouwer et al., submitted).

Our main concern for this study was to build an online tactile BCI and to compare the effects of tactor number between conditions; not to optimize classification performance. Even so, the bitrate that we achieved is modest (see for example bitrates of online visual P300 BCIs in Table V of Serby et al. (2005) where bitrates range from 2.35 to 10.88 b/m). There are several possible reasons for our relatively low bitrate. We used only eight electrodes and each model was trained on a limited dataset obtained in 4–10 min. Even though the P300 signal itself is often referred to and liked for being relatively independent on training, most studies mentioned by Serby et al. (2005) used training times from seven sessions up to a few months. We expect that acquiring more data to train the model and leaving out less representative blocks (e.g. at the start of the experiment) could increase classification performance. Furthermore, there may be a difference between P300s elicited by visual and tactile stimuli. In virtually all visual P300 BCI paradigms, participants are allowed to fixate their eyes on the target. This will likely result in a perceptual difference between (fixated) target and (non-fixated) distracters as well as the possibility for the classification algorithm to distinguish between target and distracter on the basis of low-level visual evoked potentials (Treder and Blankertz, submitted). A strategy analogous to visual fixation is not possible in the tactile modality. A third reason for our modest bitrate may be the motivation of our (naïve) participants (Nijboer et al., 2009; Kleih et al., in press). In our experimental procedure, we did not check whether participants counted the correct number of targets and the financial reward did not depend on performance. Introducing performance checks and possibly boosting their motivation by rewarding them for every correctly classified target stimulus may be ways to increase performance. Finally, our SOA of 626 ms is short compared to classic P300 studies, but long compared to P300 BCI studies, leaving a high bitrate unlikely (see Table 1 for maximal achievable bitrates).

In Experiment 2 we increased the maximal possible bitrate of the tactile P300 BCI by shortening the SOA. This was done by adjusting the stimulus on- and off-times by different amounts. Even though the effects of SOA and target-to-target interval on P300s and classification performance have been studied before (e.g. Farwell and Donchin, 1988; Serby et al., 2005; Allison and Pineda, 2006; Sellers et al., 2006; Martens et al., 2009; Mell et al., 2008), we did not encounter studies on the specific effect of stimulus on- and off-time. However, in a psychophysical discrimination study, varying these timing parameters produced differential effects (van Erp, 2005a). Participants judged relative location of tactile stimuli. Whereas for vibration durations between 50 and 350 ms, vibration duration did not have a large effect on the discriminability, discriminability increased with increasing inter stimulus intervals in the range from 0 to 200 ms.

Experiment 2

Participants

Eleven healthy participants (four female) between 19 and 27 years old volunteered to participate in Experiment 2. None of them had participated in any other BCI experiment before.

Stimuli

The same vibro-tactile stimuli were used as in Experiment 1, except that the tactors were mounted in a belt rather than a vest. Taking the participants’ waist circumference into account, six tactors were positioned at equal distances from each other along the belt. All six tactors were used in all four experimental conditions. The vibrating on- and off-times differed between conditions as explained below. As in Experiment 1, participants viewed a dimmed LCD (Dell 20 inch flatpanel, refresh rate 75 Hz) displaying instructions and a fixation cross. Again, pink noise was played to mask auditory cues.

Conditions

In Experiment 2 we used four different settings of stimulus timing (see also Table 1). The on-time and off-time in the Baseline condition were the same as those used in Experiment 1 (188 ms on, 438 ms off), making this condition exactly equal to condition Six of Experiment 1. Besides this Baseline condition, we had three other experimental conditions in which the SOA was reduced to 376 ms as compared to the 626 ms of the Baseline condition. Referring to stimulus on-time, these are called Short On (63 ms on, 313 ms off), Medium On (188 ms on, 188 ms off) and Long On (376 ms on, 0 ms off).

Design

The design was almost the same as in Experiment 1. The order of conditions was randomized for each participant. For every condition, there were two blocks to train the classification algorithm followed by one block to test it. Again, a block consisted of six sequences of stimuli. Each sequence started with one of the tactors being designated as the target. Each of the tactors served as a target once in one block, starting with the front-left tactor and every time going to the next tactor counter-clockwise. Each of the six sequences consisted of activating each tactor in that condition (the target and the five distracters) 10 times, in pseudo random order. One tactor could not vibrate twice in a row, since that would have resulted in one long vibration for the Long On condition.

The recording materials, task and procedure and Classification algorithm and analysis were exactly the same as in Experiment 1.

Results

Figure 4A shows the classification accuracy for each condition, together with an indication of chance performance (1/6 for all conditions). Figure 4B presents the classification accuracy corrected for chance. One-sample t-tests against zero showed that the classification was above chance in all conditions (all p-values 0.01 or lower). A repeated measures ANOVA showed an effect of condition [F_(3,30) = 4.51; p < 0.01]. Tukey post-hoc tests indicated that both the Baseline condition and the Medium On condition differed significantly from the Long On condition (p < 0.01 in both cases). The difference between the Short On condition and the Medium On condition approached significance (p = 0.06). Figure 4C shows the bitrate and Table 1 the numerical values. A repeated measures ANOVA indicated an effect of condition [F_(3,30) = 5.03; p < 0.01]. Tukey post-hoc tests showed that bitrate was significantly higher in the Medium On than in the Long On condition (p < 0.01). In every condition of Experiment 2, except for the Long On condition, at least two participants achieved the maximal achievable bitrate. For the Baseline and Medium On condition the number of participants achieving the maximal score was even 5 and 4 respectively.

Figure 4.

Mean performance of the classification model in the four conditions of Experiment 2 as expressed by classification accuracy (A), classification accuracy corrected for chance (B) and bitrate (C). The stars in (A) indicate chance performance. Error bars represent standard errors of the mean.

Figure 5 depicts the mean Pz EEG samples from 200 ms before stimulus onset until 800 ms after, for each condition, and separately for target and distracter presentations (solid and dashed curves respectively). In line with the classification accuracy, the P300 for the Long On condition seems relatively small.

Figure 5.

EEG averaged across participants and presented separately for targets (solid lines) and distracters (dashed lines) in the four conditions of Experiment 2. Only data from electrode Pz is presented.

Discussion

The results of Experiment 2 showed that we can reduce the SOA from 626 to 376 ms without reducing classification accuracy. The resulting bitrate could be increased by a factor 1.7 compared to the timing settings of Experiment 1. However, consistent with van Erp (2005a) it is critical how the SOA was reduced. The Medium On condition resulted in a significantly higher classification accuracy and bitrate than the Long On condition even though the SOA was 376 ms in both conditions. Because in our design, SOA and target-to-target interval were coupled, this means that target-to-target interval is at least not the only factor determining classification accuracy and bitrate.

Since shortening the SOA from 626 to 376 did not necessarily decrease classification accuracy and could result in a higher bitrate, we further reduced the SOA in Experiment 3.

Experiment 3

Participants

Eleven healthy participants (six female) between 19 and 32 years old volunteered to participate in Experiment 2. None of them had participated in any other BCI experiment before.

Conditions

As a baseline we took the best condition of Experiment 2, which was Medium On (188 ms on, 188 ms off) and called it Long On–Long Off in this experiment. We then reduced either the on-time or the off-time arriving at the following conditions: Long On–Short Off (188 ms on, 63 ms off), Medium On–Medium Off (125 ms on, 125 ms off) and Medium On–Short Off (125 ms on, 63 ms off). Table 1 summarizes the details of the four different conditions in Experiment 3. As in Experiment 2, there were six tactors in all conditions.

The stimuli, recording materials, design, task and procedure and classification algorithm and analysis were exactly the same as in Experiment 2.

Results

Figure 6A shows the classification accuracy for each condition and Figure 6B the classification accuracy corrected for chance. One-sample t-tests against zero on classification accuracy corrected for chance revealed that in all conditions except for the Medium On – Short Off condition (p = 0.052), classification performance was significantly higher than chance (p = 0.046 for Medium On–Medium Off, other p-values < 0.01). A repeated measures ANOVA showed a significant effect of condition [F_(3,30) = 2.98; p < 0.047]. Tukey post-hoc tests did not show significant differences, although the differences between condition Long on–Long Off compared to conditions Medium On–Medium Off and Medium On–Short Off approached significance (p = 0.10 and p = 0.07 respectively). Figure 6C shows the bitrate and Table 1 the numerical values. A repeated measures ANOVA indicated no effect of condition on bitrate [F_(3,30) = 0.57; p = 0.64]. In every condition of Experiment 3, except for the Medium On–Short Off condition, at least one participant achieved the maximal achievable bitrate.

Figure 6.

Mean performance of the classification model in the four conditions of Experiment 2b as expressed by classification accuracy (A), classification accuracy corrected for chance (B) and bitrate (C). The stars in (A) indicate chance performance. LL: Long On–Long Off, LS: Long On–Short Off, MM: Medium On–Medium Off, MS: Medium On–Short Off. Error bars represent standard errors of the mean.

Figure 7 depicts the mean Pz EEG for each condition. The mean P300 amplitudes seem roughly consistent with classification accuracy.

Figure 7.

EEG averaged across participants and presented separately for targets (solid lines) and distracters (dashed lines) in the four conditions of Experiment 2b. LL: Long On–Long Off, LS: Long On–Short Off, MM: Medium On–Medium Off, MS: Medium On–Short Off. Only data from electrode Pz is presented.

Discussion

By shortening on- and off-time further in Experiment 3, we did not achieve a further increase in classification accuracy or bitrate. The best results in terms of bitrate tended to be reached in the Long On–Long Off condition (188 ms on-time and 188 ms off-time, i.e. an SOA of 376 ms) and the Long On–Short Off condition (188 ms on time and 63 ms off-time, i.e. an SOA of 251 ms). The accuracy tended to decrease in this latter condition. Thus, reducing the SOA below the baseline of 376 did not have a positive effect in our experiment.

The trend in the pattern of our results did not replicate that of the psychophysical study by van Erp (2005a) which indicated that off-time affected performance more strongly than on-time. Here we find similar results for conditions with equal on-times and varying off-times and different results for conditions with different on-times and equal off-times. Thus, the tendency is rather that on-time has a stronger effect than off-time. Apparently the task used in van Erp (2005a), judging relative locations of tactile stimuli, does not directly generalize to the present task (counting tactile stimuli at a particular absolute location on the body).

General Discussion and Conclusions

The bitrate results of the three experiments are summarized in Table 1. It is difficult to compare bitrates across experiments directly, since different participants were recorded in each experiment and (as commonly found in BCI experiments) there are large individual differences in performance. In Experiment 2, participants scored on average 1.4 higher than in Experiment 1 as can be seen when comparing the Six and the Baseline condition. Comparing the Medium On condition to the Long On–Long Off condition shows that participants of Experiment 2 scored about 1.3 times better than the participants of Experiment 3. To facilitate a rough direct comparison between experiments, we computed the mean bitrate aligned to the results of Experiment 2 (last column Table 1). For this, the mean bitrates of Experiment 1 were multiplied by 1.4 and the mean bitrates of Experiment 3 were multiplied by 1.3. With this rough procedure we neglect possible effects that the context created by the other conditions in the experiment could exert. Even though the conditions are technically exactly the same, the Long On–Long Off condition in Experiment 3 could be experienced as relatively easy compared to the Medium On condition in Experiment 2 which again might have influenced the results. Keeping in mind that most differences between bitrates were not significant, the highest bitrates for our setup were achieved with an on-time of 188 and an off-time between 63 and 188 ms (where the accuracy tended to be slightly higher for the longer off-time). Lengthening the off-time did not result in a higher bitrate (compare Baseline versus Medium On in Experiment 2) since higher classification accuracies could not outweigh the longer presentation time. Reducing the on-time below 188 ms while keeping the off-time the same tended to reduce the bitrate (compare Long On–Short Off and Medium On–Short Off). The only room for further bitrate improvement through manipulation of stimulus timing parameters might be reducing the off-time below 63 ms for an on-time of 188. However, an off-time of 0 ms seems too short as indicated by the particularly bad result of the Long On condition in Experiment 2. Off-times close to 0 ms may result in spatio-temporal interactions like apparent motion (Kirman, 1974a) and therewith impede performance.

The optimal timing parameters as found in Experiment 2 and 3 using six tactors cannot be generalized to other numbers of tactors without further investigation. When using less tactors, ERP overlap effects could cause a decrease in performance for short on- and off-times. On the other hand, stimuli delivered by few evenly spaced tactors are less vulnerable to spatio-temporal interactions compared to many tactors (Kirman, 1974b).

The achieved levels of classification accuracy in this study are not high enough for most potential applications. There are several ways to possibly improve performance (both accuracy and bitrate) of our tactile P300 BCI. As already mentioned, more electrodes and training could help though obviously, these adjustments have their downsides. We also mentioned increasing user motivation. In contrast to many studies, we recorded from naïve, paid participants. The classification technique that we used may not be optimal, as well as the number of stimulus repetitions before the algorithm made a decision on the attended tactor. The optimal number of repetitions, the optimal timing parameters and classification technique may best be determined for each individual user. With respect to the number of repetitions this may be done relatively easy online – that is, the BCI could stop presenting stimuli and make a decision once it is ‘certain enough’ about the user's choice. Another potential way to (massively) increase bitrate is to adjust stimulus presentation so that different combinations of options are presented simultaneously, much like the traditional P300 speller. For example, think of adding other information than, in our navigation example, the desired movement direction such as the desired step size by adding a dimension like torso height. In this way, different heights in torso location would act as P300 speller rows while different locations around the waist act as columns. Finally, an intelligent way of presenting options, e.g. by taking into account how likely different options are to be chosen and omitting presentation of impossible options (which will be often present in navigation applications) will certainly improve accuracy and bitrate.

This study demonstrated the feasibility of a BCI controlled by EEG responses, elicited by attending to specific vibro-tactile stimuli around the waist. Tactile stimuli are particularly useful in BCIs for patients who cannot fixate or move their eyes well, or who completely lost visual perception. For other users the advantages of a tactile BCI are that it allows them to look and listen to their environment rather than to stimuli necessary for controlling the BCI, and that tactile stimuli can easily go unnoticed by others. Although a BCI with tactile stimuli to for example the hands and wrists could work as well (Satomi et al., 1995; Ito and Takamatsu, 1997; Nakajima and Imamura, 2000) and would be suitable for some applications, tactile stimuli around the waist are specifically appropriate for navigation displays (van Erp and van Veen, 2004; van Erp et al., 2005) and thus a logical choice for navigation BCIs. We tested stimulus configurations consisting of two, four and six equally distributed tactors and did not find a significant difference in classification performance. This suggests that all of these numbers of tactors could be used in such a BCI. With optimal on-times of about 188 ms and off-times of 63–188 ms, SOAs are similar to those of many visual BCIs.

Conflict of Interest Statement

Part of this study was previously published as a proceeding paper at the 4th International BCI Workshop and training course in Graz: A.-M. Brouwer, J. B. F. Erp (2008) A tactile P300 BCI and the optimal number of tactors: Effects of target probability and discriminability.