The performance of 9–11-year-old children using an SSVEP-based BCI for target selection

Abstract

In this paper, we report the performance of 9–11-year-old children using a steadystate visual evoked potential (SSVEP)-based brain-computer interface (BCI) and provide control data collected from adults for comparison. Children in our study achieved much higher performance (79% accuracy; average age 9.64 years old) than the only previous investigation of children using an SSVEP-based BCI (∼50% accuracy; average age 9.86 years old). Experiments were conducted in two phases, a short calibration phase and a longer experimental phase. An offline analysis of the data collected during the calibration phase was used to set two parameters for a classifier and to screen participants who did not achieve a minimum accuracy of 85%. Eleven of the 14 children and all 11 of the adults who completed the calibration phase met the minimum accuracy requirement. During the experimental phase, children selected targets with a similar accuracy (79% for children versus 78% for adults), latency (2.1 seconds for children versus 1.9 seconds for adults), and bitrate (0.50 bits/second for children and 0.56 bits/second for adults) as adults. This study shows that children can use an SSVEP-based BCI with higher performance than previously believed and is the first to report the performance of children using an SSVEP-based BCI in terms of latency and bitrate. The results of this study imply that children with severe motor disabilities (such as locked-in syndrome) may use an SSVEP-based BCI to restore/replace the ability to communicate.

1. Introduction

Non-invasive brain-computer interfaces (BCIs) enable users to control external devices (such as computer systems [1, 2], prosthetic arms [3, 4], and robotic vehicles [5]) using brain activity. The primary application of BCIs is to replace [6] function in those with severe motor disabilities that are the result of injury [7] or disease [8, 9]. They also, however, may be useful for rehabilitation (such as after a stroke [10]) or to supplement [11] and/or improve [12] the natural capabilities of healthy individuals.

Most current BCI systems use electroencephalography (EEG), a non-invasive and relatively inexpensive tool, to measure the brain’s naturally generated electrical activity. For example, EEG can be used to measure brain activity generated by imagined movement [8], the detection of infrequent targets [13], or spatially distinct sounds [14]. Here, we consider EEG-based BCIs that use brain activity elicited by repetitive visual stimulations. These brain signals are commonly known as steady-state visual evoked potentials (SSVEPs).

SSVEP-based BCIs rely on the fact that repetitive visual stimulation elicits brain activity at the same speed (measured in frequency or flashes per second) as the stimulus [15]. In addition, the amplitude of the brain activity elicited by the stimulus is dependent on the user’s attention [16]. This means that if there are multiple stimuli flashing at different rates, the stimulus that the user attends to (the target) will elicit a larger amplitude response than the stimuli that the user ignores. In practice, the user’s target is unknown and must be inferred through analysis of the EEG data. This analysis, known as classification, outputs a guess of the user’s target, called the predicted target. This process of the user attending to targets and the classification system predicting targets enables a user to select a specific stimulus from the available set of stimuli and is the basis for all SSVEP-based BCIs.

There are three common ways to measure the performance of an SSVEP-based BCI user. Accuracy is the proportion of times the predicted target matches the target [17]. Latency is the mean time from target onset to classification [18, 2]. Bitrate is the number of bits per second that are transmitted, and is often preferred over other measures because it accounts for both accuracy and latency. Common ways of quantifying bitrate are the information transfer rate (ITR) [19] and the Nykopp bitrate (NBR) [20]. We use NBR to quantify bitrate in this paper, since it addresses several well-known limitations of ITR [21].

Given that the goal of most SSVEP-based BCIs is to replace function in those with severe motor disabilities and that these disabilities affect many different groups of people (with different ages, genders, etc.), it is important to understand how SSVEP-based BCI performance varies among these different groups. As an example, consider that SSVEP-based BCIs are often tested with young adults, but the average age of patients who have locked-in syndrome (LIS) is approximately 50 (this estimate is based on the average age of 151 LIS patients reported by Bruno et al. [22]). Then consider that Lesenfants et al. [23] found that only one out of six LIS patients (average age 49 ± 19.7 years) could use their SSVEP-based BCI system with better than chance accuracy, but 80% of young adults could use their system.

Prior studies of these demographic differences in the performance of SSVEP-based BCIs have reported mixed results. Allison et al. [24] assessed the performance of more than 100 people between the ages of 18 and 79 at a large computer expo (CeBIT 2008). While younger people tended to perform at higher bit rates, there were no significant differences in performance between people of different ages or genders. A follow-up paper by Volosyak et al [25] also found no differences between people of different ages. Two more recent papers, however, have reported differences between young adults and older adults in the context of SSVEP-based BCIs. Hsu et al. [26] compared SSVEP amplitudes in young adults, older adults, and ALS patients. They found that young adults produced larger SSVEPs with a higher signal to noise ratio at an occipital electrode site compared to older adults and ALS patients. In addition, Volosyak et al. [18] recently investigated differences in performance between young adults (between the ages of 19 and 27) and older adults (between the ages of 54 and 76) while using an SSVEP-based BCI for text-entry. The results of this study showed that younger adults achieved a higher average bitrate than older adults.

While the aforementioned studies have considered adults of different ages, only one previous study has investigated the performance of children compared with adults when using an SSVEP-based BCI. In this study, Ehlers et al. [17] asked children of different ages to use an SSVEP-based BCI for text-entry. Ehlers at al. found that children achieved lower accuracy than adults and concluded that children were not yet able to generate a reliable SSVEP due to developmental differences. The data presented by Ehlers et al. [17] are an important contribution to the literature on SSVEP-based BCIs. However, it is not clear if external factors influenced the results. For example, the children were tested in a noisy school environment, one that may have distracted them from the task. In addition, it is not clear whether the children struggled with the SSVEP-based BCI because it was used for text-entry, a task that—even without a BCI—is harder for children than adults [27]. Previous cognitive neuroscience research conflicts with the results from Ehlers et al. [17]. For example, Birca et al. [28] first reported that there were no differences in SSVEP magnitude between children and adults, and later reported that children aged 8–11 had larger SSVEP responses over the occipital region than adults [29]. Furthermore, Ehlers et al. [17] only reported accuracy, and not the latency or the bitrate of the participants.

The small number and conflicting results of these prior studies leave open the question of whether or not children can use an SSVEP-based BCI with good performance. Resolving this question is important if we want to consider the use of SSVEP-based BCIs to replace lost function in children with severe motor disabilities, including LIS [30]. To help resolve this question, we describe a new study in this paper that compared the performance of children (aged 9–11) using an SSVEP-based BCI. Our study consisted of two phases, a short calibration phase and a longer experimental phase. Data collected during the calibration phase were used to choose parameters for our classifier and to screen out participants with low performance (11 of 14 children and 11 of 11 adults met minimum performance requirements). Data collected during the experimental phase were used to measure average accuracy, latency, and bitrate in each of the two groups.

2. Method

2.1. Participants

Twenty-six able-bodied volunteers (fifteen 9–11 year olds, Mean = 9.73 and eleven adults aged between 19–68, Mean = 39.31) participated in our study. Participants were recruited through email bulletins and word of mouth. Two adult participants had previous experience with a BCI (S01a and S09a). All subjects had normal or corrected-to-normal vision and no prior history of neurological illness ^‡. Each participant was compensated with a small gift ($5.00US or less) for their time. This study was approved by the Institutional Review Board at the University of Illinois at Urbana-Champaign.

2.2. EEG recording

EEG signals were recorded from six tin electrodes. The electrodes were placed on the surface of the scalp located at 10–5 international sites: PO3, POZ, PO4, O1, OZ, and O2 [32]. The channels were grounded at the right ear and referenced to the top of the head (location CZ). The signals were recorded at impedances of less than 10kΩ. All EEG signals were band-pass filtered from 1Hz to 30Hz, amplified using a James Long bioamplifier, and digitized at 128Hz (National Instruments Model PCI-6225). BCI2000 [33] was used to visualize and record the preprocessed EEG signals.

2.3. Experimental procedures

All experiments were conducted in a cool and sound attenuated room with dim ambient lighting. The participants were seated in a comfortable office chair between two speakers facing an LED computer monitor (24-inch BenQ XL2420T). All participants were approximately 24 inches from the monitor, but no chin rest was used to restrict head movement. After completing the consent process, each subject was asked to complete a brief survey with basic background questions, based on the questionnaire used by Allison et al. [24]. After the survey was completed, the participants completed a short calibration phase and a longer experimental phase. After the experiments, participants also completed an additional phase that will be discussed as a part of different study. During all experiments participants were asked to focus their visual attention on a target presented on the monitor. For both the calibration phase and the experimental phase the targets were three white circles (5.1cm in diameter) flashing between white and black at 6.2Hz, 7.7Hz, and 10Hz. We chose these frequencies based on previous experience [34] and because they are less likely to elicit photo-induced seizures. The second of these two reasons is especially important when working with younger individuals, who are often unaware that they are photosensitive [31].

2.4. Calibration phase

Each participant completed a calibration phase to calibrate the BCI system and to screen participants with low performance.

Data from the calibration phase was used to set free parameters of the classifier (Appendix A; Section 2.6). The participants were given verbal instructions and allowed to start the application when they were ready by pressing a key on the keyboard. Once the calibration phase started, an arrow specified the target during each trial (Figure 1a) and this target was highlighted with a yellow outline. Participants were instructed to overtly focus their attention on the target for the entire trial. Each trial lasted five seconds, with a short pause between trials. The order of the specified targets was randomized with each of the three frequencies specified as the target five times for a total of 15 trials. The calibration phase took no more than five minutes. Following the calibration phase, the participants were given time to relax while the experimenter calibrated the BCI system with the calibration data.

Figure 1:

Graphical representations of the experimental interface. (a) Shows how targets were identified with an arrow during the calibration phase. (b) Shows how the target was highlighted during the experimental phase. (c) Shows the feedback given to the user after the classifier guessed the predicted target. Note that the size of the targets and text have been enlarged to improve readability.

During the analysis of the calibration data, if the participant never achieved an 85% accuracy, they were deemed to be unable to use the SSVEP-based BCI and any data collected during the experimental phase was excluded from further analyses.

2.5. Experimental phase

During the experimental phase, users were asked to select a sequence of targets using our SSVEP-based BCI. When the experiment was started, a splash screen was displayed while the experimenter described the task. After the researcher provided instructions on how to use the application, the participant was allowed to press a key on the keyboard to begin the experiment.

Similar to the calibration phase, the interface used in the experimental phase displayed three stimuli. The target that the participant was supposed to select was the same as in the calibration phase, except there was no arrow pointing to it (Figure 1b).

Participants were instructed to select targets by overtly shifting their visual attention. If the classifier guessed a target, a check mark was shown at the location of the predicted target (Figure 1c) and a tone provided audio feedback that a selection had been made. Participants were given up to 5 seconds to select a target during each trial. If no target was selected within 5 seconds, the trial ended and the next trial began. The application paused for one second between trials. Real-time feedback on the number of trials completed was provided at the top of the interface.

The experimental phase consisted of four rounds and a bonus round at the end (data from the bonus round will be described as part of a future study). Each round contained 15 trials. The order of the targets during each round was randomized. Each of the three stimulation frequencies were specified as the target five times during each round. At the end of each round, a message was displayed indicating the round number and the system paused for six seconds. At the end of the experimental phase a message was displayed to the user, letting them know the session was completed.

2.6. Signal processing

A classifier, based on canonical correlation analysis (CCA), was used to determine the predicted target. Our algorithm for the classification of SSVEP targets using CCA was similar to the one described by Lin et al. [35] with three notable differences:

• (i)We considered only two harmonic frequencies.
• (ii)A threshold (τ) was used to enable asynchronous control.
• (iii)Two free parameters, the amount of data considered by the classifier (window-length [t], measured in seconds) and τ were set using data from the calibration phase.

For additional details on the classifier used in this study, see Appendix A.

2.6.1. Window-length and threshold

As discussed in Section 2.6, the purpose of the calibration phase (Section 2.4) was to set t and τ. After each participant completed the calibration phase, individual five-second trials were extracted from the calibration data. After trial extraction, there were five trials for each of the three target frequencies. Using this data, a search was performed to find the parameters that maximized the participant’s NBR [20]. The NBR was computed for t = [0.5,0.625,...,5] and for τ = [0,0.01,...,1]. The values of t and τ that maximized NBR were subsequently used for classifying targets during the experimental phase.

There were two differences in the way that the parameters were calculated for different subjects. First, for children, the minimum value of t was set to be 1.250 seconds. For the adults, the minimum value of t was set to be 0.5 seconds. Second, for five of the children—denoted with asterisks in Table S1—the parameters were calculated using three harmonic frequencies instead of two. The possible impact of these two differences will be considered in the results and discussion (Sections 3 and 4).

3. Results

Of the 26 people who participated in our study, 22 were able to complete the entire experiment (average age children = 9.64 years of age, average age adults = 38.00 years of age). One child was excluded due to a technical issue (software crash). Three children were excluded due to low performance. Two of the children who were excluded appeared distracted during the calibration phase. They did not pay attention to the screen or appear to attempt the task. Data from participants who did not achieve a minimum accuracy of 85% within five seconds of stimulation during the calibration phase (Section 2.4) were excluded from further analyses.

In our study, there were three primary measurements of performance.

• (i)Accuracy - Accuracy was calculated as the number of times that the predicted target was equal to the target divided by the total number of trials.
• (ii)Latency - The average amount of time (measured in seconds) that elapsed between the onset of the stimuli and the classification of the predicted target.
• (iii)NBR - Calculated using the definition of Nykopp [20, 21, 36], a quantity that describes the amount of information transmitted over a noisy channel per unit of time and reported in terms of bits/second. Our calculation of NBR is based on the formulas found in Kronegg et al. [21].

3.1. Calibration phase

We report our analysis of the calibration data in two ways. First, we describe the t and τ values used during the experiment (since there were two differences in the way that t and τ were calculated for the participants who were children; Section 2.6.1). Second, we describe a post-experiment analysis of the data. In both cases, 11 of the 14 children and all 11 adults who completed the calibration phase exceeded our threshold for being able to use an SSVEP-based BCI.

3.1.1. Parameters from experiments

The results obtained during the calibration phase are given in Tables S1 and S2. Before assessing statistical differences between the children and adults in the data from the calibration phase, the assumptions of a two-sample t-test were tested. A Shapiro-Wilk test (performed using the MATLAB command swtest by Ahmed BenSaïda from the file exchange) was used to test the assumption of normality. This test found that the data was normally distributed for τ (p = 0.85), but not for t (p = 0.001). Since the data was found to be normally distributed, τ was also tested for outliers (using the MATLAB function isoutlier; outliers were defined as being more than three standard deviations from the mean; no outliers were found) and for equal variances between the groups using Levene’s test (MATLAB function vartestn; p = 0.88). A Mann-Whitney U test with no correction for multiple comparisons found that the t values used for classification in the participants who were children (Mean = 1.545 seconds, Mdn = 1.375 seconds, SD = 0.450 seconds) were longer (p = 0.02) than those used for the participants who were adults (Mean = 1.227 seconds, Mdn = 1.000 seconds, SD = 0.849 seconds). Although, one adult (participant S05a) had a t of 3.625 seconds (Table S2), 1.375 seconds longer than any of the children who participated in the study. There was no difference in the τ values calculated for children (Mean = 0.57, Mdn = 0.58, SD = 0.15) compared with adults (Mean = 0.60, Mdn = 0.60, SD = 0.15) using a two-sample t-test (p = 0.63).

3.1.2. Post-experiment analysis

The purpose of the post-experiment analysis of the data from the calibration phase was to calculate t and τ in the exact same way for all of the participants using the classifier described in Section 2.6. Before assessing statistical differences in the data from the calibration phase between the children and adults, the assumptions (normally distributed data, no outliers, equal variance) of a two-sample t-test were tested. A Shapiro-Wilk test (performed using the MATLAB command swtest) was used to test the assumption of normality for each of the variables. This test found that the data was normally distributed for τ (p = 0.93), accuracy (p = 0.13), and bitrate (p = 0.33), but not for t (p = 0.001) or latency (p = 0.02). A test for outliers (using the MATLAB function isoutlier; outliers defined as more than three standard deviations from the mean) did not find any outliers in the data. Finally, using Levene’s test for equality of variances, the two groups in each of the variables were found to have equal variance. A Mann-Whitney U test (with no correction for multiple comparisons) of the calibration data for the included participants revealed t would have been shorter (p = 0.035) for the adults (Mean = 1.227, Mdn = 1.000, SD = 0.849) than the children (Mean = 1.432, Mdn = 1.250, SD = 0.448). Further statistical analyses did not reveal any differences in τ (two-sample t-test; p = 0.77), accuracy (two-sample t-test; p = 0.11), latency (Mann-Whitney U test; p = 0.19), or NBR (two-sample t-test; p = 0.08) between children and adults. These data are reported in Tables 1–3. Figure 2 presents the data from the calibration phase—as calculated in the post-experiment analysis—for the children and the adults graphically. When the accuracy of classification for a τ = 0 is plotted as a function of t, we can see that children and adults improve in accuracy as t increases (Figure 2a). Figure 2b shows accuracy curves for the children who were excluded from the study and the average accuracy of included and excluded children. Unlike the included children, the accuracy of excluded children (for τ = 0) does not increase as a function of t. Figures 2c (children) and 2d (adults) show accuracy as a function of t (τ = 0) for the three stimulation frequencies. Figure 2e shows the t and τ values calculated for each child overlaid on an image of the average NBR for each possible value of t and τ. A similar image for the adults is shown in Figure 2f.

Table 1:

Post-Experiment Analysis of Data from Calibration Phase (Included Children)

Subject	t	τ	Accuracy	Latency	NBR
S01c	1.375	0.75	0.87	2.40	0.51
S02c	1.125	0.70	0.87	1.38	0.79
S03c	2.125	0.40	0.87	2.41	0.44
S04c	1.375	0.47	1.00	1.57	1.01
S05c	0.875	0.75	1.00	1.58	1.00
S06c	2.250	0.35	0.87	2.60	0.46
S07c	1.125	0.63	0.93	2.14	0.69
S08c	1.250	0.58	0.93	1.70	0.78
S09c	1.250	0.64	0.93	1.70	0.78
S10c	1.875	0.36	0.87	1.92	0.62
S11c	1.125	0.73	1.00	1.83	0.86
Mean	1.432	0.58	0.92	1.93	0.72
Mdn	1.250	0.63	0.93	1.83	0.78
SD	0.448	0.16	0.06	0.40	0.20

Table 3:

Post-Experiment Analysis of Data from Calibration Phase (Included Adults)

Subject	t	τ	Accuracy	Latency	Bitrate
S01a	1.000	0.60	0.93	1.17	1.14
S02a	1.375	0.51	0.93	1.76	0.75
S03a	1.125	0.57	1.00	1.86	0.85
S04a	0.750	0.70	1.00	1.80	0.88
S05a	3.625	0.27	0.93	3.76	0.35
S06a	1.625	0.42	0.93	2.13	0.62
S07a	0.750	0.68	0.93	1.12	1.18
S08a	0.750	0.74	0.93	1.43	0.93
S09a	0.875	0.66	0.93	1.19	1.11
S10a	0.625	0.82	1.00	1.33	1.20
S11a	1.000	0.60	1.00	1.74	0.91
Mean	1.227	0.60	0.96	1.75	0.90
Mdn	1.000	0.60	0.93	1.74	0.91
SD	0.849	0.15	0.03	0.74	0.26

Figure 2:

Performance of participants from data collected during the calibration phase based on the post-experiment analysis. Accuracy of (a) included children versus included adults, (b) included children versus excluded children and the average of all of the children, (c) included children for each stimulation frequency as a function of t and τ = 0, and (d) included adults for each stimulation frequency as a function of t for τ = 0. Images showing bitrate calculated for each t and τ for (e) included children and (f) included adults. The t and τ values used for each participant in the experimental phase are denoted by a white dot.

3.2. Experimental phase

The performance of participants during the experimental phase is shown in Figure 3. The accuracy data from the experimental phase was found to meet the assumptions of the twosample t-test. Both latency (p = 0.003) and NBR (p = 0.03) were found to violate the assumption of normally distributed data, as measured using the Shapiro-Wilk test (performed using the MATLAB command swtest). Further statistical analyses did not reveal any differences between the children and adults in terms of accuracy (two-sample t-test; p = 0.84; Figure 3a), latency (Mann-Whitney U test; p = 0.15; Figure 3b), or NBR (Mann-Whitney U test; p = 0.45; Figure 3c) in the experimental phase. Both children (Mean = 79%, Mdn = 83%, SD = 14%) and adults (Mean = 78%, Mdn = 80%, SD = 11%) achieved similar levels of accuracy during the experimental phase and both groups had worse performance during the experimental phase than during the calibration phase. In terms of latency, the children (Mean = 2.106 seconds, Mdn = 2.073 seconds, SD = 0.48 seconds) were almost 0.2 seconds slower than the adults (Mean = 1.917 seconds, Mdn = 1.710 seconds, SD = 0.73 seconds), but the high variance meant that this was not significant. Children (Mean = 0.50 bits/second, Mdn = 0.49 bits/seconds, SD = 0.20 bits/second) also transmitted information at a slightly lower (but not significantly lower) NBR than adults (Mean = 0.56 bits/second, Mdn = 0.59 bits/second, SD = 0.25 bits/second).

Figure 3:

Performance of participants during the experimental phase in terms of (a) accuracy, (b) latency, and (c) NBR. The variations in the x-axis are random noise that has been added to reduce overlap in the data points and increase readability.

The data from the experimental phase were also inspected for differences in accuracy and latency of the selection of targets by frequency (Figure 4) and round (Figure 5). No differences were found.

Figure 4:

Performance of included children versus included adults during the experimental phase by stimulation frequency in terms of (a) accuracy and (b) latency. The variations in the x-axis are random noise that has been added to reduce overlap in the data points and increase readability.

Figure 5:

Performance of included children versus included adults during the experimental phase by round in terms of (a) accuracy and (b) latency.

4. Discussion

This study demonstrated that 9–11-year-old children are able to use an SSVEP-based BCI with much higher accuracy than previously reported [17] and is the first work to report the performance of children when using an SSVEP-based BCI in terms of latency and bitrate (NBR). Furthermore, the accuracies, latencies, and NBRs of the children and adults who completed the experimental phase were nearly identical. The remainder of the discussion is organized by phase: first we discuss the calibration phase, then we discuss the experimental phase, and we conclude by considering future work.

4.1. Calibration phase

A calibration phase was used to calibrate the BCI system and to screen participants for inclusion in the experimental analysis. Data from the calibration phase showed that 11 of the 14 children (Table S1) and all 11 adults (Table S2) included in this study met the minimum threshold for inclusion in the experimental analysis.

Three children did not meet the criteria for their data to be included in the experimental analysis. The experimenters noted that two of the excluded children were visibly distracted during the calibration phase. The calibration task was relatively boring. Children were asked to attend to targets without getting feedback of any kind. It is possible the distracted behavior that was observed during the calibration phase could be an indication of a lack of engagement during this session. We encourage future research to examine the impact of engagement on BCI performance. In addition, an eye-tracker could be used to determine where participants are directing their gaze. It is likely that the third excluded child was simply one of those people who are unable to use an SSVEP-based BCI [37]. By collecting more data, it may be possible to understand differences in BCI “literacy” between children and adults, this represents one direction of future work.

The data collected during the calibration phase were also used to compare the performance of children with that of adults. This comparison was performed using two separate analyses. The first analysis used the same parameters as in the experiments to compare children verses adults. In this analysis, the t values for the children were slightly longer than those used for the adults. We partially attribute this to (1) the fact that the minimum window-length permitted for children was 1.25s while the minimum window-length for adults was 0.5 seconds and (2) for several of the children, the classifier we used during the calibration phase had three harmonics instead of two. Many of the adults (Table S2) had t values of less than 1.25 seconds, which may have skewed these results. The second analysis was conducted after all of the experiments were completed and used the exact same classifier for all participants. When this analysis was performed, the t values calculated for the adults were still shorter than the t values calculated for the children. While the difference was small, this result may suggest (as proposed by [17]) that there are some differences in the ability of children versus adults to generate an SSVEP. It would be interesting to see if this difference is exaggerated in even younger children.

4.2. Experimental phase

Data from the experimental phase show that children (9–11-years-old) can use an SSVEP-based BCI for a target selection task with a similar accuracy, latency, and NBR as adults. It is well known that repetitively flickering stimuli elicit an SSVEP in both children [28] and adults [38]. While healthy adults are known to be able to use this signal to control an SSVEP-based BCI [2, 37, 1, 25, 18], there is only one previous study investigating whether children also have this ability [17]. That study observed that children completed the SSVEP-based BCI task with significantly lower accuracy than adults (median difference 24.4% for 7–11Hz stimuli) [17]. Thus, the present study (1) provides evidence that children have the ability to use SSVEP-based BCIs and (2) suggests that any differences in performance between 9–11-year-old children and adults are smaller than previously believed [17]. Since children can generate an SSVEP and use this signal to control an SSVEP-based BCI for a target selection task, it is possible that SSVEP-based BCIs could be used as a communication devices for children with severe motor disabilities, such as LIS [30].

This is, to the best of our knowledge, the first study to report the performance of 9– 11-year-old children using an SSVEP-based BCI in terms of latency and bitrate (NBR). By considering latency and bitrate (NBR)—in addition to accuracy—our study provides a more complete picture of the performance differences between children and adults. This more complete picture is important, since it is possible for two users of an SSVEP-based BCI to have a similar target selection accuracy, but very different overall performance. For example, S03c (see Table S3) and S09a (see Table S4) both had an average accuracy of 88% in the experimental phase. However, S03c had an average latency of 3.079 seconds and bitrate of 0.42 bits/second compared to S09a who had an average latency of 1.256 seconds and bitrate of 0.90 bits/second. In other words, while these two participants had identical accuracies, S09a communicated twice the information per unit of time—as measured by NBR—as S03c. Thus, the reported accuracy results of Ehlers et al. [17] may have been confounded by the latency of target selection, which they allowed to vary. Here, we show that the performance of children using an SSVEP-based BCI for a target selection task was similar to the performance of adults in terms of accuracy, latency, and NBR.

There appear to be differences in classification accuracy between the calibration phase (92% for children, 96% for adults) and the experimental phase (79% for children, 78% for adults). All of the data collected during the calibration phase was used to set the parameters for the experimental phase. This may have caused overfitting, the parameters we chose were appropriate for the calibration phase data, but did not generalize well to experimental phase data that was subsequently collected. Even with the limited data that was available, this issue may have been mitigated using cross-validation when setting the classifier parameters [39]. This represents a potential area for improvement in future studies.

This study demonstrates that children can use an SSVEP-based BCI much better than previously reported, although there are differences between our study and the only previous investigation of SSVEP-based BCIs for children. Ehlers et al. reported children (mean 9.86 years old) achieved an accuracy of approximately 50% (based on visual analysis of Figure 2 in [17]). The children in our study, on the other hand, achieved an average of 79% accuracy. There are a number of methodological differences between the two studies that could explain this difference:

• The environments in which the children completed the experiments were different. In Ehlers et al. [17] children performed the experiments in a noisy school environment, while in our experiments they were in a quiet laboratory environment. There is value in testing the performance of children under different environmental conditions. Clinical and/or home environments may be noisy in some cases (like in the experiments of Ehlers et al. [17]) and quiet in others (like the experiments reported here). Detailed investigations of the effect of environment and distraction on the performance of children using an SSVEP-based BCI may inform the design of clinical/home systems.

• The tasks were different. The children in our study used the SSVEP-based BCI to complete a target selection task. In Ehlers et al. [17], children completed a text-entry task. Text-entry is a very common platform for SSVEP-based BCI experiments [2, 1], but may not be appropriate for children [27].

• Our experiments used slightly different stimulation frequencies. The frequency of stimulation is known to impact the amplitude of the elicited SSVEP [1].

• We conducted a calibration phase to set two classifier parameters before the experimental phase.

• After low performance in the calibration phase, three children were excluded from the experimental phase of our study. Ehlers et al. [17], however, did not exclude any children from their results. Figure 2b shows—using the data from the calibration phase—that grouping the included and excluded children together may have reduced average performance. This suggests two things. First, it is important to consider the performance of individuals in these studies and identify outliers. Second, excluding the three children may have caused the differences in performance between our study and Ehlers et al. [17] to appear larger than they really are.

It is possible that any of these methodological differences could explain why the children in our study had higher performance than the similar age group of children in the study by Ehlers et al. [17].

One potential limitation of the current study is that the parameters used in the classifier were slightly different for some of the children than they were for the adults. As discussed in Section 4.1, we performed a post-experimental analysis of the calibration data to assess the impact of this difference. This analysis showed that there would have been a slight difference in the t value used for the children if the parameters used in the classifier were identical. The t values calculated for the adults, however, were lower than the t values calculated for children in either analysis. Even though there were small differences in the way the parameters t and τ were set for children versus the adults, it is most likely that these differences lowered the performance of children compared with adults (i.e., children may perform even better than we report here).

4.3. Future work

Given the high performance of 9–11-year-old children observed in this study, we encourage the investigation of SSVEP-based BCIs for younger groups of children. Data collected during the experiments of Ehlers et al. [17] did not reveal any significant differences between groups of children with average ages of 6.73, 8.08, or 9.86 years. Their data suggest a slight downward trend in accuracy with low-frequency stimuli as a function of age. In a study of SSVEPs elicited using 5Hz simulations, Birca et al. [29] show that both the magnitude and phase alignment of SSVEPs (generally) increase with age. This study, however, was limited in how it defined SSVEPs (first harmonic response only) and the stimulation frequencies tested (5Hz only). An earlier study by Birca et al. [29] used a wider range of stimulation frequencies (5, 7.5, 10, and 12.5Hz). The results of this study showed that younger children (6–9 years old) had lower amplitude SSVEPs than adults for 12.5Hz stimuli, but there were no other significant differences between SSVEP amplitude and age. As an additional confound, neither of these studies considered the effect of attention on the amplitude of the elicited SSVEPs. It seems likely—based on our results and these previous studies—that younger groups of children should also be able to use SSVEP-based BCIs. There may, however, be differences in accuracy as a function of age for specific stimulation frequencies.

Another area of future work could include the development of a more engaging interface. Both the target selection task used in our study and the speller used by Ehlers et al. [17] may have been viewed by the participants as boring. We propose that adding elements of games— called gamification—may increase the engagement of children with the SSVEP-based BCI and improve their performance. The use of gamification in lab studies has allowed researchers to improve their ability to conduct studies with children [40].

5. Conclusion

In summary, our data make several contributions to the development of SSVEP-based BCIs for children. We report that of the fourteen 9–11-year-old children who were included in our study, 11 of them achieved an 85% accuracy or above during a short calibration phase. Furthermore, of the 11 children who completed the longer experimental phase, they achieved an average accuracy of 79% in 2.1 seconds, which was very similar to data collected from adults using the same SSVEP-based BCI. This study is also the first to report the performance of children using an SSVEP-based BCI in terms of latency and bitrate. The results of this study imply that children with severe motor disabilities (such as LIS) may use an SSVEP-based BCI to restore/replace the ability to communicate. We encourage future research to follow these results by continuing to explore SSVEP-based BCI research with children.

Table 2:

Post-Experiment Analysis of Data from Calibration Phase (Excluded Children)

Subject	t	τ	Accuracy	Latency	Bitrate
E01c	0.500	0.00	0.40	0.50	0.42
E02c	0.500	0.00	0.40	0.50	0.98
E03c	0.500	0.00	0.60	0.50	1.05
Mean	0.500	0.00	0.47	0.50	0.82
Mdn	0.500	0.00	0.40	0.50	0.98
SD	0.000	0.00	0.12	0.00	0.35

Appendix A. Classification of the predicted target

These equations are based on the original description the canonical correlation for classification of SSVEP targets by Lin [35]. Assuming k stimuli at frequencies f₁...f_k, CCA considers two sets of variables X and Y_k and finds two weight matrices w_X and wy_k that maximize the correlation ρ_k between them.

$${\rho }_k=\frac{w_x^T{\mathrm{\Sigma }}_{X{Y}_k}{w}_{y_k}^T}{\sqrt{w_x^T{\mathrm{\Sigma }}_{XX}{w}_x{w}_{y_k}^T{\mathrm{\Sigma }}_{k_k{Y}_k}{w}_{y_k}}}$$ (A.1)

When CCA is used to detect an SSVEP, X represents a matrix of EEG data (m channels by n samples) and Y_k represents a matrix of sine and cosine reference waves (r reference waves by n samples) at harmonic h frequencies of stimulus k.

$$Y_k=\left[\begin{array}{c}sin(2\pi ft1)\\ cos(2\pi ft1)\\ ⋮\\ sin(2\pi fth)\\ cos(2\pi fth)\end{array}\right]$$ (A.2)

ρ_k is initially a vector with a length equal to the smaller of m and r, however, here we consider ρ_k = max(ρ_k). If

$$max(\rho )>\tau$$ (A.3)

then the classifier’s best guess (the predicted target) is

$$\widehat{k}=argmax(\rho )$$ (A.4)

Each trial of the calibration phase was analyzed in the following way. The analysis of each trial started at time 0, the time of stimulus onset. The data from time 0 to time t was then considered, if the max(ρ_k) does not exceed the threshold, then the window slid in steps of 0.125 seconds. After the window is moved, ρ_k is recomputed and compared to the threshold again. This process continues until ρ_k exceeds τ or until t extends beyond the end of the trial. In the case that ρ_k exceeded the threshold, $\widehat{k}$ is then compared to the target x^⋆. If ρ_k does not exceed τ before the end of the trial, then this was modeled in the calculation of NBR as an erasure.

Appendix B. Supplemental information

Table S1:

Data from Calibration Phase (Included Children)

Subject	t	τ	Accuracy	NBR
S01c*	1.375	0.75	0.87	0.57
S02c*	1.250	0.77	0.87	0.78
S03c	2.500	0.43	0.87	0.47
S04c	1.375	0.47	1.00	1.01
S05c*	1.250	0.57	0.94	0.94
S06c	2.250	0.35	0.87	0.46
S07c*	1.375	0.58	0.93	0.71
S08c	1.250	0.58	0.93	0.78
S09c	1.250	0.64	0.93	0.78
S10c	1.875	0.36	0.87	0.62
S11c*	1.250	0.72	1.00	0.78
Mean	1.545	0.57	0.92	0.72
Mdn	1.375	0.58	0.93	0.78
SD	0.450	0.15	0.05	0.18

^*- Denotes that 3 harmonics and erasure were used for calculation of window-length and threshold.

Table S2:

Data from Calibration Phase (Included Adults)

Subject	t	τ	Accuracy	NBR
S01a	1.000	0.60	0.93	1.14
S02a	1.375	0.51	0.93	0.75
S03a	1.125	0.57	1.00	0.85
S04a	0.750	0.70	1.00	0.88
S05a	3.625	0.27	0.93	0.35
S06a	1.625	0.42	0.93	0.62
S07a	0.750	0.68	0.93	1.12
S08a	0.750	0.74	0.93	0.93
S09a	0.875	0.66	0.93	1.11
S10a	0.625	0.82	1.00	1.20
S11a	1.000	0.60	1.00	0.91
Mean	1.227	0.60	0.96	0.90
Mdn	1.000	0.60	0.93	0.91
SD	0.849	0.15	0.04	0.25

Table S3:

Data from the Experimental Phase (Children)

Subject	Accuracy	Latency	NBR
S01c	0.92	2.211	0.62
S02c	0.82	2.432	0.49
S03c	0.88	3.079	0.42
S04c	0.85	1.522	0.67
S05c	0.87	1.402	0.84
S06c	0.47	2.513	0.14
S07c	0.78	2.282	0.41
S08c	0.73	1.749	0.43
S09c	0.83	1.858	0.57
S10c	0.60	2.044	0.26
S11c	0.93	2.073	0.64
Mean	0.79	2.106	0.50
Mdn	0.83	2.073	0.49
SD	0.14	0.48	0.20

Table S4:

Data from the Experimental Phase (Adults)

Subject	Accuracy	Latency	NBR
S01a	0.83	1.424	0.73
S02a	0.85	1.602	0.66
S03a	0.80	1.900	0.46
S04a	0.90	1.467	0.85
S05a	0.57	3.913	0.14
S06a	0.63	2.028	0.30
S07a	0.73	1.947	0.44
S08a	0.67	2.266	0.35
S09a	0.88	1.256	0.90
S10a	0.90	1.569	0.77
S11a	0.80	1.710	0.59
Mean	0.78	1.92	0.56
Mdn	0.80	1.71	0.59
SD	0.11	0.73	0.25