Skip to main content
NCBI home page
Cognitive Neurodynamics logoLink to Cognitive Neurodynamics
. 2022 Apr 25;17(1):105–118. doi: 10.1007/s11571-022-09808-z

Human factors engineering of BCI: an evaluation for satisfaction of BCI based on motor imagery

Xiaotong Lyu 1,2, Peng Ding 1,2, Siyu Li 1,2, Yuyang Dong 1,2, Lei Su 1, Lei Zhao 4, Anmin Gong 3, Yunfa Fu 1,2,✉
PMCID: PMC9871150  PMID: 36704636

Abstract

Existing brain-computer interface (BCI) research has made great progress in improving the accuracy and information transfer rate (ITR) of BCI systems. However, the practicability of BCI is still difficult to achieve. One of the important reasons for this difficulty is that human factors are not fully considered in the research and development of BCI. As a result, BCI systems have not yet reached users’ expectations. In this study, we investigate a BCI system of motor imagery for lower limb synchronous rehabilitation as an example. From the perspective of human factors engineering of BCI, a comprehensive evaluation method of BCI system development is proposed based on the concept of human-centered design and evaluation. Subjects’ satisfaction ratings for BCI sensors, visual analog scale (VAS), subjects’ satisfaction rating of the BCI system, and the mental workload rating for subjects manipulating the BCI system, as well as interview/follow-up comprehensive evaluation of motor imagery of BCI (MI-BCI) system satisfaction were used. The methods and concepts proposed in this study provide useful insights for the design of personalized MI-BCI. We expect that the human factors engineering of BCI could be applied to the design and satisfaction evaluation of MI-BCI, so as to promote the practical application of this kind of BCI.

Keywords: Human factors engineering of BCI, Human-centered BCI design, Satisfaction of BCI, MI-BCI, BCI

Introduction

The modern brain-computer interface (BCI) is a revolutionary example of human–computer interaction (Graimann et al. 2013; Wang et al. 2013). It aims to bypass the peripheral nerve and muscle system to achieve direct communication and control between the brain and the external world, so as to improve or enhance the quality of life of disease patients, disabled humans, and healthy individuals (Blankertz et al. 2010; Wolpaw and Wolpaw 2012; Zjajo 2016; Wang et al. 2017). However, there is still a gap between the research of BCI and its practical application (Kübler et al. 2020; Xu et al. 2021). To narrow this gap, human factors engineering of BCI must be fully considered to provide satisfactory BCI goods and services. The concepts and methods of human factors engineering of BCI play an important role in promoting the practicality of BCI (Lu et al. 2021).

Human factors engineering of BCI requires the design and evaluation of the BCI system with the human as the center, as shown in Fig. 1. In a human-centered BCI engineering, the designer first determines the required BCI application field according to the needs of the human, then guides the selection and development of BCI software and hardware systems, obtains the initial prototype of the BCI system, and determines the roles played by the user in their relationship with BCI. These users may fall into several categories: (1) Participants in the BCI production process (such as experimental subjects); (2) users of output BCI (specifically refers to users who use output-type BCI, and the BCI system has no input-type effect on the user); (3) users of input BCI (specifically referring to a class of objects that use the BCI system to control, adjust, and affect the user's state). The designer must determine the scales and engineering metrics of the satisfaction evaluation part of the BCI system according to the corresponding roles. The satisfaction of BCI system is then evaluated and calculated. After continuous iteration, BCI products with high user satisfaction that can be used in daily life are finally obtained. It should be noted here that this study evaluates the satisfaction of a lower limb motor imagery of BCI (MI-BCI) system from the perspective of subjects (i.e., participants in the BCI production process; healthy humans).

Fig. 1.

Fig. 1

Open in a new tab

The relationship among human factors engineering of BCI, BCI system, and the human. The figure shows the relevant elements and the relevant processes used for MI-BCI design and satisfaction evaluation

So far, some literatures have proposed user-centered design to improve the usability of BCI (Zickler et al. 2011; Zickler et al. 2013; Holz et al. 2013; Schreudera et al. 2013; Kübler et al. 2013; Kübler et al. 2014, 2019; Liberati et al. 2015; Riccio et al. 2016; Chavarriaga et al. 2017; Choi et al. 2017; Martin et al. 2018;

Wolpaw et al. 2020; Abiri et al. 2020; Branco et al. 2021). These studies have provided valuable ideas and methods. Proposals for usability of BCI mentioned in the literature mainly include: (1) the effectiveness of BCI (accuracy); (2) the efficiency of BCI (information transfer rate (ITR) + NASA task load index (NASA-TLX)); (3) the satisfaction of BCI (mainly evaluated by a visual analog scale (VAS), extended QUEST 2.0, assistive technology device predisposition assessment (ATD-PA), use in daily life, and follow-up) (Zickler et al. 2013; Holz et al. 2013; Schreudera et al. 2013; Kübler et al. 2013; Branco et al. 2021). Based on these ideas and methods, existing studies have conducted user-centered design (UCD) for steady-state visual evoked potentials of BCI (SSVEP-BCI), P300-BCI, and hybrid BCI and evaluated their usability (Zickler et al. 2013; Schreudera et al. 2013; Kübler et al. 2014; Kübler et al. 2019; Miralles et al. 2015; Choi et al. 2017; Wolpaw et al. 2020; Branco et al. 2021), but relatively few UCD and evaluations have been conducted for MI-BCI (Holz et al. 2013; Choi et al. 2017). In addition, in these studies, the various groups of human beings involved in BCI are collectively referred to as BCI users. The roles and application fields represented by these so-called users are not distinguished, and there is a lack of targeted evaluation.

This study aims to conduct a human-centered design and evaluation of a lower limb MI-BCI system based on the human factors engineering of BCI. On the basis of traditional metrics (such as accuracy and ITR) for evaluation, and further considering factors such as the environment in which BCI products are used and user needs, we combine these factors with the user satisfaction survey scales of the BCI system to jointly evaluate subjects’ satisfaction (Lu et al. 2021). After the introduction, the remainder of this paper is as follows: The second part describes human factors engineering of BCI. The third part describes the methods used, including the experimental data description, the design, and the evaluation method of the human-centered MI-BCI (Zickler et al. 2013; Holz et al. 2013; Schreudera et al. 2013; Kübler et al. 2014, 2019; Miralles et al. 2015; Choi et al. 2017; Wolpaw et al. 2020; Wang et al. 2020; Abiri et al. 2020; Branco et al. 2021). The fourth part is the MI-BCI satisfaction evaluation result. Part 5 is the discussion, and part 6 is the conclusion.

Human factors engineering of BCI

The signal source of communication or control between a BCI system and the external world is the brain signal generated by the human central nervous system. In other words, the human is the direct signal source or controller of the BCI system, and human and system are directly connected or coupled. The most remarkable feature of the system is that the human directly controls external equipment with their own brain, and the human is always in the circuit. Human experience is an important part of the effect and service quality of BCI. Therefore, it is imperative to consider human factors engineering throughout the research and development process of the entire BCI system.

Human factors engineering of BCI studies the interactions among human, hardware, and software for BCI, external machine and environment. Its elements are shown in Fig. 2, The most core elements are BCI users, hardware and software for BCI. For BCI users, different BCI users have different needs, so the BCI system should be designed and developed according to such needs. At the same time, the capability characteristics of BCI user (human parameters and biomechanical characteristics, cognitive and decision-making ability, operation and control ability, etc.) and status characteristics (mood, workload and human factor reliability, etc.) should also be considered. For BCI hardware and software, in terms of hardware, the appearance and weight of amplifier and BCI sensor greatly affect whether users choose BCI products. In addition, it is necessary to design a universal interface between BCI and external devices to facilitate the communication between various external devices and BCI. In terms of software, simple, natural and comfortable BCI psychological tasks can reduce the mental load and induce significantly different and trustworthy brain signal characteristics to reduce the ineffectiveness of BCI. Brain signal processing and decoding affect the accuracy and ITR of BCI to a large extent, and thus seriously affect the user experience. Intelligent peripheral equipment and an intelligent environment are also indispensable. More details about human factors engineering of BCI elements are detailed in the literature (Lu et al. 2021). The goal of human factors engineering of BCI is to bring the design of BCI system under different use conditions more in line with the characteristics, capabilities, and needs of the individual human. The ultimate goal of BCI design is to achieve the best match between humans (BCI users) the BCI system, and the environment, and finally to obtain a safe, reliable, and efficient BCI system. The above objectives determine that the human should be the centre of the design and evaluation of BCI.

Fig. 2.

Fig. 2

Open in a new tab

Elements to be considered in human factors engineering of BCI

Methods

EEG data description

In this study, a total of 16 healthy right-handed subjects (13 males and 3 females, aged between 20 and 30 years) participated in the research and evaluation of a lower limb synchronous rehabilitation online BCI system, as shown in Fig. 3A. All subjects were not affected by any drugs that might affect their imagination response, and experimental trials were conducted in a clear state of mind. Before starting the experimental record, all subjects were informed of the experimental process and filled in the informed consent form.

Fig. 3.

Fig. 3

Open in a new tab

A Overall framework of online BCI system for lower limb synchronous rehabilitation. STFT is short-time Fourier transform. B Experimental paradigm. C The BCI system selects the five electrode according to the international 10–20 system, which are Fp1, Fp2, C3, Cz, and C4

The system adopted a 19 conducting saline EEG cap with electrodes placed in accordance with the international 10–20 system. Five electrodes (C3, C4, CZ, FP1, and FP2) were selected to collect EEG signals, as shown in Fig. 3C. EEG data acquisition was divided into offline and online parts. In the offline part, the experiment used three types of lower limb MI paradigms: leg flexion backward with standing posture, leg extension forward with standing posture, and normal walking with standing posture. There were 60 trials in total—20 trials for each action—and each trial lasted 9 s. The experimental paradigm is shown in Fig. 3 B: (1) Preparation stage (2 S): The subjects stand in front of the computer and relax their entire body. (2) Prompt stage (4 s): A plus sign is displayed on the screen in the first 2 s, which indicates that the participant's MI is about to begin, and the next 2 s a text and picture prompt is presented, which reminds the participant of the content of the participant's MI. (3) MI stage (3 s): During this period, the screen is black, and the subjects carry out the corresponding MI according to the prompt. (4) Rest phase (2 s): The prompt text and pictures disappear, the screen shows a black screen, and the subjects stop MI, relax, and rest. Then, the collected EEG data are trained. Model matching is conducted to make each parameter of the system model adapt to the corresponding subject. During the online part, an interactive interface is opened. According to the prompts, the subjects perform one of three kinds of actions: leg flexion backward with standing posture, leg extension forward with standing posture, and normal walking with standing posture. Finally, the corresponding instructions are output to complete the target action.

Human factors engineering design of BCI

This study follows a human-centered concept and method, using the rapid prototyping method to design and evaluate the online BCI system for synchronous rehabilitation of lower limbs. To make the initial prototype of BCI quickly adapt to human needs and obtain a BCI system with high satisfaction (Millan et al. 2004; Galán et al. 2008), as shown in Fig. 4 (Guger et al. 2003; Muhammad 2011). The process includes the following steps: first, questionnaires and semi-standardized interviews are set up according to the above BCI human factors engineering elements and study (Jeunet et al. 2021) to obtain the basic needs of users with movement disorders, such as appearance, function, after-sales service and remote support, etc. Next, according to the requirements three kinds of lower limb movement imagine paradigm are designed for data collection including leg extension forward with standing posture, leg flexion backward with standing posture and normal walking with standing posture. The amplitude envelope features of EEG data were extracted, using naive Bayesian classifier (NBC), quadratic discriminant analysis (QDA) and the decision tree model to classify, selection of the optimal classification model based on accuracy. It uses of the same algorithm for real-time processing to obtain the initial online BCI system for synchronous rehabilitation of lower limbs. Then it makes the participants evaluate satisfaction BCI initial prototype (See section “Satisfaction evaluation of BCI system” for details).

Fig. 4.

Fig. 4

Open in a new tab

Schematic diagram of rapid prototyping method for BCI system development

The researchers in the subjective evaluation results on the basis of combining the traditional quantitative evaluation results (e.g., accuracy and information transfer rate) choose to give up or be corrected and improved according to the result of comprehensive analysis. To obtain improved BCI prototype (perfect BCI hardware and software functions). Then further evaluate the satisfaction of BCI users and iterate. Until the improved BCI prototype satisfies the user and uses it as the basis for the application system or application system development.

Satisfaction evaluation of BCI system

The colored box in Fig. 4 lists various metrics and scales used to evaluate the subjects' satisfaction with the BCI system, mainly including accuracy, ITR, subjects’ satisfaction rating for BCI sensors, VAS, subjects’ satisfaction rating for the BCI system (Zickler et al. 2011; Kübler et al. 2014; Colucci et al. 2021), mental workload rating for subjects manipulating the BCI system (Fu et al. 2020), ATD-PA, and interview/follow-up. Among them, the accuracy rate is the most commonly used evaluation index in BCI research and development, expressed by the correct number of trials divided by the total number of trials. ITR is another commonly used indicator in BCI research and development, which refers to the amount of information transmitted by the system in a unit of time (e.g., 1 min). In the development of KMI-BCI, research and development personnel often pay more attention to improving the accuracy and ITR of the classifier (Holz et al. 2013; Kübler 2013; Alazrai et al. 2019). They pursue high accuracy and speed, but ignore the individual differences of human. So this research pays more attention to subjective evaluation of the human.

The quality of brain signals collected by the BCI system greatly affects its performance (such as accuracy and ITR). However, there are five additional aspects that are also very important: the safety, comfort, ease of use, esthetics, and overall satisfaction of BCI sensors. They largely determine subjects’ acceptance or practicability of BCI. Table 1 depicts subjects’ satisfaction rating for BCI sensors and lists the subjects' satisfaction evaluation of the BCI sensor with regard to the above five aspects. Different BCI subjects may have different needs in the five aspects. Therefore, the weighted method is used in the scale to calculate the overall evaluation of satisfaction with the BCI sensor by different subjects according to their own preference. Table 1 was used to select the type of EEG caps that should be selected during the initial development of the BCI system and in determining how to improve the EEG caps during follow-up visits. Because humans rarely conduct psychological activities such as KMI in their daily life, they lack the experience or skills necessary to control this psychological activity, and there is serious motor imagination blindness (mainly when referring to human without MI ability or with poor MI ability) (Thompson 2019; Tian et al. 2021). Therefore, based on the above evaluation of the BCI sensor, 16 subjects were asked to conduct a preliminary test with the prototype of the developed BCI system, and 10 subjects with good effects (represented by S1–S10) were selected to participate in the follow-up satisfaction evaluation of the KMI-BCI system development process. For example, simple and fast VAS is used to evaluate the satisfaction of each task performed by the BCI system, as shown in Table 2 (Allison et al. 2012; Kübler et al. 2014). In Table 2, VAS is used to measure the satisfaction of subjects for each task. It is simple and fast, but there are a few evaluation items. It is inconvenient to find problems with the BCI systems in the rapid prototype research and development iteration process. For that, after the prototype of BCI, the subjects satisfaction rating for BCI system based on Quest 2.0 and its expansion table can be used to evaluate the satisfaction of BCI subjects with the system from different angles, as shown in Table 3 (Chan et al. 2006; Zickler et al. 2011; Kübler et al. 2014; Lu et al. 2021; Colucci et al. 2021). It should be noted that items 13–16 in Table 3 (BCI equipment professional services provided by medical personnel, BCI equipment durability, BCI equipment maintenance services, and BCI equipment follow-up consultation and tracking services) were used for the evaluation of subjects with the final BCI products. However, this study mainly focuses on the development process of a lower extremity KMI-BCI system, and thus the 10 subjects did not evaluate these four items. The qualities of items 1, 3, and 5 in Table 3 affect the subject's mental load, and excessive mental load will affect the subject’s efficiency, thus increasing the difficulty of manipulating BCI equipment, and affecting subjects' satisfaction with the BCI system. Therefore, we use the mental workload rating for subjects manipulating BCI system based on the NASA task load index to evaluate the mental load, as shown in Table 4 (Hart and Staveland 1988; Kübler et al. 2014; Branco et al. 2021).

Table 1.

Subjects’ satisfaction rating for BCI sensors

The type of BCI sensor Safety Comfort Ease of use Aesthetics Weighted average
Sensor for non-invasive BCI EEG sensor on scalp surface Conductive gel electrode 5 3.31 3.38 3.19 3.56
Physiological saline electrode 4.73 3.69 4.1 3.9 4.03
Dry electrode 4.53 3.63 4.63 4.09 4.27
NIRS sensor Emitting and detecting probes 4.57 3.37 3.9 4.11 3.89
MEG sensor Non-contact sensor for measuring magnetic field strength  ×   ×   ×   ×   × 
Other non-invasive BCI sensor Dry or wet general electrode 5 4 4 5 4.5
Sensor for invasive BCI ECoG sensor Platinum electrode array  ×   ×   ×   ×   × 
Intracortical sensor (spikes, LFP) Multi-electrode array  ×   ×   ×   ×   × 
Multi-site electrode  ×   ×   ×   ×   × 
Cone-shaped electrode  ×   ×   ×   ×   × 
Microwires array  ×   ×   ×   ×   × 
Other invasive BCI sensor  ×   ×   ×   ×   × 
Open in a new tab

1 = very dissatisfied, 2 = dissatisfied, 3 = neutral, 4 = satisfied, 5 = very satisfied. Dry or wet general electrode are only used by one person and do not participate in the analysis

Table 2.

Visual Analog Scale (VAS)

Task Satisfaction
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 Average
Leg flexion backward with standing posture 7 8 5 6 8 7 6 7 7 6 6.70
Leg extension forward with standing posture 8 7 5 7 8 6 7 8 6 7 6.90
Normal walking with standing posture 8 4 6 7 10 9 7 8 7 7 7.30
Average 7.67 6.33 5.33 6.67 8.67 7.33 6.67 7.67 6.67 6.67
Open in a new tab

Satisfaction ranges from "very dissatisfied (1)" to "Absolutely satisfied (10)"

Table 3.

Subjects’ satisfaction rating for BCI system

Evaluation item Evaluation item description Order of evaluation
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 Average
1. How satisfied are you with the comfort level of your current BCI equipment ? What is the comfort level of the BCI sensor and the comfort level of mental tasks (SSVEP, P300, MI) ? 5.00 3.00 3.00 3.00 4.00 3.00 3.00 2.00 3.00 3.00 3.20
2. How satisfied are you with the size (length, width, and height) of the current BCI equipment ? Are the sizes of BCI sensors and amplifiers ultra-miniaturized or portable ? 4.00 2.00 3.00 2.00 3.00 2.00 2.00 3.00 3.00 3.00 2.70
3. How satisfied are you with the ease of use of the current BCI equipment ? Is BCI graphical user interface simple and easy to use, and are mental tasks easy to complete? 4.00 3.00 2.00 3.00 5.00 4.00 3.00 2.00 3.00 4.00 3.30
4. How satisfied are you with whether the current use of BCI equipment can be assistive or its effectiveness ? How satisfied are you with the tasks accomplished by SSVEP-BCI P300-BCI MI-BCI ? 4.00 3.00 2.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 3.10
5. How satisfied are you with whether the current BCI equipment is easy to install and adjust? Is the software and hardware of the BCI system easy to install and adjust? Specifically, it may include whether the sensor is easy to wear and adjust, the amplifier parameter setting, whether the BCI software is easy to install and set, and whether the BCI and the external device are easy to communicate with the interface 5.00 4.00 5.00 4.00 5.00 4.00 3.00 3.00 3.00 4.00 4.00
6. How satisfied are you with the safety of BCI equipment? How safe is the invasive BCI sensor? How safe is the BCI control system? E.g., the obstacle avoidance ability of a brain-controlled wheelchair 4.00 3.00 3.00 4.00 4.00 3.00 4.00 4.00 3.00 3.00 3.50
7. How satisfied are you with the access channel and efficiency of BCI equipment? Obtain BCI after-sales service channels and service efficiency, including whether BCI can be used by independent families, and minimize the dependence on BCI technical support 3.00 2.00 2.00 3.00 4.00 3.00 3.00 3.00 3.00 3.00 2.90
8. How satisfied are you with the weight of BCI equipment currently in use? Are BCI sensors and amplifiers super light? 4.00 2.00 3.00 3.00 3.00 2.00 2.00 3.00 3.00 3.00 2.80
Average 4.13 2.75 2.88 3.13 4.00 3.00 2.88 2.88 3.00 3.25
9. How satisfied are you with the reliability of the current BCI equipment? What is the ability of the BCI system to perform specified functions without failure in a certain period of time and under certain conditions, such as reliability, failure rate and mean time between failures? 3 3 2 3 4 3 3 3 3 3 3.00
10. How satisfied are you with the response time of the current BCI equipment? How fast is the BCI system? What is the specific ITR? 5 4 3 4 5 4 3 4 4 4 4.00
11. How satisfied are you with the learnability for BCI? Is the operation of the BCI system easy to learn? This includes whether the BCI graphical user interface (GUI) and mental tasks are learnability 4 4 2 3 4 3 3 2 3 4 3.20
12. How satisfied are you with the appearance of BCI equipment? Are the graphical user interface (GUI) and sensors of the BCI system beautiful? For the BCI sensor: Is it concealed and does it match the visual aesthetics? 4 2 3 3 3 3 3 3 3 3 3.00
Average 4.00 3.25 2.50 3.25 4.00 3.25 3.00 3.00 3.25 3.50
13. How satisfied are you with the professional services of BCI equipment provided by medical staff? For the clinical application of BCI, it is necessary to evaluate the professional service quality of medical staff  ×   ×   ×   ×   ×   ×   ×   ×   ×   × 
14. How satisfied are you with the robustness and durability of BCI equipment currently in use? How robust are the BCI sensors and amplifiers?  ×   ×   ×   ×   ×   ×   ×   ×   ×   × 
15. How satisfied are you with the maintenance service of BCI equipment currently in use? What is the frequency of BCI system failure or maintenance and the quality of maintenance service? Including easy contact and maintenance efficiency  ×   ×   ×   ×   ×   ×   ×   ×   ×   × 
16. How satisfied are you with the follow-up BCI equipment consultation and tracking services provided by medical staff? For the use of BCI in follow-up daily life, we need to evaluate the quality of follow-up service of medical staff  ×   ×   ×   ×   ×   ×   ×   ×   ×   × 
Open in a new tab

The evaluation scale is "very dissatisfied (1)" to "very satisfied (5)". The user satisfaction evaluation scale Quest 2.0 and its extension table are referred to (Colucci et al. 2021)

Table 4.

Mental workload rating for subjects manipulating BCI system

Dimension Description Evaluation rating
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 Average
Mental (psychological) needs Manipulate BCI to complete the mental activities required to complete the task, whether the task is difficult 80 70 90 80 70 80 80 85 70 70 77.50
Physical (physiological) needs The physical strength required to control the BCI to complete the task, whether the muscles are tense, and the movements are relaxed 50 60 80 70 60 70 60 80 60 60 65.00
Time requirement Does the speed requirement for manipulating BCI to complete tasks make human feel nervous or panicked? 50 30 60 60 50 50 60 70 70 50 55.00
Effort level The level of effort required to control the BCI to complete the task 60 60 80 80 70 70 70 80 80 80 73.00
Performance level Whether the performance level of controlling the BCI to complete the task is satisfactory. (The better the performance, the lower the score) 20 30 40 25 25 30 25 30 25 35 28.50
Frustration The level of depression and frustration about the effectiveness of BCI manipulation 50 50 85 60 40 60 50 30 60 50 53.50
Weighted average 60.67 48.67 73.00 68.67 49.67 63.67 55.60 63.00 56.67 56.00
Open in a new tab

The rating scale is from 0 to 100. In addition to the performance level, the higher the demand, the higher the score. Because a higher performance level indicates that the BCI is easy to use. With reference to the NASA-TLX scale (Hart and Staveland 1988)

Next, ATD-PA was used to evaluate the matching between human and assistive technology (Zickler et al. 2013; Holz et al. 2013). In this study, subjects (healthy humans) evaluate their satisfaction with the online BCI system for synchronous rehabilitation of a lower limb. The subjects had high requirements for various performance of the BCI system and low demand, as a result, 10 subjects did not participate in the ATD-PA evaluation.

The evaluation results of the scale mentioned above are determined by BCI users and inevitably have certain subjectivity. Therefore, it is up to BCI developers to calculate and verification using accuracy and ITR metrics. Moreover, the accuracy rate and ITR are indirectly related to the satisfaction of BCI system. For example, only when the accuracy rate and ITR of BCI system are high, users will be satisfied with items 4, 9 and 10 in Table 3 and performance level in Table 4, so that the overall satisfaction of BCI system can achieve a good result. Therefore, accuracy and ITR are added on the basis of Table 1 to Table 4 to obtain the satisfaction evaluation results of the BCI system, as shown in Table 5. Finally, open-ended question follow-up and interviews were conducted with BCI users to improve the follow-up service level. In the next section, we discuss the research results of the above stages.

Table 5.

Summary of BCI satisfaction evaluation metrics and scale

S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
Accuracy (%) 66.7 60 53 66.7 86.7 66.7 60 60 60 60
ITR 6.49 4.17 2.37 6.49 17.24 6.49 4.17 4.17 4.17 4.17
VAS 7.67 6.33 5.33 6.67 8.67 7.33 6.67 7.67 6.67 6.67
Satisfaction rating for BCI system (1–8) 4.13 2.75 2.88 3.13 4.00 3.00 2.88 2.88 3.00 3.25
Satisfaction rating for BCI system (9–12) 4.00 3.25 2.50 3.25 4.00 3.25 3.00 3.00 3.25 3.50
Mental workload rating for manipulating BCI system 60.67 48.67 73.00 68.67 49.67 63.67 55.60 63.00 56.67 56.00
Open in a new tab

Satisfaction evaluation results of MI-BCI system

Subjects satisfaction rating for BCI sensors

For the KMI-BCI experiment, 16 subjects evaluated the conductive gel electrode, physiological saline electrode, dry electrode, emitting and detecting probes, and 1 subject evaluated the dry or wet general electrode. The evaluation of the used BCI sensor is shown in Table 1. In Table 1, an × indicates that the subjects did not use this type of BCI sensor in the MI-BCI experiment. As can be seen from Table 1, the subjects were satisfied with the dry electrode and the saline electrode (the weighted average score was between 4 and 5), the scores for safety and ease of use were above 4, and the scores of comfort and aesthetics were also above 3.6. The subjects were generally satisfied with the conductive gel electrode (weighted average score: 3.56), and its safety was high (score: 5). It should also be noted that although the weighted average score of the universal dry or wet electrodes is 4.5, only one person used them, so there is no analytical significance.

Visual analog scale

After 10 subjects performed three types of MI tasks, simple and rapid VAS was used to evaluate the satisfaction of each task performed by the online BCI system with synchronous rehabilitation of lower limbs. The scores were shown in Table 2. As can be seen from the Table, subjects S1, S5, S6, and S8 were satisfied with the BCI system (scoring between 7 and 9), while subjects S3 had a low score of 5.33. The subjects scored the highest on the task of normal walking with standing posture (7.30), while the other two tasks scored similar (6.70, 6.90).

Subjects satisfaction rating for BCI system

Table 3 depicts statistics of the satisfaction of the subjects in evaluating the use of the KMI-BCI system from different angles, where × means that 10 subjects did not evaluate these four items. From the satisfaction score in the Table, subjects S1 and S5 are satisfied with the BCI system. The average score of the 12 items is between 4 and 5, while the average score of subject S3, S7, and S8 is below 3, indicating that they are not satisfied with the BCI system. Subject S2 is dissatisfied with the evaluation by BCI system on table items 1–8, but has a medium degree of satisfaction on items 9–12. The response time (i.e., ITR) and the ease of installation and adjustment of the BCI system scored higher across these 10 subjects, being 4.00 respectively. In terms of BCI equipment size, BCI equipment weight, and BCI equipment service channel and acquisition efficiency, the scores were 2.70, 2.80, and 2.90, respectively. BCI system comfort, ease of use, accuracy, safety, reliability, and learnability satisfaction are all medium (score between 3 and 4).

Mental workload rating for subjects manipulating BCI system

Mental workload for subjects manipulating BCI system was divided into 6 dimensions for evaluation, as shown in Table 4 (Branco et al. 2021). The weighted averages of mental workload of the 10 subjects were 60.67, 48.67, 73.00, 68.67, 49.67, 63.67, 55.60, 63.00, 56.67 and 56.00. The lowest mental load was 48.67, which appeared in S2 subjects, and the highest mental load was 73.00, which appeared in S3 subjects. All subjects had higher mental (psychological) needs and effort level, and all scored above 60 points.

Summary of BCI satisfaction evaluation metrics and scale

In order to avoid the bias or contingency of the evaluation, various objective satisfaction evaluation data of the above KMI-BCI system are combined with the corresponding quantitative indicators (accuracy and ITR) for statistical analysis, as shown in Table 5. Among the subjects, S3 has the lowest accuracy (53%) and S5 has the highest accuracy (86.7%). The ITR of S3 was the slowest (2.37), and that of S5 was the fastest (17.24). S3 had the lowest VAS score (5.33), whereas that of S5 having the highest VAS score (8.67). S2 has the lowest average score (2.75) of subjects’ satisfaction rating for BCI system (1–8), and the average score (4.13) subjects’ satisfaction rating for BCI system (1–8) of S1 was the highest. S3 has the lowest mean value of 2.50 in the subjects’ satisfaction rating for BCI system (9–12), while S1 and S5 have the highest mean value of 4.00 in the subjects’ satisfaction rating for BCI system (9–12). S3 has the largest mental load (73.00), and S2 has the smallest mental workload (48.67). Except for Table 1 and Table 4, which adopts the weighted average method, the arithmetical average value of each subject is used as the final result for other scales and metrics.

Follow-up

In follow-up, subjects raised many concerns: (1) The BCI system is bulky and cannot be used independently; (2) hair should be washed after using the saline electrode cap; (3) compared with the existing traditional assistive technology, the speed is slower; (4) the appearance of the hat is eye-catching and does not conform to the aesthetic design; (5) the experimental paradigm is complex and requires high concentration and mental load; (6) the accuracy of the system needs to be improved and the degree of frustration is high; and (7) except S2, other subjects reported a great sense of pressure and discomfort when wearing EEG caps for a long time.

Discussion

At present, applications based on P300-BCI and SSVEP-BCI provide better accuracy and ITR, but MI-BCI has more application potential in daily life. This study designs and evaluates the KMI-BCI system for lower limb synchronous rehabilitation. The movement classification involved in the KMI-BCI system was more difficult to perform and classify than common left-hand and right-hand movement, and the average recognition accuracy was 63.98 ± 9.06%. However, from the human-centered perspective, this paradigm is more in line with the characteristics and use demands of rehabilitation patients. For esthetics, easy installation, and adjustment, five channels were adopted under the condition of little performance loss to improve the satisfaction of subjects with this BCI system.

Table 1 shows that the dry electrode has the highest subject satisfaction and the highest EEG cap ease of use, but the EEG signal acquisition quality was found to be poor in subsequent tests. The conductive gel electrode has the highest safety and the best quality of EEG signal acquisition, but there are huge obstacles in terms of comfort, ease of use, and esthetics, which affect user experience. Combined with the satisfaction evaluation and analysis results in Tables 2, 3, 4, 5 above, we suggest the use of physiological saline EEG caps with good signal quality and high satisfaction, or alternatively to conduct personalized customization. In Table 2, each BCI subject had the highest satisfaction with the normal walking task with standing posture, while the tasks of leg extension forward with standing posture and leg flexion backward with standing posture scored lower, mainly because the former was a MI task commonly used by subjects, while the latter two tasks were not commonly used and difficult to imagine. Subject S3 had the lowest overall satisfaction with each task performed by the BCI system, which led to the lowest accuracy rate of the BCI system in S3. Although S2 had higher scores in the tasks of leg extension forward with standing posture and leg flexion backward with standing posture, the normal walking with standing posture score is too low, which leads to the lower accuracy of the BCI system of S2, as shown in Table 5. In Table 3, 10 subjects were satisfied with the response time (i.e., ITR), ease of installation and adjustment of BCI system, but had poor experience sense of BCI equipment size, BCI equipment weight, and BCI equipment service channel and acquisition efficiency. In Table 5, the accuracy of S2 is not the lowest, but the score in subjects’ satisfaction rating for BCI system (1–8) is the lowest. Combined with Table 3 and follow-up analysis, it is found that S2 is female with high requirements for the aesthetics of the BCI system, which affected her satisfaction evaluation of the BCI system. Each subject has individual differences, so it is necessary to judge the relative weight of each factor in Table 4 in the formation of mental workload. The pairwise comparison method was adopted to evaluate the importance of each factor. With 15 groups of comparison results being obtained (namely, the weights of 6 factors were determined), and the scores of 6 factors in Table 4, the final mental workload result was obtained by weighted average. When calculating factor weights, it was found that the weight proportions of effort level, mental (psychological) needs, performance level and frustration degree were high, and the dimension loads of effort level and mental (psychological) needs were large, which may have led to a performance decline of our system, as shown by S3 in Table 5. In Table 5, S3 has the lowest accuracy, and besides subjects’ satisfaction rating for BCI system (1–8), the satisfaction evaluation result of the scale is also the worst. S5 has the highest accuracy, but the satisfaction evaluation result of subjects’ satisfaction rating for BCI system (1–8) is not the best. That is to say, the system accuracy seriously affects the availability of BCI system, but it is not the only factor. Except for a few subjects, the overall satisfaction level reached above medium level, but the subjects could not imagine using it in daily life.

Compared with traditional BCI systems, the BCI system developed by human factors engineering of BCI concept and method, with the basis of improving the accuracy and transmission rate, also pays attention to the user experience. Modular integration of the BCI system can be considered to minimize dependence on BCI technical support, so that it can be used independently. Increasing the KMI training of extending leg extension forward with standing posture and leg flexion backward with standing posture, or changing the details of the corresponding KMI paradigm, while having little effect on the system performance, can reduce the burden of the brain on the precondition. Neural feedback and adaptive methods could be introduced into the BCI system to improve the accuracy and ITR of the system (Daneshi et al. 2020; Miao et al. 2020). The performance of MI-BCI depends heavily on the performance of human MI. However, MI mental activities are not easily controlled or learned and vary greatly from individual to individual (Lee et al. 2019). By allowing the human to participate in the system development process, we can better understand the needs of BCI users, train a classification model for specific individuals, and customize a BCI system for users of output BCI or function objects of input BCI, so as to develop the most suitable BCI for individuals.

Although the present study used subjects (healthy humans) to evaluate the satisfaction of the KMI-BCI system for synchronous rehabilitation of lower limbs, the results are valid and important for the development of BCI because they clarify the prerequisites for daily use. Our next research plan is as follows: (1) To further improve the details of the paradigm of leg flexion backward with standing posture and leg extension forward with standing posture. (2) It is proposed to extract features from small samples by combining generative adversarial networks and meta-learing. (3) Recruit patients from hospitals (i.e., users of output BCI) for the study.

Summary

This study adopts the concept and method of human factors engineering of brain-computer interface to design and evaluate the KMI-BCI system for lower limb synchronous rehabilitation. It realizes the improvement of the initial BCI prototype system, which can effectively improve people's satisfaction with BCI products and promote the transformation of BCI system from laboratory research to practical application. The experimental results show that the highest recognition accuracy of the online KMI-BCI system in this study was 86.7%, and the average recognition accuracy was 63.98 ± 9.06%. The BCI system satisfaction was 3.19 ± 0.48 [subjects satisfaction rating for BCI system (1–8)] and 3.30 ± 0.45 [subjects satisfaction rating for BCI system (9–12)]. The mental workload rating for subjects manipulating the BCI system was 59.56 ± 7.79, and the overall satisfaction of the 10 subjects reached above medium level. However, there are some defects in the appearance, weight, output instruction time, and independent use of BCI system. The BCI design and evaluation method according to a human-centered framework allows comprehensive evaluation of system accuracy, ITR, and satisfaction in specific application scenarios, and provides valuable information for developers. The method can be used to improve the design of BCI systems and customize personalized BCI systems for individual human.

Author contributions

XL: experimental design, data collection and analysis, experimental results visualization and paper writing; SL and YD: data curation and analysis; PD and AG: review and revision of the paper; LS: supervision and guidance; LZ: Experimental design validation and verification; YF: Funding support and paper revision. All authors have made significant contributions to the submission and have agreed to the final version of the manuscript.

Funding

This study was supported by the National Natural Science Foundation of China (81771926, 61763022, 81470084, 61463024, 62006246, 82172058).

Data availability

The data used to support the findings of this study are included in this article, and the corresponding authors can be contacted for further inquiries.

Declarations

Conflict of interest

All authors of this article declare that there is no conflict of interest.

Ethical statement

The studies involving human participants were reviewed and approved by Medical Ethics Committee of Kunming University of Science and Technology School of Medicine. The research was conducted in accordance with the principles embodied in the Declaration of Helsinki and in accordance with local statutory requirements. All participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individuals for the publication of any potentially identifiable images or data included in this article.

Footnotes

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. Abiri R, Borhani S, Kilmarx J, Esterwood C, Jiang Y, Zhao X. A usability study of low-cost wireless brain-computer interface for cursor control using online linear model. IEEE T Hum-Mach Syst. 2020;50(4):287–297. doi: 10.1109/THMS.2020.2983848. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Alazrai R, Alwanni H, Daoud M. EEG-based BCI system for decoding finger movements within the same hand. Neurosci Lett. 2019;698:113–120. doi: 10.1016/j.neulet.2018.12.045. [DOI] [PubMed] [Google Scholar]
  3. Allison BZ, Dunne S, Leeb R, Millán JDR, Nijholt A. Towards practical brain-computer interfaces: bridging the gap from research to real-world applications. Berlin: Springer Publishing Company; 2012. [Google Scholar]
  4. Blankertz B, et al. The Berlin brain-computer interface: non-medical uses of BCI technology. Front Neurosci. 2010;4:198. doi: 10.3389/fnins.2010.00198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Branco MPA, Pels EGMA, SarsAarnoutse RHABEJA, Ramsey NFA, Vansteensel MJA, Nijboer FBC. Brain-computer interfaces for communication: preferences of individuals with locked-in syndrome. Neurorehab Neural Re. 2021;35(3):267–279. doi: 10.1177/1545968321989331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Chan S, Chan A. The validity and applicability of the Chinese version of the quebec user evaluation of satisfaction with assistive technology for people with spinal cord injury. Assist Technol. 2006;18(1):25–33. doi: 10.1080/10400435.2006.10131904. [DOI] [PubMed] [Google Scholar]
  7. Chavarriaga R, Fried-Oken M, Kleih S, Lotte F, Scherer R. Heading for new shores! Overcoming pitfalls in BCI design. Brain Comput Interfaces. 2017;4(1–2):60–73. doi: 10.1080/2326263X.2016.1263916. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Choi I, Rhiu I, Lee Y, Yun MH, Nam CS. A systematic review of hybrid brain-computer interfaces: taxonomy and usability perspectives. PLoS ONE. 2017;12(4):e0176674. doi: 10.1371/journal.pone.0176674. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Colucci M, Tofani M, Trioschi D, Guarino D, Berardi A, Galeoto G. Reliability and validity of the Italian version of quebec user evaluation of satisfaction with assistive technology 2.0 (QUEST-IT 2.0) with users of mobility assistive device. Disabil Rehabil. 2021;16(3):251–254. doi: 10.1080/17483107.2019.1668975. [DOI] [PubMed] [Google Scholar]
  10. Daneshi A, Azarnoush H, Towhidkhah F, Bernardin D, Faubert J. Brain activity during time to contact estimation: an EEG study. Cogn Neurodyn. 2020;14:155–168. doi: 10.1007/s11571-019-09563-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Fu J, Jiao X, Cao Y, Jiang J, Li Q, Feng J. EEG-based study on mental workload in multi-factor cognitive tasks space. Med Med Eng. 2020;33(1):35–44. [Google Scholar]
  12. Galán F, Nuttin M, Lew E, Ferrez PW, Vanacker G, Philips J, Millán JDR. A brain-actuated wheelchair: asynchronous and non-invasive brain-computer interfaces for continuous control of robots. Clin Neurophysiol. 2008;119(9):2159–2169. doi: 10.1016/j.clinph.2008.06.001. [DOI] [PubMed] [Google Scholar]
  13. Graimann B, Allison B, Pfurtscheller G. Brain-computer interfaces: revolutionizing human-computer interaction. Berlin: Springer Publishing Company; 2013. [Google Scholar]
  14. Guger C, Edlinger G, Harkam W. How many people are able to operate an EEG-based brain-computer interface (BCI)? IEEE Trans Neural Syst Rehabil. 2003;11(2):145–147. doi: 10.1109/TNSRE.2003.814481. [DOI] [PubMed] [Google Scholar]
  15. Hart SG, Staveland LE. Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol. 1988;52:139–183. doi: 10.1016/S0166-4115(08)62386-9. [DOI] [Google Scholar]
  16. Holz EM, Höhne J, Staiger-Sälzer P. Brain-computer interface controlled gaming: evaluation of usability by severely motor restricted end-users. Artif Intell Med. 2013;59(2):111–120. doi: 10.1016/j.artmed.2013.08.001. [DOI] [PubMed] [Google Scholar]
  17. Jeunet C, et al (2021) Modelling the acceptance of BCI-based stroke rehabilitation procedures: heading for efficiently personalised therapies. BCI Meeting 2021. https://hal.archives-ouvertes.fr/hal-03331687. Accessed Sep 2021
  18. Kübler A. Brain-computer interfacing-science fiction has come true. Brain. 2013;136(6):2001–2004. doi: 10.1093/brain/awt077. [DOI] [Google Scholar]
  19. Kübler A. The history of BCI: from a vision for the future to real support for personhood in people with locked-in syndrome. Neuroethics. 2019;13(2):1–18. [Google Scholar]
  20. Kübler A, Zickler C, Holz E, Kaufmann T, Riccio A, Mattia D. Applying the user-centred design to evaluation of brain-computer interface controlled applications. Biomed Eng-Biomed Te. 2013;58(15):3234–3234. doi: 10.1515/bmt-2013-4438. [DOI] [PubMed] [Google Scholar]
  21. Kübler A, et al. The user-centered design as novel perspective for evaluating the usability of BCI-controlled applications. PLoS ONE. 2014;9(12):e112392. doi: 10.1371/journal.pone.0112392. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kübler A, Nijboer F, Kleih S. Hearing the needs of clinical users. Handb Clin Neurol. 2020;168:353–368. doi: 10.1016/B978-0-444-63934-9.00026-3. [DOI] [PubMed] [Google Scholar]
  23. Lee MH, Kwon OY, Kim YJ, Kim HK, Lee YE, Williamson J, Fazli S, Lee SW. EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy. Giga Sci. 2019;8(5):1–16. doi: 10.1093/gigascience/giz002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Liberati G, Pizzimenti A, Simione L, Riccio A, Schettini F, Inghilleri M, Mattia D, Cincotti F. Developing brain-computer interfaces from a user-centered perspective: assessing the needs of persons with amyotrophic lateral sclerosis, caregivers and professionals. Appl Ergon. 2015;50:139–146. doi: 10.1016/j.apergo.2015.03.012. [DOI] [PubMed] [Google Scholar]
  25. Lu X, Ding P, Li S, Gong A, Zhao L, Qian Q, Su L, Fu Y. Human factors engineering of brain-computer interface and its applications: human-centered brain-computer interface design and evaluation methodology. J Biomed Eng. 2021;38(2):210–223. doi: 10.7507/1001-5515.202101093. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Martin S, Armstrong E, Thomson E, Vargiu E, Sola' M, Dauwalder S, Miralles F, Daly Lynn J. A qualitative study adopting a user-centered approach to design and validate a brain computer interface for cognitive rehabilitation for people with brain injury. Assist Technol. 2018;30(5):233–241. doi: 10.1080/10400435.2017.1317675. [DOI] [PubMed] [Google Scholar]
  27. Miao Y, Yin E, Allison BZ, Zhang Y, Chen Y, Dong Y, Wang X, Hu D, Chchocki A, Jin J. An ERP-based BCI with peripheral stimuli: validation with ALS patients. Cogn Neurodyn. 2020;14:21–33. doi: 10.1007/s11571-019-09541-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Millan JDR, Renkens F, Mourino J, Gerstner W. Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE Trans Biomed Eng. 2004;51(6):1026–1033. doi: 10.1109/TBME.2004.827086. [DOI] [PubMed] [Google Scholar]
  29. Miralles F, et al. Brain computer interface on track to home. Sci World J. 2015;2(1):1. doi: 10.1155/2015/623896. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Muhammad E. Rapid prototyping technology: principles and functional requirements. London: IntechOpen; 2011. [Google Scholar]
  31. Riccio A, et al. Interfacing brain with computer to improve communication and rehabilitation after brain damage. Prog Brain Res. 2016;228(1):357–387. doi: 10.1016/bs.pbr.2016.04.018. [DOI] [PubMed] [Google Scholar]
  32. Schreudera M, Ricciob A, Risettib M, Dähnea S, Ramsayd A, Williamsond J, Mattiab D, Tangermanne M. User-centered design in brain-computer interfaces-a case study. Artif Intell Med. 2013;59(2):71–80. doi: 10.1016/j.artmed.2013.07.005. [DOI] [PubMed] [Google Scholar]
  33. Thompson MC. Critiquing the concept of BCI illiteracy. Sci Eng Ethics. 2019;25(4):1217–1233. doi: 10.1007/s11948-018-0061-1. [DOI] [PubMed] [Google Scholar]
  34. Tian G, et al. Execution, assessment and improvement methods of motor imagery for brain-computer interface. J Biomed Eng. 2021;38(3):434–446. doi: 10.7507/1001-5515.202101037. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Wang X, Jin J, Zhang Y, Wang B. Brain control: human-computer integration control based on brain-computer interface. Acta Autom Sin. 2013;39(3):208–221. doi: 10.3724/SP.J.1004.2013.00208. [DOI] [Google Scholar]
  36. Wang Z, Chen L, He F, Wan B, Ming D. Development trend and prospect of BCI technology facing rehabilitation and assisting applications. Chin J Sci Instrum. 2017;38(6):1307–1318. [Google Scholar]
  37. Wang K, Xu M, Wang Y, Zhang S, Chen L, Ming D. Enhance decoding of pre-movement EEG patterns for brain-computer interfaces. J Neural Eng. 2020;17(1):016033. doi: 10.1088/1741-2552/ab598f. [DOI] [PubMed] [Google Scholar]
  38. Wolpaw JR, Wolpaw EW. Brain-computer interfaces: principles and practice. Oxford: Oxford University Press; 2012. [Google Scholar]
  39. Wolpaw JR, Millán JDR, Ramsey NF. Brain-computer interfaces: definitions and principles. Handb Clin Neurol. 2020;168:15–23. doi: 10.1016/B978-0-444-63934-9.00002-0. [DOI] [PubMed] [Google Scholar]
  40. Xu M, He F, Jung TP, Gu X, Ming D. Current challenges for the practical application of electroencephalography-based brain–computer interfaces. Engineering. 2021;7(12):1710–1712. doi: 10.1016/j.eng.2021.09.011. [DOI] [Google Scholar]
  41. Zickler C, et al. A brain-computer interface as input channel for a standard assistive technology software. Clin EEG Neurosci. 2011;42(4):236–244. doi: 10.1177/155005941104200409. [DOI] [PubMed] [Google Scholar]
  42. Zickler C, Halder S, Kleih SC, Herbert C, Kübler A. Brain painting: Usability testing according to the user-centered design in end users with severe motor paralysis. Artif Intell Med. 2013;59(2):99–110. doi: 10.1016/j.artmed.2013.08.003. [DOI] [PubMed] [Google Scholar]
  43. Zjajo A. Brain-machine interface: circuits and systems. Berlin: Springer Publishing Company; 2016. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data used to support the findings of this study are included in this article, and the corresponding authors can be contacted for further inquiries.

Articles from Cognitive Neurodynamics are provided here courtesy of Springer Science+Business Media B.V.

RESOURCES