Quantifying the role of motor imagery in brain-machine interfaces

Abstract

Despite technical advances in brain machine interfaces (BMI), for as-yet unknown reasons the ability to control a BMI remains limited to a subset of users. We investigate whether individual differences in BMI control based on motor imagery (MI) are related to differences in MI ability. We assessed whether differences in kinesthetic and visual MI, in the behavioral accuracy of MI, and in electroencephalographic variables, were able to differentiate between high- versus low-aptitude BMI users. High-aptitude BMI users showed higher MI accuracy as captured by subjective and behavioral measurements, pointing to a prominent role of kinesthetic rather than visual imagery. Additionally, for the first time, we applied mental chronometry, a measure quantifying the degree to which imagined and executed movements share a similar temporal profile. We also identified enhanced lateralized μ-band oscillations over sensorimotor cortices during MI in high- versus low-aptitude BMI users. These findings reveal that subjective, behavioral, and EEG measurements of MI are intimately linked to BMI control. We propose that poor BMI control cannot be ascribed only to intrinsic limitations of EEG recordings and that specific questionnaires and mental chronometry can be used as predictors of BMI performance (without the need to record EEG activity).

Non-invasive brain machine interfaces (BMI) have enabled humans to control a variety of external devices through real-time decoding of brain activity. In non-invasive BMI approaches, users are engaged in a cognitive task, such as visual attention or motor imagery (MI), while their brain signals are recorded through electroencephalography (EEG) and on-line processed to ultimately control external devices¹,². Despite important developments concerning both recording and analysis techniques of brain signals, the ability to control MI-based BMI is characterized by large inter-individual differences. The portion of users that cannot achieve successful control is usually reported to be in the range of 15–30%³. However, studies employing large number of subjects have shown that up to 50% of users are not able to achieve accuracy above 70%, a threshold commonly considered more accurate for identifying successful control⁴,⁵.

Previous work using MRI and EEG recordings have identified anatomical (volumetric and connectivity measurements)⁶,⁷, functional (extent of the cortical activation)⁸, as well as electrophysiological indices (power modulation over specific frequency bands)⁹,¹⁰,¹¹ as potential factors for this variability in BMI-control. In addition, psychological, cognitive and motor characteristics have been evaluated. These include (i) questionnaires aimed at the self-assessment of personality and psycho-pathological traits¹²,¹³, (ii) individual intrinsic characteristics such as age and the per-day amount of hand-arm movement¹⁴, and (iii) behavioral tasks assessing motor and cognitive skills, such as visuo-motor coordination and concentration¹⁵.

However, the specific ability to perform the particular cognitive task required for MI-based BMI, i.e. motor imagery (MI), has not been investigated in detail. Although different self-reporting motor imagery questionnaires have been employed in the context of BMI⁷,¹²,¹⁶,¹⁷, these have led to inconsistent results. It should be noted that as with BMI control skills, MI ability, that is the ability to mentally perform or rehearse an action without actually executing it (see for reviews)¹⁸,¹⁹, also presents marked differences across the general population. These differences have been extensively evaluated in the field of cognitive neuroscience using subjective methods such are the aforementioned self-reporting motor-imagery questionnaires, but also through different behavioral measures. These include objective measurements assessing the relationship between the duration of executed and imaged movements, such as the mental chronometry task²⁰,²¹,²². In addition, the effects of MI translate into physiological (e.g. skin conductance, heart beat variation and respiratory rate) and neurophysiological modulations that can be evaluated with neuroimaging techniques²³ and transcranial magnetic stimulation²⁴.

In the present study we directly addressed whether the variability in the ability to control a BMI and in performing MI are linked in the sense that in MI-based BMI paradigms, variations in BMI control performance can be related to the different individual skills in MI. Better understanding this potential relationship might elucidate whether BMI and MI rely on common mechanisms and whether MI ability as quantified by measures from cognitive neuroscience may be a predictor of BMI-performance. We investigated MI accuracy in high- and low-aptitude BMI users and used a reliable behavioral task that has previously been employed to collect an objective (mental chronometry) measure of MI ability²¹,²⁵. We additionally used a self-reporting questionnaire with two sub-scales that separately assess kinesthetic and visual features of MI²⁶. If variations in BMI control performance are associated with the different individual skills in MI, we predicted to find better motor MI performance (mental chronometry task) and higher MI scores in the questionnaires in high-aptitude rather than low-aptitude BMI users. Kinesthetic MI has previously been shown to activate motor-associated brain structures²³ involved in BMI control⁸ to a greater extent than visual MI. Therefore, we tested the hypothesis that kinesthetic MI would be more strongly associated with BMI aptitude as compared to visual MI. Finally, we compared neural MI correlates extracted from EEG signals in high- and low-aptitude BMI users. Since BMI-based control has been related to specific lateralized activity in bilateral motor and premotor cortices⁸,²⁷, we hypothesized that the lateralization of cortical activity recorded during pure MI (that is without visual feedback as during BMI control) would discriminate between high- and low-aptitude BMI users.

Results

In order to study the relationship between BMI proficiency and MI, we first classified participants (N = 24) as high- vs. low-aptitude BMI users, depending on their performance during the BMI task. In the MI task, participants were asked to move a cursor to the left or right side of a computer screen by imagining, respectively, left or right hand clasping movements. Based on previous BMI studies⁶,²⁸, we divided our pool of subjects into two groups according to the median value (69.5%) of the overall on-line BMI control performance. For each subject, the mean BMI performance was computed as the percentage of time frames in which the cursor moved in the cued direction. The low aptitude group consisted of 12 subjects, with performance ranging from 49% to 67%, while the high-aptitude group had a performance ranging from 72% to 87%. We note that the threshold around 70% of accuracy is in accordance with that previously proposed by others authors to discriminate effective BMI-control from chance in a 2-class paradigm⁵,²⁸.

MI behavioral abilities and questionnaire discriminate between high- and low-aptitude BMI users

MI ability was evaluated through two different measurements: the mental chronometry task²⁹, assessing the difference in duration between actual and imagined movements and a self-reporting questionnaire, the MIQ-RS²⁶, including two subscales measuring visual and kinesthetic features of MI. The mental chronometry task is considered a reliable measure of MI quality and aims at assessing individual’s ability to preserve the temporal characteristics of the movement during MI, based on the isochrony between real execution and MI²⁴,²⁹.

Questionnaire

The MIQ-RS questionnaire showed that for high aptitude users, the kinesthetic experience of MI was judged as more vivid and easier to evaluate than for low aptitude users, as shown by higher ratings in the former group in the kinesthetic MI scale (t = −2.255, p = 0.034, Fig. 1(a)). No difference between the groups was found for visual MI scale (t = −0.771, p = 0.449, Fig. 1(b)).

Figure 1. Significant differences between high- and low-aptitude users in motor imagery abilities evaluated by using a MI questionnaire and mental chronometry.

Average ratings (±standard error) among all items of the MIQ-RS questionnaire separately for the visual and kinesthetic subscales (upper panels). High aptitude users rated kinesthetic motor imagery as significantly easier than low aptitude users (a), while no difference between the groups was found for the visual MI subscale (b). Results (mean values ± standard error) related to the chronometric measurements are shown in the bottom panels. The Deviation from isochrony index (i.e. the deviation from an equal time required to execute and to imagine five hand clasping movements) was worse (higher) in low rather than high aptitude users (c). Conversely, the average time required to execute the movement did not differ between the two groups (d). Data in (c,d) are collapsed across left and right hands since they did not statically differ.

Mental chronometry

Based on the MI task used for the BMI control, we compared the time required to execute and imagine five-hand clasping movements. The temporal congruency between MI and execution is an indicator of good imagery abilities, whereas a higher discrepancy between the two modalities have been associated with poor imagery quality. The repeated measure ANOVA with imagined hand (left versus right), task (execution versus MI) and group (high- versus low- aptitude) as factors, performed on the mental chronometry data revealed a significant difference in the duration of the MI task, but, importantly, not in the execution task, in the two groups (task X group, F(1, 22) = 4.383, p = 0.048). Post hoc comparisons (Newman-Keuls tests) showed that in low-, but not in high-aptitude BMI users the imagery of hand clasping movements takes longer than the execution of the same movement regardless of the employed hand (low-aptitude BMI users p = 0.002; high-aptitude BMI users: p = 0.619). In contrast, the result in high-aptitude users is compatible with the notion of isochrony (see next paragraph). Interestingly, the duration of the execution was equal in the two groups and for both hands (p > 0.05, Fig. 1(d)).

In addition, we calculated an index of deviation from isochrony, in order to quantify the discrepancy from the temporal congruence (isochrony) in the duration of imagined and executed movement (for further details on the calculation, refer to Materials and methods). Low values of this index indicate high MI quality, in accordance with the fact that good imagery skills are associated to reduced differences between the time required to execute and imagine the movement²¹,²⁴,²⁹. The t-test performed on the deviation from isochrony index revealed a significant difference between groups (t = 2.175, p = 0.041, Fig. 1(c)), with values close to zero in high-aptitude users and greater values in low-aptitude ones, indicating a link between higher MI accuracy and better BMI control abilities.

EEG data of MI discriminate between high- and low-aptitude BMI users

In order to compare electrophysiological correlates of MI abilities in low- and high-aptitude BMI users, we analyzed EEG signals collected during pure MI without BMI (with no visual feedback, nor cursor control), at the beginning of each session (calibration session, off-line, see Fig. 2(a)). We used two analyses: we analyzed the modulation of the μ-rhythm and we exploited the information contained in the common spatial patterns (CSPs).

Figure 2. Experimental procedure during the BMI session.

(a) Calibration session, off-line: subjects were instructed to imagine clasping movements performed either with the left or the right hand as indicated by a visual cue (red arrow) without receiving any visual feedback. (b) The collected EEG data were then used to extract EEG correlates of motor imagery, to train a linear classifier and to extract subject specific spatial filters. (c) These parameters were then used in the on-line session, during which subjects were asked to control in real-time the movement of a cursor (black rectangle) by performing the same motor imagery task as in the calibration session.

μ-rhythm

In order to investigate differences in μ-rhythm power spectra between high- and low-aptitude BMI users in all subjects we considered a fixed frequency range (8–12 Hz). This range over sensorimotor regions has been classically referred to as the μ-rhythm, and it has been consistently reported to be modulated during motor imagery (see for instance³⁰). We selected two clusters of electrodes located over sensorimotor regions in each hemisphere (Fig. 3). We considered the average value of each cluster, separately for the two imagined hands. Finally, an index of laterality was computed considering the difference between the contralateral and ipsilateral cluster, for both hands. A repeated measure ANOVA with factors imagined hand (right and left) and group (high and low-aptitude BMI users) run on the index of laterality revealed a main effect of group (F(1, 22) = 5.778, p = 0.025), independently of the imagined hand (imagined hand, F(1, 22) = 0.326, p = 0.574; imagined hand X group, F(1, 22) = 0.213, p = 0.649, Fig. 4(a)). Thus, irrespectively of the hand involved in MI, high-aptitude users showed a more negative index of laterality compared to low-aptitude users (group, F(1, 22) = 5.778, p = 0.025). Note that a more negative value corresponds to a stronger difference between the contralateral and the ipsilateral activity, in the direction of a more marked suppression over the contralateral hemisphere. Thus, these results reflect a higher difference in μ-rhythm suppression in the hemisphere contralateral to the imagined hand (with respect to the ipsilateral one) in high- as compared to low-aptitude BMI users, as depicted in Fig. 3. We note that our choice of the electrodes selected to be included in the two clusters was further corroborated by results at the level of single channels (uncorrected). High-aptitude users showed more strongly suppressed activity over electrodes located in the hemisphere contralateral to the imagined hand. More specifically, for right-hand MI, we found a statically significant effect over electrodes FC5, FC3, C5, C3, CP5 and CP3, while for left-hand MI, electrodes FC2, FC4, FC6 and C4.

Figure 3. μ-rhythm suppression during motor imagery over two sensorimotor clusters.

The panel illustrates the mean power spectrum (±standard error across subjects) in the μ-band (8–12 Hz) over two clusters of electrodes located over sensorimotor regions in the left (yellow) and the right (purple) hemisphere during left (on the left) and right (on the right) hand motor imagery. Traces in the contralateral cluster with respect to the imagined hand show a clear μ-rhythm suppression, stronger in the high-aptitude (upper plots) as compared to the low-aptitude (lower plots) group. Conversely, the mean power over the ipsilateral cluster presents a weaker suppression with respect to the contralateral cluster. These effects have been quantified with the proposed index of laterality.

Figure 4. Significant differences between high- and low-aptitude BMI users in EEG-correlates of pure motor imagery.

The difference in μ-rhythm suppression (mean values ± standard error) between two clusters of electrodes, one over each hemisphere (index of laterality) was significantly more marked in the high- than in the low-aptitude group, showing the importance of a specific lateralized activation during MI for BMI control (a). The correlation (mean values ± standard error) between the right and left common spatial patterns (CSP) was higher in high- rather than low-aptitude users, indicating a more lateralized cortical activity during MI in the former group (b).

CSP analysis

CSPs are subject specific spatial filters that weigh each electrode according to its contribution to the classification³¹. We expected to identify a stronger contribution from those channels which are located in the hemisphere contralateral to the imagine hand (Supplementary Figure 1). To evaluate the presence of this lateralized pattern, we computed the correlation coefficient between one CSP referring to right-hand MI (first CSP) and the mirrored-image of one CSP relative to left-hand MI (last CSP). Thus, if this lateralized pattern is associated to a more proficient modulation of neural activity for both hands during MI, the correlation coefficient should be higher in high-aptitude BMI users. The CSP analysis showed that in high-aptitude BMI users, the first and last CSPs were more strongly correlated (t = −2.153, p = 0.043) than in low-aptitude BMI users (Fig. 4(b)), suggesting a higher lateralization during MI in the high-aptitude group.

Prediction of BMI-control abilities

Given the significant difference between high- and low-aptitude users on the kinesthetic sub-scale and the mental chronometry task, we next used a linear classifier (logistic regression algorithm) to investigate whether these subjective and behavioral indices are potential predictors of BMI-control ability. More specifically, we tested how accurately the estimated model would predict the assignment of each individual subject to either the high- or low-aptitude BMI group. This analysis lead to an average accuracy of 73.7% of correct classification and was above the threshold derived with a binomial cumulative distribution for a two-class classification problem⁵,³² (70%; sample size: 24 subjects). The control analysis, performed using as predictors the ratings on the visual subscale of the MIQ-RS and the duration of executed clasping movements acquired during the mental chronometry task, resulted in a lower average accuracy (57.1%), that it is below the significant threshold obtained with the binomial cumulative distribution.

Discussion

Our study aimed at investigating whether MI ability underlies proficiency in BMI control. We found that EEG signals during MI (as captured by μ-rhythm and CSP) as well as behavioral and subjective measurements of MI accuracy (mental chronometry; kinesthetic sub-scale of the MIQ-RS questionnaire) distinguished between users with high- and low-aptitude for BMI control. In particular, high-aptitude BMI users showed higher MI accuracy and more lateralized neural activation during MI as compared to low-aptitude BMI users. In addition, our results suggest that questionnaire and behavioral measures can be used as predictors of BMI performance, without the need for recording EEG signals.

Higher motor imagery ability in high-aptitude BMI users

We used two reliable cognitive indices of MI abilities: self-reporting scores concerning kinesthetic features of MI activity (based on MIQ-RS questionnaire) and an objective behavioral measure based on mental chronometry. Both measures exhibited a significant difference between the high- and the low-aptitude group. We will discuss the subjective questionnaire data first. In the context of our chosen questionnaire, higher ratings indicate an ease in performing the task. Kinesthetic MI ratings from proficient BMI performers were significantly higher than those from the low-aptitude group, whereas there was no such group difference for subjective ratings of visual MI, highlighting the importance of motor and kinesthetic imagery (participant’s ability to feel imagined movements) rather than visual imagery (participant’s ability to see imagined movements) in BMI control.

Different MI questionnaires (e.g. Vividness of movement imagery questionnaire in¹²,¹⁷; the revised Movement Imagery Questionnaire in⁷; The Kinesthetic and Visual Imagery questionnaire in¹⁶) have already been employed in the context of evaluating BMI abilities. However, to date only one study reported a significant relationship between BMI performance and imagery scores for visual versus kinesthetic modalities¹⁶. In line with our results, Vuckovic and collaborators¹⁶ found a stronger correlation between participants’ ratings in kinesthetic versus visual MI. Crucially, they considered the accuracy of an off-line EEG classifier using data recorded during MI, but not during BMI control as in the present study. Our study significantly extends that finding in showing the prominent role of kinesthetic imagery in explaining inter-subject differences, not only during MI, but also during on-line BMI control. We note that previous studies had underlined the importance of instructing subjects to perform kinesthetic rather than visual motor imagery in BMI paradigms⁹,³⁴. Moreover, kinesthetic MI activates similar sensorimotor areas, whose activity is classically decoded for MI-based BMI, namely primary motor³⁵,³⁶, SMA, and ventral premotor cortex²³. Compatible with the present discrimination between high- and low-aptitude BMI users in kinesthetic versus visual MI, visual MI fails in producing a clear activation in these areas³⁷ and activates more posterior regions in occipital and parietal cortex²³.

Importantly, the present study is the first to use mental chronometry to investigate difference in BMI ability. We note that MI questionnaire, despite being a validated tool in MI and previously used in BMI ability estimation, reflect a subjective assessment of MI abilities based on verbal rating scales. In contrast, mental chronometry is an objective behavioral measurement based on motor responses and thus less prone to participant biases: for this reason it has been argued to be a more reliable criterion of MI quality, compared to questionnaire scores²¹. Results from the mental chronometry task revealed that low-aptitude BMI users (as compared to high-aptitude users) show a greater difference between the time required to perform versus imagine a hand clasping movement, indicating reduced MI abilities. This interpretation is supported by an extensive body of literature showing that poor isochrony is an indicator of impaired imagery abilities, both in healthy participants²¹ and in different patient groups¹⁹,³⁸,³⁹,⁴⁰. These behavioral results demonstrate that the isochrony in MI is also relevant to BMI proficiency, suggesting that higher MI abilities are crucial for BMI control. Thus, we introduce here the mental chronometry and the kinesthetic questionnaire sub-scale as predictors of BMI control; this is particularly interesting as especially the latter measure targets specifically MI skills in a reliable way without the necessity for EEG recordings. For instance, by employing the chronometry index it is possible to evaluate and monitor MI abilities in different populations (healthy adults; patients), and over various experimental conditions or phases of training or illness. Results from the mental chronometry analysis also highlighted that the duration of the executed movement per se seems not to influence BMI ability. These data further extended prior cognitive neuroscience works by showing that not only MI but also BMI control does not exclusively depend on the ability to actually perform the movement, but also relies on the complex interplay between motor abilities and somatosensory inputs from the body part involved in the MI task⁴¹,⁴².

Despite having instructed our participants not to perform any overt movement during motor imagery and despite close observation by the experimenter, we cannot exclude that sub-threshold movements not detected by us could be present or that spinal cord circuitry could be still activated. Concerning, spinal circuitry involvement, previous studies in primates indeed have shown activity at the spinal level during movement preparation (e.g.⁴³). Moreover, in humans, a residual muscular activity and spinal cord modulation have been reported during motor imagery in some subjects and under certain conditions, as a variation of inhibition at level of spinal interneurons¹⁸. Although not likely, such potentially undetected movements and spinal cord activity could have differentially impacted BMI performance in the high- and low-aptitude group during motor imagery. However, it is improbable that these effects, if present, could account also for the differences found in our subjective and objective measurements of MI ability between the two groups. Previous studies, indeed, already reported dissimilar scores in good and poor imagers at the mental chronometry task and at the MIQ-R, that were not explained by residual muscular activity (e.g.²⁴).

EEG-signatures of motor imagery differ in high- and low-aptitude BMI users

To investigate neural signatures that might reflect differences in hemispheric lateralization during MI between low- and high-aptitude users, we found two further indices derived from EEG signals during MI without online BMI control. First, we compared the difference in μ-band suppression in two clusters of electrodes over the sensorimotor cortices and found that the laterality index was larger in the high-aptitude users compared to the low-aptitude group, pointing to the importance of the lateralization of a specific frequency band during MI for BMI control. This conclusion was further supported by another analysis, aimed at evaluating the neurophysiological information carried by the CSP (CSP index computed as the correlation coefficient between the right and left CSP, separately for each subject). Results showed that in high-aptitude BMI users, the CSP index was significantly higher than in low-aptitude users, indicating that electrodes that discriminate the most between imagery of left and right hand movements are located over the contralateral hemisphere. This confirms that this lateralized pattern of neural activity, weaker in low-aptitude BMI users, is important to achieve good BMI-control.

Previous BMI studies have extracted predictors able to differentiate between high- and low- aptitude BMI users based on EEG signals recorded not only during BMI-control, but also at rest. For instance, an index of the average alpha power recorded at rest over both C3 and C4 has been shown to be related to BMI performance⁹. Other studies focused on the importance of different frequency bands, involving sensorimotor gamma oscillations during MI⁴⁴, or negative correlations between BMI performance and theta band activity at rest¹⁰. Similarly even during on-line BMI control, a recent study showed that, the association between a laterality index extracted from two central EEG electrodes (computed as a difference between the μ-rhythm desynchronization over C3 and C4) and the BOLD signal acquired during simultaneous fMRI recordings, could be used to group BMI users into good, moderate and poor performers²⁷. Other studies focused on the importance of different frequency bands, involving sensorimotor gamma oscillations during MI⁴⁴, or negative correlations between BMI performance and theta band activity at rest¹⁰. Structural brain features such as interhemispheric connections across the corpus callosum⁴⁵ have also been proposed to be important for MI-based BMI⁶. The present data corroborate the differential involvement of the two hemispheres previously reported during BMI control or at rest in high-aptitude users. Our data suggest that this pattern in good BMI users is related to MI, showing that different lateralization during MI discriminates between high and low BMI proficiency.

These findings suggest that the ability to perform the underlying cognitive task (MI) important for BMI control, is mediated via bilateral connections driving proficient BMI aptitude. Literature on MI again offers support for this proposal. During MI, a power decrease over the contralateral hemisphere (event-related desynchronization, ERD) is often associated with a power increase (event-related synchronization, ERS) over the ipsilateral hemisphere⁴⁶. Other studies⁴⁷,⁴⁸ investigated the reciprocal influence of both hemispheres during MI, and the inhibition of the contra- versus ipsilateral hemisphere has been described in good MI subjects⁴⁹. Moreover, this effect during MI mimics the pattern of lateralization largely documented for actual unilateral movements: during the execution of unimanual movements, the contralateral motor cortex inhibits the ipsilateral one through the transcallosal pathway, inducing a decrease of ipsilateral M1 excitability⁴⁹. Similarly, directionally selective disruptions of oscillations have been also documented during movement preparation in primates⁵¹,⁵². Here we extend these neuroscience findings to the field of BMI by showing that MI-based BMI control depends not only on the modulation of the contralateral hemisphere, but more importantly, on the differences between the activation in the two hemispheres in a specific frequency band (i.e. the μ-rhythm).

Finally, results from the logistic regression analysis show the potential of our behavioral measures, not only in explaining differences between high- and low-aptitude BMI users, but also in predicting BMI control ability at the individual level. The scores on the kinesthetic scale of the MIQ-RS questionnaire and the index of isochrony on the mental chronometry task were indeed significantly above chance in classifying individual participants as high- vs. low-aptitude BMI users, whereas this was not the case for control analysis (including execution of movement measurements and visual imagery sub-scale scores). A model based on these two MI indices might present a powerful tool for screening and monitoring BMI abilities; moreover these measures do not require collecting and analyzing EEG data to estimate BMI aptitude. We note that a previous study directly investigated the use of psychological variables as predictors of BMI proficiency¹². At present, we cannot exclude that the skills underlying the tests used by these authors might be involved in the ability to perform MI or somehow relate to the tasks we employed here (some of the tests used may relate to motor prediction⁵³ and concentration⁵⁴ that are both linked to MI). Further research will be required to directly test these issues.

Conclusions

Taken together our results suggest that subjective, behavioral, and EEG measurements of MI ability are intimately linked to the ability to control a BMI. The finding that MI ability predicts BMI aptitude provides evidence against the hypothesis that BMI illiteracy can be completely ascribed to intrinsic limitations of EEG in recording brain signals, in particular due to specific neuroanatomical characteristics²⁷. We cannot exclude that these technical limitations might have an impact on BMI proficiency, however the present results reveal the importance of a specific cognitive ability - MI - in BMI control. The proposed pre-screening tool based on subjective and behavioral MI measurements aims not at excluding low-aptitude BMI users, but rather at adopting specific strategies to improve their MI abilities, and thus potentially their BMI proficiency. Different aspects can influence the ability to perform MI, among them being posture⁵⁵,⁵⁶, perspective⁵⁷,⁵⁸ and the type of MI (visual or kinaesthetic)³⁷,⁵⁸. Past MI research has proposed different strategies for improving motor imagery skills in athletes⁵⁹ and even in stroke patients³⁸, by providing, for instance, verbal information describing the sensation elicited by a given movement⁵⁹, by presenting objects that can stimulate specific affordances and sensations or by using action observation prior to the imagination task³⁸. The present data show that these neuroscience-based strategies can now also be applied to BMI- “athletes”, aiming at activating the motor representation related to the action to be imagined, thus facilitating MI, which, based on the present results, might improve BMI control.

Materials and Methods

Participants

Twenty-four right-handed (as determined by the Handedness Inventory)⁶¹ healthy participants (7 females and 17 males, mean age 26.5 years, SD ± 3.8, range 21–34) took part in the experiment which was approved by the Ethics Committee of the Faculty of Biology and Medicine of the University of Lausanne and was conducted in accordance with the Declaration of Helsinki. Written informed consent was obtained from all participants that took part in the experiment.

Experimental procedure

Two different experimental sessions were run. One session was dedicated to the EEG recording and BMI-control while the other aimed to behaviorally assess MI ability. The order of the two sessions was counterbalanced across subjects.

Motor imagery ability

Questionnaire

The MIQ-RS²⁶ is composed of two subscales aiming at evaluating participant’s ability to see (visual imagery) and feel (kinesthetic imagery) different imagined movements.

Every subscale contains seven items corresponding to seven different movements (e.g. pushing, pulling, reaching and grasping object, make a fist).

Each item entails first to perform a movement, then to imagine the same action, and finally rate the difficulty of the imagination task using a 7-point scale (from 1 = very hard to see ⁄ feel; 4 = neutral (not easy⁄not hard); 7 = very easy to see⁄feel, and intermediate levels). Questionnaires have been commonly used to asses motor imaginary ability²¹. The MIQ-RS is a suitable option for examining movement imagery abilities targeting primarily the upper extremities²⁶.

Mental chronometry

Here in accordance with the BMI task, subjects were asked to perform or imagine five hand clasping movements, either with the left or the right hand. They were instructed to imagine the feeling of performing the movement (kinesthetic MI) with eyes open as in the BMI task. There were in total 4 experimental conditions, with hand (left/right) and task (execution/imagination) as factors. Each condition was repeated 10 times to account for a reliable estimation of the actual and imaged movement duration.

We used a chronometer to measure the time required to complete the 5-hand clasping task. Subjects were instructed to verbally report the moment in which they start and complete the task.

The rationale of the mental chronometry task is based on the evidence that imagined and executed movements follow the same rules of motor control²²,³⁹ and that a temporal congruence exists between the duration of executed and imaged action. For this reason, comparing duration of motor imagery and of the real execution of the same action has been previously employed as a simple and reliable method to evaluate imagery accuracy in healthy participants and in patients²¹,²⁴,³⁸,⁴¹.

EEG acquisition and BMI loop

A 64-channel EEG system (g.tec medical engineering GmbH, Graz, Austria) was used for the EEG recording. A subset of 27 of a total of 64 available EEG channels were considered for the signal recording and the BMI loop, which refers to the processing chain from the decoding of brain activity to the feedback presentation. This subset included all the electrodes situated over the sensorimotor cortex, including fronto-parietal, central and centro-parietal electrodes. The ground was located on the forehead (AFz) and the reference was placed on the right earlobe. The EEG signal was sampled at 256 Hz and Butterworth filters (in the range of 1–30 Hz) were used during the recording.

During the EEG-BMI session participants sat comfortably in front of a computer screen onto which the stimuli and BMI visual feedback was projected. They were instructed to keep their hands on their thighs with the palm facing up (Fig. 5) and to avoid any movement during the motor imagery and BMI-control. The whole experiment included two off-line (calibration session, see below) and two online (BMI cursor control, see below) runs.

Figure 5. Experimental setup during EEG acquisition and BMI-control: A 64-channel EEG system was used for recording brain activity.

During motor imagery and BMI control, the instructions and the visual stimuli, including the BMI feedback as shown in the picture, were presented on a computer screen. Participants sat comfortably in front of the screen, keeping their hands on their thighs with the palms up.

Calibration session: pure motor imagery and classifier training

Each recording began with the two off-line runs of 40 trials each, during which the subject performed MI without visual feedback while the EEG signal was recorded. Subjects were instructed to imagine clasping movements performed either with the left or the right hand by focusing on the kinesthetic sensations of the movement (kinesthetic MI). The trial began with a fixation cross (2 seconds), followed by a red arrow (cue, 1 second), placed in the center of the screen, indicating which hand had to be imagined. Right after the cue disappeared, the subject was instructed to start imagining the movement and continue for a total duration of 6 seconds (Fig. 2(a)). After the motor imagery period, subjects had 7 seconds of rest.

The off-line runs were required in order to select subject-specific discriminant EEG features between the left and right hand imaged movements, namely the spatial filters and the linear classifier parameters to be used during the on-line control session.

The spatial filters (common spatial patterns, CSPs), were used to improve the efficiency in the discrimination of two classes (left and right hand imaged movement) and represents a method commonly used in BMI applications to increase the signal strength³¹. The CSP algorithm finds, for each subject, spatial filters (in the same number as the electrodes), each of which weights each electrode based on its contribution to the classification (Supplementary Figure 1).

In order to compute the classifier and extract the CSP, the EEG data acquired during the offline session were first visually inspected to exclude trials that might contain artifacts. Then the EEG signal were band-pass filtered between 8 and 30 Hz, to include the μ (8–12 Hz) and β (12–30 Hz) frequency bands. After this, the signal’s matrix of covariance was computed: the CSP algorithm works by maximizing the variance for one class and at the same time minimize variance for the other class. The variance of the band-pass filtered signal is equivalent to the band power of the signal and the signal variance in the frequency band exploited for the BMI directly reflects the presence of the typical motor-imagery-related patterns of activity. Based on previous studies³¹, we expected the firsts two CSPs to show a prominent contribution from electrodes located over the hemisphere contralateral to one imagined hand movement, and likewise for the last two CSPs, but for the other hand.

For each subject, the first and last two CSPs are used to construct four feature vectors³¹ and these feature vectors are then used to build the linear classifier (LDA classifier, Fig. 2(b)).

On-line session: BMI cursor control

During the on-line session subjects were instructed to mentally perform left or right hand clasping movements as in the calibration phase but with the aim of attempting to control the movement of a cursor (black rectangle) towards the cued side of the screen.

At each time frame, the output of the linear classifier provided the direction and the velocity for the cursor displacement.

As in the calibration session, each trial began with the presentation of a fixation cross (2 seconds) followed by a central arrow, (1 second, cue), pointing either to the left or to the right, indicating which hand had to be imagined (right or left) and the side of the screen towards which the subject should attempt to move the cursor (Fig. 2(c)). The hand involved in the imagined movement and the cued side of the screen were always coincident (e.g. right hand and right side of the screen). After 1 second, the cursor appeared and the subject could start immediately controlling it.

If the cursor reached the side of the screen within the timeout period of 6 seconds, then the cursor disappeared and a fixation cross was displayed for the remaining time. At the end of the cursor control period, subjects had a 7 seconds of rest. The total duration of each trial was 16 seconds. There were a total of 64 trials divided into two runs, equally divided into the two imagined hand.

Data Analysis

Analysis of behavioral data

Questionnaire

For each subject, we computed the average of the ratings for visual and kinesthetic MI items of the MIQ-RS, separately. Then, we statistically tested for differences between the two groups (independent samples t-test, two-tailed), independently for the two subscales.

Mental chronometry

Therefore, we computed for each subject the average time required to perform the actual or imaged clasping movement with the right and left hand in the mental chronometry task. We submitted these values to a repeated measure ANOVA with imagined hand (left versus right) and task (execution versus MI) as within-subjects factors and with group (high versus low-aptitude BMI users) as between-subjects factor.

Additionally, we calculated an index of deviation from isochrony by computing the ratio between the average duration of actual and imaged clasping movement for every subject and finally by considering the absolute value of the deviation of this ratio from 1, as indicated by the following formula:

A ratio equal to 1 (“isochrony” between MI and actual execution) indicates an equal time required for completing both tasks. In agreement with the “principle of isochrony” governing the relationship between imagined and actual movement duration²¹,²²,³⁹, higher values of “deviation from isochrony” correspond to lower MI abilities. We then compare statistically this index between the two groups (independent samples t-test, two-tailed).

Statistical analysis of electrophysiological data

Analysis of μ-rhythm

First we computed for each subject and separate for the right and left MI, the event shift in the μ band (8–12 Hz) power spectrum (event related spectral perturbation, ERSP⁶¹), considering as a baseline the first two seconds at the beginning of each trial, during the presentation of the fixation cross and the arrow indicating which hand to imagine. Then we considered the average over time from 1 seconds after the beginning of the MI period till the end of the 6 seconds of MI.

We first investigated the presence of the classical modulation of the μ-rhythm due to MI already reported in an extended body of literature⁴⁶. Then, as an exploratory approach we statistically tested for differences in the μ-band desynchronization between the high- and low-aptitude groups, separate for left and right MI over all electrodes, except for those lying along the central line and those over fronto-temporal, temporal and temporo-parietal regions. We then selected two subgroups of electrode, one over each hemisphere, and extracted the average value over each cluster for the two imagined hands (Fig. 3).

We computed an index of laterality, representing the different degree of activation between the two hemispheres: to do this we calculated the difference between the contralateral and ipsilateral cluster separate for the two imagined hands. This gave us a single index accounting for the difference in activation between the hemisphere contralateral to the imagined hand and the ipsilateral one over sensorimotor regions. We expected a more pronounced difference, i.e. a higher lateralization between the two hemispheres, in high aptitude users. More specifically, since the μ-band modulation is expressed in term of desynchronization with respect to the rest period at the beginning of each trial, we expect a stronger suppression, thus a more negative value, for high aptitude users than for low-aptitude users. We run a repeated measure ANOVA with imagined hand (right versus left) as within-subjects factors and with group (high versus low-aptitude BMI users) as between-subjects factor. If the ANOVA revealed a significant interaction, we performed post hoc comparisons using Newman-Keuls tests.

Analysis of Common Spatial Pattern

We decided to investigate whether the different performance between the two groups could be depicted by the topography of the first and the last CSP. This analysis was motivated by the observation that a high aptitude BMI user would likely present a clear lateralized pattern of activation (Fig. 2(b) and Supplementary Figure 1), with a more prominent contribution in the classification from electrodes placed contralateral to the imagined direction as already described elsewhere³¹. We computed then the correlation coefficient between the first CSP (right hand MI) and the mirrored-imaged of the last CSP (left hand MI): if the lateralized pattern would be present in both CSP, it would result in a higher correlation coefficient. Then, we statistically investigated (independent samples t-test, two-tailed) differences in the correlation coefficient between high and low-aptitude BMI users.

Prediction of BMI-control abilities

Moreover, we were interested in investigating whether the two behavioral indices related to MI abilities could also be exploited to predict if a subject would belong to the high- or low-aptitude group. Behavioral, rather than the neurophysiological data are investigated as predictors for two reasons. First, we wanted to avoid any possible circularity which could be caused by using as predictors, variables extracted from EEG data, being the division between high and low-aptitude users based on the BMI-control performance.

Secondly, behavioral predictors offer the great advantage of avoiding setting up an EEG recording, thus notably decreasing the acquisition time.

Thus, we used two behavioral variables extrapolated from the kinesthetic ratings to MIQ-RS questionnaire and from mental chronometry task (deviation from isochrony) as predictors for a linear classifier based on a logistic regression algorithm⁶². The training and testing of the classifier were performed using a 8-out cross validation procedure, in which the training set was chosen by leaving out, for the test set, the data from 4 subjects per group. Thus, since our pool included 24 participants, the training set was composed by 16 subjects. This division and the corresponding classification was iterated several times to account for all the possible combinations of training and test set (exhaustive cross-validation). The empirical accuracy was computed by averaging the single accuracy from every cross validation loop.

We tested the statistical significance of our accuracy by considering the binomial cumulative distribution. The binomial distribution give us an analytical threshold of the statistical significance for a two-class classification problem: only in case of an infinite sample size, the statistical significant threshold corresponds to the theoretical chance level at 50%, otherwise the real chance level depends on the sample size (see for a discussion about this topic³²).

To provide a further evidence for the relevance of our behavioral variable in predicting BMI proficiency, we performed the same analysis using two control variables: ratings of the visual subscale of the MIQ-RS and the duration of executed clasping movements acquired during the mental chronometry task (computed as the average duration of the executed movement with the right and left hand). These parameters were chosen since statistical tests comparing these variables between the high- and low-aptitude groups showed no statistical difference.

Additional Information

How to cite this article: Marchesotti, S. et al. Quantifying the role of motor imagery in brain-machine interfaces. Sci. Rep. 6, 24076; doi: 10.1038/srep24076 (2016).