Skip to main content
PMC home page
Computational Intelligence and Neuroscience logoLink to Computational Intelligence and Neuroscience
. 2017 May 4;2017:5491296. doi: 10.1155/2017/5491296

Comparison of Brain Activation during Motor Imagery and Motor Movement Using fNIRS

Alyssa M Batula 1, Jesse A Mark 2, Youngmoo E Kim 1, Hasan Ayaz 2,3,4,*
PMCID: PMC5435907  PMID: 28546809

Abstract

Motor-activity-related mental tasks are widely adopted for brain-computer interfaces (BCIs) as they are a natural extension of movement intention, requiring no training to evoke brain activity. The ideal BCI aims to eliminate neuromuscular movement, making motor imagery tasks, or imagined actions with no muscle movement, good candidates. This study explores cortical activation differences between motor imagery and motor execution for both upper and lower limbs using functional near-infrared spectroscopy (fNIRS). Four simple finger- or toe-tapping tasks (left hand, right hand, left foot, and right foot) were performed with both motor imagery and motor execution and compared to resting state. Significant activation was found during all four motor imagery tasks, indicating that they can be detected via fNIRS. Motor execution produced higher activation levels, a faster response, and a different spatial distribution compared to motor imagery, which should be taken into account when designing an imagery-based BCI. When comparing left versus right, upper limb tasks are the most clearly distinguishable, particularly during motor execution. Left and right lower limb activation patterns were found to be highly similar during both imagery and execution, indicating that higher resolution imaging, advanced signal processing, or improved subject training may be required to reliably distinguish them.

1. Introduction

Motor imagery is the imagined movement of the body while keeping the muscles still, sometimes considered to be a conscious use of unconscious preparation for an actual movement [1]. There have been numerous studies outlining the similarities between motor execution (overt movement) and motor imagery [2–8]. Of particular interest is whether motor imagery follows the same cortical layout as motor execution in the primary motor cortex (M1). Located in Brodmann's area 4 [9], M1 is subdivided into multiple sections, each responsible for control of a different area of the body, in a layout often referred to as the cortical homunculus [10, 11]. This one-to-one mapping between physical movement and activation in a particular area of the brain provides an opportunity to detect a person's actions (and, potentially, their intended actions) solely through brain recordings, making this an interesting area of brain research. It has also been partially responsible for the popularity of both motor execution and motor imagery as control methods for brain computer interfaces (BCIs), where the mental state is estimated via brain activation patterns [12–26]. Other areas of interest for motor imagery detection are the supplementary motor area and premotor cortex, located anterior to M1 and also involved in the motor network [27, 28].

Many functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) studies have indicated that there is no activation in M1 during motor imagery [29–32] or only limited activation [33–35]. Berman et al. found that training with feedback did not increase motor cortex activity for motor imagery. Additionally, subjects who saw an increase in M1 activation during motor execution feedback training also showed an increase in electromyography (EMG), indicating that the increased fMRI blood-oxygen-level dependent (BOLD) signal may be due to increased muscle activity [29]. In a meta-analysis review, Hétu et al. noted that while motor imagery seems to use similar structures to motor execution, M1 is not consistently activated during motor imagery [35]. Authors nonetheless stress that their findings do not conclusively state that M1 is not involved in motor imagery [35].

It has been proposed that the lack of M1 activation in some motor imagery studies may be due to the lower activation levels produced by motor imagery [36]. An fMRI study by Porro et al. found that motor imagery activated M1 without a significant increase in EMG recordings overall, indicating that the increase in M1 activation was not due to muscle activity [8]. Ehrsson et al. determined using fMRI that hand, foot, and tongue motor imagery follow the same organization as motor execution in M1 [37]. Wriessnegger et al. found significant activation compared to rest for both motor execution and motor imagery in the motor areas, but activation for motor imagery was slower (with an approximately 2-second delay) and smaller in magnitude [38]. While motor execution showed significantly higher activation over the sensorimotor area as compared to the anterior prefrontal areas, motor imagery showed no significant difference in activation between these areas. There were also differences in the activation over time for bilateral and contralateral activation between motor imagery and motor execution. Sitaram et al. also found that fNIRS recordings of motor imagery for left and right hand tapping were similar to motor execution recordings, but smaller in magnitude [23]. An fNIRS pilot study by An et al. compared activation from motor execution, imagery, passive movement, and movement observation for a hand grasping task and found that motor imagery induces a moderate activation in M1 [5].

The type of motor imagery can also affect the quality of motor imagery recordings. Two primary types of motor imagery are visual, where a person self-visualizes the movement, and kinesthetic, where a person imagines the feelings and sensations produced by the movement (i.e., what it “feels like” to perform the motion). Lotze and Halsband suggest that simple, highly kinesthetic tasks may increase M1 activation [27]. An fMRI study by Guillot et al. compared kinesthetic and visual motor imagery in participants with good to excellent motor imagery ability and found that kinesthetic motor imagery shares more similar neural pathways to motor execution, but both forms of motor imagery caused activation in M1 [39].

It has also been reported that motor imagery recordings in M1 are greatly affected by the motor imagery abilities of individual subjects and not just recording and experimental methods [40–42]. The use of objective questionnaires has been proposed in order to determine whether a person will be able to use motor imagery effectively [42]. Miller et al. found that activation in M1 during motor imagery, measured using electrocorticography (ECoG), can be increased via training with feedback and that in some cases it can exceed the original motor execution levels [28]. Similar improvements in motor imagery activation were found when controlling a robot with an electroencephalography (EEG) BCI [43].

A variety of motor imagery techniques have been examined for use with fNIRS. Earlier, Coyle et al. were able to distinguish imagination of squeezing a ball from rest with an accuracy of 70–90% [21]. Other studies have shown up to 89% accuracy distinguishing motor imagery of the left hand or wrist from the right hand or wrist [22, 23]. More recently, fNIRS has been used to detect motor imagery activation for a tennis arm-swinging motion [44] as well as a finger-tapping sequence [45]. The ability to distinguish motor imagery of the feet is also being explored for use in BCIs [12, 24]. While both feet are typically used together in motor imagery BCIs, recently we and other researchers have begun to use left and right feet or legs separately in fNIRS [12, 13] and EEG [46].

In the current study, we aim to explore the similarities and differences between the motor cortex activation recorded via fNIRS during simple motor imagery and motor execution tasks for both upper and lower limb movements. The four tasks under investigation are left hand, right hand, left foot, and right foot tapping as compared to a resting state. To the best of our knowledge, this is the first fNIRS study to compare motor imagery and motor execution for each foot individually. Additionally, we consider the ability of fNIRS to distinguish between left and right foot motor tasks, which could benefit future BCIs through the addition of new control tasks.

2. Materials and Methods

2.1. Participants

Thirteen healthy participants volunteered in the experiment. Subjects were aged 18–35, right-handed (according to the Edinburgh Handedness Inventory), English speaking, and with vision correctable to 20/20. No subjects reported any physical or neurological disorders or were on medication. The experiment was approved by the Drexel University Institutional Review Board, and subjects were informed of the experimental procedure and provided written consent prior to participating.

2.2. Functional Near-Infrared Spectroscopy (fNIRS)

fNIRS is a noninvasive, relatively low-cost, portable, and potentially wireless optical brain-imaging technique [47]. It uses near-infrared light to measure changes in oxygenated (HbO) and deoxygenated (HbR) hemoglobin levels due to the hemodynamic response, the rapid delivery of oxygenated blood to active cortical areas through neurovascular coupling [48]. Recordings from fNIRS are similar to fMRI [49–51], but the measurement area is limited to the outer cortex and has lower spatial resolution (mm versus cm) [49]. However, fNIRS can measure at a higher temporal resolution to capture additional frequency bands and does not require subjects to lay down in a supine position with the loud noises generated by fMRI. Due to their portability, fNIRS devices can be used in more natural settings, such as sitting at a desk and even with mobile participants walking outdoors [52], rather than solely in restrictive and artificial lab environments. Despite the lower temporal resolution and time delay of the hemodynamic response compared to EEG measurements, fNIRS provides a unique trade-off between temporal and spatial resolution and is free from most artifacts, such as muscle activity and eye blinks. It can also easily be used in conjunction with other measurement techniques such as physiological signals [53], EEG [15, 54, 55], and neurostimulation [56, 57].

In the common configuration, light sources and detectors are placed on the scalp and two wavelengths of light are transmitted through the top layer of the cerebral cortex. Light at wavelengths between approximately 700 and 900 nm can pass through skin, bone, and water, but it is absorbed primarily by HbO and HbR [58]. Because HbO and HbR have different light absorption properties, the relative changes in HbO and HbR, and therefore the change in oxygenation of the tissue, can be calculated from changes in the reflected dual-wavelenghth light using the modified Beer-Lambert law [59].

2.3. Data Acquisition

Participants sat in a chair facing a monitor that displayed the experiment cues. They were instructed to sit with both feet flat on the floor and hands in their lap or on chair arm rests with palms facing upwards.

Twenty-four optodes (measurement locations) over the primary and supplementary motor cortices were recorded using a Hitachi ETG-4000 optical topography system, as shown in Figure 1. Sensors were arranged in two separate arrays, one each for the left and right hemispheres. The arrays were placed directly next to each other, and adjacent sources and detectors within each array were 3 cm apart. The center point between the two arrays was aligned with Cz for each participant according to the International 10–20 system. Although not as high spatial resolution as fMRI, fNIRS can provide spatial resolution to capture task differences and distribution of activation over cortical areas comparable to fMRI [49, 51]. HbO and HbR levels were recorded at each location at a sampling rate of 10 Hz.

Figure 1.

Figure 1

Open in a new tab

Layout of the light sources, light detectors, and optodes (numbered 1–24). Adjacent sources and detectors are 3 cm apart.

2.4. Experimental Protocol

Motor imagery and motor execution data were recorded on two separate days in order to collect more data while keeping the session length to one hour. Both sessions included motor imagery and motor execution tasks. The protocol included five tasks: rest and tapping of the right hand, left hand, right foot, and left foot. This protocol was developed based on a preliminary study we reported previously [12, 13], and details are provided below.

2.4.1. Tasks

Participants performed five tasks under both motor execution and motor imagery conditions. During motor execution, they were instructed to tap the indicated hand or foot once per second, self-paced. Hand tapping was demonstrated as curling and uncurling the fingers towards the palm, similar to clenching an imaginary ball. Foot tapping kept the heel on the floor as the ankle bent to raise and lower the toes, and subjects were instructed to also curl and uncurl their toes during the movement. During the resting state, participants were instructed to relax, refrain from moving, and not think about anything in particular.

During motor imagery tasks, subjects were instructed to imagine the same actions as performed during motor execution but to refrain from any movement, including muscle twitches. They were also instructed to use kinesthetic imagery, that is, imagine the feelings and sensations felt during an actual movement. Subjects practiced both motor imagery and motor execution tasks, guided by the experimental program, before beginning the experiment in order to familiarize themselves with the protocol and tasks.

The trials followed the timing protocol shown in Figure 2. Each trial began with 9 seconds of rest followed by a cue to indicate the upcoming task (e.g., left foot). Subjects then performed the designated task for 15 seconds, followed by a 4-second “Trial Finished” message indicating the task was over. There were a total of 15 seconds between the end of one task and the beginning of another to allow activation to return to a baseline level. Intertrial periods as low as 10 seconds have been used in prior fNIRS motor imagery studies [5, 60, 61].

Figure 2.

Figure 2

Open in a new tab

Trial timing diagram.

2.4.2. Session Protocol

Each day was split into multiple runs, as shown in Figure 3. The session consisted of three repetitions of a run with 10 motor execution trials followed by a run with 25 motor imagery trials. The trials in each run (motor imagery or motor execution) contained an equal number of all 5 tasks presented in a pseudorandomized order, without allowing the same task to appear more than twice in a row within each run. Motor execution was interspersed between sections of motor imagery in order to improve the subject's ability to imagine performing the task, both as a reminder of the kinesthetics involved in the actual movement and to reduce fatigue caused by repeated motor imagery trials [62]. A total of 150 motor imagery and 60 motor execution trials were collected for each subject.

Figure 3.

Figure 3

Open in a new tab

Experiment protocol: each day had three repetitions of the motor execution and motor imagery runs.

2.5. Data Analysis

An outline of the analysis procedure, run separately for motor imagery and motor execution data, is shown in Figure 4. All data followed the same preprocessing methods prior to extracting individual task periods for each trial. After preprocessing, there were separate procedures for time series analysis and overall activation and statistical analysis.

Figure 4.

Figure 4

Open in a new tab

Overview of the data analysis procedure, performed separately for motor imagery and motor execution tasks.

2.5.1. Preprocessing

The recorded data were filtered by a 100th order low-pass finite impulse response filter with a 0.1 Hz cutoff frequency in order to remove high-frequency physiological signals such as heart rate and respiration. Data quality was evaluated manually by an expert and optodes with poor quality were removed, as were sections of data that contained artifacts or that were saturated. Subjects needed at least 20 trials of usable data for each of the five tasks, and no more than three optodes missing from both days. Five subjects were excluded due to insufficient data quality. Common average referencing (CAR) based spatial filtering was applied to enhance the signal quality. CAR is a method commonly used in EEG in which the average value of all optodes at each time point is used as a common reference (i.e., the average value across all channels is subtracted from each optode at that time point). This enhances changes in small sets of optodes while removing global spatial trends from the data.

2.5.2. Time Series Visualization

The 15-second task period for each trial was extracted from the data, along with a period of nontask data immediately before and after the trial. Correlation-based signal improvement (CBSI) was applied to each trial period in order to reduce artifact noise and improve signal quality [63]. Baseline correction was then performed on each trial to ensure that the beginning of each task period was approximately zero. This was accomplished by subtracting the average value of the data 0.5 seconds through 0.5 seconds after the start of the task period from every point in the task. The average motor imagery and motor execution time series for a representative optode was calculated for each task.

2.5.3. Average Activation Plots

To create activation plots, the 15-second task period of each trial was extracted immediately after preprocessing. The activation is the difference in HbO levels for each optode between the beginning and end of the trial. To calculate the activation, the average HbO level for the first two seconds of the task was subtracted from the average HbO level from the last 6 seconds of the task (seconds 9–15) for each optode, resulting in a total of 24 features for each trial. The difference in activation level from the beginning of the task was also calculated at multiple other time points during the trial in order to visualize the change in activation over time.

The average activation level for each optode during the resting state was subtracted from all features from that optode, in order to enhance the ways in which the task condition differs from the rest condition. All trials for a given task were averaged across all subjects to give an overall average activation level. These data were arranged into a spatial map according to the optodes' locations and linearly interpolated to show the activation locations.

An additional analysis was performed to show the difference between motor execution and motor imagery by subtracting the average motor imagery values from the motor execution values for the corresponding optode and task.

2.5.4. Statistical Analysis

Statistical analysis was performed using linear mixed models on the average value of the last 6 seconds of each task. Each optode was evaluated for the main effects of task (5 levels: left hand, right hand, left foot, right foot, and rest), type (2 levels: motor imagery, motor execution), and their interaction on the HbO activation level of each optode. Multiple-testing correction (false discovery rate: FDR) was applied to the resulting p values for each effect using the R p.adjust() function and the “FDR” method. Then, each optode was evaluated for the effect of task on HbO activation levels individually for motor imagery and motor execution (all p values adjusted using FDR). Optodes were also evaluated for the effect of type (motor imagery or motor execution) on HbO activation individually for each optode and task (p values adjusted using FDR).

An additional post hoc analysis was conducted to determine, for each task, which optodes had a significant increase in HbO levels from the beginning of the task. A linear mixed model compared the average value from seconds 9–15 to the average value from 0-1 seconds individually for each optode and task. The resulting p values were also adjusted using FDR.

3. Results

Task (5 levels: left hand, right hand, left foot, right foot, and rest) had a significant effect on HbO activation (p < 0.05, FDR adjusted) in fifteen optodes (1, 5, 7, 9, 10, 12, 13, 14, 16, 17, 18, 19, 21, 22, and 24). These optodes stretch across the sensory arrays over Cz (according to the International 10/20 system), corresponding roughly to what we expect to find based on the cortical homunculus layout of the motor cortex. Motor type (2 levels: motor imagery, motor execution) showed a significant effect on HbO activation (p < 0.05, FDR adjusted) in six optodes (1, 9, 13, 16, 22, and 24), all of which also showed an effect for task. Task and type had a significant interaction (p < 0.05, FDR adjusted) for ten optodes (1, 5, 7, 9, 10, 13, 16, 19, 22, and 24), including all six optodes showing an effect for motor type. A table of all results is included in Online Resource 1 (Table S1 in Supplementary Material, available online at https://doi.org/10.1155/2017/5491296).

A post hoc analysis evaluated the effect of task on each optode individually for each motor type. Nine optodes (5, 7, 9, 10, 12, 16, 19, 22, and 24) showed a significant effect of task on motor execution, and three optodes (1, 7, and 22) showed a significant effect of task on motor imagery (p < 0.05, FDR corrected). The full table of results is available in Online Resource 1 (Table S2).

An additional post hoc analysis examined the difference in HbO activation between motor execution and motor imagery individually for each task and optode. Eleven optodes showed a significant effect of motor type on HbO activation for at least one and up to three of the four different motor tasks (p < 0.05, FDR adjusted). Motor execution showed a larger increase in HbO activation than motor imagery on the contralateral hemisphere, particularly for the two hand tasks and right foot task. Additionally, the increased activation for right foot was concentrated more closely in the center, near Cz, while the increased activation for hand tasks was further from the center and closer to C3 and C4. Figure 5 shows the average difference in activation between motor execution and motor imagery for each task, determined by subtracting the average motor imagery activation level from the average motor execution level for each optode. Optodes that showed a significant effect of motor type (p < 0.05, FDR adjusted) are circled. The corresponding p values are listed in Table S3 of Online Resource 1.

Figure 5.

Figure 5

Open in a new tab

Average difference in activation between motor execution and motor imagery. Optodes with a significant difference (p < 0.05, FDR adjusted) between motor execution and motor imagery for a given task are circled.

Looking at the average activation levels for motor imagery and motor execution, it can also be seen that motor execution showed a clearer contralateral activation, while motor imagery showed more bilateral activation patterns. The right hand task had the most similar activation pattern between motor imagery and motor execution. Additionally, the right hand motor imagery task has a significant effect on an optode showing a decrease in HbO on the ipsilateral side, but none of the optodes showing an increase in HbO (on either hemisphere) during right hand motor imagery were rated as having a significant effect. Left hand and right foot motor imagery had more bilateral or ipsilateral activation patterns than during motor execution. The right foot task has two optodes that showed significant changes in activation during both motor imagery and motor execution. Additionally, left foot motor imagery shows a more expected activation pattern than for motor execution, with an area of activation and an optode with statistically significant activation in the optodes near Cz on the contralateral side.

Right hand and left hand motor execution tasks are the most easily distinguished among the tasks. Left and right foot motor execution have much more similar activation patterns, although right foot has a more contralateral activation pattern as opposed to that of left foot. Additionally, right hand and right foot motor imagery are very similar, with right foot having slightly lower activation levels and less activation in the optodes further from Cz.

Figure 6 shows the average HbO activation (contrasted against the resting state by subtracting the rest feature from each of the other tasks for each optode) across all subjects from 9–15 seconds after the start of the task. Optodes found to have a significant (p < 0.05, FDR adjusted) difference between the first second of the task and seconds 9–15 are circled.

Figure 6.

Figure 6

Open in a new tab

Average HbO activation across all subjects for motor execution (a) and motor imagery (b). Significant optodes (p < 0.05, FDR adjusted) are circled.

The timing of the spatial activation patterns also differs between motor imagery and motor execution. While both motor imagery and motor execution have relatively low, diffuse activation at the start of the right hand task, motor execution quickly shifts to highly contralateral activation (by approximately 5 seconds) and remains mostly unchanged for the duration of the task. Motor imagery shifts to contralateral activation more slowly, by about 10–15 seconds into the task. Additionally, the motor imagery activation levels never reach the strength of motor execution. The timing of HbO activation for the right hand task is shown in Figure 7. Each row shows the average activation over all subjects for a specific time period. Full-size plots for all four motor tasks (left hand, right hand, left foot, and right foot) are available in Online Resource 1 (Figures S1–S4).

Figure 7.

Figure 7

Open in a new tab

Average in HbO activation over time for motor imagery and motor execution during the right hand task.

Differences can also be seen in the average time series for a single optode for each of the four tasks. Activation in optode 16 during the right hand task shows a distinct tendency for larger and faster HbO activation during motor execution than imagery, but the latter increases towards the end of the recording period. Left and right foot (in optodes 2 and 24, resp.) show significant activation during motor execution with low or no corresponding activation during motor imagery tasks. In contrast, the left hand shows a more similar activation pattern in both time and strength between motor imagery and motor execution in optode 1. Figure 8 shows the average time series across all subjects for a single optode during each task, along with the standard error of the mean.

Figure 8.

Figure 8

Open in a new tab

Average HbO and HbR activation for a single optode for each task. Standard error of the mean is shown as a faded area around the average. White area from 0 to 15 seconds is the task period; the grey areas are the resting state before and after the task.

4. Discussion

This study examined the differences in brain activity for upper and lower limbs during motor imagery and motor execution tasks recorded using fNIRS. Motor execution and motor imagery showed differences in activation timing of HbO, with motor imagery activation levels increasing more slowly than the corresponding motor execution tasks, as has been reported previously [38]. There were significant differences for spatial distribution of activation between execution and imagery as shown in Figure 5. Moreover, motor execution also showed higher activation levels than motor imagery overall, reflected in the number of optodes with significant HbO activation during the task as well as the number of optodes showing a significant effect of task. These are also in line with previously reported findings [23, 38]. Such differences between execution and imagery could be due in part to the continuous somatosensory and visual feedback of the movement and muscle stimulation that is only present during motor execution [23, 35, 64].

Upper limb (i.e., left and right hand) tasks were the most easily distinguishable for left and right comparison among the four task types based on the spatiotemporal activation patterns. Motor execution hand tasks showed a larger increase in HbO levels, as well as more optodes where task had a significant effect on HbO levels than motor imagery hand tasks. They also showed a strong contralateral activation pattern, while motor imagery hand tasks had more bilateral activation patterns, which has also been observed previously [33].

Right hand demonstrated mostly contralateral activation patterns for both motor imagery and motor execution conditions, as shown in Figure 6, while left hand showed a much more bilateral activation during motor imagery. This could be due to the fact that all participants were right-handed, potentially making the right hand task easier to imagine. Despite its primarily contralateral activation pattern, right hand motor imagery also showed a significant decrease in HbO levels in an optode on the ipsilateral side, without a statistically significant increase for any optodes on the contralateral side. This could indicate that subjects utilized an alternative strategy rather than exactly simulating right hand tapping during the imagery. Future studies may investigate whether training of subjects and use of different mental strategies may affect activity during motor imagery.

Foot tasks had much more bilateral or ipsilateral HbO activation patterns than the hand tasks in both motor execution and motor imagery conditions. Right foot motor execution showed the most contralateral activation pattern, while during motor imagery the activation was much more ipsilateral. However, right foot was the only task to have optodes that showed a statistically significant change from the beginning of the task during both motor imagery and motor execution. The left foot task, on the other hand, showed a more contralateral activation pattern and more optodes with significant activation during motor imagery, while during motor execution the activation was highly diffuse and bilateral, with no significant activation levels at any of the optodes.

The highly bilateral activation patterns during left and right foot tasks indicate that distinguishing between left and right foot using fNIRS may prove difficult. Higher resolution may be required in order to reliably distinguish between the two feet, or they may be best used together in a single “feet” task (as done in some recent BCI studies [17, 65, 66]). Alternatively, using whole leg motor imagery, instead of toe/foot, as in the EEG study by Hsu et al. [46], could cause activation patterns more readily identifiable using fNIRS. Toe and foot motor areas are near or within the longitudinal fissure between brain hemispheres, which is more difficult to measure, while leg motor areas are further apart and closer to the surface of the scalp [10, 11]. In a recent study, we demonstrated the enhanced functional connectivity between motor-related brain regions (M1 and primary somatosensory cortex) and high-level cognitive brain regions during the transition period between rest and hand movements [67]. As dorsolateral prefrontal cortex seems to play a role in the preparation of the sensorimotor system for the task, level of motivation and practice as well as mental workload and distractors in the environment could affect motor imagery related activity levels. One limitation of the current study was the lack of motor imagery ability evaluation of the participants. Future experiments should consider screening subjects for motor imagery abilities as suggested by Marchesotti et al. [42]. Additionally, feedback training could be used to improve motor imagery abilities, as suggested by Miller et al. [28]. This could also improve the distinguishability of the two foot tasks.

5. Conclusions

Prior studies have suggested that fNIRS can detect changes in brain activity during motor imagery and motor movement hand tasks similar to fMRI. This study confirms and extends these findings to motor movement and imagery of the left and right foot. Although the activation was relatively weaker, all motor imagery tasks still showed significant levels of activation. The results further suggest that motor execution more strongly evoked the expected contralateral activation patterns compared to motor imagery, particularly in both hand tasks. The differences in spatial distribution of activation between execution and imagery highlights the need for attention when selecting classifier features for BCI use. Moreover, left and right foot activation patterns were more difficult to differentiate than the hand tasks. Differences between left and right foot activation may be made more distinct by using higher resolution imaging, advanced signal processing, such as task-related functional connectivity, or improved subject training with a specific mental strategy.

Current fNIRS-based BCI systems have primarily focused on left/right hand motor imagery tasks. This study opens the door to the use of foot imagery to complement hand imagery tasks in fNIRS-based BCI paradigms. Future BCI systems could develop new approaches to use such multiclass motor imagery to increase overall system performance and BCI usability.

Supplementary Material

The Supplementary Material accompanying this paper includes additional results in tables for the three statistical analyses. It also includes a figure comparing motor imagery and motor execution activation patterns over time for each of the four motor tasks.

5491296.f1.zip (4.5MB, zip)

Acknowledgments

This work was supported in part by the National Science Foundation Graduate Research Fellowship under Grant no. DGE-1002809. Work reported here was run on hardware supported by Drexel's University Research Computing Facility. The authors would like to thank Mr. Adrian Curtin for his help in the study.

Conflicts of Interest

fNIR Devices, LLC, manufactures optical brain-imaging instruments and licensed IP and know-how from Drexel University. Dr. Ayaz was involved in the technology development and thus offered a minor share in the new startup firm fNIR Devices, LLC. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.

References

  • 1.Jeannerod M. Mental imagery in the motor context. Neuropsychologia. 1995;33(11):1419–1432. doi: 10.1016/0028-3932(95)00073-C. [DOI] [PubMed] [Google Scholar]
  • 2.Munzert J., Lorey B., Zentgraf K. Cognitive motor processes: the role of motor imagery in the study of motor representations. Brain Research Reviews. 2009;60(2):306–326. doi: 10.1016/j.brainresrev.2008.12.024. [DOI] [PubMed] [Google Scholar]
  • 3.Sharma N., Jones P. S., Carpenter T. A., Baron J.-C. Mapping the involvement of BA 4a and 4p during Motor Imagery. NeuroImage. 2008;41(1):92–99. doi: 10.1016/j.neuroimage.2008.02.009. [DOI] [PubMed] [Google Scholar]
  • 4.Solodkin A., Hlustik P., Chen E. E., Small S. L. Fine modulation in network activation during motor execution and motor imagery. Cerebral Cortex. 2004;14(11):1246–1255. doi: 10.1093/cercor/bhh086. [DOI] [PubMed] [Google Scholar]
  • 5.An J., Jin S. H., Lee S. H., et al. Cortical activation pattern for grasping during observation, imagery, execution, FES, and observation-FES integrated BCI: an fNIRS pilot study. Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC '13); July 2013; Osaka, Japan. pp. 6345–6348. [DOI] [PubMed] [Google Scholar]
  • 6.Kuhtz-Buschbeck J. P., Mahnkopf C., Holzknecht C., Siebner H., Ulmer S., Jansen O. Effector-independent representations of simple and complex imagined finger movements: a combined fMRI and TMS study. European Journal of Neuroscience. 2003;18(12):3375–3387. doi: 10.1111/j.1460-9568.2003.03066.x. [DOI] [PubMed] [Google Scholar]
  • 7.Stippich C., Ochmann H., Sartor K. Somatotopic mapping of the human primary sensorimotor cortex during motor imagery and motor execution by functional magnetic resonance imaging. Neuroscience Letters. 2002;331(1):50–54. doi: 10.1016/S0304-3940(02)00826-1. [DOI] [PubMed] [Google Scholar]
  • 8.Porro C. A., Francescato M. P., Cettolo V., et al. Primary motor and sensory cortex activation during motor performance and motor imagery: a functional magnetic resonance imaging study. Journal of Neuroscience. 1996;16(23):7688–7698. doi: 10.1523/JNEUROSCI.16-23-07688.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Geyer S., Ledberg A., Schleicher A., et al. Two different areas within the primary motor cortex of man. Nature. 1996;382(6594):805–807. doi: 10.1038/382805a0. [DOI] [PubMed] [Google Scholar]
  • 10.Cheyne D., Kristeva R., Deecke L. Homuncular organization of human motor cortex as indicated by neuromagnetic recordings. Neuroscience Letters. 1991;122(1):17–20. doi: 10.1016/0304-3940(91)90182-S. [DOI] [PubMed] [Google Scholar]
  • 11.Lotze M., Erb M., Flor H., Huelsmann E., Godde B., Grodd W. fMRI evaluation of somatotopic representation in human primary motor cortex. NeuroImage. 2000;11(5):473–481. doi: 10.1006/nimg.2000.0556. [DOI] [PubMed] [Google Scholar]
  • 12.Batula A. M., Ayaz H., Kim Y. E. Evaluating a four-class motor-imagery-based optical brain-computer interface. Proceedings of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC '14); August 2014; Chicago, Ill, USA. pp. 2000–2003. [DOI] [PubMed] [Google Scholar]
  • 13.Batula A. M., Mark J., Kim Y. E., Ayaz H. Developing an optical brain-computer interface for humanoid robot control. In: Schmorrow D. D., Fidopiastis M. C., editors. Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience. Vol. 9743. Toronto, Canada: Springer International Publishing; 2016. pp. 3–13. (Lecture Notes in Computer Science, (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes Bioinformatics)). [Google Scholar]
  • 14.Acqualagna L., Botrel L., Vidaurre C., Kübler A., Blankertz B. Large-scale assessment of a fully automatic co-adaptive motor imagery-based brain computer interface. PLoS ONE. 2016;11(2) doi: 10.1371/journal.pone.0148886.e0148886 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Koo B., Lee H.-G., Nam Y., et al. A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery. Journal of Neuroscience Methods. 2015;244(1):26–32. doi: 10.1016/j.jneumeth.2014.04.016. [DOI] [PubMed] [Google Scholar]
  • 16.Chae Y., Jeong J., Jo S. Toward brain-actuated humanoid robots: asynchronous direct control using an EEG-Based BCI. IEEE Transactions on Robotics. 2012;28(5):1131–1144. doi: 10.1109/tro.2012.2201310. [DOI] [Google Scholar]
  • 17.Doud A. J., Lucas J. P., Pisansky M. T., He B. Continuous three-dimensional control of a virtual helicopter using a motor imagery based brain-computer interface. PLoS ONE. 2011;6(10) doi: 10.1371/journal.pone.0026322.e26322 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Lafleur K., Cassady K., Doud A., Shades K., Rogin E., He B. Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain-computer interface. Journal of Neural Engineering. 2013;10(4):711–726. doi: 10.1088/1741-2560/10/4/046003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Wang M., Daly I., Allison B. Z., et al. A new hybrid BCI paradigm based on P300 and SSVEP. Journal of Neuroscience Methods. 2015;244:16–25. doi: 10.1016/j.jneumeth.2014.06.003. [DOI] [PubMed] [Google Scholar]
  • 20.Yi W., Zhang L., Wang K., et al. Evaluation and comparison of effective connectivity during simple and compound limb motor imagery. Proceedings of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC '14); August 2014; Chicago, Ill, USA. pp. 4892–4895. [DOI] [PubMed] [Google Scholar]
  • 21.Coyle S. M., Ward T. E., Markham C. M. Brain-computer interface using a simplified functional near-infrared spectroscopy system. Journal of Neural Engineering. 2007;4(3):219–226. doi: 10.1088/1741-2560/4/3/007. [DOI] [PubMed] [Google Scholar]
  • 22.Naseer N., Hong K.-S. Classification of functional near-infrared spectroscopy signals corresponding to the right- and left-wrist motor imagery for development of a brain-computer interface. Neuroscience Letters. 2013;553:84–89. doi: 10.1016/j.neulet.2013.08.021. [DOI] [PubMed] [Google Scholar]
  • 23.Sitaram R., Zhang H., Guan C., et al. Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain-computer interface. NeuroImage. 2007;34(4):1416–1427. doi: 10.1016/j.neuroimage.2006.11.005. [DOI] [PubMed] [Google Scholar]
  • 24.Ito T., Akiyama H., Hirano T. Brain machine interface using portable Near-InfraRed spectroscopy—improvement of classification performance based on ICA analysis and self-proliferating LVQ. Proceedings of the 26th IEEE/RSJ International Conference on Intelligent Robots and Systems: New Horizon (IROS '13); November 2013; pp. 851–858. [DOI] [Google Scholar]
  • 25.Barbosa A. O. G., Achanccaray D. R., Meggiolaro M. A. Activation of a mobile robot through a brain computer interface. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '10); May 2010; pp. 4815–4821. [DOI] [Google Scholar]
  • 26.Yin X., Xu B., Jiang C., et al. Classification of hemodynamic responses associated with force and speed imagery for a brain-computer interface. Journal of Medical Systems. 2015;39(5):p. 53. doi: 10.1007/s10916-015-0236-0. [DOI] [PubMed] [Google Scholar]
  • 27.Lotze M., Halsband U. Motor imagery. Journal of Physiology Paris. 2006;99(4–6):386–395. doi: 10.1016/j.jphysparis.2006.03.012. [DOI] [PubMed] [Google Scholar]
  • 28.Miller K. J., Schalk G., Fetz E. E., Den Nijs M., Ojemann J. G., Rao R. P. N. Cortical activity during motor execution, motor imagery, and imagery-based online feedback. Proceedings of the National Academy of Sciences of the United States of America. 2010;107(9):4430–4435. doi: 10.1073/pnas.0913697107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Berman B. D., Horovitz S. G., Venkataraman G., Hallett M. Self-modulation of primary motor cortex activity with motor and motor imagery tasks using real-time fMRI-based neurofeedback. NeuroImage. 2012;59(2):917–925. doi: 10.1016/j.neuroimage.2011.07.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Hanakawa T., Immisch I., Toma K., Dimyan M. A., Van Gelderen P., Hallett M. Functional properties of brain areas associated with motor execution and imagery. Journal of Neurophysiology. 2003;89(2):989–1002. doi: 10.1152/jn.00132.2002. [DOI] [PubMed] [Google Scholar]
  • 31.Gerardin E., Sirigu A., Lehéricy S., et al. Partially overlapping neural networks for real and imagined hand movements. Cerebral Cortex. 2000;10(11):1093–1104. doi: 10.1093/cercor/10.11.1093. [DOI] [PubMed] [Google Scholar]
  • 32.Binkofski F., Amunts K., Stephan K. M., et al. Broca's region subserves imagery of motion: a combined cytoarchitectonic and fMRI study. Human Brain Mapping. 2000;11(4):273–285. doi: 10.1002/1097-0193(200012)11:4&#x0003c;273::AID-HBM40&#x0003e;3.0.CO;2-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Dechent P., Merboldt K.-D., Frahm J. Is the human primary motor cortex involved in motor imagery? Cognitive Brain Research. 2004;19(2):138–144. doi: 10.1016/j.cogbrainres.2003.11.012. [DOI] [PubMed] [Google Scholar]
  • 34.Park C., Chang W. H., Lee M., et al. Which motor cortical region best predicts imagined movement? NeuroImage. 2015;113:101–110. doi: 10.1016/j.neuroimage.2015.03.033. [DOI] [PubMed] [Google Scholar]
  • 35.Hétu S., Grégoire M., Saimpont A., et al. The neural network of motor imagery: an ALE meta-analysis. Neuroscience and Biobehavioral Reviews. 2013;37(5):930–949. doi: 10.1016/j.neubiorev.2013.03.017. [DOI] [PubMed] [Google Scholar]
  • 36.Mellet E., Petit L., Mazoyer B., Denis M., Tzourio N. Reopening the mental imagery debate: lessons from functional anatomy. NeuroImage. 1998;8(2):129–139. doi: 10.1006/nimg.1998.0355. [DOI] [PubMed] [Google Scholar]
  • 37.Ehrsson H. H., Geyer S., Naito E. Imagery of voluntary movement of fingers, toes, and tongue activates corresponding body-part-specific motor representations. Journal of Neurophysiology. 2003;90(5):3304–3316. doi: 10.1152/jn.01113.2002. [DOI] [PubMed] [Google Scholar]
  • 38.Wriessnegger S. C., Kurzmann J., Neuper C. Spatio-temporal differences in brain oxygenation between movement execution and imagery: a multichannel near-infrared spectroscopy study. International Journal of Psychophysiology. 2008;67(1):54–63. doi: 10.1016/j.ijpsycho.2007.10.004. [DOI] [PubMed] [Google Scholar]
  • 39.Guillot A., Collet C., Nguyen V. A., Malouin F., Richards C., Doyon J. Brain activity during visual versus kinesthetic imagery: an fMRI study. Human Brain Mapping. 2009;30(7):2157–2172. doi: 10.1002/hbm.20658. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Jeunet C., N'Kaoua B., Lotte F. Advances in user-training for mental-imagery-based BCI control: Psychological and cognitive factors and their neural correlates. Progress in Brain Research. 2016;228:3–35. doi: 10.1016/bs.pbr.2016.04.002. [DOI] [PubMed] [Google Scholar]
  • 41.Lebon F., Byblow W. D., Collet C., Guillot A., Stinear C. M. The modulation of motor cortex excitability during motor imagery depends on imagery quality. European Journal of Neuroscience. 2012;35(2):323–331. doi: 10.1111/j.1460-9568.2011.07938.x. [DOI] [PubMed] [Google Scholar]
  • 42.Marchesotti S., Bassolino M., Serino A., Bleuler H., Blanke O. Quantifying the role of motor imagery in brain-machine interfaces. Scientific Reports. 2016;6 doi: 10.1038/srep24076.24076 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Bauer R., Fels M., Vukelić M., Ziemann U., Gharabaghi A. Bridging the gap between motor imagery and motor execution with a brain-robot interface. NeuroImage. 2015;108:319–327. doi: 10.1016/j.neuroimage.2014.12.026. [DOI] [PubMed] [Google Scholar]
  • 44.Abdalmalak A., Milej D., Diop M., Naci L., Owen A. M., St Lawrence K. Assessing the feasibility of time-resolved fNIRS to detect brain activity during motor imagery. Clinical and Translational Neurophotonics; Neural Imaging and Sensing; and Optogenetics and Optical Manipulation; February 2016; San Francisco, Calif, USA. [DOI] [Google Scholar]
  • 45.Iso N., Moriuchi T., Sagari A., et al. Monitoring local regional hemodynamic signal changes during motor execution and motor imagery using near-infrared spectroscopy. Frontiers in Physiology. 2016;6, article 416 doi: 10.3389/fphys.2015.00416. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Hsu W.-C., Lin L.-F., Chou C.-W., Hsiao Y.-T., Liu Y.-H. E{EG} classification of imaginary lower LIMb stepping movements based on fuzzy SUPport vector MAChine with Kernel-INDuced MEMbership FUNction. International Journal of Fuzzy Systems. 2017;19(2):566–579. doi: 10.1007/s40815-016-0259-9. [DOI] [Google Scholar]
  • 47.Ayaz H., Onaral B., Izzetoglu K., Shewokis P. A., Mckendrick R., Parasuraman R. Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development. Frontiers in Human Neuroscience. 2013;7, article 871 doi: 10.3389/fnhum.2013.00871. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Villringer A., Chance B. Non-invasive optical spectroscopy and imaging of human brain function. Trends in Neurosciences. 1997;20(10):435–442. doi: 10.1016/S0166-2236(97)01132-6. [DOI] [PubMed] [Google Scholar]
  • 49.Cui X., Bray S., Bryant D. M., Glover G. H., Reiss A. L. A quantitative comparison of NIRS and fMRI across multiple cognitive tasks. NeuroImage. 2011;54(4):2808–2821. doi: 10.1016/j.neuroimage.2010.10.069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Muthalib M., Anwar A. R., Perrey S., et al. Multimodal integration of fNIRS, fMRI and EEG neuroimaging. Clinical Neurophysiology. 2013;124(10):2060–2062. doi: 10.1016/j.clinph.2013.03.018. [DOI] [PubMed] [Google Scholar]
  • 51.Noah J. A., Ono Y., Nomoto Y., et al. fMRI validation of fNIRS measurements during a naturalistic task. Journal of Visualized Experiments. 2015;no. 100 doi: 10.3791/52116.e52116 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.McKendrick R., Parasuraman R., Murtza R., et al. Into the wild: neuroergonomic differentiation of hand-held and augmented reality wearable displays during outdoor navigation with functional near infrared spectroscopy. Frontiers in Human Neuroscience. 2016;10, article 216 doi: 10.3389/fnhum.2016.00216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Zimmermann R., Marchal-Crespo L., Edelmann J., et al. Detection of motor execution using a hybrid fNIRS-biosignal BCI: a feasibility study. Journal of NeuroEngineering and Rehabilitation. 2013;10(1, article 4) doi: 10.1186/1743-0003-10-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Liu Y., Ayaz H., Curtin A., Onaral B., Shewokis P. A. D. Schmorrow. Vol. 8027. Berlin, Germany: Springer; 2013. Towards a hybrid P300-based BCI using simultaneous fNIR and EEG; pp. 335–344. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). [Google Scholar]
  • 55.Fazli S., Mehnert J., Steinbrink J., et al. Enhanced performance by a hybrid NIRS-EEG brain computer interface. NeuroImage. 2012;59(1):519–529. doi: 10.1016/j.neuroimage.2011.07.084. [DOI] [PubMed] [Google Scholar]
  • 56.McKendrick R., Parasuraman R., Ayaz H. Wearable functional near infrared spectroscopy (fNIRS) and transcranial direct current stimulation (tDCS): expanding vistas for neurocognitive augmentation. Frontiers in Systems Neuroscience. 2015;9, article A027 doi: 10.3389/fnsys.2015.00027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Muthalib M., Besson P., Rothwell J., Ward T., Perrey S. Effects of anodal high-definition transcranial direct current stimulation on bilateral sensorimotor cortex activation during sequential finger movements: an fnirs study. In: Elwell C. E., Leung T. S., Harrison D. K., editors. Advances in Experimental Medicine and Biology. Vol. 876. New York, NY, USA: Springer; 2016. pp. 351–359. [DOI] [PubMed] [Google Scholar]
  • 58.Ayaz H., Shewokis P. A., Curtin A., Izzetoglu M., Izzetoglu K., Onaral B. Using MazeSuite and functional near infrared spectroscopy to study learning in spatial navigation. Journal of Visualized Experiments. 2011;(56):1–12. doi: 10.3791/3443. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Cope M. The Development of a near Infrared Spectroscopy System and Its Application for Non Invasive Monitory of Cerebral Blood and Tissue Oxygenation in the Newborn Infants. London, UK: University of London; 1991. [Google Scholar]
  • 60.Abibullaev B., An J., Jin S.-H., Moon J. I. Classification of brain hemodynamic signals arising from visual action observation tasks for brain-computer interfaces: a functional near-infrared spectroscopy study. Measurement: Journal of the International Measurement Confederation. 2014;49(1):320–328. doi: 10.1016/j.measurement.2013.11.044. [DOI] [Google Scholar]
  • 61.Yin X., Xu B., Jiang C., et al. A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching. Journal of Neural Engineering. 2015;12(3) doi: 10.1088/1741-2560/12/3/036004.036004 [DOI] [PubMed] [Google Scholar]
  • 62.Rozand V., Lebon F., Stapley P. J., Papaxanthis C., Lepers R. A prolonged motor imagery session alter imagined and actual movement durations: potential implications for neurorehabilitation. Behavioural Brain Research. 2016;297:67–75. doi: 10.1016/j.bbr.2015.09.036. [DOI] [PubMed] [Google Scholar]
  • 63.Cui X., Bray S., Reiss A. L. Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics. NeuroImage. 2010;49(4):3039–3046. doi: 10.1016/j.neuroimage.2009.11.050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Naseer N., Hong K. fNIRS-based brain-computer interfaces: a review. Frontiers in Human Neuroscience. 2015;9, article 3 doi: 10.3389/fnhum.2015.00003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Kaiser V., Bauernfeind G., Kreilinger A., et al. Cortical effects of user training in a motor imagery based brain-computer interface measured by fNIRS and EEG. NeuroImage. 2014;85:432–444. doi: 10.1016/j.neuroimage.2013.04.097. [DOI] [PubMed] [Google Scholar]
  • 66.Ge S., Wang R., Yu D. Classification of four-class motor imagery employing single-channel electroencephalography. PLoS ONE. 2014;9(6) doi: 10.1371/journal.pone.0098019.e98019 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Cheng L., Ayaz H., Sun J., Tong S., Onaral B. Modulation of functional connectivity and activation during preparation for hand movement. IIE Transactions on Occupational Ergonomics and Human Factors. 2016;4(2-3):175–187. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

The Supplementary Material accompanying this paper includes additional results in tables for the three statistical analyses. It also includes a figure comparing motor imagery and motor execution activation patterns over time for each of the four motor tasks.

5491296.f1.zip (4.5MB, zip)

Articles from Computational Intelligence and Neuroscience are provided here courtesy of Wiley

RESOURCES