High-frequency band temporal dynamics in response to a grasp force task

Abstract

Brain-Computer Interfaces are being developed to restore reach and grasping movements of paralyzed individuals. Recent studies have shown that the kinetics of grasping movement, such as grasp force, can be successfully decoded from electrocorticography (ECoG) signals, and that the high-frequency band (HFB) power changes provide discriminative information that contribute to an accurate decoding of grasp force profiles. However, as the models used in these studies contained simultaneous information from multiple spectral features over multiple areas in the brain, it remains unclear what parameters of movement and force are encoded by the HFB signals and how these are represented temporally and spatially in the SMC. To investigate this, and to gain insight in the temporal dynamics of the HFB during grasping, we continuously modelled the ECoG HFB response recorded from 9 individuals with epilepsy temporarily implanted with ECoG grids, who performed three different grasp force tasks. We show that a model based on the force onset and offset consistently provides a better fit to the HFB power responses when compared with a model based on the force magnitude, irrespective of electrode location. This suggests that HFB power, although potentially useful for continuous decoding, is more closely related to the changes in movement. This result may potentially contribute to the more natural decoding of grasping movement in neural prosthetics.

1. Introduction

Brain-Computer Interfaces (BCIs) have been proposed as a technology to replace central nervous system output that is lost as a result of injury or disease (Brunner et al., 2015). BCI systems can provide, for example, a means of communication to people with locked-in syndrome (Vansteensel et al., 2016), or restore movement through the control of robotic devices (Ajiboye et al., 2017). To date, many BCI systems focus on neural signals acquired from the hand region in the sensorimotor cortex (SMC), as regaining arm and hand function has been described to be crucial to improve the quality of life of severely paralyzed individuals, in particular those with spinal cord injury (Anderson, 2004). Indeed, this region has been shown to encode both discrete (Branco et al., 2017; Chestek et al., 2013) and continuous parameters of hand movement (Flint et al., 2014; Nakanishi et al., 2014) and to be a straightforward target for BCI control (e.g., Wang et al., 2013; Hotson et al., 2016; Vansteensel et al., 2016; Branco et al., 2017; Ajiboye et al., 2017; Milekovic et al., 2018; for a review see also Branco et al., 2019).

In order to achieve seamless control of (robotic) hand movements using a BCI system, the continuous decoding of intended grasp movement seems to be particularly relevant. By providing grasp function through robotic hands, BCIs promise to restore this function in paralyzed individuals. Grasp movement can be described by a combination of kinematic and kinetic parameters (Krakauer et al., 1999; Todorov, 2000), such as movement force, direction, velocity and trajectory (Ashe and Georgopoulos, 1994; Evarts, 1968; Georgopoulos et al., 1982; Wessberg et al., 2000). Movement kinematics are important for accurate description of continuous finger movement and, consequently, the position of robotic hands. However, in order to provide safe object manipulation and meaningful interaction with the environment, the decoding of grasp force is also essential. A limited number of studies have focused on decoding grasp force for BCI control (Carmena et al., 2003; Chen et al., 2014; Flint et al., 2014; Pistohl et al., 2012), of which only two showed successful continuous decoding of grasp force in humans, using ECoG (Chen et al., 2014; Flint et al., 2014). Particularly, these studies showed the high-frequency band (HFB, > 70Hz) to be one of the most discriminative features for force decoding. However, in order to use these features to accurately control a BCI system, we need to understand how these signals behave temporally and spatially during exerted force tasks. Indeed, recent ECoG studies have attempted to describe the movement-related spatial-temporal dynamics of the HFB signals over the SMC during other motor tasks, both in non-human primates (Donoghue et al., 1998; Fetz et al., 1980) and humans (Conant et al., 2018; Crone et al., 1998; Salari et al., 2018). Interestingly, these studies found different types of responses broadly distributed in the SMC, including transient and sustained HFB responses. Therefore, and extending on these findings, we hypothesized that the HFB response during continuous grasp force movements could either represent the exerted force magnitude, displaying a sustained response during isometric force contractions, or, in contrast, represent the transient onset-offset profile of the grasp force movement. These findings potentially increase the understanding of the cortical representation of movement in the SMC, and may, thereby, contribute to the improvement of BCIs.

Here we modelled the temporal dynamics of the HFB ECoG responses, during three different grasp force tasks, namely fast impulse-like responses, continuous dynamic force and isometric force contractions. Using these three tasks, we were able to see how the HFB follows the force profile and to compare the HFB response between different regions of the SMC. For that, we compared two models, one based on the exerted force magnitude and another based of the change in exerted force (i.e., movement onset-offset), with the mean HFB signal over electrodes in four different cortical regions-of-interest, namely 1) all electrodes, 2) electrodes covering the primary motor cortex (M1), 3) the central sulcus (CS) and 4) the primary somatosensory cortex (S1).

2. Materials and Methods

2.1. Subjects

Data from 9 epilepsy patients (4 females, mean age 17.4 years, range 10-34 years, Table 1), who were temporarily implanted with subdural ECoG grids for removal of their focus of epilepsy, were used. Seven patients (S1-S6, and S9) were implanted with clinical grids (1 cm inter-electrode distance, 2.3 mm exposed surface diameter, AdTech, Racine, USA) covering, among other regions, the sensorimotor hand region. Two patients (S7 and S8) were also implanted with high-density (HD) ECoG grids (Table 1) over the sensorimotor hand region. For these patients, only the data recorded from the HD-grid was used in the analysis. HD-grids had 64 channels (8×8 electrode layout), with 1 mm exposed surface diameter and an inter-electrode distance of 4 mm (PMT Corporation, Chanassen, MN, USA). This study was approved by the Medical Ethical Committee of the Utrecht University Medical Center. Informed consent was obtained from all individual participants included in the study according to the Declaration of Helsinki (2013).

Table 1. Patient and ECoG grid information.

	S1	S2	S3	S4	S5	S6	S7	S8	S9
Age	14	17	10	13	14	17	34	20	18
Gender	Female	Male	Male	Male	Female	Male	Female	Male	Female
Handedness	Right	Right	Right	Right	Right	Right	Right	Right	Right
Implanted hemisphere	Right	Right	Right	Left	Right	Left	Right	Left	Right
Grid type	Clinical	Clinical	Clinical	Clinical	Clinical	Clinical	High-density	High-density	Clinical

2.2. Experimental Design and force acquisition

In order to test the linear relationship between the HFB and exerted grasp force, the subjects performed three continuous grasping tasks with biomechanical feedback, hereafter referred to as task 1, 2 and 3 (Figure 1). During these tasks, patients controlled the y-position of an icon on a screen and were instructed to have the icon follow a cue on the screen that moved vertically, by squeezing a rubber-bulb with the hand contralateral to the implanted hemisphere. The change in pressure inside the rubber-bulb can be interpreted to be proportional to the applied grasp force, as force = pressure/area and hand contact area was kept constant throughout the task. Each task consisted of 17 blocks of 20 s, 8 active blocks and 9 rest blocks. In the rest block, the participants were instructed to not squeeze the rubber-bulb and to watch the cue and icon moving automatically at the same pace in the screen. In task 1, the icon started at the bottom of the screen and ‘jumped’ to the top three times per block at a frequency of 0.75 Hz. In task 2 and 3, the icon made a sinusoidal movement over the screen at a velocity of 0.25 Hz. In task 2, a block consisted of five complete and continuous sinusoids. In task 3 the sinusoid was interrupted three times per block (at random heights), and the icon halted on screen for a random period between 1 to 2 s. In task 1 and 3, the time between ‘jumps’ and ‘halts’ varied randomly and lasted at least three seconds. Due to limited time availability of the patients, the number of completed runs varied between tasks and patients. Subjects performed at least one run of each task (Table 2). For each run and task, the force values were calibrated, such that the maximum grasp force corresponded to highest (ceiling) position of the icon on the screen. The force values were sampled at 100 Hz and subsequently upsampled to match the sampling frequency of the ECoG signals.

Figure 1. Three grasp force tasks.

Each participant performed three different grasp force tasks (task 1, 2 and 3). In all tasks the subject was asked to follow a cue (right: green circle) on the screen as accurately as possible by squeezing a rubber-bulb with the hand contralateral to the implanted hemisphere. The participant controlled the y-position of an icon on the screen (right: red circle). In task 1 the cue ‘jumped’ to the top of the screen and immediately returned to the bottom. In task 2 the cue moved up and down at a fixed frequency, leading to a sinusoidal movement of the hand with 5 cycles. In task 3, the cue performed the same sinusoidal movement in task 2 but “halted” at a y-position for small intervals of time (random between 1 and 2 s). In the left panels, the y-axis represents force (in arbitrary units, a.u.) and the x-axis represents time (in seconds, s).

Table 2. Number of runs per subject.

	S1	S2	S3	S4	S5	S6	S7	S8	S9
Task 1	1	1	2	2	1	2	3	1	1
Task 2	2	1	2	2	1	2	3	1	2
Task 3	2	1	2	2	1	2	2	1	3

Since the subjects performed three different tasks, we could assess the HFB response to force production during fast impulse responses, continuous dynamic force transition and isometric conditions. In this study we considered the response to task 1 to be representative of the individual cortical impulse response. Because an impulse response is a single short response with both a high force and high change in force, we used it to identify the electrodes that responded significantly to the task (see section 2.4), and to estimate the parameters necessary to test each of the models (see section 2.5). Task 3 was used to study the ECoG activity during constant force application, but no movement (i.e., during isometric contraction), and task 2 was used to investigate activity during a continuous, sinusoidal variation of applied force.

2.3. ECoG acquisition and data processing

ECoG signals from the clinical grids were acquired with a 128-channel Micromed recording system (Treviso, Italy; hardware band-pass filter 0.15-134.4 Hz) at 512 Hz. Data from the HD-grids were recorded using a Blackrock system (Microsystems LLC, Salt Lake City, USA) at 2000 Hz. The data were analyzed in Matlab® using the FieldTrip Toolbox (Oostenveld et al., 2011). Some electrodes were excluded from the analysis because of noisy or flat signals (which can be caused by broken leads), and were identified based on deviations from the typical power spectral density distributions, line noise values and raw voltage distributions (Liu et al., 2015). The remaining electrodes (Table 3, Figure 2) were used for common-average re-referencing. The signals were band-pass filtered between 0.15 and 130 Hz and notch filtered at 50 and 100 Hz. High-frequency band (HFB) power signals were extracted using a Morlet wavelet transform, with multiplication in the frequency domain (length equal to 3 standard deviations of the implicit Gaussian kernel and width 7 cycles). HFB power was extracted for every time point and for the frequency range of 70-125 Hz (steps of 1 Hz and averaged across all frequency bins). The resulting HFB signals were smoothed using a moving average filter with a 0.1 s window.

Table 3.

Number of electrodes used in this study: total number electrodes included in the analysis, total number of electrodes with significant response to task 1, and number of electrodes with significant responses to the task 1 located over primary motor cortex (M1), primary somatosensory cortex (S1) or central sulcus (CS). Subjects implanted with HD-grids are indicated with an asterisk.

	S1	S2	S3	S4	S5	S6	S7*	S8*	S9
# included electrodes	53	108	53	49	57	64	64	64	61
# significant electrodes	21	16	12	7	27	32	61	58	12
# M1 electrodes	5	3	0	0	5	4	31	24	1
# S1 electrodes	5	1	3	3	5	4	13	22	4
# CS electrodes	0	2	1	2	3	5	10	12	2

Figure 2. ECoG grid localization.

Brain surface renderings with projected electrode locations of the 9 individual subjects. For each subject, all electrodes displayed were included in re-referencing. Electrodes displayed in blue, orange, yellow or white colors showed a significant response to the task. Depending on their location the electrodes were grouped in M1 (blue circles), S1 (orange circles), central sulcus (CS, white circles) or remaining cortical regions (yellow circles). Electrodes with no significant response to the task are indicated with small black circles.

2.4. Electrode selection

Electrode selection was based on the individual response to task 1. Electrodes with a significant response to task 1 were selected per run with a paired t-test (p < 0.05, Bonferroni corrected) between rest and active HFB mean values (rest period -2 to -1 s and active period -1 to 0 s, both relative to the peak of force; see also Figure 3). When a subject performed more than one run, the union of the significant channels of individual runs was used. Significant electrodes were grouped into four regions-of-interest (ROIs), depending on their cortical region (Figure 2, Table 3): all electrodes (in- and out-side the SMC), electrodes over M1, electrodes over central sulcus and electrodes over S1. For this purpose, the electrodes were localized by co-registering a pre-operative T1-weighted anatomical magnetic resonance imaging (MRI) scan (Philips 3T Achieva, Best, the Netherlands) with a post-implantation computerized tomography (CT) scan (Philips Tomoscan SR7000, Best, the Netherlands). The electrode locations were corrected for brain shift and projected on a cortex surface rendering (Branco et al., 2018; Hermes et al., 2010). In total, four averaged signals were used in the following analysis, each representing the mean HFB signal across one of the four ROIs.

Figure 3. Average HFB response during task 1 for a representative subject.

The mean HFB response (gray lines) across all trials for subject S7 during task 1. The mean signals are normalized between 0 and 1 and displayed in a time window of -1.5 to 1.5 s relative to the peak of pressure. The shaded gray region represents the standard deviation. The averaged force magnitude profile across trials (squeezes and releases) is indicated with the blue lines, also normalized between 0 and 1, for comparison purposes. For most channels, an HFB peak before the peak of the force (indicated with vertical black line) and a smaller HFB peak after the force peak are visible. Each electrode was colored according to its cortical location, with blue representing M1, white the central sulcus, orange S1 and yellow outside of the SMC, and their position on the cortical surface is displayed above. Electrodes indicated by a black dot showed no significant response to the task.

2.5. Model comparison

In this study we modelled the HFB power response using two different models based on the force applied during the three grasp force tasks. We compared one model based on the magnitude of force (F(t)) exerted during the task (magnitude, model 1, m₁(t) = F(t)) with another model based on the change in force (derivative of magnitude, model 2, m₂(t)). To accommodate negative derivative values for m₂(t), given that HFB power does not have negative values, the model combines the positive values of the derivative (P(t)) with the absolute value of the negative values of the derivative (N(t)) using the following linear combination:

$${\text{m}}_2\left(t\right)=a\cdot \text{P}\left(t\right)+b\cdot \left|\text{N}\left(t\right)\right|,0.1\le b\le 1\text{and}a\ge b$$ (1)

$$\text{P}\left(t\right)=\{\begin{array}{ll}{F}^{\prime }\left(t\right),\hfill & F^{\prime }\left(t\right)\ge 0\hfill \\ 0,\hfill & F^{\prime }\left(t\right)<0\hfill \end{array},$$ (2)

$$\text{N}\left(t\right)=\{\begin{array}{ll}{F}^{\prime }\left(t\right),\hfill & F^{\prime }\left(t\right)\le 0\hfill \\ 0,\hfill & F^{\prime }\left(t\right)>0\hfill \end{array},$$ (3)

where F’(t) denotes the derivative of the magnitude of force F’(t). We considered a positive second peak when b was greater than or equal to 0.1 and, as we do not expect the second peak to be larger than the first peak (Hermes et al. (2012), Figure 3 and Supplementary Figure 1), we defined a to always be greater than or equal to b. We modelled four mean HFB power responses (electrodes averaged within each of the ROIs, see section 2.4) by means of a convolution, where each estimated HFB signal, eHFB(t), was obtained by convolving the model m_i(t) (i = 1 for model 1 and i = 2 for model 2) with a Gaussian function g(t) (Salari et al., 2018) with a specific full-width-half-maximum (FWHM, in seconds):

$${\text{eHFB}}_{\text{i}}\left(t\right)=m_i\left(t\right)\ast g\left(t\right),$$ (4)

where the symbol ‘∗’ denotes the convolution operation. The estimated HFB power was computed per subject, per task and per ROI. Since there were three tasks, we could compare the fit of each of the force models in the presence and absence of movement (i.e., isometric force contraction). The difference between the estimated HFB power (eHFB) and the measured HFB power was determined by the root-mean-squared-error (RMSE). Before computing the RMSE, both the HFB and the eHFB signals were z-scored over the whole run to avoid effects of amplitude differences. Moreover, as suggested in previous studies (Coon and Schalk, 2016; Branco et al., 2017; Ramsey et al., 2017; Salari et al., 2018), the onset of the motor response may vary in time, especially for electrodes located in different regions of the brain. To remove potential misalignment between the model and the neural responses, we computed the RMSE while shifting the eHFB signal from -0.5 to 0 s in steps of 0.02 s. The shift (τ) with the lowest RMSE score was taken as the model that best fitted the data:

$$\arg \underset{\tau }{\min }\Vert {\text{eHFB}}_{\text{i}}\left(t+\tau \right)-\text{HFB}\Vert ,\tau \in \left[-0.5,0\right]$$ (5)

where ‖ . ‖ denotes the RMSE. In order to statistically compare the fitting of both models across ROIs and tasks, we computed the difference between the RMSE scores of the two models and performed a two-way repeated-measures ANOVA with ROI and tasks as within-subject factors.

2.6. Parameter estimation

In this study we considered the HFB response to task 1 to be representative of the subject’s cortical impulse response and assumed that the response to the remaining tasks could be constructed from these impulse responses. For that reason, and to avoid circularity effects in our results, we estimated the model parameters from task 1 and applied them to task 2 and 3. We optimized in total 4 parameters (FWHM of model 1, FWHM of model 2, a and b) for task 1. For each model, we optimized all (applicable) parameters simultaneously by computing the RMSE between the eHFB and the measured HFB for every shift τ (see previous section, Eq. (5)).

For both models, the width of the Gaussian (FWHM) was estimated for every subject, ROI and run, by determining the lowest RMSE for FWHM values ranging from 0.001 s to 1 s (steps of 0.02 s). When a subject performed more than one run, the FWHM that resulted in the lowest RMSE score was taken for that subject and used in his/her remaining tasks and runs. Additionally, for model 2, besides the FWHM, also the parameters a and b were estimated by computing the RMSE for the range of values defined in Eq. (1) and in steps of 0.05. In this study we were not interested in the absolute height of the peaks but the relation between them. Therefore, we estimated the ratio between b and a (b/a) and interpreted the height of the second peak as a function of the height of first peak. This way, for a = 1 and a ratio of, for example, b/a = 0.1, b would be 0.1, which is 10% the height of the first peak. Since there was no fundamental reason why the ratio b:a should be subject- or region-specific, we estimated these values based on a group analysis and used the estimated value for all subjects, ROIs (All, M1, CS and S1) and tasks. A one-way repeated-measures ANOVA was used to estimate the statistical difference between the ratios b/a between the regions-of-interest All, M1, CS and S1. In sum, a and b were estimated through the following multi-step approach: 1) a and b were estimated independently for task 1 per each subject and ROI; 2) the ratio b/a was computed across subjects per ROI; 3) the mean ratio b/a of all ROIs combined was calculated; 4) the mean b/a ratio estimated in task 1 was used in model 2 for all tasks, subjects and ROIs.

2.7. Anatomical specificity

In order to provide additional spatial information about the RMSE scores during grasp force tasks, we re-computed the RMSE scores per model for every electrode over the SMC (M1, CS and S1) for the two subjects implanted with HD-grids (S7 and S8). We performed this analysis only for task 3, as this provided the largest difference between model 1 and model 2 and included moments of isometric force. In this analysis the model parameters were the same as estimated in section 2.6. For each electrode the FWHM of the Gaussian was set to the one computed for the ROI it belonged too (either M1, CS or S1). In order to evaluate which model provided the best fit for each electrode separately, we subtracted both the RMSE scores and their statistical significance between the two models. To assess the statistical significance of the RMSE scores, we randomly shifted the HFB trace in relation to the eHFB 5000 times in order to obtain an empirical null distribution of correlation values on random observation (Combrisson and Jerbi, 2015). RMSE scores with a p-value below 0.05 were deemed significant.

3. Results

3.1. Parameter estimation

The Gaussian width (FWHM) that resulted in the lowest RMSE value across runs of task 1 for each subject, was estimated for both models 1 and 2 and for every ROI separately (Table 4). For each subject, the optimal FWHM value per ROI and model were kept constant for task 2 and 3. Notably, the FWHM for model 1 was on average smaller than the FWHM of the force profile (~0.5 s, see also Figure 3), whereas the mean FWHM for model 2 was in the same order of the width as the derivative peaks (~0.25 s). This means that, after the convolution between the Gaussian function (g(t)) and model 1 (m₁(t)), the eHFB kept on average the FWHM of the original force profile m₁(t). This result indicates that the FWHM parameter contributed less to shape model 1, as it was smaller than the actual width of the model, than to shape model 2.

Table 4. Mean FWHM across subjects for model 1 and 2.

FWHM was extracted from task 1 and separately for the four ROIs: All – mean HFB power across significant channels; M1 – mean HFB power across significant channels over M1; CS – mean HFB power across significant channels over the central sulcus; S1 – mean HFB power across significant channels over S1; std – standard deviation over subjects.

	FWHM model 1 (mean ± std, in s)	FWHM model 2 (mean ± std, in s)
All	0.17 ± 0.18	0.26 ± 0.06
M1	0.23 ± 0.23	0.26 ± 0.12
CS	0.12 ± 0.17	0.23 ± 0.07
S1	0.15 ± 0.22	0.23 ± 0.07

For model 2, two additional parameters were estimated, a and b (see Eq. (1) in section 2.5). Assuming that the ratio between the positive and negative derivative peaks is a property of the brain response and, therefore, invariant across subjects, these two parameters were estimated based on the b/a ratios of all subjects together (Figure 4). The a and b values displayed in Figure 4B were the ones that resulted in the smallest RMSE value per subject, run and ROI. The best fitting values for a and b showed a consistent negative slope, where a was on average 0.8 and b on average -0.2. Since we were interested in estimating the ratio b/a and since no statistical difference was found between the ratios b/a for the regions-of-interest M1, CS, S1 (one-way repeated-measures ANOVA, F(2,16) = 1.64, p = 0.23, Figure 4A), we estimated the parameters a and b from the mean ratio of all regions-of-interest (All, M1, CS and S1) combined (Figure 4A, all ratios; b/a=0.3). Hence, we set a = 1 and b = -0.3, which indicates that the second peak (absolute value of the negative derivative peak) is 70% smaller than the first peak (positive derivative peak). These parameters were, hereafter, set constant for all subjects, ROIs and tasks.

Figure 4. Estimation of parameters a and b for tasks 2 and 3.

A) Boxplots of the ratios b/a for four ROIs separately (All, M1, CS and S1) and all ratios combined (All ratios, black box). The central horizontal line indicates the median and the white star the mean (over subjects, runs and ROIs). The box indicates 50% of the distribution; the thin dashed black lines indicate the maximum and minimum ratios not considered outliers, and the outliers are indicated by a '+' symbol. All – mean HFB signal across all significant electrodes; M1 – mean HFB signal across all significant electrodes over M1; CS – mean HFB signal across all significant electrodes over the central sulcus; S1 – mean HFB signal across all significant electrodes over S1. B) Individual a and b values (gray circles) for every ROI, subject and run. Mean a and b values are indicated with a star and the median with a circle. There is a consistent negative slope between a and b, with 50% of the a values between 0.65 and 0.95 and 50% of the b values between -0.10 and -0.25.

3.2. Model comparison

Using the parameters estimated above for task 1, the two models were, subsequently, fit to the HFB response for the three tasks. Results (Figure 5, left panel, and Figure 6) show a significant difference between models for all ROIs and for all three tasks (Wilcoxon signed-rank test t-test, p < 0.05, false discovery rate corrected). The significant difference between the models can be explained by the fact that the HFB power often showed a second smaller peak after the first peak, which was more pronounced in task 3, but also visible in task 1 and 2 (Figure 5, right panel), and which was more accurately modelled by model 2. Altogether, these results indicate that 1) the HFB power does not follow the magnitude of force exerted by the individual, but rather the change in force (onset-offset of the force exertion); and 2) this difference is more pronounced during isometric force tasks (task 3), where the HFB power failed to exhibit a sustained response related to constant force.

Figure 5. Comparison between models.

Left panel: two models (model 1, black solid line, and model 2, black dashed line) were fit to the HFB. The RMSE (fit) scores were computed per subject and task, for each of the models. The tasks with significant different scores between model 1 and model 2 (at p < 0.05) are indicated with an asterisk. Right panel: representative traces of the HFB power averaged across all significant electrodes, exerted force trace, eHFB computed using model 1 and eHFB computer using model 2, for three representative runs (top: S7; middle: S8; bottom: S7), are displayed.

Figure 6. Comparison between models for three regions of the SMC.

Two models (model 1, black solid line, and model 2, black dashed line) were fit to the HFB power per ROI. The RMSE (fit) scores were computed per subject, ROI and task, for each of the models. The ROIs with significant different scores between model 1 and model 2 (at p < 0.05) are indicated with an asterisk.

From the four ROIs, only three regions (M1, CS, and S1) are independent, in that each electrode is only included in one of the regions. In order to statistically compare the differences in RMSE scores between the two models across ROIs and tasks, we computed the difference between the RMSE scores of the two models (Figure 7) and performed a two-way repeated-measures ANOVA with ROI (M1, CS and S1) and tasks (task 1, 2 and 3) as within-subject factors. The results showed that the subtracted RMSE scores were significantly different between tasks (F(2,10) = 17.33, p < 0.01) and between ROIs (F(2,10) = 6.49, p < 0.05), and a significant interaction between them (F(4,20) = 4.21, p < 0.05). Additionally, contrasts on the ROI main effect showed that M1 was significantly different from S1 (F(1,5) = 8.92, p < 0.05, r = 0.80), indicating that the onset-offset effect is more pronounced in the S1 cortex, but still visible in M1 and above the central sulcus. Contrasts on the task main effect showed the largest significant difference between task 3 and the other two tasks (F(1,5) = 18.31, p < 0.01, r = 0.87).

Figure 7. RMSE difference between model 1 and model 2.

The difference in RMSE scores between model 1 and model were computed for every task and for every ROI. Each bar represents the mean RMSE differences across subjects, and the error bar indicates the standard error. There were in total four ROIs: all significant electrodes to the task (All, black bar), all significant electrodes to the task over M1 (M1, dark gray bar), all significant electrodes to the task over the central sulcus (CS, light gray bar) and all significant electrodes to the task over S1 (S1, white bar). A two-way repeated-measures ANOVA was performed to assess the effect of the task and the ROI on the model performance. Comparison was made for three ROIs only: M1, CS and S1. Statistical differences are indicated with a * for p < 0.05 and ** for p < 0.01.

3.3. Anatomical specificity

The RMSE scores were further computed per model for every electrode over the SMC in the HD-grid patients (S7 and S8). Results (Figure 8A) showed that model 2 fits the HFB signals of task 3 better in electrodes located close to the central sulcus (in both M1 and S1) than electrodes further from the central sulcus. No clear difference between M1 and S1 was visible, indicating that both cortices seem to respond similar to the grasp force tasks (as can be also seen in Figure 3). In order to evaluate if any electrode had a preference for model 1, we compared the RMSE scores of both models per electrode. Results show that none of the electrodes preferred model 1 over model 2 (Figure 8B) and that the significance level for model 2 was always higher (p-value lower) than for model 1, with the biggest differences observable for electrodes close to the central sulcus (Figure 8B). Of note, as can be seen in Figure 8, the M1 region comprised more electrodes than the S1 region, and, therefore displayed more electrodes that were less well fitted by the models than S1. Additionally, in order to assess how the model fits the HFB signals over the SMC, we compared the values of the optimal temporal shifting between the HFB trace and the eHFB trace for the best fitting model (model 2) in the regions M1, CS and S1. Results (Figure 9) showed that electrodes over S1 on average responded later (-160 for S7 and -90 ms for S8) than electrodes over M1 (-130 for S7 and -40 ms for S8), with CS electrodes responding between S1 and M1 electrodes. Together, the above findings indicate that M1 and S1 electrodes present similar onset-offset responses but shifted in time.

Figure 8. Spatial mapping of the fitting scores for task 3.

A) RMSE scores for model 2 per electrode were displayed on the brain surface of subject S7 and S8, for task 3 and per run. The RMSE scores were normalized per run between 0 and 1, where 0 (dark blue on the color scale) represents the lowest RMSE and the best fit, and 1 (yellow on the color scale) the opposite. B) RMSE score differences between model 1 and model 2, where a positive difference indicates RMSE model 1 is larger than RMSE of model 2 and therefor model 2 fits better. Differences in RMSE are color-coded between -0.2 (dark blue) and 0.2 (dark red), with all values > 0.2 displayed in dark red. All panels: the electrodes considered above S1 are enclosed by an orange line, while electrode above M1 are enclosed by blue line. The central sulcus electrodes are the ones between the S1 and M1 regions. Black dot s indicated electrodes excluded from the analysis.

Figure 9. Temporal shift between model 2 and HFB signal for task 3.

Model 2 was fit per electrode for a range of temporal shifts from -0.5 to 0 s in steps of 0.02 s, where 0 is the movement onset. The shift that yielded the lowest RMSE between model 2 and HFB signal is displayed for task 3 and for two subjects implanted with HD-grids (S7 and S8). Subject S7 performed 2 runs, whereas S8 only performed one run. The difference in shifts indicate that the best fit in S1 channels was on average later than in M1. The boxplot indicates 50% of the distribution of the shifts of all electrodes, with the horizontal black line indicating the median and the vertical black lines indicating the maximum and minimum ratios not considered outliers. The outliers are indicated by the '+' symbol. Each boxplot is color-coded according to three cortical ROIs: M1 – mean HFB signal across all significant electrodes over M1 (dark gray box); CS – mean HFB signal across all significant electrodes over the central sulcus (light gray box); S1 – mean HFB signal across all significant electrodes over S1 (white box).

4. Discussion

Correct understanding of how movements are generated and what movement parameters are represented in the SMC are crucial to improve the decoding of motor-related brain signals, for instance for Brain-Computer Interfaces. Here, we investigated whether the shape of the HFB responses to continuous grasp force evidences a transient or a sustained profile and how these responses compare between regions of the SMC. For that, we modelled the HFB power changes during three grasp force tasks that employed an impulse-response force profile (task 1), a dynamic force profile (task 2) and an isometric force profile (task 3), with a model based on the force magnitude and a model based on force changes (derivative of force).

The results showed that during grasp force movements, the HFB power shows a transient response during flexion of the fingers (contraction) followed by a 70% smaller transient response during extension of the fingers (release). This transient response was represented in the model based on the derivative of the force (onset-offset) and absent in the model based on the force magnitude. Consequently, the model based on the derivative of the force fitted the HFB signals better than a model based on the force magnitude, irrespective of the task and the electrode location. We found that, on average, the onset-offset model fitted S1 signals significantly better than M1 signals. As we modeled averaged signals across electrodes over M1 and S1, one could suggest that differences found between M1 and S1 could be due to the smaller number of M1 channels that showed a second peak (and a preference for model 2), compared with S1. Furthermore, when fitting the onset-offset model to each channel separately, we found that it had a preference for electrodes closer to the central sulcus (see Figure 3 and 8 for examples), and that only the shift between the model and the HFB signals (relative to movement onset) differed between M1 and S1: S1 channels displayed a larger shift in time between the HFB signal and the model than M1 channels, as also reported in other studies (e.g., Hermes et al., 2012).

Overall, the lowest RMSE scores were obtained for task 1. This was expected, since this was the task used to construct the model and to extract the optimal model parameters. Although a significant difference between the models was found for multiple sensorimotor brain regions for this task, the largest differences were observed for task 3. This difference is likely due to the task design, which includes moments of static isometric force contraction, during which the HFB signals did not exhibit a sustained response.

4.1. Interpreting the onset-offset model

Evidence for the representation of force in the SMC has been reported in the past in non-human primate single-cell studies during precision grip and grasp tasks (Smith et al., 1975; Wannier et al., 1991; Boudreau and Smith, 2001; Hendrix et al., 2009). These studies showed that most cells relate to the derivative of force (phasic response) and generally discharge prior to muscle activity onset (Smith et al., 1975). During the period of static isometric force (maintained force), some cells increased discharge proportionally to the applied force and around or after muscle activity onset. This relationship between cell activity and static force levels was described as monotonic for some levels of force but not to the forces on the extremes of the range, yielding an so-called “S-shaped” curve (Cheney and Fetz, 1980; Evarts et al., 1983). Other cells presented a mix of phasic and tonic discharge responses during the dynamic (increase in force) and static (constant force) conditions (Smith et al., 1975; Cheney and Fetz, 1980).

The above studies suggest that both constant force levels and onset-offset of movement are encoded to some extent in the non-human primate sensorimotor cortex. Evidence from previous human ECoG studies showed that HFB power contributes substantially to the reconstruction of kinetics parameters, such as isometric grip force and EMG activity, when added as a feature to sophisticated regression methods (Chen et al., 2014; Flint et al., 2014; Shin et al., 2012), but that it provides limited information for the decoding of force levels associated with different object weights (Pistohl et al., 2012). Adding to these findings, we suggest that the HFB signals themselves likely provide information especially about the change in force, as our results showed that HFB power does not seem to encode moments of sustained static isometric force contractions, but rather onset and offset transitions of the movement. This result is illustrated by the presence of a two-peak response associated with flexion and extension of the fingers across the SMC. In order to infer whether the HFB power signals specifically relate to the derivative of the force profile and depend on the timing and duration of the behavioral response, we have tested an alternative model to model 1, which convolved two Gaussians (the heights of which followed the ratio we have found in this study, a=1, b=0.3) separated by a distance and in which the FWHM and distance were the fitting parameters. This alternative model has more fitting parameters than model 1, which also addresses any bias that may have occurred due to a difference in the number of fitting parameters between model 1 and 2. Results showed no statistical difference between the mean RMSE differences model 1 - model 2 (Figure 7) and the mean RMSE differences alternative model - model 2, across tasks or ROIs. Because of the fixed distance between Gaussians across trials, the alternative model converged to a similar shape as model 1 by merging both peaks into one. The difference between the alternative model and model 2 were best appreciated in task 3, where the sustained force moments are still present in the alternative model. The worse performance of model 3 compared to model 2 confirms that the two peaks we found closely follow the derivative of the grasping movement. Notably, this double-peak response in both motor and sensory cortices has also been previously reported during non-force related tasks, such as flexion and extension in non-human primates (Fetz et al., 1980) and in humans (Hermes et al., 2012), and during intended and pinching movement of the fingers in humans (Wang et al., 2012; Flint et al., 2017).

An interesting question is how HFB features of the ECoG signal relate to the single-cell recordings and how sustained force levels are encoded in the ECoG signals recorded from the SMC. A possible explanation for these results could be that ECoG electrodes do not pick up the neuronal tonic responses because they record from a large-population of neurons (each presenting either tonic, phasic or mixed responses). As phasic responses were more prominent at a single-cell level than tonic responses, these would be, therefore, the most detected by HFB ECoG signals. Another possible theory would be that tonic responses are not mapped in the HFB features but may, instead, be represented in other temporal-spectral features of the ECoG signal. Two candidate features that have been previously shown to be informative for the decoding of force and movement (e.g., Pistohl et al., 2012; Flint et al., 2014; Chen et al., 2014) are the low-pass filtered ECoG signals or local motor potentials and the low-frequency band signals. Finally, it cannot be excluded that the onset-offset responses contain a representation of the force magnitude used during sustained movements, as the derivative model also comprises information about magnitude.

Altogether, the evidence points towards a transient representation of the onset and offset of the movement in the ECoG HFB signals (coupled or not with force exertion) rather than encoding moments of sustained force. Since muscle activity is known to have the same amplitude modulation as the exerted force (Lawrence and De Luca, 1983; Smith et al., 1975; van Duinen et al., 2007), our results further suggest that the HFB signals are more likely to represent kinematics of the movements, such as velocity, whereas the representation of kinetic parameters such as force or muscle magnitude is weaker. Whether or not the magnitude of sustained force is encoded within the onset-offset modulation of the HFB signals or in other signal features remains to be investigated. Further research will be necessary to understand how other movement parameters of the force grasping movement are captured by ECoG signals recorded from the SMC, and how can these be used for BCI control.

4.2. Interpreting the spatial representation

Even though transient responses were found throughout the SMC, this onset-offset representation of movement was more consistent over S1 than in M1 across subjects and tasks. Interestingly, this result is also in line with previous observations in early non-human primate studies, where transient discharge was less visible in M1 cells when compared to S1 cells (Wannier et al., 1991), and in human ECoG studies that showed a more pronounced second peak related to extension of the fingers in S1 when compared with M1 (Hermes etal., 2012). Nevertheless, we showed that even though S1 responses are better fitted by model 2, likely indicating the presence of more predominant second peak across S1 electrodes, the second peak is also clearly present in most of the M1 channels (e.g., Figure 3 and Figure 8). Additionally, significant responses to the task, presenting again a second peak, were also found outside of the SMC, especially for the subjects implanted with standard ECoG grids. These regions (see Figure 2) included motor related areas, namely the supplementary motor area, posterior parietal cortex, cingulate motor areas and the premotor cortex, which have been implicated in the visuomotor pathway (Dum and Strick, 1993; Gold and Mazurek, 2002; Hamzei et al., 2002) and to participate in the production of grasping movements (Fagg and Arbib, 1998).

4.3. Limitations

In this study we approximated the exerted force during a grasping task by the pressure variations inside a rubber bulb. Even though this approximation is valid, as the movements were constrained to holding the rubber-bulb at all times, we cannot rule out small disparities between measured pressure and force. Like many studies, we analyzed force during an executed movement task, which, inevitably, involves a superimposition of both kinetic and kinematics parameters, and, therefore, we can only speculate what the true relation between the applied force and the HFB signals are. Notably, the task used in this study was designed for other purposes and is therefore suboptimal to fully address this question. For that, the experiment should be repeated in a more systematic, sophisticated and controlled setting, where static isometric, dynamic isometric and movement conditions are compared in a larger population of subjects and with enough trial repetitions. To investigate these questions, a task with different force levels, executed for the different durations, in the present and absence of movement should be developed. This alternative design would also allow to perform a more systematic statistical regression analysis on individual electrodes (as suggested in Salari et al., 2018), which could potentially reveal a more complex combination of transient and sustained HFB responses distributed over the SMC.

Another limitation of this study is that some of the results are highly driven by two subjects with high-density grids, who had considerably more electrodes included over the SMC compared to all other subjects. For one subject (S7), there was a visible difference in the overall magnitude of the RMSE difference between run 1 and 2. This variance can be expected due to the nature of the data. The most obvious sources of variability between runs in this study are the performance to the task and the calibration values for the (air) pressure-balloon (computed per run). Additionally, neuropsychological factors such as motivation, attention, medication, fatigue, which are known to vary within and across the days in this patient population, are also known to have an influence on the task performance and the quality of the neuronal data. Lastly, besides HFB signals, other ECoG features have been shown to be informative for the decoding of force profiles, such as local motor potentials (Flint et al., 2014). It would be interesting to also study the relation between these, the HFB response and the force patterns.

5. Conclusion

In this study we show that sensorimotor high-frequency band signals elicited during a grasp force task are strongly coupled with the onset and offset of the movement. We show that for three different grasp force tasks, with impulse, dynamic and isometric force profiles, the HFB signals are closely related to the periods of actual movement, and that moments of sustained force contractions are less clearly represented. Even though in literature no consensus has been reached regarding which kinematic or kinetic parameters of grasp movement are encoded in the sensorimotor cortex, our results suggest that the HFB components of the ECoG signals encode kinematic parameters, such as velocity. This information can be taken into consideration in understanding and decoding neural signals, for instance for BCI. Moreover, these results also support the evidence that ECoG signals, and in particular HFB signals, provide rich and discriminative features for the decoding of grasping movements.