Compartmentalized dynamics within a common multiarea mesoscale manifold represent a repertoire of human hand movements

Nikhilesh Natraj; Daniel B Silversmith; Edward F Chang; Karunesh Ganguly

doi:10.1016/j.neuron.2021.10.002

. Author manuscript; available in PMC: 2023 Jan 5.

Published in final edited form as: Neuron. 2021 Oct 21;110(1):154–174.e12. doi: 10.1016/j.neuron.2021.10.002

Compartmentalized dynamics within a common multiarea mesoscale manifold represent a repertoire of human hand movements

Nikhilesh Natraj ^1,^2,^*, Daniel B Silversmith ^1,², Edward F Chang ^3,⁴, Karunesh Ganguly ^1,^2,^4,^*

PMCID: PMC9701546 NIHMSID: NIHMS1751787 PMID: 34678147

Abstract

The human hand is unique in the animal kingdom for unparalleled dexterity, from complex prehension to fine finger individuation. How does the brain represent such a diverse repertoire of movements? We evaluated mesoscale neural dynamics across the human “grasp network”, using electrocorticography and dimensionality reduction methods, for a repertoire of hand movements. Strikingly, we found that the “grasp network” represented both finger and grasping movements alike. Specifically, the manifold characterizing the multi-areal neural covariance structure was preserved during all movements across this distributed network. In contrast, latent neural dynamics within this manifold were surprisingly specific to movement-type. Aligning latent activity to kinematics further uncovered distinct submanifolds despite similarities in synergistic coupling of joints between movements. We thus find that despite preserved neural covariance at the distributed network level, mesoscale dynamics are compartmentalized into movement-specific submanifolds; this mesoscale organization may allow flexible switching between a repertoire of hand movements.

Keywords: Human, Hand, Motor control, Grasp, Finger, ECoG, Spatiotemporal, Synergy, Manifold, Dimensionality Reduction

eTOC Blurb: In brief

How does the human brain flexibly support a remarkably diverse repertoire of hand movements? Natraj et al. show that mesoscale activity, for movements ranging from grasps to finger individuation, lie within a common multi-area manifold spanning the “grasp-network”. However, latent dynamics within this manifold were movement-specific and compartmentalized into distinct behaviorally-relevant submanifolds.

Introduction

The human hand is unique in the animal kingdom for demonstrating remarkable dexterity that far exceeds non-human primates (Napier, 1960). Not only can the human hand perform complex prehensile postures, but it is also capable of individuating fingers with precision, especially the opposable thumb (Young, 2003, Faisal et al., 2010). How does the human brain flexibly switch between such a diverse repertoire of hand movements? Classic lesion and inactivation studies along with electrophysiological recordings in non-human primates have highlighted the role of a large-scale network, otherwise called the ‘grasp network’, encompassing premotor, sensorimotor, and parietal regions (Schaffelhofer and Scherberger, 2016, Davare et al., 2011, Brochier and Umiltà, 2007, Jeannerod et al., 1995, Rizzolatti and Luppino, 2001, Taira et al., 1990, Lemon, 1993, Fagg and Arbib, 1998). The nodes of the network are also believed to be compartmentalized into distinct “cortical areas” linked by reciprocal connections; compartmentalization in humans may be distinct even from non-human primates (Koch et al., 2010, Changeux et al., 2020, Grèzes et al., 2003, Lemon, 2008). While this suggests that flexible hand control depends on spatiotemporal network activity (Hattori et al., 2009, Wheaton et al., 2005, Filimon, 2010), it also raises the fundamental question of how such distributed activity patterns flexibly support a diverse repertoire of movements. To address this question, we turned to a prominent current hypothesis which postulates that latent activity restricted to low-dimensional subspaces support computations (Santhanam et al., 2009, Churchland et al., 2012, Gallego et al., 2018, Hall et al., 2014, Stavisky et al., 2019, Bouchard et al., 2013, Sadtler et al., 2014, Briggman et al., 2005, Athalye et al., 2017, Flint et al., 2020). The low-dimensional subspace is called a neural manifold (Fig. 1). Activity captured within this manifold, calculated by projecting high-dimensional data, constitute latent time-varying neural population dynamics (Gallego et al., 2017, Jazayeri and Afraz, 2017, Veuthey et al., 2020). Notably, whether and how mesoscale activity in the grasp network can be represented as a manifold is poorly understood; this is especially the case when examining a repertoire of movements. Our study aimed to determine whether and how mesoscale manifolds spanning the grasp network represent hand movements ranging from finger individuation to grasping, using mesoscale electrocorticography (ECoG, Fig. 1A) and dimensionality reduction methods.

Fig. 1. — A) Multi-area ECoG grid coverage in grasp network colored by anatomical region (MFGc – caudal medial frontal gryus along the dorsal premotor cortex, Pars Operc. – pars opercularis along the ventral premotor cortex, Post. Central – post central gyrus or primary S1, Pre Central – pre central gyrus or primary M1, Sup. Parietal – superior parietal gyrus along the posterior parietal cortex, STG – superior temporal gyrus along temporal cortex, SMG – supramarginal gyrus along inferior parietal lobe), along with a visualization of 3D kinematic positions (wrist joint, MCP – metacarpal segment, PIP – proximal interphalangeal joint, DIP – distal interphalangeal joint, End pt. – end point segment).

B) Time-varying neural activity (blue solid line) in a high-dimensional space defined by activity of individual channels. Due to neural covariance between channels over distributed regions, we can identify multi-areal low-dimensional manifolds for a repertoire of hand movements (post-cards) using PCA. Manifold axes are neural PCs (“neural modes”, red), representing channels with covarying neural activity. High-dimensional channel activity projected onto manifold (black dotted line) constitute latent dynamics.

C) Snapshots of the hand during each of the eight movements in the study.

D) First possibility on manifolds and latent dynamics (via dPCA) postulates that temporal pattens of latent dynamics within a common manifold are shared across movements. Aligning each movement’s latent activity to kinematics using CCA leads to highly overlapping submanifolds given that all movements arise from a common appendage, the hand.

E) The second possibility postulates that latent dynamics are distinct and compartmentalized. Specifically, temporally distinct movement-specific patterns of latent dynamics constitute a higher proportion of the overall neural variance. Aligning each movement’s latent activity to kinematics leads to compartmentalized submanifolds.

An important first question is whether a repertoire of hand movements ranging from finger individuation to grasping recruits a distributed network; the “grasp network” has exclusively been studied in context of grasping. It remains unclear, however, whether single finger movements might share a common neural manifold with grasping movements. Evidence for the presence of a common distributed network would permit comparisons among each movement’s “manifold”, a subspace in high-dimensional ECoG channel space that spans the network (Fig. 1B). This manifold, identified using Principal Component Analysis (PCA), represents covariance patterns (Gallego et al., 2018, Kobak et al., 2016, Athalye et al., 2017, Churchland et al., 2012, Sadtler et al., 2014, Hall et al., 2014) due to interactions in the grasp network during movement (Fig. 1B). Each axis of the manifold, a neural mode or neural PC, represent a network of covarying multi-areal activity during movement. Such manifolds can be individually identified for the repertoire of hand movements (Fig. 1B-C). Given all the movements here arise from a common kinematic basis, the hand (Todorov and Ghahramani, 2004, Santello et al., 2013, Häger-Ross and Schieber, 2000), it is possible that the manifold might be preserved across movements. The overarching hypothesis underlying our approach is that a common low-dimensional manifold, based on multi-areal neural covariance, underlies all hand movements.

We then focus on the latent neural dynamics i.e., temporal activity patterns captured in the common manifold; how might these compare across movements? One intriguing possibility is that the latent dynamics might be temporally similar across movements (Fig. 1D). This might especially be true for the human hand where there is significant temporal, neuronal and biomechanical coupling between joints during movement (Häger-Ross and Schieber, 2000, Todorov and Ghahramani, 2004, Schieber, 1995, Indovina and Sanes, 2001, Schieber, 1990, Sanes et al., 1995, Ejaz et al., 2015, Leo et al., 2016), suggesting that the output from the brain may have some degree of similarity for the different movements. This possibility is also suggested by studies in non-human primates performing either gross arm movements or a set of reach-to-grasp and wrist tasks wherein latent dynamics in primary motor areas were highly preserved across movements (Gallego et al., 2018, Kaufman et al., 2016, Churchland et al., 2012). Alternatively, another possibility is that neural dynamics might be temporally distinct for different movements (Fig. 1E). This would suggest that streams of spatiotemporal mesoscale activity are kept segregated within the grasp network. We used demixed Principal Component Analysis (dPCA, (Kobak et al., 2016)) to understand whether a significant proportion of latent dynamics were temporally common across movements or if temporal patterns of latent dynamics were kept segregated (Fig. 1D-E). Distinguishing between these two types of grasp-network representations can provide insight into how it compartmentalizes mesoscale dynamics by movement-type.

We then further assessed how aligning each movement’s neural dynamics to its joint kinematics reflects on behaviorally relevant submanifolds. Recent work suggests that aligning neural dynamics to movements can provide further insights into compartmentalization (Sani et al., 2020, Jazayeri and Afraz, 2017). If, for example, temporal patterns of latent neural dynamics were shared between movements, then aligning neural data to kinematics at behaviorally relevant time-scales would result in highly overlapping “kinematically-aligned submanifolds” within the larger neural subspace. In this scenario, the subspaces that track kinematics are largely shared between movements and might represent intracortical processing of action regardless of the specific hand movement (Fig. 1D). Alternatively, if latent neural dynamics were compartmentalized by movement-type, then aligning each movement’s latent dynamics to its kinematics would result in distinct kinematically-aligned submanifolds. Each movement’s kinematically-relevant latent neural trajectories would therefore be compartmentalized in distinct submanifolds within the common larger manifold (Fig 1E). We used Canonical Correlation Analysis (CCA (Ewerbring, 1990, Wang et al., 2020) to align each movement’s latent neural dynamics to its kinematics and identify the aligned submanifolds separately for each hand movement-type.

Strikingly, our results revealed that while there is indeed a common multi-areal manifold (based on PCA) across a repertoire of hand movements, there are also submanifolds (identified by CCA) highly specific to the kinematics of each hand movement. Such distinct submanifolds could not be accounted for by possible somatotopic differences between movements. Surprisingly, this was even the case for both finger and grasping movements, highlighting the notion that even apparent single finger individuation recruits the grasp network. Rather, our results revealed that the distinct aligned submanifolds were due to distinct temporal patterns of latent dynamics in the common manifold. Our results thus provide evidence that although the overall multiarea manifold is preserved and that the movements themselves arise from a common kinematic basis, there is remarkable compartmentalization of mesoscale dynamics into distinct submanifolds for each hand movement. Such compartmentalization may underlie the remarkable ability of humans to flexibly select and execute a specific action from a repertoire of movements that arise from the hand.

Results

Experimental design

To understand multi-area manifolds underlying hand movements, we recorded high-density ECoG signals in the grasp network from four human subjects monitored for epilepsy. We also measured continuous 3D position data of all the joints in the hand using LeapMotion. Participants in this study performed eight self-paced pantomimed movements in a block design: flexion/extension of each of the five individual fingers and three common grasp movements. The three grasps were the pinch grasp where the index and thumb come together (instructed to “pantomime picking up a small object”), the tripod grasp involving the index, middle and thumb fingers (pantomime “pick up an object like a marker”), and the “power” grasp (pantomime “pick up a large object that requires closing your whole hand”). An individual trial constituted of a complete cycle of flexion/extension (fingers) and opening/closing for the three grasps. Subjects had full control of their hand in space, with their elbow and forearm supported by a pillow. The null position of the hand between cycles of movements was when all fingers were fully outstretched. On average, participants performed 23 trials per movement [bootstrapped 95% C.I. 21 − 25 trials].

Grasp network and low frequency oscillations

ECoG signals have been characterized by multiple distinct frequencies of oscillations. For our study, we primarily focused on Low Frequency Oscillations or LFOs (Ramanathan et al., 2018, Bansal et al., 2011, Rickert et al., 2005, Agashe et al., 2015), defined in this study to be δ band activity (0.5–4 Hz). LFOs are widely evident in mesoscale ECoG recordings when decoding arm and hand kinematics (Pistohl et al., 2012, Paek et al., 2014, Ramanathan et al., 2018, Pistohl et al., 2008, Acharya et al., 2010) and are distributed over cortical motor regions (Miller et al., 2007, Kubanek et al., 2009, Ganguly et al., 2009). Indeed, we confirmed that the envelope of distributed LFOs (we imply envelope when mentioning LFOs unless we explicitly mention the oscillation profile) in the mesoscale grasp-network tracked cycles of flexion/extension of individual finger movements (Fig. S1).

While our main study objective was to establish a dynamic view (i.e., modeling spatiotemporal activity patterns), we also wondered whether a static property of LFOs (i.e. mean activity of all channels) encoded movement information. For example, prior fMRI research has shown that the distance between finger movements’ mean voxel-wise activity in local M1 circuits exhibit a ‘representational structure’ characterized by a unique set of similarity patterns between movements (Ejaz et al., 2015). However, such a representational structure in mean cortical activity has not been shown at the level of the mesoscale grasp network and in relation to both finger and grasping movements. We evaluated the pairwise distance between movements’ mean (time-averaged) LFO activity using the Mahalanobis distance, which is the static distance between the location of movements’ neural data in high-dimensional channel space i.e., distance between neural centroids, Fig. 2A. Formally, it measures the distance between multi-dimensional channel means scaled by the pooled multi-dimensional variance. An example of the mean LFO amplitudes across the network for two movements is shown in Fig. 2B. Analysis of the average pairwise Mahalanobis distance matrix (Fig. 2C) and the clustering of Mahalanobis distances between movements (Fig. 2D) revealed a representational structure was present (mean correlation of r = 0.38, p < 1 × 10⁻³ between participants’ individual distance matrices). First, the three grasp movements clustered together. Next, the index, middle, ring and pinky fingers clustered together. Notably, the thumb was overall similar to grasping actions than other finger movements (Fig. 2D). We found a similar representational structure when using a linear classifier (Support Vector Machine (Fan et al., 2008)) to discriminate time-points around the centroid of each movement’s grasp-network channel LFOs (Fig. S2A-D). Thumb movements were again closer to grasping (Fig. S2C, by 2.3%, mixed effects t(26) = 2.4, p = 0.024). The distance between movements’ mean LFO activity was significant enough to achieve high accuracies in SVM classification, either in a pairwise (Fig. S2E) or multi-class scheme (Fig. S2F).

Fig. 2. — A) We evaluated the Mahalanobis distance between the static locations of movements’ neural data in high-dimensional channel space. The mean activity, by averaging LFOs over time (left), defines the static location or centroid of channel LFOs (solid circles in the plot on right); the multi-dimensional variance determines the spread around the centroid (ellipsoid dotted lines).

B) Mean channel LFO amplitudes across the ECoG grid overlaid on the brain for a subject (EC169). Increasing electrode radii denoting higher mean LFO amplitude values, color denotes anatomical location.

C) The average pairwise Mahalanobis distance matrix between movements’ centroids.

D) Hierarchical clustering of the average distance matrix in C) revealed a representational structure.

Overall, our results showed that LFOs across the grasp network carry significant movement-related information, both in terms of tracking kinematics and in their static location in high-dimensional channel space, and therefore are a good candidate feature for characterizing grasp-network mesoscale manifolds.

The multi-areal neural manifold is preserved across movements

We then used PCA to identify the “manifold” i.e., the multi-area mesoscale covariance structure; this analysis was performed individually for each movement within each subject. Note that in all subsequent manifold analysis that involves dimensionality reduction based on a covariance structure, the mean activity of each channel was removed as it does not inform on covariations. Results revealed that the majority of variance could be captured by a low-dimensional 45D mesoscale manifold (~75% of the overall neural variance, S.D range: 71.4% to 80.52% across movements, subjects) achieving a >5-fold dimensionality reduction (Fig. 3A). Hereafter, the term “manifold” refers to the general low-dimensional subspace captured using PCA; in contrast, ‘submanifolds’, discussed in later sections, refers to when we align latent dynamics to kinematics using CCA.

Fig. 3. — A) PCA was used identify a manifold for each movement from LFOs. Average neural variance accounted for (VAF) by the PCs or neural modes is shown in the figure.

B) (Left) Similarity in the orientation between movements’ 45D manifolds was evaluated in pairwise manner using the method of principal angles; smaller angles imply highly overlapping manifolds. (Right) Principal angles between thumb manifold and the other seven movements in the repertoire is shown for an example subject. The black dotted line represents the lower bound of the null distribution of principal angles (2.5^th percentile for α= 0.05) and the average first principal angle is highlighted (8.34 degrees).

C) Average normalized histogram of number of significant principal angles between movements’ 45D manifolds (angles computed pairwise), where each circle represents a single subject.

D) Ratios of across-movement VAF to within-movement VAF in real (orange) and control data (light blue).

E) Cortical channel representation of an exemplar neural mode (1^st neural PC weights, EC189-thumb). Increasing sizes of electrode represent increasing spatial covariance.

F) Boxplots of the average channel weight magnitudes within each node of the grasp-network for all 45 neural modes (across movements and subjects). Edge of blue box correspond to 25^th and 75^th percentile of data; red horizontal line corresponds to the median and the whiskers extend to the entire data spread not considered outliers.

We evaluated whether the manifold, identified individually for each movement, was preserved across movements given that the movements themselves arise from a common kinematic appendage. To do this, we used the method of principal angles and principal directions to evaluate whether the 45D manifolds were oriented similarly (Fig. 3B). If two manifolds are oriented similar to each other in high-dimensional channel space, then there will always be some linear combinations of each manifolds’ individual neural modes that result in very small angles. These linear combinations of the neural modes are called the principal directions within each manifold and the resulting angles are called the principal angles. There are as many principal directions and principal angles as the manifold dimensionality. The principal angles between the thumb and the other seven movements’ manifolds are shown in Fig. 3B for one participant. The principal angles suggested similar orientation between manifolds, with the first principal angle between the thumb and other movement manifolds having a mean of 8.34 degrees (Fig. 3B, [95% C.I. 8.15 – 8.52 degrees]).

Under the null hypothesis, we would not expect a difference between the observed principal angles between movements from a null distribution of principal angles. We used the tensor maximum entropy method (TME, (Elsayed and Cunningham, 2017, Gallego et al., 2018), see Methods) to simulate surrogate tensors of neural data without a covariance structure (R^{channel×time×mvmt − type}) 1000 times, thereby allowing us to compute null distributions of principal angles from the surrogate data. For example, the threshold for significance of the principal angles is shown as a black dotted line in Fig. 3B (α = 0.05). It can be seen that the principal angles between the thumb and other seven movements were much lower than what could be expected by chance (Fig. 3B). There are 28 total pairwise comparisons between movements per subject given the eight movements in our study. For each such pairwise comparison, we can evaluate the number of significant principal angles between the two 45D manifolds. Results revealed that a minimum of 44 of the 45 principal angles were significantly smaller than would be expected by chance (Fig. 3C, α = 0.05, False Discovery Rate (FDR) corrected (Benjamini and Hochberg, 1995)). The manifold was therefore oriented in high-dimensional channel space similarly across movements.

The similar orientations between movement manifolds suggested that the multi-area neural covariance structure was shared across movements. To verify this, we computed the ratio of across-movement VAF to within-movement VAF (Gallego et al., 2018). The within-movement VAF is the amount of variance captured by each movement’s own 45D manifold. The across-movement VAF is the neural variance captured by projecting a movement’s data onto the 45D PCA manifold of another movement. If all movements shared variance in a common low dimensional manifold based on a similar neural covariance structure, then the ratio of the across-movement VAF to within-movement VAF would be close to one. Indeed, the average ratio of the VAF was 0.79 across subjects and movements [95% bootstrapped C.I. 0.775–0.8]. The distribution of these ratios was significantly different from control neural VAF ratios (0.294, [0.268 − 0.316]) wherein the across-movement VAF was obtained by projecting data onto random 45D manifolds (Fig. 3D, p ≤ 0.01, 2-sided Kolmogorov-Smirnov, KS test between distribution of ratios in the real data versus control). We then examined the spatial extent of the common multi-areal manifold. Each axis of the 45D manifold, the PCA neural mode, is a vector in high-dimensional channel space whose weights identify multi-areal channels with covarying neural activity. The weight magnitudes of the first neural mode during thumb movements is shown in Fig. 3E for a subject. We computed the average channel weight magnitude within each node of the grasp network for each movement’s 45D manifold. Results revealed that the modes of the 45D manifolds engaged all regions of the grasp-network equally (Fig. 3F).

Overall, our results showed that the manifold characterizing multi-areal covariance was remarkably preserved and a common grasp-network subspace represented all hand movements, from finger individuation to grasping.

Latent dynamics within the common subspace are distinct for the movement repertoire

Having identified a common mesoscale manifold, we examined the temporal patterns of latent dynamics within it using demixed Principal Component Analysis or dPCA (Kobak et al., 2016, Gallego et al., 2018). Here we use dPCA to examine both movement-specific and movement-independent latent dynamics within the common manifold. Given our experimental design, the objective function of dPCA finds two types of dPC neural modes: 1) time dPC modes that capture time-varying activity that is common to all movements, i.e. movement-independent latent activity and 2) movement dPC modes that capture time-varying activity that is distinct to movement-type, therefore identifying movement-dependent latent activity (Gallego et al., 2018). The two types of dPCA modes therefore de-mix the neural data and capture either temporally common or temporally distinct latent dynamics. The VAF by each type of dPC neural mode can be estimated in terms of a reconstruction error of the original neural data and sorted by variance (Kobak et al., 2016). The VAF of each dPC mode delineates what proportion of the overall variance of all eight movements is due to either temporally consistent neural dynamics or temporally distinct neural dynamics between movements. We performed dPCA on trial-averaged data to more reliably uncover the dPCA modes.

An example of the VAF by the first 15 time and movement-specific dPCA modes are shown in Fig. 4A for a participant. Interestingly, in contrast to findings using spiking data in local M1 circuits across a set of movements (Gallego et al., 2018, Kaufman et al., 2016), the movement-specific dPCA modes contributed to the majority of the variance in dPCA analysis, by a factor of almost ~5:1 (83.62% to 16.38% on average) in each participant over the time dPCA modes (Fig. 4A-B). Therefore, the majority of variance in the neural data was driven by the fact that each movement was characterized by its own distinct temporal pattern of latent dynamics in a common subspace. From a physiological perspective, this result implies that there are distinguishable phases or differential timing of activity within the nodes of the grasp-network between movements (see Fig. S3 for plot of mean LFO traces across the grasp network for all movements).

We then visualized both common and movement-specific latent dynamics by projecting LFOs onto a few exemplar dPCA modes for one participant (Fig. 4D). The common neural dynamics (Fig. 4D, bottom) seemed to be qualitatively highly linked to cycles of movement (Fig. 4C). For instance, activity projected onto the first time dPCA mode exhibited a peak in LFO envelope right before full flexion or grasp closing. Similarly, activity projected onto the second time dPCA mode exhibited a peak just after full flexion or grasp closing. Activity in another exemplar time dPCA mode seemed to preserve phasic cycles of LFO power in both phases of flexion/extension or opening/grasping. Conversely, projecting activity onto the first three movement-specific dPCA modes for a participant highlights the temporal differences in latent neural dynamics between movements (Fig. 4D, top). The starting location of the neural dynamics was different for each movement, as was the time-varying trajectory in the latent subspace. Our main results of temporally distinct latent dynamics within a common multi-areal subspace persisted regardless of how we epoched our neural data, i.e., epoching data either around peak flexion/closing or via normalizing the entire length of the trial.

Manifolds and dynamics of high-gamma follow low-frequency oscillations

To what extent are our results dependent on our choice of LFOs, given that the framework of neural manifolds during behavior had primarily been developed with spiking activity in M1 ensembles (Santhanam et al., 2009, Kaufman et al., 2016, Churchland et al., 2012, Gallego et al., 2018, Athalye et al., 2017)? It should be noted that prior studies in animals have identified LFO field potentials as a correlate of coordinated population-level spiking activity (Hall et al., 2014, Ramanathan et al., 2018). Although mesoscale ECoG recordings are fundamentally different signatures of neural activity compared to spiking activity, a putative correlate of underlying population spiking activity at an ECoG recording channel is best represented by high-gamma envelopes ( $γ_{h}$ , broadband activity ≥ 70 Hz (Chang, 2015)). We therefore first investigated the relationship between LFOs and $γ_{h}$ envelopes at individual channels across the grasp network during movement (Canolty et al., 2006).

To investigate the relationship between LFOs and $γ_{h}$ , we followed the following phase-based analyses. Specifically, raw ECoG signal was first filtered into two frequency bands: LFO oscillations (0.5–4Hz δ band filtered activity, Fig. 5A) and high-gamma oscillation between 70–150Hz (Fig. 5A). We then extracted the envelope of the 70–150Hz oscillation using the Hilbert transform to generate $γ_{h}$ envelopes (hereafter, the envelope is implied when mentioning $γ_{h}$ ). The $γ_{h}$ signal was further filtered within the low-frequency δ band of 0.5–4Hz, thereby extracting the low-frequency component of $γ_{h}$ (hereafter called $γ_{h}^{L F O}$ , Fig. 5A, orange). If mesoscale $γ_{h}$ were to exhibit a relationship with LFOs, then there should be coupling between the phase of LFO oscillations and the phase of $γ_{h}^{L F O}$ . We thus extracted the respective phases of LFO oscillations and $γ_{h}^{L F O}$ using the Hilbert transform and evaluated the preferred phase difference (Fig. 5A). The circular mean of the preferred phase differences between the two signals is the Phase Locking Value (PLV) (Canolty et al., 2011). The PLV is a complex number; its magnitude represents the trial-to-trial phasic consistency, and its angle represents the preferred phase relationship. Analyses of the PLV magnitudes showed that a significant phasic relationship existed (assessed by circularly shuffling the phase of LFOs); a single M1 channel shown in Fig. 5A (polar histogram). On average, ~70% of channels across our subjects exhibited such a significant relationship (Fig. 5B, α = 0.05, FDR corrected; see a single subject example of significant channels in Fig. S4). A summary figure of the preferred phase-angle between LFOs and $γ_{h}^{L F O}$ is shown in Fig. 5C. The time-course of LFOs across the network were thus closely related to low-frequency $γ_{h}$ .

Fig. 5. — A) Example of phase-locking value (PLV) between LFO oscillations and the low-frequency component of $γ_{H}$ ( $γ_{H}^{L F O}$ ) at example M1 channel.

B) The percentage of channels with significant PLV aggregated across all eight movements shown for each individual subject (open circle).

C) Summary plot across all subjects showing the circular distribution of preferred phase angles between LFO oscillations and $γ_{H}^{L F O}$ (the angle of PLV) across channels within each node of the grasp network.

D) Example of the principal angles between the thumb’s 45D $γ_{H}^{L F O}$ neural manifold and the 45D $γ_{H}^{L F O}$ manifolds of the other seven movements in the repertoire is shown for an example subject.

The black dotted line represents the lower bound of the null distribution of principal angles (2.5^th percentile for α = 0.05).

E) Average normalized histogram of the number of significant principal angles between movements’ 45D $γ_{H}^{L F O}$ manifolds (angles computed pairwise). Given the eight movements in the study across the four subjects, there were a total of 112 unique pairwise comparisons.

F) Ratios of the across-movement VAF to within-movement VAF for $γ_{H}^{L F O}$ manifolds (orange) compared to control data (light blue).

G) We used dPCA to fit a common $γ_{H}^{L F O}$ neural manifold for all eight movements and identify time and movement-specific dPCs.

H) Example of the neural VAF from the dPCA analyses on $γ_{H}^{L F O}$

I) Comparison of the neural VAF due to the two types of dPC modes across all four subjects (each filled circle represents an individual subject).

J) (Left) Shared variance between the two signals’ dPCA manifolds was computed as the proportion of variance in a signal’s dPCA manifold that is also captured by the other signal’s dPCA manifold. (Right) Shared variance between LFO and $γ_{H}^{L F O}$ dPCA manifolds shown for each subject, compared to a surrogate distribution of shared variance between either of the two signals’ dPCA manifolds with random manifolds.

Having established the relationship between LFOs and $γ_{h}^{L F O}$ we then sought to analyze the manifolds and latent dynamics of $γ_{h}^{L F O}$ . To this end, we first identified a 45D $γ_{h}^{L F O}$ manifold for each movement using PCA. Across movements and subjects, the 45D $γ_{h}^{L F O}$ manifold captured a median of 58.02% of the neural VAF [S.D. range 51% to 65.1%]. We then contrasted the identified 45D $γ_{h}^{L F O}$ manifolds between movements pairwise to verify if they were similarly oriented to each other in high-dimensional channel space, in a manner similar to earlier analyses with LFOs. Results revealed that the 45D $γ_{h}^{L F O}$ manifold was indeed preserved across movements; the principal angles between movements’ manifolds were significantly smaller than chance (single subject example in Fig. 5D) with a median of 43 out of 45 significant principal angles across all pairwise comparisons (28 pairwise comparisons in each subject for a total of 112 comparisons, Fig 5E). To verify that variance in this multi-areal $γ_{h}^{L F O}$ neural manifold was shared across movements, we computed the ratio of across-movement neural VAF to within-movement neural VAF by projecting $γ_{h}^{L F O}$ data from one condition onto the $γ_{h}^{L F O}$ manifold of another condition. Results revealed that this ratio was high (0.69 [95% bootstrapped C.I. 0.67–0.705]) and significantly greater than would be expected by projecting data on random manifolds (0.38 on average for random manifolds, p ≤ 0.01 on 2-sided KS test between distributions of ratios for real and control data, Fig. 5F). Thus, a common $γ_{h}^{L F O}$ manifold represented all hand movements.

We then used dPCA to evaluate similarities in latent neural dynamics between movements in the common 45D manifold (on trial-averaged $γ_{h}^{L F O}$ data) (Fig. 5G). Like earlier results with LFOs, there was little temporal commonality in latent $γ_{h}^{L F O}$ dynamics in the grasp-network subspace. The majority of neural variance in $γ_{h}^{L F O}$ , by a ratio of ~7:1 (87.65%:12.35% on average), was due to the movement-specific modes. A single example of the neural VAF due to either movement-specific or time modes is shown in Fig. 5H; the overall VAF due to these two modes for four subjects is shown in Fig. 5I. Therefore, like LFOs, each movement was primarily characterized by its own temporally distinct latent $γ_{h}^{L F O}$ dynamics rather than temporally shared latent activity patterns.

Given the similar manifold and latent dynamics properties of both grasp-network LFOs and $γ_{h}^{L F O}$ for the hand-movement repertoire and their significant phasic relationship, we then wondered how similar the two respective subspaces were. We thus compared the two signals’ manifolds. Specifically, we evaluated the proportion of neural variance in one manifold that was captured by the other (Fig. 5J). If both signals largely share a common subspace, then the proportion of shared variance would be significantly greater than chance (Degenhart et al., 2020). Indeed, results revealed that for each subject, the percentage of shared variance between the two signals’ dPCA manifolds (average 31.1%) was significantly greater (p ≤ 0.01, FDR corrected) than with random manifolds (on average 20.01% [18.5 − 21.23 bootstrapped C.I.], Fig. 5J). Similarly, we found that the PCA manifolds of both signals shared significant variance with each other (42.7% [38.4 − 45.2 bootstrapped C.I.], Fig. S5) as compared to sharing variance with random manifolds (21.1% [19.5 − 22.41 bootstrapped C.I.], p ≤ 0.01 on 2-sided KS test between distributions). The 45D multi-areal subspaces and low-dimensional covariance structure of both signals were therefore more similar to each other than expected by chance.

Kinematic recordings of the hand-movement repertoire

Our model of compartmentalized grasp-network activity (Fig. 1E) also makes the prediction that aligning the temporally distinct latent dynamics to kinematics (separately for each movement) will also result in distinct kinematically-aligned submanifolds. We first sought to quantify the kinematics across the repertoire. We used the LeapMotion system to measure continuous 3D kinematic position data from all joints in each of the five fingers (3D position of wrist, MCP, PIP, DIP, end-point bone in each finger), resulting in 75 kinematic dimensions that were referenced to the palm center (see Methods and see Fig. S6A-C for processed whole-hand kinematics). We also validated the LeapMotion system using a separate magnetic tracker system (Fig. S6D). Given the biomechanical and neuronal coupling in the hand, there is significant temporal covariation between joints. Such temporal covariation patterns amongst joints can be represented by elemental whole-hand control postures, called kinematic “synergies” (Santello et al., 1998, Ingram et al., 2008, Todorov and Ghahramani, 2004); these appear to be preferentially encoded in the brain over individual joints or muscles (Leo et al., 2016, Ejaz et al., 2015). Here, we used PCA individually on each movement’s and participant’s trial-concatenated kinematic dataset to identify synergies and their temporal activation functions (Methods, (Todorov and Ghahramani, 2004, Ingram et al., 2008, Flint et al., 2016, Leo et al., 2016)). The top three such synergies and their trial-averaged activations are shown in Fig. 6A-B for the tripod grasp and thumb movements respectively (see Fig. S6E for activity in synergy space).

Fig. 6. — A) First three postural kinematic synergies and temporal activations, shown for tripod grasp.

B) First three postural kinematic synergies and temporal activation, shown for thumb movement.

C) The PC weights for each of the 75 dimensions (3D position of each the 5 joints of each of the 5 fingers) of the first thumb synergy.

D) The PC weights for each of the 75 dimensions of the second thumb synergy.

E) The PC weights for each of the 75 dimensions of the third thumb synergy.

F) Number of significant synergies for each eight movement, calculated per subject (open circle), using the Marchenko-Pastur bound on the eigenvalues of kinematic covariance matrix.

G) Contribution of non-dominant fingers towards weights of first three finger movement synergies (e.g., proportion of weights for index, middle, ring and pinky during thumb movements). Star sign represents significance at the 0.05 level.

How might the synergies of finger individuation compare to grasping? If finger movements did not involve a whole-hand control strategy, then there should be a lower number of significant synergies compared to grasping; these finger synergies should also be restricted only to joints of the individuated digit. However, while the first synergy was indeed specific to the dominant digit (Fig. 6B), subsequent synergies exhibited multi-jointed covariation (Fig. 6D-E). Could these secondary synergies be a significant source of variation in the kinematic data? While examining the variance accounted for by PCs can determine the number of important synergies, we instead opted for a statistical method based on random matrix theory (the Marchenko-Pastur (MP) bound (Marchenko and Pastur, 1967)) to identify the number of ‘statistically significant’ synergies (Fig. S6A). Interestingly, there were no differences in the number of significant synergies between individual finger and grasping movements (Fig. 6F, mixed effect model, t(30) = 1.07, p = 0.29). Across movements and subjects, the median number of significant synergies was three; they were sufficient to capture ~95% of the variation. We found that a similar number of significant synergies was required to capture ~95% VAF when we analyzed kinematic data in the joint angle space (Fig. S6F-G). To further quantify whether the finger synergies involved multi-jointed covariation across the entire hand or only along the segments of the individuated finger, we examined the representation of the non-dominant fingers weights (Fig. 6G). For example, the non-dominant fingers during thumb would consist of the index, middle, ring and pinky fingers. While the first synergy had lower representation of the non-dominant fingers (43.1%, [95% C. I. 36.3% − 49.67%]), the second synergy had significantly higher representation of the non-dominant fingers (78.83% [75.11% − 82.31%], mixed effect models, t(38) = −10, p = 3.4 × 10⁻¹²). A similar result was obtained when comparing the representation of the non-dominant fingers in the third synergy (68.23% [62.7% − 73.4%], mixed effect models, t(38) = −5.88, p = 8.16 × 10⁻⁷, Fig. 6G).

Our results therefore revealed that the synergies of even ‘simple’ individuated human finger movements were characterized by whole-hand covariation patterns – see also (Kirsch et al., 2014, Schieber, 1995) for a similar finding in nonhuman primates (Fig. S6H-I). This might also explain why the neural manifold is preserved for grasping actions as well as apparent single finger movements.

Kinematically-aligned submanifolds

To identify movement-aligned submanifolds, we used Canonical Correlation Analysis (CCA (Uurtio et al., 2017, Ewerbring, 1990)) to align each movement’s latent LFO dynamics within the common 45D PCA manifold to its top three multi-jointed kinematic synergies (Fig. 7A). CCA discovers three pairs of neural and kinematic modes such that the time-dependent activation of any pair of neural and kinematic CCA modes, i.e., the neural and kinematic trajectories, are maximally correlated while being orthogonal to other CCA mode-pairs. As we built the CCA individually for each movement, this orthogonality constraint applies only to CCA mode-pairs within a movement and not across movements. The CCA neural modes constitute the 3D aligned submanifold within the larger PCA neural manifold. Hereafter, “submanifolds” refers to the CCA kinematically aligned subspace. Moreover, when we mention neural trajectories, we refer to the behaviorally-relevant neural trajectories as a result of the CCA alignment. Cross-validation analyses showed that the CCA modes were significantly predictive for held out neural and kinematic data (Fig. 7B, p ≤ 0.05, FDR corrected). The mean cross validated correlation between the dominant CCA neural and kinematic trajectories across participant and movements was 0.54 (95% C. I [0.49 − 0.583]) and the value across all the second and third pairs of CCA neural and kinematic trajectories across participants and movements was 0.41 (95% C. I [0.39 − 0.433]). Notably, CCA kinematic trajectories were more strongly correlated with mixtures of 489 synergies for finger movements, whereas they were more correlated with individual synergy 490 activity during grasping movements (Fig. S7).

To understand whether the aligned CCA neural submanifolds were distinct for movement-type (Fig. 7C), we used the method of principal angles. The principal angles between the thumb CCA submanifold and the other movements’ CCA submanifolds is shown in Fig. 7C. It can be seen that the principal angles between the submanifolds was quite large, with the first principal angle between submanifolds having a mean of 67.6 degrees [95% C.I. 66.5 – 68.8] across all pairwise comparisons between movements. It should be noted that although these angles are large, there are not completely orthogonal due to some degree of overlap given the statistics of the neural data. However, these principal angles between CCA submanifolds in the real-data were not significantly smaller than the null distribution (at the α = 0.05 level, Fig. 7D, using TME), quantifying that CCA aligned submanifolds were not more similar than expected by chance. We found a similar result with the submanifolds of $γ_{h}^{L F O}$ (Fig. S8A-B).

To illustrate that aligned CCA submanifolds compartmentalize each movement’s behaviorally relevant neural dynamics, we cross-projected each movement’s latent dynamics within the larger common manifold onto the CCA aligned submanifolds of other movements. As shown in Fig. 7E, LFO latent dynamics of the middle finger was projected onto its own CCA aligned submanifold (preserved CCA neural modes); this is compared to the projection of latent dynamics of another movement (pinch grasp) onto the middle finger CCA submanifold (swap CCA neural modes). As illustrated, cross-projection of the other seven movement’s latent dynamics in the 45D manifold onto the middle submanifold degraded the temporal structure of the movements’ kinematically relevant neural trajectories. Given that the objective function of CCA is to find temporally correlated neural and kinematic trajectories for each movement, we evaluated the effect of neural submanifold swapping on the correlation between neural and kinematic trajectories. We found that projecting any movement’s dynamics onto another movement’s 3D CCA aligned submanifold resulted in lower correlation with its three CCA kinematic trajectories as compared to projecting neural data onto its own submanifold (Fig 8F, 2-sided KS test between distributions of correlations from original neural modes vs. swapped neural modes, p ≤ 0.05, FDR corrected). To ensure the robustness of the results concerning the distinctiveness of submanifolds, we also implemented a robust version of CCA and projected the submanifolds onto the column spaces of each other rather than just swap them (see Methods); our main findings remained unchanged.

Fig. 8. — A) Dropping LFO channels with highest weights within first CCA neural mode significantly decreases correlation with synergies. Scale of y-axis is normalized to the maximum squared correlation when using all channels.

B) The mean effect of dropping LFO channels based on their weight magnitude in the first neural mode for all subjects and movements (with 95% bootstrapped C.I. shading).

C) Example of significant channels across all CCA neural modes for thumb movements.

D) Projection of significant channels onto the cortical surface created LFO cortical channel density maps shown for two movements. Color scale represents the smoothed density of channels on the cortical surface.

E) The average pairwise cosine distance between movements’ LFO cortical channel density maps.

F) Hierarchical clustering of average distance matrix revealed no significant separation between cortical channel density maps.

G) Projection of significant channels across subjects onto an average MNI brain template separately for all finger and grasp movements. Boundaries of Brodmann anatomical regions are color coded.

H) Pairwise comparison of the proportion of significant channels within each anatomical region across subjects revealed no differences between finger and grasping movements.

To verify that the distinctiveness of CCA submanifolds were not an artifact of their dimensionality, we aligned latent neural dynamics to just the first synergy, at which point CCA reduces to multiple-regression. The time-course of the first synergy activation was also highly correlated across movements as it captures the dominant cycle of flexion/extension in the case of fingers or opening/closing in the case of grasping (mean correlation between movements for the 1^st synergy was r = 0.83 [95% C. I 0.81 − 0.84] in trial concatenated data). However, even in this highly constrained scenario where a common kinematic output was being regressed onto neural data, the 1D submanifold was highly specific to movement-type (Fig. 7G-H). Swapping the 1D submanifold between movements caused a significant loss in kinematic correlation with the first synergy (example for middle finger movement shown in Fig. 7G and summary statistics across all movements and subjects shown in Fig. 7H, 2-sided KS test, p ≤ 0.01). Overall, our CCA results confirmed that kinematically relevant neural trajectories were compartmentalized by movement-type into distinct submanifolds within the common multi-areal mesoscale manifold.

The aligned submanifolds span the grasp network equally

The finding that distinct CCA aligned submanifolds lie within a common multi-areal subspace is suggestive of a distributed representation for hand motor control across the grasp network. Could such a distributed architecture of the aligned CCA submanifolds persist if we were to align high-dimensional LFOs to synergies directly without the intermediate PCA step? Such a finding would confirm that computations in submanifolds spanning a distributed cortical network represent the kinematics of the movement repertoire. We thus applied CCA to align high-dimensional channel level LFOs to the top three synergies directly without the pre-processing step of first projecting channel LFOs onto the common neural manifold. Each of the three resultant CCA neural modes are now weighted linear combination of channel LFOs, rather than weighted linear combinations of the neural modes (PCs).

To identify a cortical network of ‘significant’ channels, we first established the importance of each channel’s weight magnitude within each CCA neural mode; results revealed that dropping channels with the highest weights in a CCA neural mode caused greater loss in correlation than dropping channels with the lowest weights (Fig. 8A-B, Fig. S9). To generate a surrogate distribution of channel weights within each CCA neural mode, we simulated each channel’s LFOs 2000 times prior to running CCA. We then identified significant channels within each CCA neural mode if a channel’s weight magnitude exceeded the null distribution at the α = 0.01 level (FDR corrected for multiple comparisons). Significant channels identified within each CCA neural mode were isolated and then pooled together to identify a binary cortical map of each movement’s aligned submanifold. On average, 30.8% (95% C. I [28% − 34.13%]) of channels were significant at the α = 0.01 level (FDR corrected). An example of such a cortical network of channels during thumb movement is shown for one participant in Fig. 8C. The distributed nature of the network of channels appeared to be consistent across the various hand movements (Fig. S10). To statistically evaluate the similarity between each movement’s cortical network, we first projected the significant channels onto the 3D cortical surface (Hamilton et al., 2017) using a Gaussian function with a cortical spread of 1cm, creating LFO cortical channel density maps (Fig. 8D). We then evaluated the similarities between movements’ LFO cortical channel density maps using the pairwise cosine distance metric. The average pairwise similarity matrix, depicted in Fig. 8E, suggested that there was no separation between movements’ density maps, and that the cortical maps were not distinct (hierarchical clustering, Fig. 8F, p = 0.963).

Our results suggested that the spatial extent of each kinematically aligned manifolds spanned a distributed network, even when directly aligning high-dimensional channel LFOs to synergies. To visualize this network, we projected all significant channels from all participants onto an average brain template (Hamilton et al., 2017), separately for grasping and finger movements (Fig. 8G). The network for both finger and grasping movements spanned the grasp network equally, even though one might have expected the kinematics of finger movements to be driven more by primary sensorimotor areas (Indovina and Sanes, 2001, Sanes et al., 1995). Indeed, there were no statistical differences between finger and grasping movements (Fig. 8H, mixed effect models, 0.05 level, multiple comparison corrected, S1: t(30) = −1.04, p = 0.30, M1: t(30) = −1.75, p = 0.09, parietal cortex: t(30) = −1.61, p = 0.11, supramarginal gyrus: t(30) = −1.16, p = 0.25, ventral premotor cortex: t(30) = −2.5, p = 0.018 and dorsal premotor cortex: t(30) = −1.43, p = 0.163). Importantly, we also found that the distributed cortical structure of the aligned manifold persisted even when finger movements were executed in a non- rhythmic, cue-based manner (Fig. S11).

Discussion

Our study aimed to understand how multi-areal manifolds and latent dynamics in the “grasp network” represent a repertoire of hand movements. Our results showed that although the manifold based on mesoscale covariance is preserved across a repertoire of hand movements, latent dynamics within this common multi-areal subspace are compartmentalized into distinct submanifolds spanning the entre grasp network instead of relying on shared low-dimensional patterns. Notably, these manifold results held true for both LFOs and the low-frequency patterns of $γ_{H}$ , with unique phasic relationships between the two signals. Such distinct compartmentalized dynamics within a common multi-areal subspace could correspond to the visuomotor transformations necessary to plan, select and execute the appropriate synergistic gestures for hand movements sharing a common kinematic basis (Schaffelhofer and Scherberger, 2016, Michaels et al., 2020). This mesoscale level finding stands in contrast to population spiking activity in local M1 circuits (Kaufman et al., 2016, Gallego et al., 2018). In addition to grasp-network dynamics, analyses of the static property of the grasp network revealed a representational structure characterized by a unique set of similarity patterns in the mean activity.

A common low-dimensional mesoscale manifold for grasping and finger movements

Understanding the neural control of the hand has been of exceeding interest in motor control research, from classical stimulation and electrophysiology work in non-human primates (Muir and Lemon, 1983, Buys et al., 1986, Lemon et al., 1987, Lemon, 1988, Schieber and Hibbard, 1993) to neuroimaging in humans (Sanes et al., 1995, Indovina and Sanes, 2001, Colebatch et al., 1991). However, the neural computations of how latent multi-area dynamics in the grasp network, represent a repertoire of hand movements was unclear. Dimensionality reduction methods and multivariate methods in general provide excellent means of studying latent dynamics and neural manifolds of a neuronal population and have been widely used in neuroscience (Humphrey et al., 1970, Briggman et al., 2005, Gallego et al., 2017, Jazayeri and Afraz, 2017). Moreover, multivariate brain-behavior correlational methods such as CCA might better uncover behaviorally relevant dynamics rather than characterizing the neural manifold alone (Sani et al., 2020, Jazayeri and Afraz, 2017). CCA specifically uncovers simultaneous neural and kinematic transformations when identifying maximally correlated neural and kinematic trajectories; these appear to be significantly more revealing than one-to-one or many-to-one regression-based methods.

An interesting finding in our study was the fact that the neural manifolds of both finger and grasping spanned the “grasp network”. In contrast, past work had dichotomized the neural control of individual fingers as local to primary motor cortex (Beisteiner et al., 2001, Indovina and Sanes, 2001, Sanes et al., 1995, Schieber and Hibbard, 1993), while viewing grasping actions to be controlled by a distributed network. A potential reason for this view is the general belief that unlike “single finger movements”, grasping actions involve multi-jointed whole hand postures (Schaffelhofer and Scherberger, 2016, Leo et al., 2016, Borra et al., 2017). However, early biomechanical data hinted that even single finger movements do in fact require as many multi-jointed synergies as prehension postures (Todorov and Ghahramani, 2004, Schieber, 1995, Häger-Ross and Schieber, 2000). Indeed, our kinematic data here revealed that single finger movements involved active covarying control of the entire hand and required as many significant synergies as grasping, see also (Kirsch et al., 2014). It is therefore likely that complex hand control, whether it be finger individuation or grasping, may necessitate activity in the entire grasp network.. While parietofrontal and sensorimotor nodes of the grasp network have been well defined for hand control, of particular interest is the role of supramarginal gyrus. From a functional neuroanatomical point of view, the supramarginal gyrus is close to the angular gyrus in the inferior parietal cortex and is thought to encode self-awareness of the hand. For example, disruption of activity in the supramarginal gryus using cortical stimulation resulted in finger agnosia (Roux et al., 2003) and prior fMRI research has shown consistent activity in the SMG during the execution of intransitive finger movements (Jonas et al., 2007). Interestingly, activity in the supramarginal gyrus has also been shown to encode the control of kinematic hand synergies in humans (Leo et al., 2016) and the inferior parietal area has been well defined in both human and non-human primates during grasping ((Castiello, 2005, Buxbaum et al., 2006)). Our data thus provide novel evidence that the SMG encodes the kinematic synergies of simple finger movements in a manner like grasping. It should be noted however, that participants in our study performed free-moving hand movements; further work is necessary to understand whether there is a difference in grasp network recruitment for prone hand movements vs. hand movements in space where accessory muscles might be recruited for stabilization during individuation. We note however, that our rather stringent criteria for synergy detection uncovered multiple kinematic synergies during finger individuation in both joint position and angle space.

Our results here also highlight how an entire repertoire of complex hand movements, including finger individuation, can be represented without a rigid somatotopy by a common distributed network spanning multiple cortical areas, specifically outside of the “hand knob” in primary motor cortex (Ejaz et al., 2015, Sanes et al., 1995, Schieber and Hibbard, 1993). Of particular interest is the distributed neural control of the human thumb. The use of an opposable thumb is thought to be a defining property of dexterous hand actions, not just in humans (Napier, 1955, Napier, 1960, Young, 2003) but also in other primates species, notably in the capuchin monkey (Mayer et al., 2019, Truppa et al., 2016). The human thumb has a tendinous and musculoskeletal structure distinct from the other four fingers, and allows greater simultaneous rotation and flexion/extension (Napier, 1955). Not only did thumb movements recruit activity over the entire distributed grasp network, but our representational analyses also revealed that the centroid of the thumb movement’s LFO distribution was the most discriminative compared to all other movements. It was also closer to grasping than the other four fingers, highlighting the importance of the opposable thumb in complex prehensile movements.

Low frequency oscillations

In this study, we used LFOs as a correlate of neural activity; LFOs in ECoG are widespread over varied motor cortical regions (Kubanek et al., 2009, Miller et al., 2007), have been shown to be highly involved during motor control (Ramanathan et al., 2018, Bansal et al., 2011, Rickert et al., 2005, Pistohl et al., 2008, Paek et al., 2014) and can be used to decode hand synergies (Acharya et al., 2010)); similar low-frequency components in MEG (magnetoencephalography) have also been shown to be coherent with hand movement profiles (Jerbi et al., 2007). Given the fact that the repertoire of hand movements utilized a common whole-hand control scheme, especially given the temporal and biomechanical coupling between joints, it wouldn’t have been surprising if the dynamics of grasp-network LFOs were stable and preserved for the movement repertoire. Yet, although the multi-areal neural covariance structure was preserved across movements, latent neural dynamics were compartmentalized by movement-type with distinct behaviorally relevant submanifolds for each hand movement. Our results thus show that multi-area mesoscale dynamics during a movement repertoire, even in low frequency oscillations, best represent the discrete hand action being performed than a readout of motor output traces. These results have applications with respect to ECoG in human brain machine interface (BCIs) trials (Silversmith et al., 2020, Benabid et al., 2019). Specifically, a hybrid decoding approach, such as decoding both the neural dynamics corresponding to the user’s discrete cognitive motor state in conjunction with the readout of real-time kinematic motor output, can potentially improve BCI decoding performance. Our results likely have clinical relevance for stroke given that LFOs reorganize in perilesional cortex (Ramanathan et al., 2018) and are correlated with abnormal flexor muscle synergies after stroke (Godlove et al., 2016). Understanding how neural manifolds of LFOs represent multiple normal and abnormal hand synergies post-stroke can aid in designing rehabilitation schemes such as low-frequency cortical stimulation to preferentially modulate hand control (Khanna et al., 2021) or understanding the effects of peripheral nerve stimulation on cortical LFO manifolds (Tu-Chan et al., 2017).

Differences between single-area cortical spiking and distributed mesoscale dynamics

While distributed mesoscale ECoG signals are fundamentally different signatures of neural activity than single and multiunit spiking activity in primary motor cortex, it is quite possible that there are sufficient similarities to warrant a comparison. Notably, a single mesoscale ECoG channel likely reflects the heterogenous activity of around 10⁵ neurons (Chang, 2015). While it is difficult to relate ECoG LFOs to single neurons, it is possible to suggest population level similarities. More specifically, it is possible that there is an analogy between LFOs measured using local field potential (LFP) recordings with microwire arrays (electrode tips are high impedance) and those measured using lower impedance ECoG electrodes. For example, prior studies in animals have shown that LFOs in LFP to correspond to population level spiking activity (Ramanathan et al., 2018, Hall et al., 2014). In these studies, the local single unit firing activity was closely related to ensemble population dynamics, e.g., the spike population perievent time histogram (PETH) or simply the sum of all recorded spiking activity. Because in both studies the LFOs could be recorded across the array (albeit with spatiotemporal phase offsets), it is quite likely that the ECoG LFO is an aggregate and related mesoscale measure of similar phenomenon. Moreover, our data suggests that mesoscale LFOs have a phase relationship with the low-frequency component of $γ_{H}$ , a putative correlate of population spiking activity at the recording channel. While further work is required to fully understand precisely how neural spiking over a relatively large cortical regions (e.g., 2–4 mm in size) is associated with ECoG LFOs and $γ_{H}$ , we suggest the possibility that population level spiking activity and LFPs recorded using high impedance electrodes are related to our ECoG mesoscale LFO signals.

Our key finding of distinct latent dynamics for the hand-movement repertoire is quite different from what is evident for movement related spiking activity in primary motor cortex (M1); where temporal patterns of latent neural dynamics are highly preserved across varied movements (Gallego et al., 2018, Kaufman et al., 2016, Churchland et al., 2012). One possibility for this difference is that the shared low-frequency population level neural dynamics in M1 spiking activity might function as a more general means of transmitting information to subcortical circuitry irrespective of the actual movement-type (Kaufman et al., 2016, Russo et al., 2018). Perhaps consistent with this possibility is a recent study in rodents that found that movement triggered low-frequency spiking and LFP activity in M1 were tightly phase locked with similar activity in the striatum, which is a single synapse downstream of M1 (Lemke et al., 2019). Also consistent with this model is the recent finding that injection of LFO frequency electrical currents after stroke in perilesional premotor cortex can increase neural cofiring and improve finger control (Khanna et al., 2021); computational modelling in this paper suggested that this allowed enhanced propagation of activity patterns to presumed cortical and subcortical targets.

How then can we compare our findings using ECoG to those based on local M1 spiking? In contrast to population level activity in local circuits, distributed ECoG LFOs do not demonstrate temporally-preserved, movement-triggered single-trial modulations in power at the population level as robustly as spiking activity for the repertoire of actions. Instead, grasp-network dynamics are distinct with distinguishable phases or inter-areal timing differences in mesoscale activity for the repertoire. Rather than serve primarily to transmit downstream information to subcortical regions, ECoG LFOs might correspond to cross-area population coupling within a distributed network. Recent work suggests that such cortico-cortical coupling represents a distinct communication subspace which can be separated from movement potent signals (Veuthey et al., 2020, Semedo et al., 2019). Interestingly, inactivation of an upstream area showed that the movement subspace is distinguishable from the communication subspace (Veuthey et al., 2020). Thus, it is quite possible that low-frequency ECoG mesoscale cortical dynamics are generally weighted towards communication subspaces across the grasp network necessary to discretely select and plan the appropriate synergistic gestures for each hand movement. This further suggests that there are separable and consistent “pathways” for the propagation of information across the distributed grasp network for each movement type. This might allow rapid and precise feedforward communication for each movement type; it might also permit a straightforward means to incorporate sensory feedback.

It should also be noted that notwithstanding the differences between local population spiking activity and multi-areal mesoscale recordings, it might also be the case that the complexity of hand movements might elicit mesoscale dynamics that are fundamentally different from M1 population spiking data during well-learned and stereotyped upper arm movements. For instance, recent studies have shown that the latent dynamics of even local population activity in M1 tends to be highly variable from movement to movement during grasping actions (Suresh et al., 2020, Rouse and Schieber, 2018). Although participants in our study performed pantomime movements, it may be that multi-area neural dynamics might differ if the hand were to actually interact with objects during both grasping and finger movements (e.g., using the index finger to flip a switch). Interaction with an object can further alter the compartmentalization of neural dynamics in the grasp network (Michaels et al., 2020, Russo et al., 2020) and the resultant extrinsic inputs can result in more tangling of the latent mesoscale neural dynamics (Russo et al., 2018). This further suggests that perhaps greater training and practice with objects during dexterous hand control, especially after stroke, might result in more temporally common rather than distinct dynamics.

Summary

In conclusion, we present here a neural framework highlighting how distinct manifolds of mesoscale ECoG dynamics within the grasp network represent a repertoire of human hand movements; this might be a mechanism through which humans can rapidly switch among a repertoire of complex hand movements that are kinematically similar. Extending our framework to clinical populations and to naturalistic hand interactions with daily objects can aid in further understanding the function of large-scale neural manifolds for dexterous human hand control.

STAR Methods

Resource availability

Lead contact

Further information and requests for resources should be addressed to and will be fulfilled by the lead contact of the study, Karunesh Ganguly, M.D, PhD, Karunesh.ganguly@ucsf.edu

Materials Availability

This study did not generate new unique reagents.

Data and code availability

All data reported in this paper will be shared by the lead contact upon reasonable request.
The dPCA algorithm is publicly available (https://github.com/machenslab/dPCA) and the tensor maximum entropy algorithm (TME) is publicly available (https://github.com/gamaleldin/TME). Cortical visualization of ECoG activity was done using a publicly available package (https://github.com/libertyh/img_pipe/tree/matlab). Interfacing of LeapMotion to Matlab for real-time synchronized data acquisition of kinematic and neural data was done using publicly available packages (https://github.com/jeffsp/matleap) and via the LeapMotion SDK (https://developer.leapmotion.com/). All analyses were conducted in Matlab using previously published methods and hand visualization analyses was done using proprietary software (Poser, Bondware, https://www.posersoftware.com/). For modifications of publicly available code packages as pertaining to this study, for real-time data acquisition code synchronizing LeapMotion, Matlab and the ECoG TDT systems, for ECoG neural signal processing, for representational-structure based static analyses of grasp network activity, for neural manifold based analyses, for phase coupling analyses and for synergy based kinematic analyses, all enquiries will be fulfilled upon request to the lead contact of this study.
Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

Experimental model and subject details

Participants

We recruited in total five patients (two male, three female, mean age 31.2 years) undergoing clinical monitoring for epilepsy at UCSF’s Department for Neurological Surgery. Each patient was implanted with high-density electrocorticographic surface grids (256 channels, 4mm pitch) and electrode strips covering the motor, sensory, frontal and parietal cortical regions. The placement of the electrodes was for the purposes of identifying foci of seizures. The coverage of the grids was left hemispheric for three of the five participants. Four of the five participants performed all self-paced grasping and individual finger movements; these four subjects form the main study dataset to address our hypothesis of grasp-network neural activity. Two of these four participants also performed cue-based thumb movements and the final 5^th subject was able to perform only cue-based thumb movements due to clinical time restraints. All hand movements were contralateral to the recording ECoG grid. All patients gave their informed consent to participate in the study protocol as approved by the UCSF Committee on Human Research and all procedures were approved by the UCSF Institutional Review Board (IRB).

Method details

Experimental design and data acquisition

The task required participants to perform the following eight self-paced movements: flexion/extension of each finger and pantomime of three types of grasping movement (pinch, tripod and power grasp). For the pinch grasp, participants were instructed to imagine picking up a small object using their thumb and index finger. For the tripod grasp, participants were instructed to imagine picking up a larger object such as a pen/marker with their thumb, index and middle fingers. For the power grasp, participants were instructed to imagine picking up/grabbing a larger object using all their fingers. The evolution of these three specific types of grasp movements are thought to be essential for human prehension, and are thought to underlie all human hand grasping actions in general (Young, 2003, Napier, 1960). Participants performed the self-paced movements sequentially and not randomly. For example, participants performed cycles of self-paced thumb movements continuously, rested for a few minutes, then performed index movements and so on, finishing with the grasping movements. Two of the four participants also performed cue-based thumb movements and the final fifth participant was able to perform only cue-based thumb movements due to clinical time constraints. In the cue-based movements, participants were instructed to perform the movement when they viewed a ‘Go’ cue and they were given 3s to perform the movement. Prior to the Go cue, a ‘Ready’ cue was presented for a variable length between 1.5–2s to alert them of the Go cue. Following the completion of 3s after the Go cue, a ‘Rest’ cue was presented for 3s. This cue-based design was repeated for about 25 trials on average per participant. Participants performed free unconstrained hand movements in space, although their forearm and elbow was supported by a pillow.

Continuous neural data was sampled at 3052Hz with a PZ2 amplifier connected to a RZ2 Tucker-Davis recording systems (TDT) and was hard-referenced to a separate reference electrode outside of the recording grid. Concurrent kinematic data was simultaneously recorded using the LeapMotion system at a sampling rate of approximately 100Hz. Custom code in Matlab was written to use the Arduino board to generate synchronized time-markers online in the kinematic and neural data streams. All analysis and processing of data were performed in Matlab (Mathworks Inc.)

Sample size estimation

We did not perform a priori sample size estimation, especially given the rarity in obtaining invasive brain data of this nature. We relied on previously published reports in the literature for similar work that utilized a comparable sample size. The majority of our analyses were performed on single subjects and replicated across subjects. Our analyses did not require randomization or blinding, and subjects were recruited based on ECoG grid coverage that was implanted purely for the purposes of clinical monitoring for epilepsy and subjects were excluded if there were any clinical constraints or if their grid coverage did not encompass cortical areas specific to our study.

ECoG signal processing

Collected neural data were first down sampled to 508Hz, and notch filters were used to remove line noise at 60Hz and its relevant harmonics. Data were then visually inspected to identify bad channels that did not record any meaningful, noise-free neural activity and were removed from further analyses. We also visually examined data for artifactual epochs; these time-periods were marker for further removal after synchronization with kinematic data. We then applied a common median reference to the raw ECoG signals. Low frequency oscillations (LFOs) were defined in this study to be ECoG activity within the δ frequency band of 0.5–4Hz. To extract LFOs, we applied a 4^th order IIR band-pass filter to ECoG data within the frequency range of 0.5 and 4Hz. To extract the envelope of the LFO, we applied the Hilbert transform and extracted the analytic amplitude at each channel. An additional processing step involved z-scoring channel data that is very commonly used in ECoG-based analyses (Silversmith et al., 2020). For representational similarity analyses, channel data were z-scored across movements to preserved differences in mean activity between movements. For manifold based analyses (such as PCA or CCA) that depends on covariance, channel data were z-scored (thereby mean-centered) within each dataset individually, as the mean does not inform on covariations. Note that z-scoring has the additional step of re-scaling each channel’s data by the standard deviation around its mean; however we found that re-scaling did not have any effect on either our representational or manifold based analyses as all channels’ raw or filtered activity exhibited very similar variation around its mean in all hand movement datasets. With regard to high-gamma analyses (Chang, 2015), we applied a bandpass filter to the raw ECoG data with limits of 70–150Hz. The Hilbert transform was then applied to bandpass-filtered data and high-gamma amplitudes, $γ_{H}$ , was given by the analytical amplitude of the Hilbert transform. For analyzing the relationship between LFOs and $γ_{H}$ , the sampling rate of the data was maintained at 508Hz. To extract the low-frequency component of $γ_{H}$ , i.e., $γ_{H}^{L F O}$ , a second filtering step was applied wherein high gamma amplitudes were band pass filtered within the 0.5–4Hz range using the 4^th order IIR band-pass filter. Here on out, when we mention LFOs we imply the envelope of the δ band oscillation and when we specifically mention LFO oscillations we refer to the δ band oscillation alone without the envelope. Similarly when we mention $γ_{H}^{L F O}$ we refer to the low-frequency component of high gamma amplitudes.

For exploring the relationship between the manifolds of neural activity (either LFOs or $γ_{H}^{L F O}$ ) and kinematics, data for both were down sampled to 25Hz in time-synchrony with each other followed by Savitzky-Golay smoothing (2^nd order filter with a span of 400ms, (Pistohl et al., 2008)). After synchronization with kinematic data, whenever artifactual epochs or time-periods were removed in either neural or kinematic data streams, the corresponding epoch was removed from the other data stream. Using the kinematic data, we created neural epochs or trials based on a full cycle of flexion/extension in the case of self-paced finger movements or a full cycle of opening/closing in the case of self-paced grasping movements. In the self-paced experiments, there was variability in the exact amount of time it took to start and execute a full self-paced cycle as there was no experimental cue for each trial. For this reason, we normalized the length of each trial to 3s and accordingly interpolated each epoch of neural data to have 75 samples at 25Hz (0s to 2.96s with a Δt = 40ms). We then were able to average across trials with a consistent time-scale or alternatively concatenated data across trials based on the analysis. For the cue-based movements, we extracted epochs of data from 500ms before the Go cue to 3s after the Go cue; each epoch was therefore consistently 3.5s of length. Note that our results were robust to how we epoched our neural data i.e., our main findings remained unchanged even when we epoched data around peak flexion/closing without trial length normalization.

Movement related information in grasp-network LFO amplitudes

Representational structure in grasp-network LFO amplitudes

To establish the relationship between LFOs and movement, we evaluated whether there was a representational structure in mean grasp-network LFO activity i.e., the static property of the grasp network corresponding to the location of each movement in high-dimensional channel space. Specifically, we wondered whether the movements’ mean LFO activity exhibited a unique set of similarity patterns similar to prior fMRI work in local M1 regions (Ejaz et al., 2015), that had shown for instance, that mean voxel-wise activity for pinky and ring movements were more similar than to thumb movements. Notably, Ejaz et. al. had shown that such neural similarity patterns follow the kinematical coupling pattern between fingers, where the thumb is individuated distinctly while the pinky and ring fingers are usually co-active. However, such a representational structure in mean cortical activity has not been shown at the level of the mesoscale grasp network and in relation to both finger and grasping movements simultaneously. To ascertain the representational structure in grasp-network LFOs, we evaluated the pairwise statistical distance D_ij between the centroids of any two movements’ channel LFO distributions (assumed to be multivariate Gaussian) within a participant using the Mahalanobis distance metric:

D_{i j} = [μ_{i} - μ_{j}] {[\frac{Σ_{i} + Σ_{j}}{2}]}^{- 1} {[μ_{i} - μ_{j}]}^{⊤}

Here, $μ_{i} \in R^{c h \times 1}$ and $μ_{j} \times R^{c h \times 1}$ are vectors denoting the multidimensional mean LFO amplitudes across channels, averaged over time for movements i and j, and $Σ_{i} \in R^{c h \times c h}$ and $Σ_{j} \in R^{c h \times c h}$ are the corresponding multidimensional variance-covariance matrices between channels’ time-varying LFO amplitudes. The Mahalanobis measure therefore is the pairwise distance between any two movements’ multidimensional channel means scaled by the pooled multidimensional variance. For accurate comparisons and as detailed earlier, we utilized a common baseline to z-score channel LFO amplitudes (z-scoring channel data across all movements) to preserve between-movement differences in mean LFOs. The above procedure resulted in the construction of a symmetrical distance matrix for each participant D ∈ R^8×8 whose entries contained all the pairwise distances between movements. We then assessed the similarity between participants’ distance matrices. Specifically, upper-diagonal entries of each participant’s distance matrix (note that the distance matrix is symmetric) were collapsed into one vector and correlated with similar vectors from other participants (Ejaz et al., 2015). The mean correlation value formed the statistic of similarity in participants’ representational content. Significance was assessed by a permutation procedure wherein the movement labels were swapped within each participant when computing the distance matrix and subsequently the mean correlation value. This procedure was performed 1000 times to generate a null distribution of mean correlation values under the null hypothesis that there is no representational structure in LFO distributions across participants. To parcellate the similarity patterns between movements’ LFO distributions, we performed hierarchical clustering on the average Mahalanobis distance matrix using Ward’s criterion (Theodoridis and Koutroumbas, 2003) and depicted the result by a scaled dendrogram.

Apart from the Mahalanobis distance analyses, we also utilized a more traditional classification approach to evaluate the representational structure in LFOs. We used a linear Support Vector Machine (SVM (Fan et al., 2008)) with channels as features and instantaneous LFO amplitudes as observations. Here, the instantaneous LFO amplitudes are fluctuations around mean δ band envelope for each channel, and the SVM linear classifier evaluated the separability between movements in terms of the channels’ fluctuations around the mean activity. A pairwise SVM is a linear hyperplane that maximizes the margins between the observations of two datasets and aims to discriminate time-periods when a participant was performing a particular type of hand movement from time-periods corresponding to the second movement. The linear classifier is essentially a vector W_SVM that weights each channel’s LFO amplitudes X_LFO(t) at any time t; a two-class SVM discriminates the two movements (with class labels 1 and −1) based on the sign of the inner product between channels LFO activity and W_SVM, i.e.,

c l a s s_{A} : W_{S V M}^{⊤} X_{L F O} (t) > 0

c l a s s_{B} : W_{S V M}^{⊤} X_{L F O} (t) < 0

We contrasted each movement against all other movements individually in a pairwise manner (e.g. thumb vs. index, thumb vs. middle etc.), within each participant. Within each contrast, we averaged results from four-fold cross validation wherein within each fold 70% of the data was randomly selected to train the SVM and the remaining 30% formed the testing set. The SVM’s slack parameter was computed from the 70% training data. Importantly, although each movement had a distinct number of sample points, the training dataset was balanced with an equal proportion of both movements’ time-periods. However, since the testing dataset was imbalanced, we evaluated the classifier performance via the term balanced accuracy which is the average of the sensitivity and specificity of the SMV model’s performance on held out data. The SVM model’s sensitivity is given by the ratio of and its specificity is given by the ratio of

s e n s i t i v i t y = \frac{T r u e P o s t i v e s}{T r u e P o s i t i v e s + F a l s e N e g a t i v e s}

and its specificity is given by

s p e c i f i c i t y = \frac{T r u e N e g a t i v e s}{T r u e N e g a t i v e s + F a l s e P o s i t i v e s}

both computed from the held out test data samples. Balanced accuracy is then the average of these two numbers. To obtain confidence intervals and test the statistical significance of the SVM model, we used Bayesian statistics. Specifically, we modeled both sensitivity and specificity as Binomial distributions for which the beta distribution is the conjugate prior (Bishop, 2006). For example, the beta distribution for sensitivity is proportional to:

p (μ) \propto μ^{a - 1} {(1 - μ)}^{b - 1}

where μ is the random variable describing the distribution of values around true sensitivity. The mean or expected value of this variable is given by

E (μ) = \frac{a}{a + b}

which as it can be seen shares a close relationship with the actual formula for sensitivity i.e., a = No. of True Positives and b = No. of False Negatives. Thus p(μ) defines a distribution around the estimated value of sensitivity; the width of this distribution depends on the number of held-out test samples. The parameters a and b are thus hyperparameters and are initially chosen to be both equal to 1 which results in the beta distribution being a flat or uniform prior i.e., p(μ) is a constant for all possible values of μ. Given the number of samples from the held out data, the parameters a and b are then updated or added with the number of true positives and false negatives. In this manner, we can update the sensitivity of the model i.e., E(μ) and the pdf p(μ) around the mean sensitivity from the held out test samples. Similarly, we can generate another pdf P(α) for the specificity of the model along with E(α) from the held out test samples where α is a random variable for the possible values of specificity. Note that these pdfs need to be appropriately normalized so that they sum to one (see chapter two of (Bishop, 2006) for full details). The balanced accuracy can then be computed as the average of the random variables μ and α defining sensitivity and specificity respectively. The average (or sum) of two random variables is a linear operation which results in the convolution of the pdfs of the two random variables. The pdf for balanced accuracy is thus given by:

p (η) = p (μ) * p (α)

where η is the random variable describing the possible values of balanced accuracy, E(η) is the mean of this distribution and ∗ is the convolution operator. As before, this pdf should be normalized so that it sums to one. In this manner we can generate pdfs of sensitivity, specificity and balanced accuracy of the SVM model from its performance on testing data. The significance of the SVM model and p value can then be easily obtained by evaluating the integral of the pdf (η) to the left of 0.5 or 50%. This procedure of computing the balanced accuracy of a pairwise SVM model and its associated p value was performed for all pairwise comparisons across all subjects.

The obtained pairwise balanced accuracies were then used to build a symmetric matrix D ∈ R^8×8 where any entry D_ij is the balanced accuracy of the model when discriminating movements i and j. We built such matrices for each subject and averaged across subjects to obtain the average classification distance between all movements. We then examined the averaged matrix for a representational structure i.e., if the activity around mean grasp-network LFOs of some movements were closer to others and more likely to be misclassified between them. For instance we can examine if pinky finger movements tended to be misclassified more as ring finger movements. To this end, we performed hierarchical clustering on the average pairwise matrix using Ward’s criterion and statistically evaluated the misclassifications using mixed effect models. We also used the individual pairwise classifiers in a multi-classification framework using a max-vote strategy wherein the movement that won the maximum number of classifications from each pairwise classifier was assigned to the testing data-point.

Phase-coupling of ECoG LFOs to kinematics

To further establish the relationship between LFOs and movement, we first evaluated whether the amplitude of LFOs peaked at preferred phases of movement. For finger movements, the main phases of movement corresponded to flexion and extension, and for grasping movements they corresponded to opening and closing of the hand. A phase of π was considered as full flexion (fingers) or full closing (grasps) and a phase of 0 or 2π was considered full extension (fingers) or full opening (grasps), phases in between 0 and π constituted the flexion/closing cycle and phases in between π and 2π constituted the extension/opening cycle. Within each trial and for each channel, we then identified the preferred phase of movement when LFO amplitude peaked. We identified similar such preferred movement phases when LFO amplitudes peaked for all trials. Under the null hypothesis, one would expect a circularly uniform density of preferred phases, suggestive of no relationship between LFOs and kinematics. We tested this hypothesis at each channel using the Rayleigh test in the circular statistics toolbox (Berens, 2009). In this manner, we were able to identify channels whose LFO envelopes peaked at preferred phases of movement. A nominal significance threshold of α = 0.05 was used to identify the cortical network of channels whose LFOs were significantly phase-locked to movement. We performed this analysis for each movement and subject individually. Channels that exhibited significant phasic relationships were assigned a value of 1 and non-significant channels were assigned a value of 0. Such a binary mask of significant channels was identified for each movement and subsequently aggregated across movements. Channels that consistently exhibited significant phase relationships across movements would have higher aggregated values and therefore deemed to have relatively higher locking to LFOs to kinematics than channels that were significant in only a few of the eight movements.

Neural manifold analyses

To understand the neural manifold based on the multi-area neural covariance structure, PCA was applied to trial-concatenated, mean-centered data individually for each participant and movement. Let $X_{a} \in R^{t \times c h}$ denote an individual z-scored neural dataset for a particular movement a, where t denotes the time-samples across trial concatenated data, with the number of channels given by ch. PCA uncovers an orthonormal basis $D \in R^{c h \times c h}$ that captures directions of maximal variance wherein each column of D is a neural mode or principal component identifying multi-areal channels with covarying neural activity. The variance accounted for by k PCs for the movement a can be estimated using the reconstruction formula:

R {(a)}_{k}^{2} = \frac{{‖ X_{a} ‖}^{2} - {‖ X_{a} - X_{a} D_{k} D_{k}^{⊤} ‖}^{2}}{{‖ X_{a} ‖}^{2}}

Here, the norm of the matrix terms in the above equation is the sum of the squared values of the elements of the matrix. Results revealed that on average across movements 45 PCs captured at least 75% of the neural VAF in trial concatenated data, resulting in a low-dimensional neural manifold given by the first 45PCs i.e., $D_{a}^{45} \in R^{c h \times 45}$ for movement a. Note that the neural modes can be visualized on the cortical network as it assigns a weight for each channel, allowing us to compute the average weight within each node of the grasp network. Specifically, we defined rough Brodmann Area-based regions of interest (ROI) identifying primary motor cortex (M1), sensory cortex (S1), ventral and dorsal premotor cortices (PMv, PMd), parietal cortex (PPC) and supramarginal gyrus (SMG) and quantified the average channel weight within each ROI across all 45 neural modes.

Having identified the manifold for each movement, we then computed whether the 45D manifolds were oriented similarly in high-dimensional channel space using the method of principal angles (Björck and Golub, 1973, Gallego et al., 2018, Meyer, 2000). In the cases of two lines, this trivially reduces to the angle between the lines. In the case of subspaces or higher-dimensional manifolds, the angles are not so trivial as there are infinite combinations of lower dimensional flats between the manifolds. However, one can find principal directions within the manifold (which are linear combinations of the basis of the manifold) such that they produce the smallest possible angles between the two manifolds. In our data here, the neural modes form an orthonormal basis for the manifold as a consequence of the PCA step. We then computed the singular value decomposition (SVD) of the dot product matrix between the two manifolds for any two movements a and b by the formula

P_{a} S P_{b} = S V D (D_{a}^{45^{⊤}} D_{b}^{45})

where S is a diagonal matrix whose entries are the cosine of the 45 ordered principal angles (smallest to largest) i.e.,

S = d i a g (\cos (θ_{1}), \cos (θ_{2}) \dots \cos (θ_{45}))

P_a and P_b of size R^45×45 are the ordered principal directions i.e., directions within the manifolds of movements a and b that have the associated principal angles between the two manifolds. As it can be seen, there are as many principal angles as the dimensionality of the manifold. Using the SVD, we computed the principal angles between all pairs of neural manifolds across all four participants. Given the 8 movements in the study, there are 28 such pairwise comparisons per participant. Under the null hypothesis that the manifolds would not be more similar to each other than what would be expected b chance, we would not expect a difference between the observed principal angles from a null distribution of principal angles. To construct the null distribution of principal angles, we turned to the tensor maximum entropy method (TME, (Elsayed and Cunningham, 2017)) that was developed and applied in the context of simulating smoothed spiking waveforms in M1 when a monkey performed varied tasks (Gallego et al., 2018, Elsayed and Cunningham, 2017). Full details on the method and associated code can be found in the original paper by Elsayed and Cunningham, here we detail the main approach. Briefly, each participant’s neural data can be considered as a 3D tensor of dimensions R^ch×t×mvmt. The first two dimensions are channels and time samples and the third dimension represents the 8 hand movements in the study. We used the TME method to simulate null tensors of data while respecting the first and second order moments (i.e., mean and covariance) along the second and third tensor but not along the channel tensor. By not simulating the channel covariance structure in the real data, we were able to simulate data without a channel covariance structure but respecting all other 1^st and 2^nd order statistics of the data. After simulating a tensor of data, we applied PCA to identify the manifolds in the simulated data and evaluated the principal angles between the simulated manifolds. By iterating through this procedure 1000 times, we were able to generate a null distribution of principal angles and test the hypothesis of a preserved neural manifold in the real data. Given that the exact number of trials and hence the time-samples of trial-concatenated data were not similar movement to movement, we periodically subsampled trials when computing principal angles and running the TME code to ensure that the 3D tensor was consistent. If two manifolds were similar to each other, then their principal angles would be smaller than the null distribution of principal angles, assessed at the α = 0.01 level.

To evaluate the whether the manifolds across movements shared variance with each other in a common multi-areal subspace, we computed the ratio of the across-movement VAF to within-movement VAF (Gallego et al., 2018). The within-movement VAF is nothing but the variance captured when projecting each movement’s data onto its own 45D manifold. The across-movement VAF is the variance captured by projecting data from movement a onto the 45D manifold of movement b. The equation for computing the across-task VAF is given by:

R^{2} (a, b) = \frac{{‖ X_{a} ‖}^{2} - {‖ X_{a} - X_{a} D_{b} P_{b} {(D_{b} P_{b})}^{⊤} ‖}^{2}}{{‖ X_{a} ‖}^{2}}

The ratio of across-movement to within-movement VAF, $\frac{R^{2} (a, b)}{R^{2} (a)}$ was then computed for all pairwise comparisons. To obtain a surrogate distribution of the maximal across-movement to within movement ratios, we first generated random 45D manifolds and used the QR decomposition to obtain an orthonormal basis for the random 45D manifold. Data from each movement were then projected onto the orthonormal basis of this random manifold to compute the across-movement variance. We iterated this procedure 1000 times and within each iteration computed the maximal ratio of the null across-movement to within-movement VAF. We carried the above manifold analyses steps for both LFOs and $γ_{H}^{L F O}$ in the grasp network.

demixed Principal Component Analysis to evaluate latent dynamics

Given that our results showed that all movements shared a common neural manifold, we then sought to understand whether latent neural dynamics within the common multi-areal neural subspace were temporally shared by the different hand movements using demixed Principal Component Analysis or dPCA (Kobak et al., 2016). Full details on the method can be found in the paper by Kobak et. al., here we focused on the main details of the method as it pertains to our data. By design, the objective function of dPCA is to build a common neural subspace or all movements and find dPC neural modes that correspond to two forms of neural activity: 1) time-varying activity that is common to all movements and is independent of the movement-type i.e., via time dPCA modes and 2) time-varying activity that is specific to the type of movement being performed outside of the common activity i.e., via movement-specific modes. The two types of dPCA modes therefore de-mix the neural data and projecting LFOs onto these modes capture latent neural dynamics that are either common to, or dependent on, the type of movement. dPCA was performed on trial-averaged data to better uncover latent modes. Note that the use of dPCA is justified and meaningful given our finding that a common multi-areal manifold represented all hand movements. We fit a dPCA manifold of the same dimensionality as the PCA manifold for consistency in analyses (which was 45D).

For each participant, the neural data for all movements was arranged in a 3D tensor $X \in R^{c h \times 8 \times t}$ , where ch is the number of neural channels and t is the 75 time samples corresponding to 3s of data (trial averaged) at 25Hz. The data matrix X was then marginalized into the following terms: X_t that represents the time-varying average neural activity across all 8 hand movements, X_m that represents the multivariate centroid of each movement and X_tm that represents the movement specific neural activity. The size of the X_t, X_m, X_tm are all the same as X by replicating values. Pertinent to analysis here, X_t represents the movement-independent time-varying common neural activity in a trial, and X_m, X_tm represent movement-specific neural activity above and beyond the common time-varying activity. Our primary interest here was to understand X_t and X_tm the common time and movement-specific time-varying neural dynamics. Prior representational analyses have already emphasized the properties of, each movement’s centroid of channel distributions. We thus z-scored each channel’s LFO activity for each movement, thereby placing the centroid of each movement at the origin (X_m = 0). As detailed earlier in the ECoG signal processing section, re-scaling via the z-scoring step did not influence dPCA results as all channels exhibited similar variations around its mean across all hand movement datasets. The core dPCA function was then used to approximate the original data matrix and the overall covariance of the tensor via the marginals i.e.,

X = X_{t} + X_{t m} + X_{n o i s e}

C = C_{t} + C_{t m} + C_{n o i s e}

where C is the total covariance matrix, and C_t, C_tm are the covariance matrices for the marginals. The loss function of dPCA is then given by:

L_{t} = {‖ X_{t} - F_{t} D_{t} X ‖}^{2}

L_{t m} = {‖ X_{t m} - F_{t m} D_{t m} X ‖}^{2}

where D_t, D_tm are the time and movement-specific dPCA modes respectively. Projection of neural data onto these modes i.e.,

N_{t} = D_{t} X,

N_{t m} = D_{t m} X

are the latent movement-independent neural dynamics and latent movement-specific neural dynamics respectively. F_t, F_tm are encoders transforming low-dimensional neural dynamics back into full channel space. The dPCA solvers for estimating D_t, D_tm, F_t, F_t uses the reduced rank regression algorithm and the user has to specify the number of dPCA modes to be identified by reduced rank regression. We used 45 modes to keep our analyses consistent with results from the neural manifold analyses. To estimate the variance accounted for by the two types of dPCA modes, we used the same formula as in the dPCA paper, where the subscript $ϕ \in (t, t m)$ and ‖ denotes the sum of the squares of all the entries in a matrix:

R_{ϕ}^{2} = \frac{{‖ X ‖}^{2} - {‖ X - F_{ϕ} D_{ϕ} X ‖}^{2}}{∥ X ∥^{2}}

We carried out the dPCA analyses on both LFOs and $γ_{H}^{L F O}$ in the grasp network. If the latent 1408 neural dynamics were distinct for the hand movement repertoire, then one would expect the variance due to the movement-specific modes to be higher than the variance due to the time dPC modes.

Relationship between LFOs and $γ_{H}$

Phase-coupling of ECoG LFO oscillations to the low frequency component of $γ_{H}$

To evaluate the relationship between LFOs and $γ_{H}$ at individual channels across the ECoG grid, we utilized the phase-locking value metric (Canolty et al., 2012, Canolty et al., 2006) to investigate the coupling between LFO oscillatory phase and the low frequency phase of high gamma amplitudes i.e., $γ_{H}^{L F O}$ . As detailed earlier in the ECoG signal processing section, $γ_{H}^{L F O}$ is generated by performing two filtering steps on the raw ECoG data i.e., δ band pass filtering the amplitude of high gamma oscillations (70–150Hz). We performed an additional Hilbert transform and extracted the phase time series of $γ_{H}^{L F O}$ from the angle of the Hilbert transform, thereby generating the time series of the low-frequency phase of high gamma amplitudes, $ϕ_{γ_{H}}^{L F O} (t)$ . At the same time, raw ECoG data were band-pass filtered in the LFO range of 0.5–4Hz alone to create the δ(t) signal i.e., LFO oscillations. The phase time-series was extracted from δ(t) via the Hilbert transform to obtain $ϕ_{L F O} (t)$ .

We now have two time series, $ϕ_{L F O} (t)$ which is the phase of LFO oscillations and $ϕ_{γ_{H}}^{L F O} (t)$ which is the low-frequency phase of high gamma amplitudes. If there is a relationship between LFO oscillations and high gamma amplitudes, then these two phase time-series would be phase locked to each other consistently across trials. This phasic relationship between the two signals can be evaluated by the Phase Locking Value (PLV), as follows. First we computed the angular difference between $ϕ_{L F O}^{k} (t)$ and $ϕ_{γ_{H}}^{L F O^{k}} (t)$ averaged over all time-points $t = 1, 2 \dots n$ within a given single trial k to obtain single trial estimates of the preferred phase angle (a single number in radians) between the two signals. We then evaluated the circular mean of the preferred phase angles across all $k = 1, 2 \dots m$ trials to obtain the PLV. The PLV is thus a single complex number computed for every channel; the strength of the coupling between the two signals was obtained from the magnitude of PLV and the preferred phase difference between LFO oscillations and high gamma amplitudes was obtained from the angle of PLV. If all trials have the same length, then the PLV can be obtained from the following formula:

P L V = \frac{1}{m} \sum_{k = 1}^{m} \frac{1}{n} \sum_{t = 1}^{n} e x p [i (ϕ_{L F O}^{k} (t) - ϕ_{γ_{H}}^{L F O^{k}} (t))]

P A C_{m a g} = | P L V |

P A C_{a n g l e} = ∠ P L V

If there is indeed a consistent preferred phase difference between the two signals across trials, then the |PLV| (PAC, phase amplitude coupling relationship between LFO phase and high gamma amplitudes) will be significantly greater than what could be expected by chance. To test for the significance in the strength of phasic coupling at each channel, surrogate or null distributions were constructed by randomizing the LFO phase time-series $ϕ_{L F O} (t)$ prior to estimate the PLV. This randomization procedure was repeated 1000 times to generate a surrogate distribution of the PLV at each channel, allowing us to test the strength of PAC_mag between the two signals at individual channels (α = 0.05, FDR corrected). We performed the phase-based analyses at each electrode and individually within each movement for each subject. For each subject, we quantified the percentage of channels that exhibited such significant PLV in at least one of the eight movements and aggregated significant channels across movements. Channels that consistently exhibited significant phase relationships across movements would have higher aggregated values and therefore deemed to have relatively higher PLV between the two signals.

Shared variance between the manifolds of LFOs and $γ_{H}^{L F O}$

We evaluated the shared variance between the 45D multi-areal manifolds of both signals, LFOs and $γ_{H}^{L F O}$ , using the percentage of variance metric (Degenhart et al., 2020), which computes the percentage of variance in one signal’s manifold that can be captured by the second signal’s manifold. If the manifolds of both signals shared significant variance with each other, this percentage should be significantly greater than chance. We evaluated the shared variance between both signals’ dPCA manifold and their manifolds from the PCA analysis. With respect to the dPCA manifolds, the shared variance in the dPCA manifold of $γ_{H}^{L F O}$ that is also captured by the dPCA LFO manifold is given by the following percentage of variance captured formula:

p_{c a p}^{d P C A} = \frac{t r a c e (U_{L F O} U_{L F O}^{⊤} Q_{γ_{H}^{L F O}} Q_{γ_{H}^{L F O}}^{⊤} U_{L F O} U_{L F O}^{⊤})}{t r a c e (Q_{γ_{H}^{L F O}} Q_{γ_{H}^{L F O}}^{⊤})}

where $U_{L F O}$ is an orthonormal basis of the 45D dPCA LFO manifold (obtained by the QR decomposition of the dPCA manifold) and $Q_{γ_{H}^{L F O}}$ is the 45D dPCA manifold of $γ_{H}^{L F O}$ . Similarly, we can also compute the shared variance in the dPCA manifold of LFO that is captured by the dPCA $γ_{H}^{L F O}$ manifold by finding an orthonormal basis for the $γ_{H}^{L F O}$ dPCA manifold. Note that by dPCA manifold, we refer to both the encoder and decoder subspaces from the dPCA analyses; we therefore computed the shared variance between the two signal’s encoder and decoder subspaces using the $p_{c a p}^{d P C A}$ formula relative to each other and averaged the four numbers to obtain the shared variance between the two signals’ dPCA manifolds. The null distribution of the shared variance is computed by finding $p_{c a p}^{d P C A}$ of the dPCA manifolds of either LFOs or $γ_{H}^{L F O}$ with random manifolds of the same size across 1000 iterations. We can then assess the significance level of the shared variance between the two signals’ manifolds relative to the null distribution. We performed this analyses for each of the four subjects’ individual dPCA manifolds.

Similarly, it is also possible to find the shared variance between the two signal’s manifolds from the PCA analyses. Note that in this case the analyses has to be performed movement-wise to contrast the two signals’ individual manifolds across all movements and subjects (8 movements × 4 subjects for a total of 32 comparisons). As the neural modes from the PCA step already define an orthonormal basis, the shared variance between the two signals can be obtained by evaluating the percentage of variance in one signal’s manifold that is also captured by the second signal’s manifold for each individual movement’s neural data and with the following formula:

p_{c a p}^{P C A} = \frac{t r a c e (D_{L F O} D_{L F O}^{⊤} D_{γ_{H}^{L F O}} D_{γ_{H}^{L F O}}^{⊤} D_{L F O} D_{L F O}^{⊤})}{t r a c e (D_{γ_{H}^{L F O}} D_{γ_{H}^{L F O}}^{⊤})}

where $D_{γ_{H}^{L F O}}$ is the 45D manifold of $γ_{H}^{L F O}$ and $D_{L F O}$ is the 45D manifold with LFOs. While the above formula gives the shared variance relative to $γ_{H}^{L F O}$ , we can swap the terms in the numerator and denominator and obtain the shared variance relative to LFOs and take the average of the two numbers. Note that the encoder and decoder matrices from the PCA analyses are the transpose of each other and equivalent; we therefore evaluated the shared variance in the decoder subspace alone. The null distribution of the shared variance is obtained by assessing $γ_{H}^{L F O}$ of either the manifold of LFOs or $γ_{H}^{L F O}$ with random manifolds of the same size across many iterations. We can then assess the significance level of the shared variance between the two signals’ manifolds relative to the null distribution. We performed this analyses for each movement across the four subjects.

Kinematic recordings of the repertoire of human hand movements

We used the LeapMotion system to record instantaneous 3D position data of each joint in the human hand, the center of the palm and the wrist. By default, the data is referenced to the world-coordinate system at the LeapMotion IR sensor. As a denoising step to filter out kinematic data not pertaining to the actual movement, the data in the world coordinate system was projected onto the palm normal, a 3D vector pointing orthogonal to the inner surface of the palm, and subsequently referenced to the palm center. This preprocessing step ensures that irrespective of the orientation and position of the hand in 3D space, only task-specific joint data is collected. Data were then visually inspected for artifactual epochs that were then removed. The kinematic dimension of data was 75, corresponding to the 3D position of 5 segments: the wrist, interphalangeal joints (MCP, PIP, DIP) and the endpoint bone, for each of the 5 fingers of the hand. Given the anatomy of the thumb, the MCP and PIP were considered equivalent. Kinematic data was high-pass filtered above 0.1Hz and smoothed with a Savitzky-Golay filter (2^nd order filter with a span of 400ms, (Pistohl et al., 2008)) to remove drift. Kinematic data were then down sampled to 25Hz in conjunction with neural data. Similar to neural data, we normalized the length of each kinematic trial to be 3s i.e. 75 data samples at 25Hz.

Hand movements are known to exhibit significant temporal and biomechanical coupling between joints and are thus of much lower dimension that the original joint data. These covariation patterns are called synergies in the kinematic literature; it is thought that the brain preferentially represents synergies rather than individual muscles or joints (Ejaz et al., 2015, Leo et al., 2016), thereby greatly reducing the complexity of control given the large number of degrees of freedom in the hand. To identify kinematic synergies, we applied PCA to the trial-concatenated kinematic data matrix $D \in R^{t \times 75}$ , where t is the number of time samples across the 75 kinematic dimensions (Ingram et al., 2008, Santello et al., 1998, Todorov and Ghahramani, 2004, Mason et al., 2001, Leo et al., 2016). We performed the kinematic PCA analysis individually for each subject and movement. The resulting kinematic PCs or synergies represent multi-jointed whole hand covariation patterns. Each coefficient in a kinematic PC assigns a weight to an individual joint in one of the 3D spaces. Apart from quantifying the number of PCs it required to account for 95% of the kinematic variance, we also sought to understand the number of PCs statistically different from noise. To identify the number of statistically significant synergies from a dataset, we used the Marchenko-Pastur (Marchenko and Pastur, 1967) criteria on the eigenvalues from the PCA step. Specifically, based on random matrix theory, the bound for significance on the eigenvalues of the kinematic data matrix is given by

λ_{t h r e s h} = σ^{2} {(1 + \sqrt{\frac{1}{q}})}^{2}

Here, σ² is the variance of the overall kinematic dataset and q is a ratio between the number of observations/time-samples to the kinematic dimensions (which was 75 in this study). Any eigenvector or PC that had an eigenvalue $λ > λ_{t h r e s h}$ was identified as representing a significant kinematic covariation pattern or synergy in the dataset. Having identified the k number of significant synergies $S_{i} \in R^{75 \times 1}$ , $i = 1, 2 \dots k$ , the temporal activation of each synergy A_i was computed by projecting the kinematic data through the corresponding eigenvector, i.e. $A_{i} = D S_{i}$ , $i = 1, 2, \dots k$ . We averaged the temporal activation across trials to obtain the mean time course of synergy activity and used the bootstrap method to obtain confidence intervals of the mean (trials were randomly sampled with replacement multiple times before computing the average in a bootstrap iteration, with 1000 total iterations). For visualization of the hand and synergy postures, we used Poser (Bondware Inc.) to animate the hand. We also identified synergies in joint angle data apart from joint position data. To do this we used inverse kinematics to map joint position data to joint angle data using non-linear least squares; the dimension of the joint angle data was 25 corresponding to angular displacements along each of the five segments for all five fingers. PCA on the matrix of joint angle data uncovered the joint angle synergies.

Having identified each movement’s significant synergies, we then evaluated whether the kinematic subspace spanned by the significant synergies involved whole-hand covariation for all hand movements, even for finger individuation. Note that we can also examine the PC weights of the joints within each significant synergy; this allowed understanding whether finger synergies involved covariation only along the segments of the finger being individuated or whether it involved multi-jointed covariation across the hand. In addition, we used the methods of principal angles similar to our earlier procedure with the neural manifolds. Specifically, we computed the SVD of the dot product between the k dimensional synergy subspaces of any two movements a and b as:

P_{a} V P_{b}^{⊤} = S V D (S_{a}^{k^{⊤}} S_{b}^{k})

where $S$ is a diagonal matrix whose entries are the cosine of the k ordered principal angles (smallest to largest) i.e.,

S = d i a g (\cos (θ_{1}), \cos (θ_{2}) \dots \cos (θ_{k}))

P_a and P_b of size $R^{k \times k}$ are the ordered principal directions i.e., directions within the k dimensional synergy subspace of movements a and b that have the associated principal angles between them. Using the SVD, we computed the principal angles pairwise between the synergy subspaces of all movements. To construct the null distribution of principal angles, we turned to the tensor maximum entropy method (TME, (Elsayed and Cunningham, 2017)) to simulate null tensors of kinematic data without joint position covariance while respecting all other 1^st and 2^nd order statistics of the data. After simulating a tensor of data, we applied PCA to identify the k dimensional synergy subspace in the simulated data and evaluated the principal angles between the simulated synergy subspaces. By iterating through this procedure 1000 times, we were able to generate a null distribution of principal angles and test the hypothesis of a common multi-jointed kinematic subspace. Given that the exact number of trials and hence the time-samples of trial-concatenated data were not similar movement to movement, we periodically subsampled trials when running the TME code to ensure that the 3D tensor was consistent.

Kinematically aligned submanifold analyses

Canonical correlation analysis (CCA)

To understand kinematically aligned submanifolds, we used canonical correlation analyses (CCA) that is part of a family of low-rank linear regression methods, such as reduced rank regression, partial least squares etc. CCA was performed for each movement individually wherein time-varying latent neural dynamics within the movement’s 45D manifold ( $X_{a} \in R^{t \times 45}$ ) was aligned to its top k kinematic synergies ( $X_{b} \in R^{t \times k}$ , where k = 3 as used three synergies in this study). The main objective of CCA is to uncover pairs of neural ( $w_{a} \in R^{45 \times k}$ ) and kinematic CCA modes ( $w_{b} \in R^{k \times k}$ ) such that the time-dependent activation of the modes generates maximally correlated pairs of low-dimensional neural and kinematic trajectories

z_{a} = X_{a} w_{a},

z_{b} = X_{b} w_{b}

The CCA neural modes therefore parcellate the larger neural subspace into a submanifold that captures kinematically-relevant neural dynamics. Note that while the pairwise correlation between the neural and kinematic trajectories is maximized, the trajectories of any given pair are orthogonal to other trajectory pairs:

z_{a}^{⊤} z_{a} = I \in R^{k \times k}

z_{b}^{⊤} z_{b} = I \in R^{k \times k}

thereby uncovering distinct aspects of control. The pairs of CCA modes are arranged in decreasing order of correlation between their projections. The dimensionality of the CCA modes is equal to the dataset with the lower dimensionality which in our case was always the synergy data.

The solution for CCA, i.e., w_a and w_b, can be obtained in a number of ways and details on the proofs can be found elsewhere (Uurtio et al., 2017); here we utilized a solver based on the singular value decomposition (Ewerbring, 1990) that allows for an implementation with robust statistics, as follows.

First, the sample auto and cross covariance matrices are computed: $C_{a a} = \frac{1}{n - 1} X_{a}^{⊤} X_{a}$ , $C_{b b} = \frac{1}{n - 1} X_{b}^{⊤} X_{b}$ and $C_{a b} = \frac{1}{n - 1} X_{a}^{⊤} X_{b}$ .
Maronna’s M-estimators can be used to determine robust versions of the above covariance matrices, especially $C_{a b}$ to correct for homoscedasticity and outliers automatically, thereby producing a robust version of CCA (Maronna, 1976, Maronna and Zamar, 2002).
Next, the square root or Cholesky factors of $C_{a a}$ and $C_{b b}$ are computed, $C_{a a}^{\frac{1}{2}}$ and $C_{b b}^{\frac{1}{2}}$ respectively.
Define a new matrix $A = C_{a a}^{- \frac{T}{2}} C_{a b} C_{b b}^{- \frac{1}{2}}$
Decompose $A = U S V^{⊤}$ using the SVD algorithm
The modes are then given as $w_{a} = C_{a a}^{\frac{- 1}{2}} U$ and by $w_{b} = C_{b b}^{\frac{- 1}{2}} V$
Correlations between pairwise of projections of $X_{a}$ and $X_{b} 〈 X_{a} w_{a}^{r}, X_{b} w_{b}^{r} 〉$ , $\forall r = 1, 2, \dots \min (k, c h)$ , is given by the singular values $s^{r}$ from the $r^{t h}$ diagonal entry of S.

To assess the predictive power of CCA, we used 10-fold cross validation, wherein within each fold 70% of the dataset was partitioned for training and the remaining 30% was partitioned for testing. The weight matrices $w_{a}^{t r a i n}$ , $w_{b}^{t r a i n}$ obtained from training the CCA model on the training data $X_{a}^{t r a i n}$ , $X_{b}^{t r a i n}$ were applied to the testing data to create cross validated trajectories $Z_{a}^{t e s t} = X_{a}^{t e s t} w_{a}^{t r a i n}$ and $z_{b}^{t e s t} = X_{b}^{t e s t} w_{b}^{t r a i n}$ . The cross validated correlation was computed as $d i a g (z_{a}^{{t e s t}^{⊤}} z_{b}^{t e s t})$ to obtain cross-validated r_i values for each CCA trajectory $i = 1, 2 \dots k$ . To assess the significance of the correlation values, we used a permutation procedure commonly used in regression-based analyses. Specifically, we broke the temporal relationship between $X_{a}^{t e s t}$ and $X_{b}^{t e s t}$ by randomly permuting the time-course of each synergy activation in $X_{b}^{t e s t}$ . These permutation procedures were done 1000 times prior to computing the correlation between $z_{a}^{t e s t}$ and $z_{b}^{t e s t}$ to get a null distribution of r_i values. Significance was assessed by counting the proportion of null samples that exceeded the true value of $r_{i}^{2}$ , with multiple comparison correction via the False Discovery Rate (FDR) procedure (Benjamini and Hochberg, 1995).

Distinctiveness of aligned submanifolds

To understand the distinctiveness of the aligned submanifolds, we used the method of principal angles to contrast movements’ submanifolds in a pairwise manner. The first step involves finding an orthonormal basis for the column space of the submanifold by applying the QR decomposition; unlike PCA the CCA neural modes are not constrained to be orthonormal even though the CCA trajectories are orthogonal. We then computed the dot product matrix between the orthonormal bases of any two movements and applied the SVD to obtain principal angles. Specifically,

P_{a} S P_{b} = S V D (Q_{a}^{⊤} Q_{b})

where $Q_{a}$ is the basis of the submanifold for movement a, $Q_{a}$ is the basis of the submanifold for movement b and S is a diagonal matrix whose entries are the cosine of the k ordered principal angles (smallest to largest) i.e.,

S = d i a g (\cos (θ_{1}), \cos (θ_{2}) \dots \cos (θ_{k}))

where P_a and P_b of size $R^{k \times k}$ are linear combinations of the k orthonormal basis vectors of each movement’s submanifold respectively. The resultant combinations of the basis vectors constitute the principal directions within each manifold. We computed the principal angles between all pairs of aligned neural submanifolds across participants. Given the 8 movements in the study, there are 28 such pairwise comparisons per participant. Under the null hypothesis, we would not expect a difference between the observed principal angles between movements from a null distribution of principal angles. To construct the null distribution of principal angles, we turned to the tensor maximum entropy method (TME, (Elsayed and Cunningham, 2017)). Each participant’s neural data can be considered as a 3D tensor of dimensions $R^{c h \times t \times m v m t}$ . The first two dimensions are channels and time samples and the third dimension represents the 8 hand movements in the study. We used the TME method to simulate null tensors of data while respecting the first and second order moments (i.e., mean and covariance) along every tensor. Given that the exact number of trials and hence the time-samples of trial-concatenated data were not similar movement to movement, we periodically subsampled trials when running the TME code to ensure that the 3D tensor was consistent. We then used CCA to align the simulated neural data to the synergy activations and thereby identified the null distribution of principal angles between the simulated CCA aligned neural manifolds. If two aligned manifolds were not similar to each other, then their principal angles would not be distinct from the null distribution of principal angles, assessed at the α = 0.01 level. We carried the above analyses to understand the distinctiveness of the aligned submanifold analyses for both LFOs and $γ_{H}^{L F O}$ in the grasp network.

Compartmentalization of behaviorally-relevant neural dynamics by the submanifolds

To understand whether the kinematically relevant neural dynamics were compartmentalized by the submanifolds for different movements, we projected data from one movement onto the submanifold of another movement as follows. Let $X_{a}^{(1)}$ and $X_{b}^{(1)}$ be the neural and synergy activation data for movement 1, and $X_{a}^{(2)}$ and $X_{b}^{(2)}$ be the neural and synergy activation data for movement 2. The corresponding CCA neural and kinematic modes for both movements are $w_{a}^{(1)}$ , $w_{b}^{(1)}$ and $w_{a}^{(2)}$ , $w_{b}^{(2)}$ respectively. By swapping CCA neural modes, we generate new low-dimensional neural trajectories $z_{a}^{(1) s w a p} = X_{a}^{(1)} w_{a}^{(2)}$ while the preserved neural trajectories remain to be $z_{a}^{(1)} = X_{a}^{(1)} w_{a}^{(1)}$ . As the objective function of CCA is to align low-dimensional neural dynamics with synergy admixtures, we evaluated the result of swapping CCA neural modes on kinematic correlation $< z_{a}^{(1) s w a p}$ , $z_{b}^{(1)} >$ , where $z_{b}^{(1)} = X_{b}^{(1)} w_{b}^{(1)}$ . We performed such pairwise mode swapping between all movements and within each subject. We compared the kinematic correlation for swapped and preserved modes to evaluate the generalizability of the CCA neural modes. To ensure the robustness of our results, we also projected data onto the column space of the submanifold rather than just swapping CCA neural modes. In this scenario, $z_{a}^{(1) s w a p} = X_{a}^{(1)} w_{a}^{(2)} {(w_{a}^{{(2)}^{⊤}} w_{a}^{(2)})}^{- 1} w_{a}^{{(2)}^{⊤}} w_{a}^{(1)}$ , explicitly testing how the range of the twomanifolds are related to each other.

Multiplexing of synergies

In our CCA formulation, there are some interesting properties of the CCA kinematic mode or kinematic mixing matrix w_b. First, note that if the synergy data are z-scored in the CCA model (making the variance of all synergies to be equal to one), then the covariance of X_b is essentially the identity matrix $I \in R^{k \times k}$ as the synergies by definition are orthogonal to each other as a consequence of the PCA step. Given that the columns of z_b are also orthogonal to each other, this leads to the kinematic mixing matrix $w_{b}$ to be a square orthogonal rotation matrix of dimension $R^{k \times k}$ . To see this, substitute $z_{b} = X_{b} w_{b}$ in the CCA constraint $Z_{b}^{⊤} Z_{b} = I$ , leading to

w_{b}^{⊤} X_{b}^{⊤} X_{b} w_{b} = I,

resulting in w_{b}^{⊤} w_{b} = I .

Moreover, the coefficients of $w_{b} (w_{b} (l, m)$ in the l^th row and m^th column) are nothing but the pairwise correlations between the synergy activations $X_{b}^{l}$ and CCA kinematic trajectories $z_{b}^{m}$ , $\forall l, m = 1, 2, \dots k$ . More specifically,

X_{b}^{⊤} z_{b} = X_{b}^{⊤} X_{b} w_{b} = w_{b}

Hence the coefficients of each column of w_b correspond to the admixture of synergies that generates a single CCA kinematic trajectory. Therefore, the more that w_b resembles the identity matrix, the lesser the admixture of synergies, as each CCA trajectory is better correlated to the activation of an individual synergy. Alternatively, higher off-diagonal magnitudes indicate that a CCA trajectory is correlated to the activations of multiple synergies. We can thus evaluate the squared correlation terms in the diagonal and off-diagonal elements of w_b to understand how the admixture of each movement’s synergies is cortically encoded by LFOs.

Spatial map of the CCA neural modes

As we had aligned latent neural dynamics to synergies, the CCA neural modes correspond to a linear combination of the axes of the manifold i.e., $X_{a} w_{a}$ in the input side of the CCA model can be rewritten as $D_{a} P C_{a} w_{a}$ wherein $D_{a}$ is the original data matrix in high dimensional channel space, $P C_{a}$ are the PCs of $D_{a}$ that defined the 45D neural manifold and w_a are the CCA neural modes. We had earlier shown the PCs to span a common multi-areal neural subspace; the submanifolds defined by linear combinations of these PCs will thus also span the same distributed network. Could such a distributed network persist if we were to align high-dimensional neural data directly to synergies without the intermediate step of projecting onto the neural manifold? Such a finding would suggest that the aligned manifold that covaries with kinematic synergies spans a common distributed network. To this end, we re-ran the CCA analyses where high-dimensional channel data were directly mapped to k most significant synergies. To identify the cortical network of channels associated with each kinematically aligned manifold, we focused on the channel weight within each CCA neural mode and sought to identify the most significant channels within each CCA neural mode based on the weight magnitude.

To verify that the weights assigned to the LFO channels in the CCA neural modes are meaningful, we measured the drop in cross-validated CCA $r_{i}^{2}$ values between each neural and kinematic trajectory when either the channels with the smallest weights were dropped first (ascending order of weight magnitude) or when channels with the largest weights were dropped first (descending order of weight magnitude). To assess the weight of a LFO channel in the neural mode of a given hand movement, and considering the fact that both the neural and synergy datasets consist of quasi-periodic oscillations, we modeled each channel’s LFO oscillation (0.5–4Hz filtered activity) as an p^th order auto-regressive (AR) process (Hayes, 1996), given by

L F O_{t}^{c h} = β_{0} + \sum_{i = 1}^{p} β_{i} L F O_{t - i}^{c h} + ϵ_{t}

Here, each channel’s LFO time-course is modeled as the output of filtering zero-mean unit variance gaussian white-noise $ϵ_{t} \in N (0, 1)$ through an all-pole filter. The β terms are the coefficients of the AR process and given that data were mean-centered, the β₀ term was zero. We estimated the coefficients of the AR process using the forward-backward least squares approach. To determine the appropriate model order, we computed the Bayesian Information Criterion (BIC) for model orders ranging from 1 to 100 and chose the order that gave the lowest BIC. We were then able to simulate LFOs at any given channel, for any hand movement and participant by simply filtering generated white noise through the AR filter. A run-off time was included to overcome the lag effect from the model order. In this manner, we were able to simulate artificial, randomly generated LFOs and their envelopes that had the same spectral and energy characteristics as measured LFOs. By using CCA to align simulated LFO data to synergy activations, a null distribution of weight magnitudes were generated. The significance of channel weights within each CCA neural mode $j = 1, 2 \dots k$ was measured by comparing the weight magnitude in $w_{a}^{j}$ to the null distribution of weight magnitudes from 2000 AR simulations. The FDR procedure was used for multiple comparison correction given that a statistical test was performed at each channel. Significant channels from each neural mode were pooled together to create a final binary mask of significant channels over the ECoG grid. This procedure was carried for each participant and movement to identify each aligned manifold’s cortical network of channels.

The mask of significant channels from each participant and movement was projected onto the cortical surface using 3D gaussian functions centered around each electrode, with a spreading parameter of 1cm (Hamilton et al., 2017), resulting in LFO cortical channel density maps. The cosine distance metric was used measure the similarity between movements’ pairwise cortical density maps, resulting in a similarity matrix per participant. The similarity matrices were then averaged, and hierarchical clustering was performed on the average similarity matrix using Ward’s criterion. If there was a true similarity between movements’ cortical density maps i.e., a significant cluster, then the lowest level that any two movements merge would be much smaller than the final aggregate cluster distance. We evaluated this hypothesis by comparing the ratio of the highest to lowest cluster distance to a null distribution of similar ratios obtained by shuffling movement labels within each participant for 1000 iterations prior to averaging and clustering.

We also defined rough Brodmann Area-based regions of interest (ROI) identifying primary motor cortex (M1), sensory cortex (S1), ventral and dorsal premotor cortices (PMv, PMd), parietal cortex (PPC) and supramarginal gyrus (SMG) and quantified the proportion of significant channels within each ROI. Each participant’s MRI was also warped to the MNI average template brain (Hamilton et al., 2017); this aided in projecting significant individual participant electrodes onto the average brain template as a density map by convolving each significant electrode with a 3D Gaussian function (spreading parameter of 1cm). The MRI of right hemispheric patients was mirrored so that the left hemisphere was always the common hemisphere.

Quantification and Statistical Analysis

Wherever appropriate, we employed parametric tests of difference of means using mixed effect models and non-parametric tests of significance using permutation tests. Confidence intervals were computed using the bootstrapping procedure. All the statistical tests utilized in this study are detailed in the text of the Results section in the main paper and expanded upon in the STAR ’methods details’ section for each type of analysis. Central measures (mean or median) as well measures of data spread (S.D., C.I.) are reported in the Results section and Method details for the relevant analysis and figures. Non-parametric tests of significance make no assumptions on the distribution of data; for parametric models, we did not perform tests to evaluate whether the data fit the assumptions of the statistical model. However, we confirmed the findings of parametric statistical models via equivalent permutation or shuffling tests. All statistical analyses were performed in Matlab (Mathworks Inc.).

Supplementary Material

NIHMS1751787-supplement-2.pdf^{(58.1MB, pdf)}

KEY RESOURCES TABLE

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Antibodies





Bacterial and virus strains





Biological samples





Chemicals, peptides, and recombinant proteins





Critical commercial assays





Deposited data
ECoG Recordings	This paper	N/A
Kinematic Recordings	This paper	N/A



Experimental models: Cell lines





Experimental models: Organisms/strains






Oligonucleotides





Recombinant DNA





Software and algorithms
MATLAB 2020b	MathWorks	https://www.mathworks.com/
Poser 11	Bondware	https://www.posersoftware.com/
demixed Principal Component Analysis	(Kobak et al., 2016)	https://github.com/machenslab/dPCA
Tensor Maximum Entropy	(Elsayed and Cunningham, 2017)	https://github.com/gamaleldin/TME
ECoG Cortical Visualization	(Hamilton et al., 2017)	https://github.com/libertyh/img_pipe/tree/matlab
LeapMotion SDK	LeapMotion	https://developer.leapmotion.com/
LeapMotion to Matlab Interface	LeapMotion	https://github.com/jeffsp/matleap
Other
RZ2 BioAmp Processor	Tucker-Davis Technologies	http://www.tdt.com/
PZ5M-512 Neurological Amplifier	Tucker-Davis Technologies	http://www.tdt.com/

Open in a new tab

Highlights.

We evaluated how the human brain supports a repertoire of finger and grasping actions

A common mesoscale “grasp-network” manifold represented all hand movements

However, latent neural dynamics within the manifold were specific to movement-type

Kinematically-relevant dynamics were compartmentalized into distinct submanifolds

Acknowledgements

We thank Kate Derosier, Reza Abiri and Preeya Khanna for valuable feedback and Sarah Seko for help with kinematic data, specifically for nonlinear inverse kinematic analyses. This work was supported by: Grant 2013101 from the Doris Duke Charitable Foundation, the National Institutes of Health through the NIH Director’s New Innovator Award [1 DP2 HD087955], the Weill Neurohub, and the National Institute of Neurological Disorders and Stroke (K02NS093014).

Footnotes

Declaration of interests

The authors declare no competing interests.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

ACHARYA S, FIFER MS, BENZ HL, CRONE NE & THAKOR NV 2010. Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand. Journal of neural engineering, 7, 046002. [DOI] [PMC free article] [PubMed] [Google Scholar]
AGASHE HA, PAEK AY, ZHANG Y & CONTRERAS-VIDAL JL 2015. Global cortical activity predicts shape of hand during grasping. Frontiers in neuroscience, 9, 121. [DOI] [PMC free article] [PubMed] [Google Scholar]
ATHALYE VR, GANGULY K, COSTA RM & CARMENA JM 2017. Emergence of coordinated neural dynamics underlies neuroprosthetic learning and skillful control. Neuron, 93, 955–970. e5. [DOI] [PubMed] [Google Scholar]
BANSAL AK, VARGAS-IRWIN CE, TRUCCOLO W & DONOGHUE JP 2011. Relationships among low-frequency local field potentials, spiking activity, and three-dimensional reach and grasp kinematics in primary motor and ventral premotor cortices. Journal of neurophysiology, 105, 1603–1619. [DOI] [PMC free article] [PubMed] [Google Scholar]
BEISTEINER R, WINDISCHBERGER C, LANZENBERGER R, EDWARD V, CUNNINGTON R, ERDLER M, GARTUS A, STREIBL B, MOSER E & DEECKE L 2001. Finger somatotopy in human motor cortex. Neuroimage, 13, 1016–1026. [DOI] [PubMed] [Google Scholar]
BENABID AL, COSTECALDE T, ELISEYEV A, CHARVET G, VERNEY A, KARAKAS S, FOERSTER M, LAMBERT A, MORINIÈRE B & ABROUG N 2019. An exoskeleton controlled by an epidural wireless brain–machine interface in a tetraplegic patient: a proof-of-concept demonstration. The Lancet Neurology, 18, 1112–1122. [DOI] [PubMed] [Google Scholar]
BENJAMINI Y & HOCHBERG Y 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57, 289–300. [Google Scholar]
BERENS P 2009. CircStat: a MATLAB toolbox for circular statistics. J Stat Softw, 31, 1–21. [Google Scholar]
BISHOP CM 2006. Pattern recognition and machine learning, springer. [Google Scholar]
BJÖRCK Ȧ & GOLUB GH 1973. Numerical methods for computing angles between linear subspaces. Mathematics of computation, 27, 579–594. [Google Scholar]
BORRA E, GERBELLA M, ROZZI S & LUPPINO G 2017. The macaque lateral grasping network: a neural substrate for generating purposeful hand actions. Neuroscience & Biobehavioral Reviews, 75, 65–90. [DOI] [PubMed] [Google Scholar]
BOUCHARD KE, MESGARANI N, JOHNSON K & CHANG EF 2013. Functional organization of human sensorimotor cortex for speech articulation. Nature, 495, 327–332. [DOI] [PMC free article] [PubMed] [Google Scholar]
BRIGGMAN KL, ABARBANEL HD & KRISTAN WB 2005. Optical imaging of neuronal populations during decision-making. Science, 307, 896–901. [DOI] [PubMed] [Google Scholar]
BROCHIER T & UMILTÀ MA 2007. Cortical control of grasp in non-human primates. Current opinion in neurobiology, 17, 637–643. [DOI] [PubMed] [Google Scholar]
BUXBAUM LJ, KYLE KM, TANG K & DETRE JA 2006. Neural substrates of knowledge of hand postures for object grasping and functional object use: Evidence from fMRI. Brain research, 1117, 175–185. [DOI] [PubMed] [Google Scholar]
BUYS E, LEMON R, MANTEL G & MUIR R 1986. Selective facilitation of different hand muscles by single corticospinal neurones in the conscious monkey. The Journal of physiology, 381, 529–549. [DOI] [PMC free article] [PubMed] [Google Scholar]
CANOLTY RT., CADIEU CF., KOEPSELL K., GANGULY K., KNIGHT RT. & CARMENA JM. 2012. Detecting event-related changes of multivariate phase coupling in dynamic brain networks. Journal of neurophysiology, 107, 2020–2031. [DOI] [PMC free article] [PubMed] [Google Scholar]
CANOLTY RT, CADIEU CF, KOEPSELL K, KNIGHT RT & CARMENA JM 2011. Multivariate phase–amplitude cross-frequency coupling in neurophysiological signals. IEEE Transactions on Biomedical Engineering, 59, 8–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
CANOLTY RT, EDWARDS E, DALAL SS, SOLTANI M, NAGARAJAN SS, KIRSCH HE, BERGER MS, BARBARO NM & KNIGHT RT 2006. High gamma power is phase-locked to theta oscillations in human neocortex. science, 313, 1626–1628. [DOI] [PMC free article] [PubMed] [Google Scholar]
CASTIELLO U 2005. The neuroscience of grasping. Nature Reviews Neuroscience, 6, 726–736. [DOI] [PubMed] [Google Scholar]
CHANG EF 2015. Towards large-scale, human-based, mesoscopic neurotechnologies. Neuron, 86, 68–78. [DOI] [PMC free article] [PubMed] [Google Scholar]
CHANGEUX J-P, GOULAS A & HILGETAG CC 2020. A Connectomic Hypothesis for the Hominization of the Brain. Cerebral Cortex. [DOI] [PMC free article] [PubMed]
CHURCHLAND MM, CUNNINGHAM JP, KAUFMAN MT, FOSTER JD, NUYUJUKIAN P, RYU SI & SHENOY KV 2012. Neural population dynamics during reaching. Nature, 487, 51–56. [DOI] [PMC free article] [PubMed] [Google Scholar]
COLEBATCH J, DEIBER M, PASSINGHAM R, FRISTON K & FRACKOWIAK R 1991. Regional cerebral blood flow during voluntary arm and hand movements in human subjects. Journal of neurophysiology, 65, 1392–1401. [DOI] [PubMed] [Google Scholar]
DAVARE M, KRASKOV A, ROTHWELL JC & LEMON RN 2011. Interactions between areas of the cortical grasping network. Current opinion in neurobiology, 21, 565–570. [DOI] [PMC free article] [PubMed] [Google Scholar]
DEGENHART AD, BISHOP WE, OBY ER, TYLER-KABARA EC, CHASE SM, BATISTA AP & BYRON MY 2020. Stabilization of a brain–computer interface via the alignment of low-dimensional spaces of neural activity. Nature Biomedical Engineering, 1–14. [DOI] [PMC free article] [PubMed]
EJAZ N, HAMADA M & DIEDRICHSEN J 2015. Hand use predicts the structure of representations in sensorimotor cortex. Nature neuroscience, 18, 1034. [DOI] [PubMed] [Google Scholar]
ELSAYED GF & CUNNINGHAM JP 2017. Structure in neural population recordings: an expected by product of simpler phenomena? Nature neuroscience, 20, 1310. [DOI] [PMC free article] [PubMed] [Google Scholar]
EWERBRING LM 1990. Canonical correlations and generalized SVD: applications and new algorithms. Advances in Parallel Computing. Elsevier. [Google Scholar]
FAGG AH & ARBIB MA 1998. Modeling parietal–premotor interactions in primate control of grasping. Neural Networks, 11, 1277–1303. [DOI] [PubMed] [Google Scholar]
FAISAL A, STOUT D, APEL J & BRADLEY B 2010. The manipulative complexity of Lower Paleolithic stone toolmaking. PloS one, 5. [DOI] [PMC free article] [PubMed] [Google Scholar]
FAN R-E, CHANG K-W, HSIEH C-J, WANG X-R & LIN C-J 2008. LIBLINEAR: A library for large linear classification. Journal of machine learning research, 9, 1871–1874. [Google Scholar]
FILIMON F 2010. Human cortical control of hand movements: parietofrontal networks for reaching, grasping, and pointing. The Neuroscientist, 16, 388–407. [DOI] [PubMed] [Google Scholar]
FLINT RD, ROSENOW JM, TATE MC & SLUTZKY MW 2016. Continuous decoding of human grasp kinematics using epidural and subdural signals. Journal of neural engineering, 14, 016005. [DOI] [PMC free article] [PubMed] [Google Scholar]
FLINT RD, TATE MC, LI K, TEMPLER JW, ROSENOW JM, PANDARINATH C & SLUTZKY MW 2020. The representation of finger movement and force in human motor and premotor cortices. Eneuro, 7. [DOI] [PMC free article] [PubMed] [Google Scholar]
GALLEGO JA, PERICH MG, MILLER LE & SOLLA SA 2017. Neural manifolds for the control of movement. Neuron, 94, 978–984. [DOI] [PMC free article] [PubMed] [Google Scholar]
GALLEGO JA, PERICH MG, NAUFEL SN, ETHIER C, SOLLA SA & MILLER LE 2018. Cortical population activity within a preserved neural manifold underlies multiple motor behaviors. Nature communications, 9, 1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
GANGULY K, SECUNDO L, RANADE G, ORSBORN A, CHANG EF, DIMITROV DF, WALLIS JD, BARBARO NM, KNIGHT RT & CARMENA JM 2009. Cortical representation of ipsilateral arm movements in monkey and man. Journal of Neuroscience, 29, 12948–12956. [DOI] [PMC free article] [PubMed] [Google Scholar]
GODLOVE J, GULATI T, DICHTER B, CHANG E & GANGULY K 2016. Muscle synergies after stroke are correlated with perilesional high gamma. Annals of clinical and translational neurology, 3, 956–961. [DOI] [PMC free article] [PubMed] [Google Scholar]
GRÈZES J, ARMONY JL, ROWE J & PASSINGHAM RE 2003. Activations related to “mirror” and “canonical” neurones in the human brain: an fMRI study. Neuroimage, 18, 928–937. [DOI] [PubMed] [Google Scholar]
HÄGER-ROSS C & SCHIEBER MH 2000. Quantifying the independence of human finger movements: comparisons of digits, hands, and movement frequencies. Journal of Neuroscience, 20, 8542–8550. [DOI] [PMC free article] [PubMed] [Google Scholar]
HALL TM, DE CARVALHO F & JACKSON A 2014. A common structure underlies low-frequency cortical dynamics in movement, sleep, and sedation. Neuron, 83, 1185–1199. [DOI] [PMC free article] [PubMed] [Google Scholar]
HAMILTON LS, CHANG DL, LEE MB & CHANG EF 2017. Semi-automated anatomical labeling and inter-subject warping of high-density intracranial recording electrodes in electrocorticography. Frontiers in neuroinformatics, 11, 62. [DOI] [PMC free article] [PubMed] [Google Scholar]
HATTORI N, SHIBASAKI H, WHEATON L, WU T, MATSUHASHI M & HALLETT M 2009. Discrete parieto-frontal functional connectivity related to grasping. Journal of neurophysiology, 101, 1267–1282. [DOI] [PMC free article] [PubMed] [Google Scholar]
HAYES MH 1996. Statistical digital signal processing and modeling.
HUMPHREY DR, SCHMIDT E & THOMPSON W 1970. Predicting measures of motor performance from multiple cortical spike trains. Science, 170, 758–762. [DOI] [PubMed] [Google Scholar]
INDOVINA I & SANES JN 2001. On somatotopic representation centers for finger movements in human primary motor cortex and supplementary motor area. Neuroimage, 13, 1027–1034. [DOI] [PubMed] [Google Scholar]
INGRAM JN, KÖRDING KP, HOWARD IS & WOLPERT DM 2008. The statistics of natural hand movements. Experimental brain research, 188, 223–236. [DOI] [PMC free article] [PubMed] [Google Scholar]
JAZAYERI M & AFRAZ A 2017. Navigating the neural space in search of the neural code. Neuron, 93, 1003–1014. [DOI] [PubMed] [Google Scholar]
JEANNEROD M, ARBIB MA, RIZZOLATTI G & SAKATA H 1995. Grasping objects: the cortical mechanisms of visuomotor transformation. Trends in neurosciences, 18, 314–320. [PubMed] [Google Scholar]
JERBI K., LACHAUX J-P., KARIM N., PANTAZIS D., LEAHY RM., GARNERO L. & BAILLET S. 2007. Coherent neural representation of hand speed in humans revealed by MEG imaging. Proceedings of the National Academy of Sciences, 104, 7676–7681. [DOI] [PMC free article] [PubMed] [Google Scholar]
JONAS M, SIEBNER H, BIERMANN-RUBEN K, KESSLER K, BÄUMER T, BÜCHEL C, SCHNITZLER A & MÜNCHAU A 2007. Do simple intransitive finger movements consistently activate frontoparietal mirror neuron areas in humans? Neuroimage, 36, T44–T53. [DOI] [PubMed] [Google Scholar]
KAUFMAN MT, SEELY JS, SUSSILLO D, RYU SI, SHENOY KV & CHURCHLAND MM 2016. The largest response component in the motor cortex reflects movement timing but not movement type. Eneuro, 3. [DOI] [PMC free article] [PubMed] [Google Scholar]
KHANNA P, TOTTEN D, NOVIK L, ROBERTS J, MORECRAFT RJ & GANGULY K 2021. Low-frequency stimulation enhances ensemble co-firing and dexterity after stroke. Cell, 184, 912–930. e20. [DOI] [PMC free article] [PubMed] [Google Scholar]
KIRSCH E, RIVLIS G & SCHIEBER MH 2014. Primary motor cortex neurons during individuated finger and wrist movements: correlation of spike firing rates with the motion of individual digits versus their principal components. Frontiers in neurology, 5, 70. [DOI] [PMC free article] [PubMed] [Google Scholar]
KOBAK D, BRENDEL W, CONSTANTINIDIS C, FEIERSTEIN CE, KEPECS A, MAINEN ZF, QI X-L, ROMO R, UCHIDA N & MACHENS CK 2016. Demixed principal component analysis of neural population data. Elife, 5, e10989. [DOI] [PMC free article] [PubMed] [Google Scholar]
KOCH G, CERCIGNANI M, PECCHIOLI C, VERSACE V, OLIVERI M, CALTAGIRONE C, ROTHWELL J & BOZZALI M 2010. In vivo definition of parieto-motor connections involved in planning of grasping movements. Neuroimage, 51, 300–312. [DOI] [PubMed] [Google Scholar]
KUBANEK J, MILLER KJ, OJEMANN JG, WOLPAW JR & SCHALK G 2009. Decoding flexion of individual fingers using electrocorticographic signals in humans. Journal of neural engineering, 6, 066001. [DOI] [PMC free article] [PubMed] [Google Scholar]
LEMKE SM, RAMANATHAN DS, GUO L, WON SJ & GANGULY K 2019. Emergent modular neural control drives coordinated motor actions. Nature neuroscience, 22, 1122–1131. [DOI] [PMC free article] [PubMed] [Google Scholar]
LEMON R 1988. The output map of the primate motor cortex. Trends in neurosciences, 11, 501–506. [DOI] [PubMed] [Google Scholar]
LEMON R 1993. Cortical control of the primate hand. Experimental Physiology, 78, 263–301. [DOI] [PubMed] [Google Scholar]
LEMON R, MUIR R & MANTEL G 1987. The effects upon the activity of hand and forearm muscles of intracortical stimulation in the vicinity of corticomotor neurones in the conscious monkey. Experimental brain research, 66, 621–637. [DOI] [PubMed] [Google Scholar]
LEMON RN 2008. Descending pathways in motor control. Annu. Rev. Neurosci, 31, 195–218. [DOI] [PubMed] [Google Scholar]
LEO A, HANDJARAS G, BIANCHI M, MARINO H, GABICCINI M, GUIDI A, SCILINGO EP, PIETRINI P, BICCHI A & SANTELLO M 2016. A synergy-based hand control is encoded in human motor cortical areas. Elife, 5, e13420. [DOI] [PMC free article] [PubMed] [Google Scholar]
MARCHENKO VA & PASTUR LA 1967. Distribution of eigenvalues for some sets of random matrices. Matematicheskii Sbornik, 114, 507–536. [Google Scholar]
MARONNA RA 1976. Robust M-estimators of multivariate location and scatter. The annals of statistics, 51–67.
MARONNA RA & ZAMAR RH 2002. Robust estimates of location and dispersion for high-dimensional datasets. Technometrics, 44, 307–317. [Google Scholar]
MASON CR, GOMEZ JE & EBNER TJ 2001. Hand synergies during reach-to-grasp. Journal of neurophysiology, 86, 2896–2910. [DOI] [PubMed] [Google Scholar]
MAYER A., BALDWIN MK., COOKE DF., LIMA BR., PADBERG J., LEWENFUS G., FRANCA JG. & KRUBITZER L. 2019. The multiple representations of complex digit movements in primary motor cortex form the building blocks for complex grip types in capuchin monkeys. Journal of Neuroscience, 39, 6684–6695. [DOI] [PMC free article] [PubMed] [Google Scholar]
MEYER CD 2000. Matrix analysis and applied linear algebra, Siam.
MICHAELS JA, SCHAFFELHOFER S, AGUDELO-TORO A & SCHERBERGER H 2020. A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping. Proceedings of the National Academy of Sciences, 117, 32124–32135. [DOI] [PMC free article] [PubMed] [Google Scholar]
MILLER KJ, LEUTHARDT EC, SCHALK G, RAO RP, ANDERSON NR, MORAN DW, MILLER JW & OJEMANN JG 2007. Spectral changes in cortical surface potentials during motor movement. Journal of Neuroscience, 27, 2424–2432. [DOI] [PMC free article] [PubMed] [Google Scholar]
MUIR R & LEMON R 1983. Corticospinal neurons with a special role in precision grip. Brain research, 261, 312–316. [DOI] [PubMed] [Google Scholar]
NAPIER J 1955. The form and function of the carpo-metacarpal joint of the thumb. Journal of Anatomy, 89, 362. [PMC free article] [PubMed] [Google Scholar]
NAPIER JR Studies of the hands of living primates. Proceedings of the Zoological Society of London, 1960. Wiley Online Library, 647–657. [Google Scholar]
PAEK AY, AGASHE H & CONTRERAS-VIDAL JL 2014. Decoding repetitive finger movements with brain activity acquired via non-invasive electroencephalography. Frontiers in neuroengineering, 7, 3. [DOI] [PMC free article] [PubMed] [Google Scholar]
PISTOHL T, BALL T, SCHULZE-BONHAGE A, AERTSEN A & MEHRING C 2008. Prediction of arm movement trajectories from ECoG-recordings in humans. Journal of neuroscience methods, 167, 105–114. [DOI] [PubMed] [Google Scholar]
PISTOHL T, SCHULZE-BONHAGE A, AERTSEN A, MEHRING C & BALL T 2012. Decoding natural grasp types from human ECoG. Neuroimage, 59, 248–260. [DOI] [PubMed] [Google Scholar]
RAMANATHAN DS, GUO L, GULATI T, DAVIDSON G, HISHINUMA AK, WON S-J, KNIGHT RT, CHANG EF, SWANSON RA & GANGULY K 2018. Low-frequency cortical activity is a neuromodulatory target that tracks recovery after stroke. Nature medicine, 24, 1257–1267. [DOI] [PMC free article] [PubMed] [Google Scholar]
RICKERT J, DE OLIVEIRA SC, VAADIA E, AERTSEN A, ROTTER S & MEHRING C 2005. Encoding of movement direction in different frequency ranges of motor cortical local field potentials. Journal of Neuroscience, 25, 8815–8824. [DOI] [PMC free article] [PubMed] [Google Scholar]
RIZZOLATTI G & LUPPINO G 2001. The cortical motor system. Neuron, 31, 889–901. [DOI] [PubMed] [Google Scholar]
ROUSE AG & SCHIEBER MH 2018. Condition-dependent neural dimensions progressively shift during reach to grasp. Cell reports, 25, 3158–3168. e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
ROUX F-E, BOETTO S, SACKO O, CHOLLET F & TRÉMOULET M 2003. Writing, calculating, and finger recognition in the region of the angular gyrus: a cortical stimulation study of Gerstmann syndrome. Journal of neurosurgery, 99, 716–727. [DOI] [PubMed] [Google Scholar]
RUSSO AA, BITTNER SR, PERKINS SM, SEELY JS, LONDON BM, LARA AH, MIRI A, MARSHALL NJ, KOHN A & JESSELL TM 2018. Motor cortex embeds muscle-like commands in an untangled population response. Neuron, 97, 953–966. e8. [DOI] [PMC free article] [PubMed] [Google Scholar]
RUSSO AA, KHAJEH R, BITTNER SR, PERKINS SM, CUNNINGHAM JP, ABBOTT LF & CHURCHLAND MM 2020. Neural trajectories in the supplementary motor area and motor cortex exhibit distinct geometries, compatible with different classes of computation. Neuron, 107, 745–758. e6. [DOI] [PMC free article] [PubMed] [Google Scholar]
SADTLER PT, QUICK KM, GOLUB MD, CHASE SM, RYU SI, TYLER-KABARA EC, BYRON MY & BATISTA AP 2014. Neural constraints on learning. Nature, 512, 423–426. [DOI] [PMC free article] [PubMed] [Google Scholar]
SANES JN, DONOGHUE JP, THANGARAJ V, EDELMAN RR & WARACH S 1995. Shared neural substrates controlling hand movements in human motor cortex. Science, 268, 1775–1777. [DOI] [PubMed] [Google Scholar]
SANI OG, ABBASPOURAZAD H, WONG YT, PESARAN B & SHANECHI MM 2020. Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification. Nature Publishing Group. [DOI] [PubMed]
SANTELLO M, BAUD-BOVY G & JÖRNTELL H 2013. Neural bases of hand synergies. Frontiers in computational neuroscience, 7, 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
SANTELLO M, FLANDERS M & SOECHTING JF 1998. Postural hand synergies for tool use. Journal of Neuroscience, 18, 10105–10115. [DOI] [PMC free article] [PubMed] [Google Scholar]
SANTHANAM G., YU BM., GILJA V., RYU SI., AFSHAR A., SAHANI M. & SHENOY KV. 2009. Factor-analysis methods for higher-performance neural prostheses. Journal of neurophysiology, 102, 1315–1330. [DOI] [PMC free article] [PubMed] [Google Scholar]
SCHAFFELHOFER S & SCHERBERGER H 2016. Object vision to hand action in macaque parietal, premotor, and motor cortices. Elife, 5, e15278. [DOI] [PMC free article] [PubMed] [Google Scholar]
SCHIEBER MH 1990. How might the motor cortex individuate movements? Trends in neurosciences, 13, 440–445. [DOI] [PubMed] [Google Scholar]
SCHIEBER MH 1995. Muscular production of individuated finger movements: the roles of extrinsic finger muscles. Journal of Neuroscience, 15, 284–297. [DOI] [PMC free article] [PubMed] [Google Scholar]
SCHIEBER MH & HIBBARD LS 1993. How somatotopic is the motor cortex hand area? Science, 261, 489–492. [DOI] [PubMed] [Google Scholar]
SEMEDO JD, ZANDVAKILI A, MACHENS CK, BYRON MY & KOHN A 2019. Cortical areas interact through a communication subspace. Neuron, 102, 249–259. e4. [DOI] [PMC free article] [PubMed] [Google Scholar]
SILVERSMITH DB, ABIRI R, HARDY NF, NATRAJ N, TU-CHAN A, CHANG EF & GANGULY K 2020. Plug-and-play control of a brain–computer interface through neural map stabilization. Nature Biotechnology, 1–10. [DOI] [PubMed]
STAVISKY SD, WILLETT FR, WILSON GH, MURPHY BA, REZAII P, AVANSINO DT, MEMBERG WD, MILLER JP, KIRSCH RF & HOCHBERG LR 2019. Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis. Elife, 8, e46015. [DOI] [PMC free article] [PubMed] [Google Scholar]
SURESH AK, GOODMAN JM, OKOROKOVA EV, KAUFMAN M, HATSOPOULOS NG & BENSMAIA SJ 2020. Neural population dynamics in motor cortex are different for reach and grasp. ELife, 9, e58848. [DOI] [PMC free article] [PubMed] [Google Scholar]
TAIRA M, MINE S, GEORGOPOULOS A, MURATA A & SAKATA H 1990. Parietal cortex neurons of the monkey related to the visual guidance of hand movement. Experimental brain research, 83, 29–36. [DOI] [PubMed] [Google Scholar]
THEODORIDIS S & KOUTROUMBAS K 2003. Pattern recognition, Elsevier. [Google Scholar]
TODOROV E & GHAHRAMANI Z Analysis of the synergies underlying complex hand manipulation. The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2004. IEEE, 4637–4640. [DOI] [PubMed] [Google Scholar]
TRUPPA V, SPINOZZI G, LAGANÀ T, PIANO MORTARI E & SABBATINI G 2016. Versatile grasping ability in power- grip actions by tufted capuchin monkeys (Sapajus spp.). American Journal of Physical Anthropology, 159, 63–72. [DOI] [PubMed] [Google Scholar]
TU-CHAN AP, NATRAJ N, GODLOVE J, ABRAMS G & GANGULY K 2017. Effects of somatosensory electrical stimulation on motor function and cortical oscillations. Journal of neuroengineering and rehabilitation, 14, 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
UURTIO V, MONTEIRO JM, KANDOLA J, SHAWE-TAYLOR J, FERNANDEZ-REYES D & ROUSU J 2017. A tutorial on canonical correlation methods. ACM Computing Surveys (CSUR), 50, 1–33. [Google Scholar]
VEUTHEY T, DEROSIER K, KONDAPAVULUR S & GANGULY K 2020. Single-trial cross-area neural population dynamics during long-term skill learning. Nature communications, 11, 1–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
WANG H-T, SMALLWOOD J, MOURAO-MIRANDA J, XIA CH, SATTERTHWAITE TD, BASSETT DS & BZDOK D 2020. Finding the needle in a high-dimensional haystack: Canonical correlation analysis for neuroscientists. NeuroImage, 116745. [DOI] [PubMed]
WHEATON LA, NOLTE G, BOHLHALTER S, FRIDMAN E & HALLETT M 2005. Synchronization of parietal and premotor areas during preparation and execution of praxis hand movements. Clinical Neurophysiology, 116, 1382–1390. [DOI] [PubMed] [Google Scholar]
YOUNG RW 2003. Evolution of the human hand: the role of throwing and clubbing. Journal of Anatomy, 202, 165–174. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

NIHMS1751787-supplement-2.pdf^{(58.1MB, pdf)}

Data Availability Statement

All data reported in this paper will be shared by the lead contact upon reasonable request.
The dPCA algorithm is publicly available (https://github.com/machenslab/dPCA) and the tensor maximum entropy algorithm (TME) is publicly available (https://github.com/gamaleldin/TME). Cortical visualization of ECoG activity was done using a publicly available package (https://github.com/libertyh/img_pipe/tree/matlab). Interfacing of LeapMotion to Matlab for real-time synchronized data acquisition of kinematic and neural data was done using publicly available packages (https://github.com/jeffsp/matleap) and via the LeapMotion SDK (https://developer.leapmotion.com/). All analyses were conducted in Matlab using previously published methods and hand visualization analyses was done using proprietary software (Poser, Bondware, https://www.posersoftware.com/). For modifications of publicly available code packages as pertaining to this study, for real-time data acquisition code synchronizing LeapMotion, Matlab and the ECoG TDT systems, for ECoG neural signal processing, for representational-structure based static analyses of grasp network activity, for neural manifold based analyses, for phase coupling analyses and for synergy based kinematic analyses, all enquiries will be fulfilled upon request to the lead contact of this study.
Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.

[R1] ACHARYA S, FIFER MS, BENZ HL, CRONE NE & THAKOR NV 2010. Electrocorticographic amplitude predicts finger positions during slow grasping motions of the hand. Journal of neural engineering, 7, 046002. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R2] AGASHE HA, PAEK AY, ZHANG Y & CONTRERAS-VIDAL JL 2015. Global cortical activity predicts shape of hand during grasping. Frontiers in neuroscience, 9, 121. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R3] ATHALYE VR, GANGULY K, COSTA RM & CARMENA JM 2017. Emergence of coordinated neural dynamics underlies neuroprosthetic learning and skillful control. Neuron, 93, 955–970. e5. [DOI] [PubMed] [Google Scholar]

[R4] BANSAL AK, VARGAS-IRWIN CE, TRUCCOLO W & DONOGHUE JP 2011. Relationships among low-frequency local field potentials, spiking activity, and three-dimensional reach and grasp kinematics in primary motor and ventral premotor cortices. Journal of neurophysiology, 105, 1603–1619. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R5] BEISTEINER R, WINDISCHBERGER C, LANZENBERGER R, EDWARD V, CUNNINGTON R, ERDLER M, GARTUS A, STREIBL B, MOSER E & DEECKE L 2001. Finger somatotopy in human motor cortex. Neuroimage, 13, 1016–1026. [DOI] [PubMed] [Google Scholar]

[R6] BENABID AL, COSTECALDE T, ELISEYEV A, CHARVET G, VERNEY A, KARAKAS S, FOERSTER M, LAMBERT A, MORINIÈRE B & ABROUG N 2019. An exoskeleton controlled by an epidural wireless brain–machine interface in a tetraplegic patient: a proof-of-concept demonstration. The Lancet Neurology, 18, 1112–1122. [DOI] [PubMed] [Google Scholar]

[R7] BENJAMINI Y & HOCHBERG Y 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57, 289–300. [Google Scholar]

[R8] BERENS P 2009. CircStat: a MATLAB toolbox for circular statistics. J Stat Softw, 31, 1–21. [Google Scholar]

[R9] BISHOP CM 2006. Pattern recognition and machine learning, springer. [Google Scholar]

[R10] BJÖRCK Ȧ & GOLUB GH 1973. Numerical methods for computing angles between linear subspaces. Mathematics of computation, 27, 579–594. [Google Scholar]

[R11] BORRA E, GERBELLA M, ROZZI S & LUPPINO G 2017. The macaque lateral grasping network: a neural substrate for generating purposeful hand actions. Neuroscience & Biobehavioral Reviews, 75, 65–90. [DOI] [PubMed] [Google Scholar]

[R12] BOUCHARD KE, MESGARANI N, JOHNSON K & CHANG EF 2013. Functional organization of human sensorimotor cortex for speech articulation. Nature, 495, 327–332. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R13] BRIGGMAN KL, ABARBANEL HD & KRISTAN WB 2005. Optical imaging of neuronal populations during decision-making. Science, 307, 896–901. [DOI] [PubMed] [Google Scholar]

[R14] BROCHIER T & UMILTÀ MA 2007. Cortical control of grasp in non-human primates. Current opinion in neurobiology, 17, 637–643. [DOI] [PubMed] [Google Scholar]

[R15] BUXBAUM LJ, KYLE KM, TANG K & DETRE JA 2006. Neural substrates of knowledge of hand postures for object grasping and functional object use: Evidence from fMRI. Brain research, 1117, 175–185. [DOI] [PubMed] [Google Scholar]

[R16] BUYS E, LEMON R, MANTEL G & MUIR R 1986. Selective facilitation of different hand muscles by single corticospinal neurones in the conscious monkey. The Journal of physiology, 381, 529–549. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R17] CANOLTY RT., CADIEU CF., KOEPSELL K., GANGULY K., KNIGHT RT. & CARMENA JM. 2012. Detecting event-related changes of multivariate phase coupling in dynamic brain networks. Journal of neurophysiology, 107, 2020–2031. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R18] CANOLTY RT, CADIEU CF, KOEPSELL K, KNIGHT RT & CARMENA JM 2011. Multivariate phase–amplitude cross-frequency coupling in neurophysiological signals. IEEE Transactions on Biomedical Engineering, 59, 8–11. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R19] CANOLTY RT, EDWARDS E, DALAL SS, SOLTANI M, NAGARAJAN SS, KIRSCH HE, BERGER MS, BARBARO NM & KNIGHT RT 2006. High gamma power is phase-locked to theta oscillations in human neocortex. science, 313, 1626–1628. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R20] CASTIELLO U 2005. The neuroscience of grasping. Nature Reviews Neuroscience, 6, 726–736. [DOI] [PubMed] [Google Scholar]

[R21] CHANG EF 2015. Towards large-scale, human-based, mesoscopic neurotechnologies. Neuron, 86, 68–78. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R22] CHANGEUX J-P, GOULAS A & HILGETAG CC 2020. A Connectomic Hypothesis for the Hominization of the Brain. Cerebral Cortex. [DOI] [PMC free article] [PubMed]

[R23] CHURCHLAND MM, CUNNINGHAM JP, KAUFMAN MT, FOSTER JD, NUYUJUKIAN P, RYU SI & SHENOY KV 2012. Neural population dynamics during reaching. Nature, 487, 51–56. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R24] COLEBATCH J, DEIBER M, PASSINGHAM R, FRISTON K & FRACKOWIAK R 1991. Regional cerebral blood flow during voluntary arm and hand movements in human subjects. Journal of neurophysiology, 65, 1392–1401. [DOI] [PubMed] [Google Scholar]

[R25] DAVARE M, KRASKOV A, ROTHWELL JC & LEMON RN 2011. Interactions between areas of the cortical grasping network. Current opinion in neurobiology, 21, 565–570. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R26] DEGENHART AD, BISHOP WE, OBY ER, TYLER-KABARA EC, CHASE SM, BATISTA AP & BYRON MY 2020. Stabilization of a brain–computer interface via the alignment of low-dimensional spaces of neural activity. Nature Biomedical Engineering, 1–14. [DOI] [PMC free article] [PubMed]

[R27] EJAZ N, HAMADA M & DIEDRICHSEN J 2015. Hand use predicts the structure of representations in sensorimotor cortex. Nature neuroscience, 18, 1034. [DOI] [PubMed] [Google Scholar]

[R28] ELSAYED GF & CUNNINGHAM JP 2017. Structure in neural population recordings: an expected by product of simpler phenomena? Nature neuroscience, 20, 1310. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R29] EWERBRING LM 1990. Canonical correlations and generalized SVD: applications and new algorithms. Advances in Parallel Computing. Elsevier. [Google Scholar]

[R30] FAGG AH & ARBIB MA 1998. Modeling parietal–premotor interactions in primate control of grasping. Neural Networks, 11, 1277–1303. [DOI] [PubMed] [Google Scholar]

[R31] FAISAL A, STOUT D, APEL J & BRADLEY B 2010. The manipulative complexity of Lower Paleolithic stone toolmaking. PloS one, 5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R32] FAN R-E, CHANG K-W, HSIEH C-J, WANG X-R & LIN C-J 2008. LIBLINEAR: A library for large linear classification. Journal of machine learning research, 9, 1871–1874. [Google Scholar]

[R33] FILIMON F 2010. Human cortical control of hand movements: parietofrontal networks for reaching, grasping, and pointing. The Neuroscientist, 16, 388–407. [DOI] [PubMed] [Google Scholar]

[R34] FLINT RD, ROSENOW JM, TATE MC & SLUTZKY MW 2016. Continuous decoding of human grasp kinematics using epidural and subdural signals. Journal of neural engineering, 14, 016005. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R35] FLINT RD, TATE MC, LI K, TEMPLER JW, ROSENOW JM, PANDARINATH C & SLUTZKY MW 2020. The representation of finger movement and force in human motor and premotor cortices. Eneuro, 7. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R36] GALLEGO JA, PERICH MG, MILLER LE & SOLLA SA 2017. Neural manifolds for the control of movement. Neuron, 94, 978–984. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R37] GALLEGO JA, PERICH MG, NAUFEL SN, ETHIER C, SOLLA SA & MILLER LE 2018. Cortical population activity within a preserved neural manifold underlies multiple motor behaviors. Nature communications, 9, 1–13. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R38] GANGULY K, SECUNDO L, RANADE G, ORSBORN A, CHANG EF, DIMITROV DF, WALLIS JD, BARBARO NM, KNIGHT RT & CARMENA JM 2009. Cortical representation of ipsilateral arm movements in monkey and man. Journal of Neuroscience, 29, 12948–12956. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R39] GODLOVE J, GULATI T, DICHTER B, CHANG E & GANGULY K 2016. Muscle synergies after stroke are correlated with perilesional high gamma. Annals of clinical and translational neurology, 3, 956–961. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R40] GRÈZES J, ARMONY JL, ROWE J & PASSINGHAM RE 2003. Activations related to “mirror” and “canonical” neurones in the human brain: an fMRI study. Neuroimage, 18, 928–937. [DOI] [PubMed] [Google Scholar]

[R41] HÄGER-ROSS C & SCHIEBER MH 2000. Quantifying the independence of human finger movements: comparisons of digits, hands, and movement frequencies. Journal of Neuroscience, 20, 8542–8550. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R42] HALL TM, DE CARVALHO F & JACKSON A 2014. A common structure underlies low-frequency cortical dynamics in movement, sleep, and sedation. Neuron, 83, 1185–1199. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R43] HAMILTON LS, CHANG DL, LEE MB & CHANG EF 2017. Semi-automated anatomical labeling and inter-subject warping of high-density intracranial recording electrodes in electrocorticography. Frontiers in neuroinformatics, 11, 62. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R44] HATTORI N, SHIBASAKI H, WHEATON L, WU T, MATSUHASHI M & HALLETT M 2009. Discrete parieto-frontal functional connectivity related to grasping. Journal of neurophysiology, 101, 1267–1282. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R45] HAYES MH 1996. Statistical digital signal processing and modeling.

[R46] HUMPHREY DR, SCHMIDT E & THOMPSON W 1970. Predicting measures of motor performance from multiple cortical spike trains. Science, 170, 758–762. [DOI] [PubMed] [Google Scholar]

[R47] INDOVINA I & SANES JN 2001. On somatotopic representation centers for finger movements in human primary motor cortex and supplementary motor area. Neuroimage, 13, 1027–1034. [DOI] [PubMed] [Google Scholar]

[R48] INGRAM JN, KÖRDING KP, HOWARD IS & WOLPERT DM 2008. The statistics of natural hand movements. Experimental brain research, 188, 223–236. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R49] JAZAYERI M & AFRAZ A 2017. Navigating the neural space in search of the neural code. Neuron, 93, 1003–1014. [DOI] [PubMed] [Google Scholar]

[R50] JEANNEROD M, ARBIB MA, RIZZOLATTI G & SAKATA H 1995. Grasping objects: the cortical mechanisms of visuomotor transformation. Trends in neurosciences, 18, 314–320. [PubMed] [Google Scholar]

[R51] JERBI K., LACHAUX J-P., KARIM N., PANTAZIS D., LEAHY RM., GARNERO L. & BAILLET S. 2007. Coherent neural representation of hand speed in humans revealed by MEG imaging. Proceedings of the National Academy of Sciences, 104, 7676–7681. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R52] JONAS M, SIEBNER H, BIERMANN-RUBEN K, KESSLER K, BÄUMER T, BÜCHEL C, SCHNITZLER A & MÜNCHAU A 2007. Do simple intransitive finger movements consistently activate frontoparietal mirror neuron areas in humans? Neuroimage, 36, T44–T53. [DOI] [PubMed] [Google Scholar]

[R53] KAUFMAN MT, SEELY JS, SUSSILLO D, RYU SI, SHENOY KV & CHURCHLAND MM 2016. The largest response component in the motor cortex reflects movement timing but not movement type. Eneuro, 3. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R54] KHANNA P, TOTTEN D, NOVIK L, ROBERTS J, MORECRAFT RJ & GANGULY K 2021. Low-frequency stimulation enhances ensemble co-firing and dexterity after stroke. Cell, 184, 912–930. e20. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R55] KIRSCH E, RIVLIS G & SCHIEBER MH 2014. Primary motor cortex neurons during individuated finger and wrist movements: correlation of spike firing rates with the motion of individual digits versus their principal components. Frontiers in neurology, 5, 70. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R56] KOBAK D, BRENDEL W, CONSTANTINIDIS C, FEIERSTEIN CE, KEPECS A, MAINEN ZF, QI X-L, ROMO R, UCHIDA N & MACHENS CK 2016. Demixed principal component analysis of neural population data. Elife, 5, e10989. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R57] KOCH G, CERCIGNANI M, PECCHIOLI C, VERSACE V, OLIVERI M, CALTAGIRONE C, ROTHWELL J & BOZZALI M 2010. In vivo definition of parieto-motor connections involved in planning of grasping movements. Neuroimage, 51, 300–312. [DOI] [PubMed] [Google Scholar]

[R58] KUBANEK J, MILLER KJ, OJEMANN JG, WOLPAW JR & SCHALK G 2009. Decoding flexion of individual fingers using electrocorticographic signals in humans. Journal of neural engineering, 6, 066001. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R59] LEMKE SM, RAMANATHAN DS, GUO L, WON SJ & GANGULY K 2019. Emergent modular neural control drives coordinated motor actions. Nature neuroscience, 22, 1122–1131. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R60] LEMON R 1988. The output map of the primate motor cortex. Trends in neurosciences, 11, 501–506. [DOI] [PubMed] [Google Scholar]

[R61] LEMON R 1993. Cortical control of the primate hand. Experimental Physiology, 78, 263–301. [DOI] [PubMed] [Google Scholar]

[R62] LEMON R, MUIR R & MANTEL G 1987. The effects upon the activity of hand and forearm muscles of intracortical stimulation in the vicinity of corticomotor neurones in the conscious monkey. Experimental brain research, 66, 621–637. [DOI] [PubMed] [Google Scholar]

[R63] LEMON RN 2008. Descending pathways in motor control. Annu. Rev. Neurosci, 31, 195–218. [DOI] [PubMed] [Google Scholar]

[R64] LEO A, HANDJARAS G, BIANCHI M, MARINO H, GABICCINI M, GUIDI A, SCILINGO EP, PIETRINI P, BICCHI A & SANTELLO M 2016. A synergy-based hand control is encoded in human motor cortical areas. Elife, 5, e13420. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R65] MARCHENKO VA & PASTUR LA 1967. Distribution of eigenvalues for some sets of random matrices. Matematicheskii Sbornik, 114, 507–536. [Google Scholar]

[R66] MARONNA RA 1976. Robust M-estimators of multivariate location and scatter. The annals of statistics, 51–67.

[R67] MARONNA RA & ZAMAR RH 2002. Robust estimates of location and dispersion for high-dimensional datasets. Technometrics, 44, 307–317. [Google Scholar]

[R68] MASON CR, GOMEZ JE & EBNER TJ 2001. Hand synergies during reach-to-grasp. Journal of neurophysiology, 86, 2896–2910. [DOI] [PubMed] [Google Scholar]

[R69] MAYER A., BALDWIN MK., COOKE DF., LIMA BR., PADBERG J., LEWENFUS G., FRANCA JG. & KRUBITZER L. 2019. The multiple representations of complex digit movements in primary motor cortex form the building blocks for complex grip types in capuchin monkeys. Journal of Neuroscience, 39, 6684–6695. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R70] MEYER CD 2000. Matrix analysis and applied linear algebra, Siam.

[R71] MICHAELS JA, SCHAFFELHOFER S, AGUDELO-TORO A & SCHERBERGER H 2020. A goal-driven modular neural network predicts parietofrontal neural dynamics during grasping. Proceedings of the National Academy of Sciences, 117, 32124–32135. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R72] MILLER KJ, LEUTHARDT EC, SCHALK G, RAO RP, ANDERSON NR, MORAN DW, MILLER JW & OJEMANN JG 2007. Spectral changes in cortical surface potentials during motor movement. Journal of Neuroscience, 27, 2424–2432. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R73] MUIR R & LEMON R 1983. Corticospinal neurons with a special role in precision grip. Brain research, 261, 312–316. [DOI] [PubMed] [Google Scholar]

[R74] NAPIER J 1955. The form and function of the carpo-metacarpal joint of the thumb. Journal of Anatomy, 89, 362. [PMC free article] [PubMed] [Google Scholar]

[R75] NAPIER JR Studies of the hands of living primates. Proceedings of the Zoological Society of London, 1960. Wiley Online Library, 647–657. [Google Scholar]

[R76] PAEK AY, AGASHE H & CONTRERAS-VIDAL JL 2014. Decoding repetitive finger movements with brain activity acquired via non-invasive electroencephalography. Frontiers in neuroengineering, 7, 3. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R77] PISTOHL T, BALL T, SCHULZE-BONHAGE A, AERTSEN A & MEHRING C 2008. Prediction of arm movement trajectories from ECoG-recordings in humans. Journal of neuroscience methods, 167, 105–114. [DOI] [PubMed] [Google Scholar]

[R78] PISTOHL T, SCHULZE-BONHAGE A, AERTSEN A, MEHRING C & BALL T 2012. Decoding natural grasp types from human ECoG. Neuroimage, 59, 248–260. [DOI] [PubMed] [Google Scholar]

[R79] RAMANATHAN DS, GUO L, GULATI T, DAVIDSON G, HISHINUMA AK, WON S-J, KNIGHT RT, CHANG EF, SWANSON RA & GANGULY K 2018. Low-frequency cortical activity is a neuromodulatory target that tracks recovery after stroke. Nature medicine, 24, 1257–1267. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R80] RICKERT J, DE OLIVEIRA SC, VAADIA E, AERTSEN A, ROTTER S & MEHRING C 2005. Encoding of movement direction in different frequency ranges of motor cortical local field potentials. Journal of Neuroscience, 25, 8815–8824. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R81] RIZZOLATTI G & LUPPINO G 2001. The cortical motor system. Neuron, 31, 889–901. [DOI] [PubMed] [Google Scholar]

[R82] ROUSE AG & SCHIEBER MH 2018. Condition-dependent neural dimensions progressively shift during reach to grasp. Cell reports, 25, 3158–3168. e3. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R83] ROUX F-E, BOETTO S, SACKO O, CHOLLET F & TRÉMOULET M 2003. Writing, calculating, and finger recognition in the region of the angular gyrus: a cortical stimulation study of Gerstmann syndrome. Journal of neurosurgery, 99, 716–727. [DOI] [PubMed] [Google Scholar]

[R84] RUSSO AA, BITTNER SR, PERKINS SM, SEELY JS, LONDON BM, LARA AH, MIRI A, MARSHALL NJ, KOHN A & JESSELL TM 2018. Motor cortex embeds muscle-like commands in an untangled population response. Neuron, 97, 953–966. e8. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R85] RUSSO AA, KHAJEH R, BITTNER SR, PERKINS SM, CUNNINGHAM JP, ABBOTT LF & CHURCHLAND MM 2020. Neural trajectories in the supplementary motor area and motor cortex exhibit distinct geometries, compatible with different classes of computation. Neuron, 107, 745–758. e6. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R86] SADTLER PT, QUICK KM, GOLUB MD, CHASE SM, RYU SI, TYLER-KABARA EC, BYRON MY & BATISTA AP 2014. Neural constraints on learning. Nature, 512, 423–426. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R87] SANES JN, DONOGHUE JP, THANGARAJ V, EDELMAN RR & WARACH S 1995. Shared neural substrates controlling hand movements in human motor cortex. Science, 268, 1775–1777. [DOI] [PubMed] [Google Scholar]

[R88] SANI OG, ABBASPOURAZAD H, WONG YT, PESARAN B & SHANECHI MM 2020. Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification. Nature Publishing Group. [DOI] [PubMed]

[R89] SANTELLO M, BAUD-BOVY G & JÖRNTELL H 2013. Neural bases of hand synergies. Frontiers in computational neuroscience, 7, 23. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R90] SANTELLO M, FLANDERS M & SOECHTING JF 1998. Postural hand synergies for tool use. Journal of Neuroscience, 18, 10105–10115. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R91] SANTHANAM G., YU BM., GILJA V., RYU SI., AFSHAR A., SAHANI M. & SHENOY KV. 2009. Factor-analysis methods for higher-performance neural prostheses. Journal of neurophysiology, 102, 1315–1330. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R92] SCHAFFELHOFER S & SCHERBERGER H 2016. Object vision to hand action in macaque parietal, premotor, and motor cortices. Elife, 5, e15278. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R93] SCHIEBER MH 1990. How might the motor cortex individuate movements? Trends in neurosciences, 13, 440–445. [DOI] [PubMed] [Google Scholar]

[R94] SCHIEBER MH 1995. Muscular production of individuated finger movements: the roles of extrinsic finger muscles. Journal of Neuroscience, 15, 284–297. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R95] SCHIEBER MH & HIBBARD LS 1993. How somatotopic is the motor cortex hand area? Science, 261, 489–492. [DOI] [PubMed] [Google Scholar]

[R96] SEMEDO JD, ZANDVAKILI A, MACHENS CK, BYRON MY & KOHN A 2019. Cortical areas interact through a communication subspace. Neuron, 102, 249–259. e4. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R97] SILVERSMITH DB, ABIRI R, HARDY NF, NATRAJ N, TU-CHAN A, CHANG EF & GANGULY K 2020. Plug-and-play control of a brain–computer interface through neural map stabilization. Nature Biotechnology, 1–10. [DOI] [PubMed]

[R98] STAVISKY SD, WILLETT FR, WILSON GH, MURPHY BA, REZAII P, AVANSINO DT, MEMBERG WD, MILLER JP, KIRSCH RF & HOCHBERG LR 2019. Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis. Elife, 8, e46015. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R99] SURESH AK, GOODMAN JM, OKOROKOVA EV, KAUFMAN M, HATSOPOULOS NG & BENSMAIA SJ 2020. Neural population dynamics in motor cortex are different for reach and grasp. ELife, 9, e58848. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R100] TAIRA M, MINE S, GEORGOPOULOS A, MURATA A & SAKATA H 1990. Parietal cortex neurons of the monkey related to the visual guidance of hand movement. Experimental brain research, 83, 29–36. [DOI] [PubMed] [Google Scholar]

[R101] THEODORIDIS S & KOUTROUMBAS K 2003. Pattern recognition, Elsevier. [Google Scholar]

[R102] TODOROV E & GHAHRAMANI Z Analysis of the synergies underlying complex hand manipulation. The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2004. IEEE, 4637–4640. [DOI] [PubMed] [Google Scholar]

[R103] TRUPPA V, SPINOZZI G, LAGANÀ T, PIANO MORTARI E & SABBATINI G 2016. Versatile grasping ability in power- grip actions by tufted capuchin monkeys (Sapajus spp.). American Journal of Physical Anthropology, 159, 63–72. [DOI] [PubMed] [Google Scholar]

[R104] TU-CHAN AP, NATRAJ N, GODLOVE J, ABRAMS G & GANGULY K 2017. Effects of somatosensory electrical stimulation on motor function and cortical oscillations. Journal of neuroengineering and rehabilitation, 14, 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R105] UURTIO V, MONTEIRO JM, KANDOLA J, SHAWE-TAYLOR J, FERNANDEZ-REYES D & ROUSU J 2017. A tutorial on canonical correlation methods. ACM Computing Surveys (CSUR), 50, 1–33. [Google Scholar]

[R106] VEUTHEY T, DEROSIER K, KONDAPAVULUR S & GANGULY K 2020. Single-trial cross-area neural population dynamics during long-term skill learning. Nature communications, 11, 1–15. [DOI] [PMC free article] [PubMed] [Google Scholar]

[R107] WANG H-T, SMALLWOOD J, MOURAO-MIRANDA J, XIA CH, SATTERTHWAITE TD, BASSETT DS & BZDOK D 2020. Finding the needle in a high-dimensional haystack: Canonical correlation analysis for neuroscientists. NeuroImage, 116745. [DOI] [PubMed]

[R108] WHEATON LA, NOLTE G, BOHLHALTER S, FRIDMAN E & HALLETT M 2005. Synchronization of parietal and premotor areas during preparation and execution of praxis hand movements. Clinical Neurophysiology, 116, 1382–1390. [DOI] [PubMed] [Google Scholar]

[R109] YOUNG RW 2003. Evolution of the human hand: the role of throwing and clubbing. Journal of Anatomy, 202, 165–174. [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Compartmentalized dynamics within a common multiarea mesoscale manifold represent a repertoire of human hand movements

Nikhilesh Natraj

Daniel B Silversmith

Edward F Chang

Karunesh Ganguly

Abstract

eTOC Blurb: In brief

Introduction

Fig. 1. Latent dynamics and neural manifolds for hand movement repertoire.

Results

Experimental design

Grasp network and low frequency oscillations

Fig. 2. Representational structure in mean, static activity across the grasp network.

The multi-areal neural manifold is preserved across movements

Fig. 3. Common multi-areal manifold represents the movement repertoire.

Latent dynamics within the common subspace are distinct for the movement repertoire

Fig. 4. Latent dynamics within the common manifold are distinct for the repertoire.

Manifolds and dynamics of high-gamma follow low-frequency oscillations

Fig. 5. LFO oscillations are phase-locked to high-gamma (γH) and share a common subspace with the low-frequency component of high gamma (γHLFO).

Kinematic recordings of the hand-movement repertoire

Fig. 6. Kinematic recordings of the hand movement repertoire.

Kinematically-aligned submanifolds

Fig. 7. Behaviorally-relevant dynamics are compartmentalized into distinct kinematically aligned submanifolds.

Fig. 8. Kinematically aligned submanifolds span the grasp network equally for both finger movements and grasping.

The aligned submanifolds span the grasp network equally

Discussion

A common low-dimensional mesoscale manifold for grasping and finger movements

Low frequency oscillations

Differences between single-area cortical spiking and distributed mesoscale dynamics

Summary

STAR Methods

Resource availability

Lead contact

Materials Availability

Data and code availability

Experimental model and subject details

Participants

Method details

Experimental design and data acquisition

Sample size estimation

ECoG signal processing

Movement related information in grasp-network LFO amplitudes

Representational structure in grasp-network LFO amplitudes

Phase-coupling of ECoG LFOs to kinematics

Neural manifold analyses

demixed Principal Component Analysis to evaluate latent dynamics

Relationship between LFOs and γH

Phase-coupling of ECoG LFO oscillations to the low frequency component of γH

Shared variance between the manifolds of LFOs and γHLFO

Kinematic recordings of the repertoire of human hand movements

Kinematically aligned submanifold analyses

Canonical correlation analysis (CCA)

Distinctiveness of aligned submanifolds

Compartmentalization of behaviorally-relevant neural dynamics by the submanifolds

Multiplexing of synergies

Spatial map of the CCA neural modes

Quantification and Statistical Analysis

Supplementary Material

Highlights.

Acknowledgements

Footnotes

References

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases

Fig. 5. LFO oscillations are phase-locked to high-gamma ( $γ_{H}$ ) and share a common subspace with the low-frequency component of high gamma ( $γ_{H}^{L F O}$ ).

Relationship between LFOs and $γ_{H}$

Phase-coupling of ECoG LFO oscillations to the low frequency component of $γ_{H}$

Shared variance between the manifolds of LFOs and $γ_{H}^{L F O}$