Abstract
Denervation due to amputation is known to induce cortical reorganization in the sensorimotor cortex. Although there is evidence that reorganization does not lead to a complete loss of the representation of the phantom limb, it is unclear to what extent detailed, finger-specific activation patterns are preserved in motor cortex, an issue which is also relevant for development of brain-computer interface solutions for paralyzed people. We applied machine learning to obtain a quantitative measure for the functional organisation within the motor and adjacent cortices in amputees, using high-resolution functional MRI and attempted hand gestures.
Subjects with above-elbow arm amputation (n = 8) and non-amputated controls (n = 9) made several gestures with either their right or left hand. Amputees attempted to make gestures with their amputated hand. Images were acquired using 7 tesla functional MRI. The sensorimotor cortex was divided into four regions, and activity patterns were classified in individual subjects using a support vector machine.
Classification scores were significantly above chance for all subjects and all hands, and were highly similar between amputees and controls in most regions. Decodability of phantom movements from primary motor cortex reached the levels of right hand movements in controls. Attempted movements were successfully decoded from primary sensory cortex in amputees, albeit lower than in controls but well above chance level despite absence of somatosensory feedback. There was no significant correlation between decodability and years since amputation, or age.
The ability to decode attempted gestures demonstrates that the detailed hand representation is preserved in motor cortex and adjacent regions after denervation. This encourages targeting sensorimotor activity patterns for development of brain-computer interfaces.
Keywords: amputee, sensorimotor cortex, fMRI, hand representation
Introduction
The sensorimotor areas of the human brain are somatotopically organized, with regions of the primary motor cortex (M1) and primary sensory cortex (S1) being associated with movement and sensory representations of various body parts. It has become clear that this somatotopic organization is quite detailed and that representations of individual fingers (Dechent and Frahm, 2003; Siero et al., 2014) and even separate muscles (Hadoush et al., 2011) can be identified.
Denervation due to amputation or nerve damage disrupts normal sensorimotor function. Subsequent cortical reorganization in the sensorimotor area has been reported in numerous animal studies, where intact body parts ‘invade’ areas associated with the missing limb (Donoghue and Sanes, 1987; Merzenich et al., 1984; Wu and Kaas, 1999). Sensorimotor reorganization occurs also in humans, as evidenced by transcranial magnetic stimulation studies in amputees describing increased excitability of motor areas contralateral to the amputated limb, where stump muscles demonstrate higher response amplitudes which can be induced from a larger scalp area than responses in the intact arm (Cohen et al., 1991; Röricht et al., 1999). Also, magnetoencephalography and fMRI studies with upper limb amputees have reported a shift of lip (Lotze et al., 2001), chin (Elbert et al., 1994), and shoulder (Dettmers et al., 2001) representation into the deafferented cortical hand area has been demonstrated.
Increasing evidence demonstrates that denervation does not result in a complete loss of representation of the affected limb, as the sensorimotor cortex still appears to be engaged in so-called ‘attempted movements’. When amputees attempt moving their phantom limb, the corresponding sensorimotor areas show fMRI activation similar to executed movements in able-bodied subjects (Lotze et al., 2001; Roux et al., 2003; Turner et al., 2001). Moreover, for postcentral and parietal regions, it has been shown that this persistent representation is relatively detailed. For example, in a tetraplegic patient using intracranial recordings, movement goals and trajectories have been successfully decoded from posterior parietal cortex (Aflalo et al., 2015). Using microstimulation, a persistent hand representation in S1 was also found in a long-term spinal cord injury patient (Flesher et al., 2016). In amputees, an individual finger topography of the phantom hand has been reported in the somatosensory cortex (Kikkert et al., 2016).
Although abovementioned studies provide evidence that a persistent hand representation still exists after denervation, they primarily focus on somatosensory areas, using simple (one-finger or coarse hand) movements. It is unknown whether the intact representation also applies to for example the primary motor cortex (M1), and to what extent it allows for the decoding of composite (multiple-finger) movements after denervation. This is not only relevant for understanding plasticity, but also for development of clinical brain-computer interface (BCI) solutions for severe paralysis, the feasibility of which was recently reported (Vansteensel et al., 2016).
We have previously shown that the sensorimotor organization in able-bodied subjects allows for a quantitative discrimination of four gestures from the American Manual Alphabet, based on electrocorticography and on 7T fMRI activation patterns (Bleichner et al., 2013; 2016). Gestures are especially suitable for testing persistent hand representations, because their differentiation constitutes a comprehensive evaluation of discrimination between spatial activity patterns. In the current 7T fMRI study, we investigated the discriminability of cortical representations of attempted gestures in arm amputees, to study and quantify the detailed integrity of the denervated sensorimotor cortex. Similar to our previous studies with able-bodied subjects (Bleichner et al., 2013; 2016), we used a machine learning method for ‘decoding’, which refers to identifying movements based on their cortical activation pattern. Such decoding is highly sensitive to the discriminability, hence spatial integrity, of cortical hand representations and can be used not only to reveal effects of denervation, but also to quantitatively compare discriminability to control subjects. Since all fingers are represented in the relatively small hand knob on M1 (Siero et al., 2014), any change in representation is likely to cause an increase in correlations between individual finger foci, resulting in reduced discriminability of different gestures and thus in a decline in decoding performance. Another advantage of a decoding approach is that it does not require an a priori model of the cortical organisation, which is known to be challenging especially in M1 (Graziano and Aflalo, 2007; Hluštík et al., 2001). We specifically investigated four regions of the sensorimotor system: the primary motor cortex, the primary sensory cortex, the anterior precentral gyrus, and the posterior postcentral gyrus.
Subjects/Materials and Methods
Subjects
Eight subjects with arm amputation were recruited (age 52 ± 12 years, 1 female). All subjects had transhumeral arm amputation (7 right arm, 1 left arm amputation), acquired 16.4 ± 11.5 years ago (range: 1.7–31.1 years ago). Nine control subjects were also recruited (no arm amputation, age 44 ± 21 years, 4 females). All subjects were right-handed or were right-handed before amputation according to the Edinburgh Handedness Inventory (Oldfield, 1971). An overview of all subjects is given in Table 1.
Table 1. Subject details.
Code | Sex | Age | Amputation side | Years since amputation [yrs] | Phantom paina | Reason for amputation | |
---|---|---|---|---|---|---|---|
Controls | C1 | m | 50 | ||||
C2 | f | 23 | |||||
C3 | m | 28 | |||||
C4 | m | 21 | |||||
C5 | f | 20 | |||||
C6 | f | 63 | |||||
C7 | m | 60 | |||||
C8 | f | 75 | |||||
C9 | m | 53 | |||||
Amputees | A1 | m | 52 | Right | 1.7 | 2 | traffic accident |
A2 | f | 67 | Right | 6.2 | 4 | cancer | |
A3 | m | 62 | Right | 37.1 | 1 | traffic accident | |
A4 | m | 60 | Right | 6.3 | 6 | cancer | |
A5 | m | 49 | Right | 21.8 | 8 | machine accident | |
A6 | m | 30 | Right | 23.1 | 0 | accident | |
A7 | m | 52 | Right | 17.1 | 2 | post-traumatic dystrophy | |
A8 | m | 40 | Left | 17.7 | 4 | machine accident |
On the day of scanning, subjects rated their momentary phantom pain on a scale from 0 (no pains) to 10 (heavy pains).
The study was approved by the medical-ethical committee of the University Medical Center Utrecht and all subjects gave their written informed consent in agreement with the declaration of Helsinki (2013).
Experimental design
Data acquisition
MRI data were recorded using a Philips Achieva 7T MRI system with a 32-channel head coil. Anatomical T1- and PD-weighted images were acquired first (TR/TE = 6/1.4 ms, FA = 8º, voxel size = 1 × 1 × 1 mm3). A functional Localizer task was performed to ensure the hand area was within the imaging field of view, followed by the Gesture task. Prism glasses allowed subjects to look at the screen located at the end of the scanner bore, on which the tasks were presented. Finger positions of both hands in control subjects and of the intact hand in amputees were recorded using MRI-compatible data gloves (5DT Inc., Irvine, USA).
Localizer task and analysis
Subjects were instructed to repeatedly open and close their hands during presentation of a green cue (‘move block’), and rest during a red cue (‘rest block’). Control subjects were asked to open and close both hands, while amputated subjects only used their intact hand. The task consisted of 3 rest blocks and 2 move blocks (30 seconds each).
EPI images were acquired (TR/TE = 2000/27 ms, FA = 70º, acquisition matrix size = 104 × 129, 33 slices, voxel size = 1.6 × 1.6 × 1.6 mm) during this task. The results were analyzed in real time using Philips IVIEWBOLD analysis software, and were used to optimally position the fMRI field of view for the Gesture task. For two subjects (C1 and C2), the Localizer task was not yet part of the protocol. For these two subjects, the positioning of the Gesture task EPI scans was based on the anatomical location of the hand knob in transversal and sagittal planes.
Gesture task
Six gestures, shown in Figure 1, were selected from the American Manual Alphabet. These gestures were chosen for maximum differences in flexion-extension combinations.
One of the six characters was presented every 15.6 seconds for subjects C1 and C2, and 16 seconds for all others. The character was presented for 6 seconds, followed by a fixation cross. Subjects were instructed to make the corresponding gesture as soon as a character appeared and hold it until the character disappeared. Subjects performed 4 runs (each with 60 trials; 10 per character): two with their right hand (runs ‘R1’ and ‘R2’) and two with their left hand (runs ‘L1’ and ‘L2’). Control subjects (C1-9) used executed movements, whereas amputated subjects (A1-8) used attempted movement with their phantom hand, and executed movements with their intact hand. All subjects were naive to sign language and practiced at home daily for 15 minutes during one week.
An EPI sequence was used (TR/TE = 1300/27 ms for subjects C1 and C2, TR/TE = 1600/27 ms for all other subjects, FA = 70º, acquisition matrix size = 104 × 129, 26 slices, no gap, voxel size = 1.6 × 1.6 × 1.6 mm3). All EPI scans were acquired in transversal orientation, such that both the left and right hand region were in the field of view.
Data analysis
FMRI preprocessing and Statistical maps
Preprocessing and first level analysis were performed with SPM12 (http://www.fil.ion.ucl.ac.uk/spm/) for each combined pair of runs (L1 with L2, and R1 with R2). Functional images were slice-time corrected, realigned to the mean functional image, and coregistered with the T1-weighted anatomical scan. For a group visualization, the data was smoothed using an 8 mm kernel and normalized to MNI using SPM12. For the decoding analysis, no normalization and no smoothing was applied, to preserve the detail required for classification. A design matrix was fitted using a general linear model (GLM), entering the two runs as separate sessions. Twelve regressors were used (one for each of the six gestures for each run), from which six t-maps for the six gestures were derived.
Regions of interest
Gestures were classified from four parallel regions of interest (ROIs) per hemisphere. The main focus was on the primary motor (M1) and primary sensory (S1) cortices, which were defined as the walls of the pre- and postcentral gyrus inside the central sulcus, as we know from our previous work that most informative voxels for classification are located in the central sulcus (Bleichner et al., 2013). To prevent the risk of missing information located futher from the central sulcus (Martuzzi et al., 2012), two additional ROIs were defined: the anterior part of the precentral gyrus (pre-M1), and the posterior part of the postcentral gyrus (post-S1, roughly corresponding with Brodmann area 2).
The ROIs were obtained from volumetric parcellation using FreeSurfer (http://surfer.nmr.mgh.harvard.edu/), by combining the Desikan-Killiany atlas (Desikan et al., 2006), with regions ‘PrecentralGyrusDKA’ and ‘PostcentralGyrusDKA’, and the Destrieux atlas (Destrieux et al., 2010), with regions ‘PrecentralSulcusDA’, ‘PrecentralGyrusDA’, ‘CentralSulcusDA’, ‘PostcentralGyrusDA’, and ‘PostcentralSulcusDA’. For the definition of ROIs in this paper, see Table 2. A combined mask ‘GRAND’ was defined as the union of M1, S1, pre-M1, and post-S1. In individuals, the exact boundary between M1 and S1 may not always be located exactly in the fondus of the central sulcus. Therefore, to verify that classification scores from M1 are not due to S1 activity, we also calculated the classification scores for a conservative definition of M1, M1no-fondus, where a substantial part of the central sulcus was left out.
Table 2. ROI definitions based on the Desikan-Killiany atlas (subscript DKA) and Destrieux atlas (subscript DA), where ∪ denotes the voxel-wise union, and ∩ the intersect of ROIs.
ROI name | Definition |
---|---|
M1: | PrecentralGyrusDA ∪ (CentralSulcusDA ∩ PrecentralGyrusDKA) |
S1: | PostcentralGyrusDA ∪ (CentralSulcusDA ∩ PostcentralGyrusDKA) |
pre-M1: | PrecentralSulcusDA |
post-S1: | PostcentralSulcusDA |
‘GRAND’: | M1 ∪ S1 ∪ pre-M1 ∪ post-S1 |
M1no-fondus | PrecentralGyrusDA |
Classification procedure and statistics
Patterns of BOLD activation were classified using a multi-voxel pattern analysis approach with a support vector machine (SVM) classifier (Haxby et al., 2001). Classification was performed for each of the four ROIs and ‘GRAND’ contralateral to the hand that the subject was instructed to move. For classification the number of voxels was constrained to avoid inclusion of voxels not involved in the task (Bleichner et al., 2013). Therefore, for each ROI separately, as well as for ‘GRAND’, we only kept the 250 voxels with the highest t-values across the t-maps for the different gestures (Mitchell et al., 2004). The BOLD signal in each included voxel was subsequently detrended and transformed into z-scores. The amplitude of the BOLD response for each trial was calculated for each of the 250 voxels by taking the mean signal over scans 5, 6, and 7 for subjects C1 and C2, and scans 4, 5, and 6 for all other subjects. These windows were chosen because previous decoding studies have shown maximum decodability around 6 to 8 seconds after stimulation onset (Andersson et al., 2011; Bleichner et al., 2013).
Because an SVM is a binary classifier, six SVMs were combined in a ‘one versus all others’ approach: the SVMs were trained to distinguish each gesture from the combined set of all other gestures. In this approach, when testing an unknown sample, the class for which the distance from the data point to the decision boundary is largest wins. All SVM classifiers used a linear kernel with regularization parameter (‘soft margin’) C = 1.
The classifier was trained and validated using a leave-6-out cross validation scheme (20 folds), in which for each fold, one trial for each gesture was left out (validation set), while the classifier was trained on the remaining trials (the training set). Classification results are reported as mean and standard deviation. To evaluate the effects for ROI, hand, and group, we applied a three-way repeated-measures GLM, with two within-subject factors (ROI: 4 levels, and Hand: 2 levels) and two groups (amputees and controls) at significance level 0.05. To investigate whether there was an effect of amputation in any ROI of the denervated hemisphere in amputees, only significant interactions were followed with a post-hoc two-way GLM and individual two-sample t-tests for comparing between ROIs.
Spatial extent of features inside ROIs
To assess the spatial layout of the selected features, the ROIs (M1 and S1 only) were first mapped to a normalised space as follows. The borders of the sensorimotor cortex were extracted from a flat map parcellation, and three polynomials were fitted: through the central sulcus, through the anterior border of the precentral gyrus, and through the posterior border of the postcentral gyrus. Interpolating between these polynomials resulted in a 28 × 84 tiled mesh of the sensorimotor area. The volumetric M1 and S1 ROIs, used for classification, were remapped onto the normalized tiles. Spatial extent of activity was quantified for M1 and S1 by calculating the median distance of each selected voxel to its ROI’s center of mass. The spatial extent of features was compared between hemispheres and groups, using a repeated measures GLM with two measures (S1 and M1), with hemisphere as within-subject factor (two levels: left and right) and group as between-subject factor.
BOLD response analysis
We assessed the effect of denervation on the BOLD response amplitude. Per voxel and per trial, each time point in the detrended BOLD signal was converted into percent signal change with respect to the BOLD signal of the first scan of each trial. Per subject, BOLD responses were then averaged over trials for each of the ROIs.
The BOLD responses were then averaged between 4 and 8 seconds (comparable to the window used for classification) and were statistically compared using a three-way repeated-measures GLM and post-hoc tests analogous to the analysis of the classification scores.
Correlation with phantom pain, age, and years since amputation
Amputees rated their ability to make the gestures with their phantom hand on a score from 0 (“very difficult”) to 10 (“very easy”). They rated their average everyday phantom pain on a score from 0 (“no pain”) to 10 (“heavy pains”). To investigate any relationship between phantom movement ability, phantom pain, age, or years since amputation and the classification of the phantom hand, Pearson correlations were calculated using a significance level of 0.05 (Bonferroni corrected for five tests).
Gesture execution performance and data glove amplitude
Excessive movements of the hand that should be kept still during a task could possibly influence classification scores. Therefore, the amount of motion of the still hand was compared to the amount of motion of the moving hand. The amplitude of the finger flexion sensors of the data glove was chosen as measure for movement.
Results
GLM analysis
Significant activity was found for all subjects for the contrast of all gestures versus baseline in M1 and S1 (P < 0.05, Bonferroni corrected for total numbers of voxels in the imaged volume). Figure 1B displays activity at individually tailored thresholds to indicate foci with strongest activity.
Support vector machine classification
Mean classification scores for the six gestures were significantly above chance level (binomial test, P < 0.001) for all contralateral ROIs, in controls and in amputees, for both hands (Figure 2). For left and right hands of controls, classification scores ranged from 23% to 91%, depending on ROI. Intact hand scores of amputees ranged between 35% and 96%, and phantom hand scores between 25% and 84%.
Three-way repeated-measures GLM
The three-way repeated measures GLM revealed a significant main effect of ROI [F(3,13) = 29.7, P < 0.001], a significant two-way interaction between ROI and Group [F(3,13) = 4.29, P = 0.026] and a significant three-way interaction between Hand, ROI, and Group [F(3,13) = 20.64, P < 0.001].
As post-hoc test, a two-way repeated-measures GLM was performed on each of the two hands separately. For the left/intact hand, there was a significant effect of ROI [F(3,13) = 24.79, P < 0.001], but no significant interaction between ROI and Group [F(3,13) = 1.38, n.s.]. For the right/phantom hand, there was a significant effect of ROI [F(3,13) = 20.37, P < 0.001], and a significant interaction between ROI and Group [F(3,13) = 14.70, P < 0.001].
Lastly, independent t-tests were used post-hoc to compare the decodability between controls and amputees for each ROI in the right/phantom hand. The ROIs that demonstrated a significant difference in decodability were S1 [t(15) = 2.77, P = 0.014] and pre-M1 [t(15) = - 2.18, P = 0.046]. Other ROIs did not significantly differ between groups [M1: t(15) = -0.49, n.s.; post-S1: t(15) = -1.30, n.s.].
Classification on ROI ‘GRAND’
Gestures were also decoded from the combined ROI ‘GRAND’. In controls, decodability was 77% ± 13% for the right hand and 70% ± 15% for the left hand. In amputees, scores were 64% ± 14% for the phantom hand and 79% ± 11% for the intact hand. A two-way repeated-measures GLM indicated a significant interaction between Group and Hand [F(1,15) = 12.95, P = 0.003].
Spatial extent of features inside ROIs
Figure 3 displays the spatial characteristics of cortical activity. The standardized location of the center of mass within M1 and S1 was highly comparable across hands and groups (Figure 3B-C). GLM analyses demonstrated no effect of group or hand on the extent of features in M1 and S1, indicating that there was no difference in extent of spatial distribution of the highest activated voxels between hemispheres, or between controls and amputees (Figure 3D).
BOLD response analysis
All subjects showed comparably shaped BOLD responses in all ROIs (Figure 4). BOLD response amplitudes were compared per ROI between and within groups.
The three-way repeated measures GLM revealed a significant main effect of ROI [F(3,13) = 24.6, P < 0.001], significant two-way interactions between ROI and Group [F(3,13) = 5.39, P = 0.012] and ROI and Hand [F(3,13) = 5.58, P = 0.011], and a significant three-way interaction between Hand, ROI, and Group [F(3,13) = 6.36, P = 0.011].
A two-way repeated-measures GLM was performed to follow up effects within each of the two hands separately. For the left/intact hand, there was a significant effect of ROI [F(3,13) = 33.7, P < 0.001], but no significant interaction between ROI and Group [F(3,13) = 2.17, n.s.]. For the right/phantom hand, there was a significant effect of ROI [F(3,13) = 11.9, P = 0.001], and a significant interaction between ROI and Group [F(3,13) = 4.50, P = 0.022].
Independent t-tests to compare the BOLD responses between controls and amputees for each ROI only in the right/phantom hand revealed that only S1 demonstrated a significant difference [t(15) = -2.24, P = 0.040], whereas the other ROIs did not [M1: t(15) = -0.60, n.s.; pre-M1: t(15) = 0.81, n.s.; post-S1: t(15) = -0.69, n.s.].
Correlation with phantom pain, age, and years since amputation
We did not find significant correlations in any ROI between the ‘phantom movement ability score’ or the ‘phantom pain score’ and the classification score of the phantom hand (Pearson correlation) (Figure 5). There was no significant correlation between age and classification score in any of the groups, hands, or ROIs. Although all ROIs showed a negative trend, with higher classification scores found for people with the most recently acquired amputation, no ROI showed a significant correlation of classification score with years since amputation, using a significance level of P = 0.05, Bonferroni corrected for five tests.
Gesture execution performance and data glove amplitude
Gesture execution performance was assessed by classification of the data glove recordings of the intact hand in amputees, and of both hands in control subjects. The classification scores were significantly above chance level (94% ± 6%, P < 0.001) with a minimum score of 79%.
Analysis of the data glove flexion sensor amplitude demonstrated minimal motion in the hand that should be kept still during the task compared to the hand that should be moving, with subject A2 as the only exception (Figure 6). In both amputees and controls, there was no significant correlation between still hand amplitude and classification score from ROI ‘GRAND’.
Conservative definition of M1
To verify that M1 activity was not due to S1 activity, classification scores were calculated from a conservative definition of M1, M1no-fondus. The resulting classification scores (controls left: 46% ± 12%, controls right: 43% ± 15%, amputees intact: 55% ± 12%, amputees phantom: 48% ± 19%) all remained well above chance level, as with the ‘full’ M1.
Discussion
We investigated whether the detailed topographic representation of the hand is preserved in people with above-elbow arm amputation. As a measure of (preserved) representation, we used the decodability of attempted complex hand gestures from four different contralateral ROIs of the sensorimotor hand area in amputees (M1, S1, pre-M1, post-S1), and compared this to the decodability of gestures executed with the intact hand. Data of able-bodied controls were used as an extra reference. In controls and amputees, classification scores for both hands were significantly above chance in all ROIs.
M1 hand representation after amputation
For M1, there was no difference between the classification scores for attempted movement of the phantom hand in amputees and executed right hand movement in controls. BOLD responses of amputees were similar to those of the control subjects. Previous fMRI studies have shown that phantom hand movements result in a clear activation of the contralateral M1, similar to executed intact hand movement in amputees and controls (Ersland et al., 1996; Raffin et al., 2012; 2016; Roux et al., 2003), but did not investigate whether also the representation of composite finger movements in M1 remains intact. Although our classification approach does not directly assess topography in the traditional sense, it does provide an insight in the representation of hand movements, as it inherently relies on spatial patterns. Moreover, we cannot entirely exclude the possibility that the representations of complex hand movements change after denervation, but since we are able to decode composite hand movements from the cortex, the most straightforward conclusion is that the hand representations in M1 contralateral to the phantom hand are unaffected by denervation, also at the high level of detail associated with complex hand movements involving multiple fingers simultaneously.
The exact border between M1 and S1 can vary between subjects and does not necessarily have to be located exactly at the fondus of the central sulcus, leading to a possible contribution of S1 activity on the decodability of M1. However, we tested the classification on ROI M1no-fondus, where the fondus of the central sulcus has been left out, and they remained well above chance level, indicating that decoding is not due to activity in S1 cortex.
Decodability and BOLD response in S1
Interestingly, S1 demonstrated the highest decodability, with classification scores being significantly higher than from other ROIs for both hands of controls, and for the intact hand of amputees. Even for the phantom hand in amputees, S1 decodability was similar to that of the other ROIs, albeit significantly lower than S1 scores of the right hand of controls. As such, our findings confirm and extend previous studies that showed persistent body part representation (at a more coarse level) after amputation or spinal cord injury by 1) activation within S1 during attempted foot (Cramer et al., 2005; Hotz-Boendermaker et al., 2008; 2011) and hand (Gharabaghi et al., 2014; Raffin et al., 2012) movement and 2) preserved finger somatotopy of the phantom hand in S1 by moving the phantom hand (Kikkert et al., 2016) or by microstimulation (Flesher et al., 2016).
We hypothesize that the preservation of decodability is associated with the role of S1, which is thought to reflect not only somatosensory feedback, but also anticipatory information necessary for rapid movement correction (Helmholtz, 1924). Indeed, activity in S1 has been found in subjects whose proprioceptive feedback has been disabled by an ischemic nerve block (Christensen et al., 2007). Moreover, a movement-associated activity increase in S1, preceding M1 activation and the actual movement, has been demonstrated using electrocorticography (Sun et al., 2015). Our results support the notion that the decodability of complex hand movements in S1 can be viewed as a combination of feed-forward and feedback processes. The fact that the attempted gestures in amputees, which only generate feed-forward influences on S1 activity, could be decoded from S1 at the same level as decoding from M1 in phantom and actual movements, further strengthens the notion that primary sensorimotor cortex is hardly (if at all) affected by amputation.
The difference between decodability and BOLD amplitude for the phantom hand compared to actual movements may also be associated with amputation-induced structural changes. Indeed, it has been demonstrated that grey matter volume is reduced in the denervated cortex of amputees (Makin et al., 2013), and that reduced BOLD responses may be associated with grey matter thinning (Taylor et al., 2009). Future studies may elucidate the influence of structural changes on the decodability of complex movements.
Delayed reorganization after amputation
There is evidence for both immediate and delayed reorganizational changes after amputation (Pearson et al., 2003; Wall et al., 2002). Animal studies have indicated that within minutes to hours after amputation, the receptive fields of an amputated digit become responsive to stimulation of neighbouring parts of the hand (Calford and Tweedale, 1988). The few human studies on this topic suggest that reorganization occurs rapidly as well (Weiss et al., 2000). Delayed effects of amputation have been demonstrated in the case of a new shoulder representation in denervated forelimb cortex of rats over a period of weeks (Pearson et al., 2003). In addition, for human lower-limb amputees, a negative correlation between cortical thickness in V5/MT+, as well as white matter integrity in areas involved in visuospatial processing, and years since amputation have been described (Jiang et al., 2015; 2016), suggesting that slow reorganizational processes continue to occur long after amputation. Although the ROIs showed a negative trend in the correlation between post-S1 decodability and years since amputation, this correlation was never significant when corrected for multiple comparisons. Therefore, the data suggest that the effect of time (if any) on the phantom hand representation in M1 and S1 is small, keeping in mind that the number of subjects limits the power of the analysis.
Maximizing classification scores
The data of our control subjects agree with, and extend, a previous fMRI study from our group that showed that it is possible to classify multiple executed hand gestures from the sensorimotor areas in able-bodied people with high accuracy (Bleichner et al., 2016). The above-chance classification we observed for all ROIs is suggestive for the presence of a detailed hand representation in each of the four studied regions of the sensorimotor system in controls and amputees. Whether or not a discrete somatotopy for individual fingers exists in M1 is subject to debate [see for example (Graziano and Aflalo, 2007) for an overview of possible organization principles of M1]. In cases where within-limb somatotopy was found in M1 and compared to that of S1, the latter demonstrated a more discrete and segregated organization (Cunningham et al., 2013; Hluštík et al., 2001). This difference in organization has been attributed to the more integrative role M1 plays in motor control (Cunningham et al., 2013) and could explain why, in the present study, classification scores were highest in S1, as the method used here is inherently based on spatial activity patterns. Unfortunately, straightforward inference of the different organisational principles in M1 and S1 is not possible in our study due to the nature of machine learning.
We are aware of several potential confounds that can contribute to the classification accuracy and their comparisons between controls and amputees. First, it is known that stump muscles even in above-elbow amputees can be activated when attempting to move the phantom hand, and that these activations are reproducible and different from activation patterns of the stump itself (Reilly et al., 2006). However, it is unknown whether the same effect of this peripheral reorganisation is present for the fine-grained finger movements like the ones that were used here. Therefore, we cannot completely rule out the possible influence of this confound. Second, the control group is overall younger than the amputee group. However, since we were still able to decode successfully in amputees, we believe that our conclusions would remain valid in spite of this age difference. Third, we compared the phantom hand of amputees with the dominant (right) hand of controls. Although it can be argued that the dominant hand of amputees is now their intact hand by definition, we did not want to make any assumptions about the effects of change of hand dominance on decoding from both hemispheres, especially since all amputees (except for one) acquired their amputation in adulthood.
Compared to our previous gesture decoding study in able-bodied volunteers, we observed better classification scores. When combining all ROIs, the mean score for 4 gestures was 63% (chance level 25%) (Bleichner et al., 2013), whereas we now obtained 64 – 79 % (depending on hand and group) for a chance level of 16.7%. An interesting aspect is whether the subjects’ movements or activation patterns would become better decodable after a period of training. For this report, subjects practised the task daily in the week before scanning, to gain fluency in making the gestures. A longer training session might reveal a learning curve in decoding accuracy. Also, training can be offered using fMRI neurofeedback, a technique in which information about brain activity patterns is provided to the subject in real-time. Neurofeedback has proven to be an effective method to shape brain activity in certain areas (Weiskopf, 2012), and may improve discriminative power of activity patterns.
The high classification scores we obtained for attempted gestures in amputees, especially from the combined ROI, indicate that the sensorimotor hand region of these patients may be, and remain, a suitable source of signals for BCI applications, such as multidimensional arm prosthesis control, also years after denervation. For such a BCI application, signal acquisition methods need to be wearable and accommodate high spatial detail, both of which can be accomplished with high density electrocorticography (ECoG) (Bleichner et al., 2016; Branco et al., 2017). We have shown previously that fMRI BOLD activation demonstrates good spatial correspondence with ECoG (Hermes et al., 2012; Siero et al., 2014). For an ECoG-based BCI application, decoding from either M1 or S1 separately may be of interest, since limiting the size of an implant is beneficial in terms of limiting the surgical risk. Since the classification scores from both areas is high, it means that they both serve as promising targets for ECoG-based BCI.
In conclusion, our results demonstrate that complex attempted hand movements can be decoded well from primary motor cortex in people with arm amputation, suggesting that even years after denervation, this area has not lost its detailed spatial representation integrity associated with combined finger movements, and still contain a sufficient level of information for decoding. The same holds for adjacent sensory and premotor regions. Given the similar classification results for amputees and able-bodied subjects, it may be speculated that when having to resort to able-bodied people for BCI research (when inclusion of the target population is difficult because of low numbers or vulnerability), the use of executed movements provides useful insight in the organization and behaviour of the cortical hand region in patients.
Acknowledgements
Thanks to all participants for their contribution, and to dr. M.A.H. Brouwers and prof. dr. C.K. van der Sluis for their assistance in subject recruitment.
Funding
This work was made possible by the ERC Advanced Grant 320708 (iCONNECT) granted to NFR.
Abbreviations
- A1-A8
amputated subject 1 to 8
- BCI
brain-computer interface
- BOLD
blood-oxygen-level dependent signal
- C1-C9
control subject 1 to 9
- ECoG
electrocorticography
- fMRI
functional magnetic resonance imaging
- GLM
general linear model
- M1
primary motor cortex
- post-S1
posterior part of the postcentral gyrus
- pre-M1
anterior part of the precentral gyrus
- ROI
region of interest
- S1
primary sensory cortex
- SVM
support vector machine
References
- Aflalo T, Kellis S, Klaes C, Lee B, Shi Y, Pejsa K, et al. Decoding motor imagery from the posterior parietal cortex of a tetraplegic human. Science. 2015;348:906. doi: 10.1126/science.aaa5417. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Andersson P, Pluim JPW, Siero JCW, Klein S, Viergever MA, Ramsey NF. Real-Time Decoding of Brain Responses to Visuospatial Attention Using 7T fMRI. PLoS ONE. 2011;6:e27638. doi: 10.1371/journal.pone.0027638. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bleichner MG, Freudenburg ZV, Jansma JM, Aarnoutse EJ, Vansteensel MJ, Ramsey NF. Give me a sign: decoding four complex hand gestures based on high-density ECoG. Brain Structure and Function. 2016;221:203. doi: 10.1007/s00429-014-0902-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bleichner MG, Jansma JM, Sellmeijer J, Raemaekers M, Ramsey NF. Give Me a Sign: Decoding Complex Coordinated Hand Movements Using High-Field fMRI. Brain Topography. 2013;27:248. doi: 10.1007/s10548-013-0322-x. [DOI] [PubMed] [Google Scholar]
- Branco MP, Freudenburg ZV, Aarnoutse EJ, Bleichner MG, Vansteensel MJ, Ramsey NF. Decoding hand gestures from primary somatosensory cortex using high-density ECoG. NeuroImage. 2017;147:130. doi: 10.1016/j.neuroimage.2016.12.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Calford MB, Tweedale R. Immediate and chronic changes in responses of somatosensory cortex in adult flying-fox after digit amputation. Nature. 1988;332:446. doi: 10.1038/332446a0. [DOI] [PubMed] [Google Scholar]
- Christensen MS, Lundbye-Jensen J, Geertsen SS, Petersen TH, Paulson OB, Nielsen JB. Premotor cortex modulates somatosensory cortex during voluntary movements without proprioceptive feedback. Nature Neuroscience. 2007;10:417–419. doi: 10.1038/nn1873. [DOI] [PubMed] [Google Scholar]
- Cohen LG, Bandinelli S, Findley TW, Hallett M. Motor reorganization after upper limb amputation in man. Brain. 1991;114:615. doi: 10.1093/brain/114.1.615. [DOI] [PubMed] [Google Scholar]
- Cramer SC, Lastra L, Lacourse MG, Cohen MJ. Brain motor system function after chronic, complete spinal cord injury. Brain. 2005;128:2941. doi: 10.1093/brain/awh648. [DOI] [PubMed] [Google Scholar]
- Cunningham DA, Machado A, Yue GH, Carey JR, Plow EB. Functional somatotopy revealed across multiple cortical regions using a model of complex motor task. Brain Research. 2013;1531:25. doi: 10.1016/j.brainres.2013.07.050. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dechent P, Frahm J. Functional somatotopy of finger representations in human primary motor cortex. Human Brain Mapping. 2003;18:272. doi: 10.1002/hbm.10084. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Desikan RS, Ségonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, et al. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. NeuroImage. 2006;31:968. doi: 10.1016/j.neuroimage.2006.01.021. [DOI] [PubMed] [Google Scholar]
- Destrieux C, Fischl B, Dale A, Halgren E. Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. NeuroImage. 2010;53:1. doi: 10.1016/j.neuroimage.2010.06.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dettmers C, Adler T, Rzanny R, van Schayck R, Gaser C, Brückner L, et al. Increased excitability in the primary motor cortex and supplementary motor area in patients with phantom limb pain after upper limb amputation. Neuroscience Letters. 2001;307:109. doi: 10.1016/s0304-3940(01)01953-x. [DOI] [PubMed] [Google Scholar]
- Donoghue JP, Sanes JN. Peripheral nerve injury in developing rats reorganizes representation pattern in motor cortex. Proceedings of the National Academy of Sciences. 1987;84:1123. doi: 10.1073/pnas.84.4.1123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Elbert T, Flor H, Birbaumer N, Knecht S, Hampson S, Larbig W. Extensive reorganization of the somatosensory cortex in adult humans after nervous system injury. NeuroReport. 1994;5:2593. doi: 10.1097/00001756-199412000-00047. [DOI] [PubMed] [Google Scholar]
- Ersland L, Rosén G, Lundervold A, Smievoll AI, Tillung T, Sundberg H. Phantom limb imaginary fingertapping causes primary motor cortex activation: an fMRI study. NeuroReport. 1996;8:207. doi: 10.1097/00001756-199612200-00042. [DOI] [PubMed] [Google Scholar]
- Flesher SN, Collinger JL, Foldes ST, Weiss JM, Downey JE, Tyler-Kabara EC, et al. Intracortical microstimulation of human somatosensory cortex. Science Translational Medicine. 2016;8:361–141. doi: 10.1126/scitranslmed.aaf8083. [DOI] [PubMed] [Google Scholar]
- Gharabaghi A, Naros G, Walter A, Roth A, Bogdan M, Rosenstiel W, et al. Epidural electrocorticography of phantom hand movement following long-term upper-limb amputation. Frontiers in Human Neuroscience. 2014;8 doi: 10.3389/fnhum.2014.00285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Graziano MSA, Aflalo TN. Mapping Behavioral Repertoire onto the Cortex. Neuron. 2007;56:239. doi: 10.1016/j.neuron.2007.09.013. [DOI] [PubMed] [Google Scholar]
- Hadoush H, Sunagawa T, Nakanishi K, Endo K, Ochi M. Motor somatotopy of extensor indicis proprius and extensor pollicis longus. NeuroReport. 2011;22:559. doi: 10.1097/WNR.0b013e328348e750. [DOI] [PubMed] [Google Scholar]
- Haxby JV, Gobbini MI, Furey ML, Ishai A, Schouten JL, Pietrini P. Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex. Science. 2001;293:2425. doi: 10.1126/science.1063736. [DOI] [PubMed] [Google Scholar]
- von Helmholtz H. Helmholtz’s Treatise on Physiological Optics. 1924. [Google Scholar]
- Hermes D, Miller KJ, Vansteensel MJ, Aarnoutse EJ, Leijten FSS, Ramsey NF. Neurophysiologic correlates of fMRI in human motor cortex. Human Brain Mapping. 2012;33:1689. doi: 10.1002/hbm.21314. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hluštík P, Solodkin A, Gullapalli RP, Noll DC, Small SL. Somatotopy in Human Primary Motor and Somatosensory Hand Representations Revisited. Cerebral Cortex. 2001;11:312. doi: 10.1093/cercor/11.4.312. [DOI] [PubMed] [Google Scholar]
- Hotz-Boendermaker S, Funk M, Summers P, Brugger P, Curt A, Kollias SS. Preservation of motor programs in paraplegics as demonstrated by attempted and imagined foot movements. NeuroImage. 2008;39:383. doi: 10.1016/j.neuroimage.2007.07.065. [DOI] [PubMed] [Google Scholar]
- Hotz-Boendermaker S, Hepp-Reymond M-C, Curt A, Kollias SS. Movement Observation Activates Lower Limb Motor Networks in Chronic Complete Paraplegia. Neurorehabilitation and Neural Repair. 2011;25:469. doi: 10.1177/1545968310389184. [DOI] [PubMed] [Google Scholar]
- Jiang G, Li C, Wu J, Jiang T, Zhang Y, Zhao L, et al. Progressive Thinning of Visual Motion Area in Lower Limb Amputees. Frontiers in Human Neuroscience. 2016;10 doi: 10.3389/fnhum.2016.00079. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jiang G, Yin X, Li C, Li L, Zhao L, Evans AC, et al. The Plasticity of Brain Gray Matter and White Matter following Lower Limb Amputation. Neural Plasticity. 2015;2015:1. doi: 10.1155/2015/823185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kikkert S, Kolasinski J, Jbabdi S, Tracey I, Beckmann CF, Johansen-Berg H, et al. Revealing the neural fingerprints of a missing hand. eLife. 2016;2016 doi: 10.7554/eLife.15292. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lotze M, Flor H, Grodd W, Larbig W, Birbaumer N. Phantom movements and pain An fMRI study in upper limb amputees. Brain. 2001;124:2268. doi: 10.1093/brain/124.11.2268. [DOI] [PubMed] [Google Scholar]
- Makin TR, Scholz J, Filippini N, Slater DH, Tracey I, Johansen-Berg H. Phantom pain is associated with preserved structure and function in the former hand area. Nature Communications. 2013;4:1570. doi: 10.1038/ncomms2571. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Martuzzi R, van der Zwaag W, Farthouat J, Gruetter R, Blanke O. Human finger somatotopy in areas 3b, 1, and 2: A 7T fMRI study using a natural stimulus. Human Brain Mapping. 2012;35:213. doi: 10.1002/hbm.22172. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Merzenich MM, Nelson RJ, Stryker MP, Cynader MS, Schoppmann A, Zook JM. Somatosensory cortical map changes following digit amputation in adult monkeys. The Journal of Comparative Neurology. 1984;224:591. doi: 10.1002/cne.902240408. [DOI] [PubMed] [Google Scholar]
- Mitchell TM, Hutchinson R, Niculescu RS, Pereira F, Wang X, Just M, et al. Learning to Decode Cognitive States from Brain Images. Machine Learning. 2004;57:145. [Google Scholar]
- Oldfield RC. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia. 1971;9:97. doi: 10.1016/0028-3932(71)90067-4. [DOI] [PubMed] [Google Scholar]
- Pearson PP, Li CX, Chappell TD, Waters RS. Delayed reorganization of the shoulder representation in forepaw barrel subfield (FBS) in first somatosensory cortex (SI) following forelimb deafferentation in adult rats. Experimental Brain Research. 2003;153:100. doi: 10.1007/s00221-003-1625-z. [DOI] [PubMed] [Google Scholar]
- Raffin E, Mattout J, Reilly KT, Giraux P. Disentangling motor execution from motor imagery with the phantom limb. Brain. 2012;135:582. doi: 10.1093/brain/awr337. [DOI] [PubMed] [Google Scholar]
- Raffin E, Richard N, Giraux P, Reilly KT. Primary motor cortex changes after amputation correlate with phantom limb pain and the ability to move the phantom limb. NeuroImage. 2016;130:134. doi: 10.1016/j.neuroimage.2016.01.063. [DOI] [PubMed] [Google Scholar]
- Reilly KT, Mercier C, Schieber MH, Sirigu A. Persistent hand motor commands in the amputees' brain. Brain. 2006;129:2211. doi: 10.1093/brain/awl154. [DOI] [PubMed] [Google Scholar]
- Roux F-E, Lotterie J-A, Cassol E, Lazorthes Y, Sol J-C, Berry I. Cortical Areas Involved in Virtual Movement of Phantom Limbs: Comparison with Normal Subjects. Neurosurgery. 2003;53:1342–1353. doi: 10.1227/01.neu.0000093424.71086.8f. [DOI] [PubMed] [Google Scholar]
- Röricht S, Meyer BU, Niehaus L, Brandt SA. Long-term reorganization of motor cortex outputs after arm amputation. Neurology. 1999;53:106. doi: 10.1212/wnl.53.1.106. [DOI] [PubMed] [Google Scholar]
- Siero JCW, Hermes D, Hoogduin H, Luijten PR, Ramsey NF, Petridou N. BOLD matches neuronal activity at the mm scale: A combined 7T fMRI and ECoG study in human sensorimotor cortex. NeuroImage. 2014;101:177. doi: 10.1016/j.neuroimage.2014.07.002. [DOI] [PubMed] [Google Scholar]
- Sun H, Blakely TM, Darvas F, Wander JD, Johnson LA, Su DK, et al. Sequential activation of premotor, primary somatosensory and primary motor areas in humans during cued finger movements. Clinical Neurophysiology. 2015;126:2150. doi: 10.1016/j.clinph.2015.01.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Taylor KS, Anastakis DJ, Davis KD. Cutting your nerve changes your brain. Brain. 2009;132:3122. doi: 10.1093/brain/awp231. [DOI] [PubMed] [Google Scholar]
- Turner JA, Lee JS, Martinez O, Medlin AL, Schandler SL, Cohen MJ. Somatotopy of the motor cortex after long-term spinal cord injury or amputation. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2001;9:154. doi: 10.1109/7333.928575. [DOI] [PubMed] [Google Scholar]
- Vansteensel MJ, Pels EGM, Bleichner MG, Branco MP, Denison T, Freudenburg ZV, et al. Fully Implanted Brain–Computer Interface in a Locked-In Patient with ALS. New England Journal of Medicine. 2016;375:2060. doi: 10.1056/NEJMoa1608085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wall JT, Xu J, Wang X. Human brain plasticity: an emerging view of the multiple substrates and mechanisms that cause cortical changes and related sensory dysfunctions after injuries of sensory inputs from the body. Brain Research Reviews. 2002;39:181. doi: 10.1016/s0165-0173(02)00192-3. [DOI] [PubMed] [Google Scholar]
- Weiskopf N. Real-time fMRI and its application to neurofeedback. NeuroImage. 2012;62:682. doi: 10.1016/j.neuroimage.2011.10.009. [DOI] [PubMed] [Google Scholar]
- Weiss T, Miltner WHR, Huonker R, Friedel R, Schmidt I, Taub E. Rapid functional plasticity of the somatosensory cortex after finger amputation. Experimental Brain Research. 2000;134:199. doi: 10.1007/s002210000456. [DOI] [PubMed] [Google Scholar]
- Wu CWH, Kaas JH. Reorganization in Primary Motor Cortex of Primates with Long-Standing Therapeutic Amputations. Journal of Neuroscience. 1999;19:7679–7697. doi: 10.1523/JNEUROSCI.19-17-07679.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]