Enhancing Hebbian Learning to Control Brain Oscillatory Activity

Abstract

Sensorimotor rhythms (SMR, 8–15 Hz) are brain oscillations associated with successful motor performance, imagery, and imitation. Voluntary modulation of SMR can be used to control brain–machine interfaces (BMI) in the absence of any physical movements. The mechanisms underlying acquisition of such skill are unknown. Here, we provide evidence for a causal link between function of the primary motor cortex (M1), active during motor skill learning and retention, and successful acquisition of abstract skills such as control over SMR. Thirty healthy participants were trained on 5 consecutive days to control SMR oscillations. Each participant was randomly assigned to one of 3 groups that received either 20 min of anodal, cathodal, or sham transcranial direct current stimulation (tDCS) over M1. Learning SMR control across training days was superior in the anodal tDCS group relative to the other 2. Cathodal tDCS blocked the beneficial effects of training, as evidenced with sham tDCS. One month later, the newly acquired skill remained superior in the anodal tDCS group. Thus, application of weak electric currents of opposite polarities over M1 differentially modulates learning SMR control, pointing to this primary cortical region as a common substrate for acquisition of physical motor skills and learning to control brain oscillatory activity.

Introduction

Repetitive practice is required to learn physical skills like playing sports or a musical instrument (Lashley et al. 1951; Brady 2008). Practice is also required to learn more abstract skills irrespective of physical movements, like control of brain oscillatory activity (Koralek et al. 2012) present with successful performance of motor and cognitive functions like attention (Schafer and Moore 2011), gating of sensory information (Klimesch et al. 2007), movement planning (Joundi et al. 2012), speech (Saarinen et al. 2006), and reasoning (Sheth et al. 2009).

Brain oscillations may be topographically specific, like occipital alpha rhythms related to visual processing (Chatrian et al. 1959) or sensorimotor rhythms (SMR, 8–15 Hz) predominantly localized over primary sensorimotor regions and related to motor actions (Pfurtscheller and Aranibar 1979). In the motor domain, performance, imagination, and/or imitation of movements is associated with event-related desynchronization of SMR (SMR-ERD) (Pfurtscheller and Aranibar 1979; Pfurtscheller and Neuper 1997) reflecting activity in a distributed sensorimotor cortico-subcortical network that includes the primary motor cortex (M1) (Stocco et al. 2010; Buch et al. 2012), a region crucial for motor control and motor learning (Reis et al. 2009). Previous work demonstrated that healthy volunteers and patients with brain lesions can learn a novel operant brain–machine interface task in which they are required to modulate SMR-ERD, rather than executing a physical movement, to obtain reward through associative (Hebbian) learning (Buch et al. 2008; Soekadar et al. 2011). This type of abstract learning is important because it allows, for example, neural control of robotic and prosthetic devices (Ganguly et al. 2011; Buch et al. 2012) in the absence of physical movements (Hebbian control). Sensory feedback is critical to learn this task because when successful generation of SMR-ERD did not result in any feedback during training, participants did not learn (Soekadar et al. 2011; Censor et al. 2013).

There is, however, a gap in knowledge on the mechanisms and neural substrates underlying such abstract learning and it is unclear whether targeting primary cortical regions can modulate acquisition of such skill. Here, we studied the involvement of M1 as a substrate of learning to control SMR oscillations in the absence of physical movements. We explored this idea by evaluating polarity-specific effects of M1 stimulation on learning SMR control across multiple days. Previous work indicated that repeated stimulation can stabilize the long-lasting effects of transcranial direct current stimulation (tDCS) (Reis et al. 2009) and result in larger cumulative effects when applied daily (Alonzo et al. 2012). We reasoned that repeated application of anodal tDCS over M1, shown to increase motor cortical excitability (Nitsche and Paulus 2000; Galea et al. 2009) and acquisition of physical motor skills (Nitsche and Schauenburg et al. 2003; Reis et al. 2009; Stagg, Jayaram et al. 2011), would improve Hebbian learning to control brain oscillatory activity relative to sham tDCS, and that application of cathodal tDCS, known to decrease motor cortical excitability (Nitsche and Paulus 2000), would reduce the acquisition of this skill relative to sham tDCS.

Methods

Participants and Study Design

Thirty-six young adults (28.6 ± 8.2 years, 15 males) participated in a prescreening session to evaluate handedness and baseline SMR-ERD during motor imagery. All participants were recruited at the laboratory of the Human Cortical Physiology and Neurorehabilitation Section (HCPS), National Institute of Neurological Disorders and Stroke (NINDS) and were able to elicit detectable SMR-ERD of at least 20% during imagery (Soekadar et al. 2011). All participants were right handed as assessed by the Edinburgh Handedness Inventory (Oldfield 1971), had no physical or neurological symptoms, had no past history of neurological or psychiatric diseases, and did not take any medication on a regular basis. Four participants were excluded because they were not able to elicit detectable SMR-ERD of at least 20% during imagery. Two participants retracted from the study after the prescreening due to the required time commitment. All participants gave written informed consent to participate in the study before the experiment, which was approved by the NINDS IRB, and were assigned to 1 of 3 different groups that received anodal, cathodal, or sham tDCS for a period of 20 min in association with each training session (Fig. 1a,b). All 3 groups were trained for a total of 1 h daily on 5 consecutive days. A follow-up test was done 30 days later. During training, the participants were instructed to imagine opening their left hand in response to a GO signal, which elicits SMR-ERD (Buch et al. 2008; Soekadar et al. 2011). Successful production of SMR-ERD through the imagery task resulted in online contingent left hand passive opening motions driven by an orthotic device attached to the participants' hand (Fig. 2a). Electromyographic activity was recorded from multiple muscle groups (finger and arm extensors and flexors) to assess possible muscle contractions during the task. The endpoint measure of control of neural oscillatory activity was the amount of SMR-ERD, which was computed for every single trial (Fig. 2b).

Figure 1.

(a) Experimental design: subjects participated in 5 daily training sessions (Day 1–Day 5). After determination of baseline (BAS) ability to control SMR event-related desynchronization SMR-ERD, subjects in each group underwent 20 min of anodal, cathodal, or sham transcranial direct current stimulation applied over the primary motor cortex (identified by TMS) immediately before each session (b). Reference electrode was placed over the left supraorbital region. Each training session consisted of 5 blocks (23 trials each). In each trial, subjects were instructed to respond to an auditory GO signal by imagining left hand opening motions according to a well described protocol (Buch et al. 2008; Soekadar et al. 2011) inducing SMR-ERD (Pfurtscheller and Aranibar 1979; Pfurtscheller and Neuper 1997). Successful SMR-ERD generation resulted in online contingent left hand passive opening motions driven by an orthotic device attached to the subject's hand. The end of each imagery trial was signaled by an auditory STOP signal. Trials were separated by 5 s and blocks by 60 s. 30 days later participants returned for a follow-up test (F).

Figure 2.

Experimental paradigm. (a) Brain oscillatory activity was recorded from a 275-channel whole-head MEG while the participants's left hand was attached to an orthotic device. (b) In each trial, SMR-ERD reflecting successful control (red) resulted in proportional online contingent left hand passive opening motions driven by the orthotic device. In the individual participant example above, there was no SMR-ERD in the first trial. During trial 2, SMR-ERD was detectable resulting in positive SMR-ERD values (red) and contingent passive hand motions (feedback).

Task Description and Procedure

Participants were trained to control SMR oscillations recorded with a whole-head magnetoencephalography (MEG) sensor array composed of 275 gradiometers (CTF Systems, Canada) (Fig. 2a) over a 5-day period. Brain oscillatory activity was recorded at a sampling rate of 600 Hz with a band-width of 0–300 Hz. Synthetic 3rd gradient balancing was used for online removal of background noise. In contrast to magnetometers, use of a high number of gradiometers allows detection of neuromagnetic activity right below the sensors with high topographical specificity to cortical sources (Hillebrand and Barnes 2002) being relatively insensitive to deeper sources. The 3 sensors showing highest SMR-ERD values were selected for the following training. Successful production of SMR-ERD through the imagery task resulted in online contingent left hand passive opening motions driven by an orthotic device attached to the subject's hand (see Fig. 2a, Supplementary Fig. 1 and Soekadar et al. 2011 for detailed description). Thus, they observed and felt their hand opening upon eliciting successful SMR-ERD. Online electromyography (EMG) was monitored from finger and arm flexors and extensors (m. brachioradialis and m. flexor carpi ulnaris, m. biceps brachii, and m. triceps brachii) of both arms using radio-translucent surface electrodes (Biopac^®, interelectrode distance: 2 cm). Trials in which EMG activity during imagery exceeded that recorded during rest were excluded from further analysis. Head movements were continuously recorded using individually registered head coils to account for different head shapes (Wilson 2004; Stolk et al. 2013). Trials were interrupted and discarded from further analysis when head motion exceeded 5 mm. For detection and online translation of SMR-ERD, the platform BCI2000 was used (Schalk et al. 2004; Soekadar et al. 2011). Sessions consisted of 5 blocks with 23 trials each (Fig. 1a). Start and end of each trial was indicated by auditory stimuli. Before the first training Day 2 baseline blocks were recorded in the absence of any stimulation (BAS).

Computation of ERD included the power spectrum estimation (an autoregressive model of order 16 using the Yule–Walker algorithm) of each incoming sample at the frequency found to be optimal in each participant (optimal defined as frequency showing highest SMR-ERD values during motor imagery in the range of 8–15 Hz; 9 Hz in 2 participants, 11 Hz in 27 participants and 13 Hz in one participant). Resulting values were compared with mean power values of the preceding intertrial intervals continuously updating the ERD threshold according to the method of Pfurtscheller and Aranibar (1979):

$$\text{RV}=\frac{1}{|T_{\mathrm{ref}}|}\sum _{t\in T_{\mathrm{ref}}}{P}_t,$$ (1)

$$\text{ERD(}t\text{) = }\frac{P_t}{\text{RV}}-1,t\in T_{\mathrm{task}},$$ (2)

where t represents the recorded sample block, T_ref the event-related task condition period and P_t the power estimate in a given frequency band of t. RV (reference value) represents power estimates during the rest (task-free) condition. Positive feedback consisted of hand opening and was delivered online during each trial when SMR-ERD was detected (Fig. 2b). Correlation of changes in SMR-ERD and the amount of time positive feedback (hand opening) was provided (amount of feedback) were calculated offline to confirm proper translation of successful SMR-ERD control into amount of feedback (see Supplementary Fig. 1).

Transcranial Direct Current Stimulation (tDCS)

tDCS was applied via 2 conducting 5 × 5 cm electrodes covered with a saline-soaked sponge using a Phoresor II Auto^® (model PM850, IOMED^®, Salt Lake City, UT). A bipolar electrode montage (right M1 and left supraorbital area) was used (Fig. 1b) to deliver a current of 1 mA for 20 min (current density 0.04 mA/cm²; total charge 0.048 C/cm²), previously shown to have polarity dependent (Nitsche and Paulus 2000; Stagg, Jayaram et al. 2011) and focal (Faria et al. 2011) long-lasting effects (Nitsche and Paulus 2001) on motor cortical excitability, learning, and retention of skill (Abe et al. 2010; Censor et al. 2010; Schambra et al. 2011). In the sham group, anodal tDCS was applied for 30 s and at the offset, the current was decreased in a ramp-like fashion, a method shown to achieve a good level of blinding between sessions (Gandiga et al. 2006). In the 3 groups, the active electrode was placed over the right M1. Prior to training Day 1, the M1 hand-area was localized in all participants based on the motor evoked potential (MEP) hotspot of the first digit's interosseus muscle (FDI) using transcranial magnetic stimulation (TMS). Participants and investigators performing testing and data analysis were blind to group assignment.

Offline Data Analysis

For all outcome measures, assumption of a normal distribution (Shapiro–Wilk test of normality) was tested. Parametric tests were corrected by Greenhouse–Geisser estimates if Mauchly's sphericity test indicated significance. Mixed-model repeated-measures ANOVAs (rmANOVAs) with factors “group” (anodal/cathodal/sham) as between-subject factor and “training day” (Day 1–Day 5) as within-subject variable was performed for SMR-ERD. Multiple pairwise comparisons with Bonferroni adjustments were calculated to compare changes of baseline-corrected SMR-ERD values across groups at each time point. To evaluate differences in successful SMR-ERD control, baseline-corrected SMR-ERD values of training Day 1 were compared with training Day 5 in each group using a paired-samples t-test. Effect size of anodal and cathodal tDCS was calculated using Cohen's d transformed into a regression coefficient r, where r < 0.3 is considered as small, r < 0.5 is considered as medium, and r > 0.5 is considered as large effect (Cohen 1988). To detect differences in successful SMR-ERD control between stimulation groups, independent sample t-tests were applied. To calculate topographic specificity of learned SMR control, centro-parietal sensors were sub-categorized into those in immediate proximity (PS1, 10 sensors) and extended proximity (PS2, 26 sensors) to the sensors selected for training (training sensors, TS) (Supplementary Fig. 2). Topographical specificity was calculated as the ratio between SMR-ERD learning recorded by the neighboring centro-parietal sensors and the 3 training sensors (PS1/TS and PS2/TS, respectively). For comparison between groups, a 2-way ANOVA with “group” (anodal/cathodal/sham) and “sensors” (TS/PS1/PS2) as independent variables was conducted. To evaluate differences in topographical specificity of learned SMR-control between stimulation conditions, post hoc t-tests for independent samples were used. All analyses were performed in SPSS 17.0. Significance level was set to P < 0.05. Variance is expressed as standard errors of the means (SEM).

Results

Learning to Control SMR Across Groups

At baseline, the ability to control SMR-ERD was comparable between the 3 groups (one-way ANOVA, F_2,177 = 1.404, P = 0.248). A mixed-model rmANOVA revealed main effects of “stimulation” (F_2,177 = 4.127, P < 0.05), “training day” (F_3.589,635.17 = 2.546, P < 0.05) and their interaction (F_7.718,635.17 = 4.796, P < 0.001) on SMR-ERD, indicating a different time course of learning control of neural oscillatory activity between the groups (see Fig. 3 for individual subject examples and Fig. 4 for group data). In the sham and anodal tDCS groups, training led to a progressive increase in SMR-ERD from training Day 1 to training Day 5, pointing to successful learning control of neural oscillatory activity (P < 0.05 and P < 0.001, respectively, independent-samples t-test, Fig. 4b). During the training task, the number of excluded trials in which EMG activity present in arm muscles was comparable across groups (one-way ANOVA, F_2,149 = 0.632, P = 0.533), and thus could not account for group differences in learning.

Figure 3.

Hebbian learning to control SMR oscillations in a representative participant of each group receiving anodal (a), cathodal (b) or sham (c) stimulation. Note the progressive improvement in SMR-ERD control in the participant receiving sham (P < 0.05) and anodal (P < 0.01) stimulation. The participant receiving cathodal tDCS, on the contrary, shows a progressive and significant worsening of SMR-ERD control over the 5 training days. Whiskers indicate one standard deviation above and below the mean of the data in each block. Trials with SMR-ERD values exceeding one standard deviation are represented by dots. (d) Shows the topographic distribution of SMR-ERD in the same participants at training Day 1, Day 5 and follow-up test after 30 days (F) (blue = positive SMR-ERD values, red = negative SMR-ERD values; red arrows indicate the location of sensors used for SMR-ERD control). *P < 0.05; **P < 0.01.

Figure 4.

Group control of SMR oscillations across training days (a). Note the different time-course of learning to control SMR-ERD in the 3 groups (a). Learning across training days was more prominent in the anodal tDCS compared with the sham (P < 0.05 on Day 4, Day 5 and F) and cathodal tDCS compared with the sham group (P < 0.01 on Day 3, Day 4, Day 5 and F). (b) SMR-ERD improved significantly with training from Day 1 to Day 5 in the sham group (P < 0.05). Anodal tDCS improved SMR-ERD beyond levels seen in the sham tDCS group (P < 0.001) resulting in significantly better learning in the anodal group compared with sham (P < 0.05). Cathodal tDCS blocked the training-induced improvements in the sham group (P < 0.05). Variance is shown as SEM. *P < 0.05; **P < 0.01; ***P < 0.001.

Increase in SMR-ERD from training Day 1 to training Day 5 (ΔDay 1/Day 5) was significantly larger in the anodal than in the sham (P < 0.05; independent samples t-test) or cathodal (P < 0.001) tDCS groups. Cathodal tDCS significantly reduced the benefits of training on SMR-ERD control relative to the sham group (P < 0.01). Thus, anodal tDCS increased, while cathodal tDCS reduced the beneficial effects of training as evidenced in the sham group (Fig. 4b). Calculation of the effect size indicated large effects for both, anodal (r = 0.742) and cathodal (r = −0.785) stimulation.

Retention of Skill

Learned SMR-ERD control at the follow-up test 30 days later (where no stimulation was applied) was better in the anodal tDCS group compared with the sham (P < 0.05) and cathodal (P < 0.01) groups, and better in the sham compared with the cathodal tDCS (P < 0.05) group, indicating long-lasting polarity-specific effects of M1 stimulation on the learned skill. At this time point, only participants in the anodal tDCS group showed significantly larger SMR-ERD relative to the first training day (P < 0.01). Of note, SMR-ERD control of participants trained under cathodal stimulation at follow-up was not different from SMR-ERD control of participants trained under anodal (P = 0.78) or sham (P = 0.74) stimulation on training Day 1.

Task-related Changes of Brain Oscillatory Activity in Other Frequency Bands and Topographical Specificity of Learned SMR Control

In relation to other frequency bands, a rmANOVA revealed no significant effects of “stimulation”, “training day” or their interaction on theta, 4–8 Hz: F_2,122 = 0.22, P = 0.80, beta, 17–30 Hz: F_2,122 = 1.91, P = 0.153, or gamma, 30–150 Hz: F_2,122 = 3.06, P = 0.45) rhythms (Supplementary Fig. 3). For evaluation of gamma activity a band was chosen that transcends 30–120 Hz following previous studies that indicated strongest cross-frequency coupling in the 80–150 Hz band (Canolty et al. 2006). Analysis of topographical specificity of learned SMR-control using a 2-way ANOVA indicated a significant main effect for “condition” (F_2,54 = 38.27, P < 0.01) and, sensors' (F_1,54 = 132.23, P < 0.01) with no interaction between factors (F_2,42 = 0.74, P = 0.241). Specificity was higher in the anodal compared with the cathodal (P < 0.01) tDCS group, and similar compared with the sham group (P = 0.43) which also yielded higher specificity when compared with the cathodal tDCS group (P < 0.01) (Supplementary Fig. 2 and Table 1).

Discussion

We studied the contribution of M1 to learning control of oscillatory brain activity in healthy humans. We found that the combination of anodal tDCS over M1 with training improved learning of this skill to a larger extent than cathodal or sham tDCS. This effect was polarity-specific as cathodal tDCS significantly reduced skill gains relative to sham. Retention of the newly acquired skill 1 month later remained superior with anodal compared with cathodal or sham tDCS.

Control of brain oscillatory activity constitutes a qualitatively fundamentally different skill than the ability to perform visuomotor sequences (Reis et al. 2009), finger movements (Onal-Hartmann et al. 2012), or adapting to perturbations (Shadmehr et al. 2010), all of which require physical movements (Koralek et al. 2012). Successful SMR oscillatory control through imagery in this task results in positive passive rewarding feedback, consistent of contingent online passive hand motions driven by an orthotic device (Buch et al. 2008; Soekadar et al. 2011). With its distinct features (Wolpaw et al. 2002), successful control of brain oscillatory activity represents a skill that is relevant for successful performance of motor movements (Joundi et al. 2012), speech (Saarinen et al. 2006), and imitation (Pfurtscheller and Neuper 1997) and may be useful in the management of seizure activity (Sterman and Friar 1972), attention deficits (Thompson and Thompson 1998; Monastra et al. 2005), and stroke (Birbaumer et al. 2009). In spite of being ubiquitous in successful normal and abnormal behavior and of having been recently investigated in animal models (Koralek et al. 2012), the neural substrates underlying learned control of brain oscillatory activity in humans are incompletely understood.

The contribution of the primary motor cortex in learning this abstract skill was evaluated by modulating neural activity in M1 using tDCS. The combination of anodal tDCS over M1 with training led to better learned control of SMR oscillations than cathodal or sham tDCS over the 5 training days. There was no indication in our data that resting baseline power differed across the differently stimulated groups (Supplementary Fig. 4). Inspection of Figure 4a shows that significant group differences did not become obvious until training Day 3. This indicates that the effects of tDCS on this type of skill acquisition require cumulative days to become detectable, and do not operate simply through SMR-ERD modulation elicited by passive hand movements (Matsumoto et al. 2010). An additional study in which training was continued over the time of 10 days indicated that learning in the absence of stimulation does not reach a ceiling effect by Day 5 (Supplementary Fig. 5) and is comparable to the improvements identified in the sham tDCS group (Fig. 4a). While anodal tDCS increased the effects of training on SMR control, cathodal tDCS blocked them (Fig. 4a), altogether providing a causal link between M1 function and successful learning of this skill. Such findings indicate that it is possible to modulate abstract learning to control brain oscillatory activity using noninvasive brain stimulation targeting a primary cortical region.

Within our experimental design, the effects of tDCS were specific to the trained SMR, since tDCS did not affect other frequency bands such as theta (4–8 Hz), upper beta (17–30 Hz), and gamma (30–150 Hz) (Supplementary Fig. 3). More generally, our results show for the first time that application of weak electric currents (1 mA) of opposite polarities over a relevant primary cortical region differentially modulate Hebbian learning to control oscillatory brain activity across multiple days partially originated in that region. Learned SMR control showed to be spatially specific to the sensors selected for training (Supplementary Fig. 2 and Table 1).

Four weeks after the end of training, skill was superior within the anodal tDCS group relative to baseline and also relative to the other 2 groups. This finding is consistent with the idea that tDCS in combination with training influence neural processes beyond immediate effects on cortical mechanisms (Matsumoto et al. 2010;Kasashima et al. 2012), e.g. long-term potentiation (LTP) (Fritsch et al. 2010) that is N-methyl-d-aspartate (NMDA) receptor dependent (Liebetanz et al. 2002; Nitsche, Fricke et al. 2003; Koralek et al. 2012) or modulation of the GABAergic system (Stagg, Bachtiar et al. 2011). Interestingly, learning to control firing rates in single neurons in rodents' M1 is also, at least partially, NMDA receptor dependent because blocking NMDA receptor function in striato-cortical connections disrupts acquisition of the skill (Koralek et al. 2012).

Our results provide causal evidence for the involvement of the primary motor cortex in learning a skill that does not involve physical movements, possibly as part of a more extensive basal-ganglia-thalamo-cortical circuit influenced by feedback and reward (Hinterberger et al. 2005; Stocco et al. 2010; Buch et al. 2012). Moreover, they point to common substrates between acquisition and retention of motor skills (Abe et al. 2010; Censor et al. 2010; Schambra et al. 2011) and learned brain control. Learned SMR control was topographically specific to sensors overlying the primary sensorimotor cortex, which is interconnected with premotor regions involved in motor planning and shaping of the hand (Murata et al. 1997). While likely to be of small effect, use of a different sphericity test than Mauchly's test could have been advantageous to avoid type II errors in small samples. Future work should address the relative contribution of tangential and perpendicular current sources to anodal and cathodal tDCS effects (Manola et al. 2007) using a multimodal approach with complementary sensitivity towards tangential and perpendicular current sources, e.g. recording electroencephalography and MEG simultaneously during transcranial electric current stimulation (Soekadar et al. 2013).

The findings presented here might have important implications for the development and design of novel rehabilitation strategies aiming at restoration of brain function (“neurorestoration”) using brain–machine interfaces (BMI), e.g. in the context of stroke recovery (Ramos-Murguialday et al. 2013).

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/

Funding

This work was supported by the Intramural Research Program (IRP) of the National Institute of Neurological Disorders and Stroke (NINDS), Bethesda, MD, USA; the Center for Neuroscience and Regenerative Medicine (CNRM), Uniformed Services University of Health Sciences, Bethesda, MD, USA; the German Federal Ministry of Education and Research (BMBF, grant number 01GQ0831, 16SV5838K to S.R.S. and N.B.); the European Commission under the project WAY (grant number 288551 to S.R.S. and N.B.); the Deutsche Forschungsgemeinschaft (DFG, grant number SO932-1 to S.R.S. and Reinhart Koselleck Project support to N.B.); Volkswagenstiftung (VW) and the Baden-Württemberg Stiftung.