New neural activity patterns emerge with long-term learning

Significance

Consider a skill you would like to learn, like playing the piano. How do you progress from “Chopsticks” to Chopin? As you learn to do something new with your hands, does the brain also do something new? We found that monkeys learned new skilled behavior by generating new neural activity patterns. We used a brain–computer interface (BCI), which directly links neural activity to movement of a computer cursor, to encourage animals to generate new neural activity patterns. Over several days, the animals began to exhibit new patterns of neural activity that enabled them to control the BCI cursor. This suggests that learning to play the piano and other skills might also involve the generation of new neural activity patterns.

Abstract

Learning has been associated with changes in the brain at every level of organization. However, it remains difficult to establish a causal link between specific changes in the brain and new behavioral abilities. We establish that new neural activity patterns emerge with learning. We demonstrate that these new neural activity patterns cause the new behavior. Thus, the formation of new patterns of neural population activity can underlie the learning of new skills.

Our understanding of learning is grounded in the concepts of synaptic plasticity (1, 2) and cortical map plasticity (3, 4). However, we lack an explanation for how such changes give rise to new behavioral capacities. Establishing a causal link from learning-related changes in the brain to new behavioral capacities would require knowing which neurons drive behavior, as well as the relationship between the activity of those neurons and behavior. Then, any observed change in behavior can be attributed to an observed change in the neural activity. A brain–computer interface (BCI) enables us to link changes in neural activity directly to learning because the relationship between the neural activity and the behavior is known exactly, and only the neurons we record directly influence behavior (5).

We hypothesize that learning new skills can be achieved by the formation of new patterns of neural activity, where patterns of neural activity are defined as the joint firing rate of a population of neurons measured during a brief time window (SI Appendix, Materials and Methods). To demonstrate that new neural activity patterns drive learning, we must achieve three objectives: First, we need to encourage new neural activity patterns to form. Second, we need to detect new patterns, should they appear. Third, we need to show that the new patterns directly cause the new behavioral capacities that emerge with learning. A BCI learning paradigm provides a framework whereby we can achieve these objectives.

In our BCI paradigm, a monkey moves a computer cursor from the center of a screen to one of eight peripheral targets by volitionally modulating the activity of a population of ∼90 neural units recorded with a multielectrode array chronically implanted in the arm region of primary motor cortex (Fig. 1A and SI Appendix, Materials and Methods). Each experiment begins by presenting the monkey with an “intuitive mapping” that relates neural activity patterns to cursor velocities in a way that provides proficient control without requiring the animal to learn. We then induce learning by presenting the monkey with a novel mapping from neural activity to cursor velocity (6–8).

Fig. 1.

Using a brain–computer interface to study learning. (A) Schematic of the BCI system. Monkeys controlled a BCI cursor (yellow) to acquire one (cyan) of eight possible (gray) targets by modulating their neural activity. In our BCI system, ∼90D neural activity is first mapped into the 10D intrinsic manifold, and then to 2D cursor velocity. (B) A simplified, conceptual schematic of neural activity patterns (dots). The neural activity patterns tend to lie in a low-dimensional subspace, termed the intrinsic manifold (gray plane). Monkeys move the BCI cursor by volitionally modulating their neural activity. At the beginning of each experiment, cursor velocities were determined by an intuitive BCI mapping (black arrow). Here, patterns colored green would move the cursor to the right and purple patterns would move the cursor to the left. To induce learning, we changed the mapping to an OMP mapping (blue arrow). Under the perturbed mapping, neural activity patterns map to different cursor velocities than under the intuitive mapping. This encourages the monkeys to learn. (C) Cursor trajectories for successful trials during a representative multiday OMP experiment (i.e., OMP 1; the multiday OMP experiment beginning on June 17, 2016). “Intuitive” trajectories show 40 consecutive trials with the intuitive mapping as an example of proficient cursor control. The second column shows the first 40 trials after switching to the OMP mapping. Performance is impaired. The third column shows the best 40 consecutive trials on day 1. The fourth column shows the best 40 consecutive trials after 6 d of practice. (D) Quantifying the amount of learning for single-day OMP (blue), multiday OMP, incremental training (green), and single-day WMP (red) experiments. An amount of learning of 1 indicates complete learning, and a value of 0 indicates no learning. Vertical lines indicate the mean of each distribution. (E) Learning curves for the six experiments with the greatest amount of learning. The example in C is highlighted in black.

Previously, we have shown that the structure of neural population activity limits the learning that can occur within a single day (8). We characterize how neurons naturally covary as the “intrinsic manifold” (Fig. 1B). In our BCI system, neural activity is mapped into the intrinsic manifold and then to cursor velocity (Fig. 1 A and B). This enables us to study learning by constructing two types of novel BCI mappings. A novel BCI mapping that is consistent with the intrinsic manifold, i.e., a “within-manifold perturbation” (WMP), can be well learned within a single day. The neural strategy for learning WMPs does not involve the formation of new neural activity patterns. Instead, learning occurs by reassociating preexisting patterns of neural activity with different intended movements (9). In contrast, novel BCI mappings that are inconsistent with the intrinsic manifold, i.e., “outside-manifold perturbations” (OMPs), are not well learned within a single day. OMPs encourage the formation of new neural activity patterns. Thus, we achieve the first objective of our study by challenging animals to learn to use OMP mappings given extended practice over several days. To this end, we use the framework of the intrinsic manifold to repeatedly and reliably construct novel BCI mappings that encourage the formation of new patterns of neural activity and ask whether the new mappings are learnable. Through subsequent analyses, we achieve the second and third objectives of our study, demonstrating that OMP mappings are indeed learned by the formation of new patterns of neural activity, and that those patterns directly drive the new behavior.

Results

We conducted 15 multiday OMP learning experiments (ranging from 6 to 16 d per experiment, average 9.2 d) across two monkeys. Each of these novel OMP mappings was learned over several days (Fig. 1 C and D). With multiday practice, the amount of OMP learning is substantially greater than single-day OMP learning (Fig. 1D; t test P < 10⁻⁴), and is comparable to the single-day learning we previously observed for WMPs (8) (Fig. 1D; t test P = 0.53). To facilitate learning, we employed an incremental training paradigm (10) (SI Appendix, Materials and Methods and Fig. S1 A–D). Multiday exposure to an OMP with no incremental training led to inconsistent learning (SI Appendix, Fig. S1 E–G). The incremental training approach was not effective within a single day (SI Appendix, Fig. S1G). For most experiments, learning proceeded in a manner that resembles skill learning (11): gradual improvement over the course of many days, with some dips and rebounds in performance (Fig. 1E and SI Appendix, Fig. S1 C and D). The dips and rebounds likely reflect some combination of the natural skill-learning process, motivation during a difficult task, and day-to-day recording instabilities (12–14).

The second objective of our study is to detect whether new neural activity patterns emerged during multiday OMP learning. By construction, forming new patterns of neural activity is the optimal neural strategy for learning to control the cursor under an OMP mapping because this would lead to the fastest cursor speeds. However, it is possible that the brain is unable to form new patterns because constraints exist on the patterns of neural activity that a population of neurons can exhibit (8, 9, 15–18). If this is the case, the monkey could still show some limited behavioral improvements by learning to reassociate preexisting patterns of neural activity with different intended movements (9). Thus, behavioral improvements alone are not sufficient to conclude that new patterns of neural activity have emerged.

Detecting the appearance of a new pattern of neural activity in the high-dimensional neural space is difficult because we observe only a limited number of patterns relative to the dimensionality of the space. Instead, we leveraged our BCI framework to look for the emergence of new neural activity patterns within the low-dimensional space specified by the BCI mapping. We started by defining the patterns of neural activity observed before learning as the “intuitive neural repertoire” (9). Then we projected the ∼90D neural activity patterns comprising the intuitive neural repertoire into the 2D velocity space defined by the OMP mapping (Fig. 2A). This defines the limits on cursor velocities the monkey could produce through the OMP mapping if he only expressed patterns of neural activity from within his intuitive neural repertoire (Fig. 2B and SI Appendix, Materials and Methods). We term this the neural “speed limit.” Any cursor velocities we observe after learning that exceed the speed limit must have arisen from neural activity patterns that are outside of the intuitive neural repertoire, and thus are new (Fig. 2 B and C). Over the course of many days, monkeys learned to move the cursor at velocities that exceeded the speed limit for many targets (Fig. 2 D–F). The percentage of neural activity patterns that are new significantly increased over days (t test, P = 0.0015; Fig. 2G). This shows that the brain can generate new neural activity patterns when learning to perform a new skill, but that it takes several days to do so.

Fig. 2.

New neural activity patterns emerge with long-term BCI learning. (A–C) A schematic of the technique used to identify outside-repertoire activity patterns. The OMP maps ∼90D neural population activity patterns to 2D cursor velocities. Here we illustrate using 3D neural activity patterns and a 2D OMP mapping. (A) The neural activity patterns (orange dots) generated by the animal while using the intuitive mapping define an intuitive neural repertoire (dark gray ellipsoid). Each neural activity pattern maps to a cursor velocity (orange X, as one example) through the OMP mapping (blue plane). (B, Left) The velocities predicted from the intuitive neural repertoire (orange Xs) through the OMP mapping define a speed limit (dashed gray ellipse). (Right) After learning, cursor velocities are observed that exceed the speed limit (three Xs outside of gray ellipse). (C) If the monkey produces cursor velocities that exceed the speed limit, those velocities were generated by neural activity patterns that lie outside of the intuitive neural repertoire and thus are new. (D–F) Using the speed limit to detect new neural activity patterns for an example experiment (OMP 1 from Fig. 1). (D) Velocities generated from the intuitive neural repertoire mapped through the OMP mapping. Each dot is the velocity resulting from one neural activity pattern (45-ms bin). Dots are colored by instructed target location (Inset). The speed limit is defined as the 95% convex hull (gray dashed line). By definition, 5% of the neural activity patterns are outside of the speed limit. (E) Day 1 velocities, generated while using the OMP mapping, mostly fell within the speed limit. Dots shown are from the 40 consecutive trials when behavior was the best on day 1. On these trials, 6% of the neural activity patterns were outside of the speed limit. (F) On day 8, some velocities exceeded the speed limit (e.g., patterns corresponding to purple, blue, and teal targets). Same conventions as in E. (G) The percentage of neural activity patterns that were new on the last day of OMP learning exceeds the percentage seen on the first day of learning for most experiments. Each symbol is one multiday OMP learning experiment. OMP1 is indicated in black.

The third objective of our study is to show that the new neural activity patterns caused the behavioral improvements. In conventional learning studies, neural changes are observed that accompany learning, but it has been difficult to know if those changes are directly responsible for the learned behavior. A BCI allows us to assess the behavioral consequence of any given neural activity pattern. To assess the behavioral impact of each neural activity pattern, we measured the component of cursor velocity in the direction of the target, which we term “progress” (Fig. 3A). Higher progress indicates straighter and/or faster cursor movements. Over the course of a multiday experiment, progress improved (Fig. 3B). Increases in progress were positively correlated with the emergence of more new neural activity patterns (Fig. 3C). This indicates that the monkey learned to move the cursor faster and straighter to the target in part by producing new neural activity patterns.

Fig. 3.

New neural activity patterns drive behavioral improvements. (A) Illustration of the progress metric. Progress is defined as the component of cursor velocity in the cursor-to-target direction. The + represents the center of the screen. Gray circles are cursor positions at previous time points. (B) Mean progress toward each target on day 1 (thin) and day 8 (thick) for an example experiment (OMP 1). (C) The cursor movements showed more progress when there was a larger percentage of neural activity patterns that were new (Pearson correlation coefficient r = 0.76, P = 6 × 10⁻¹⁶), and generally increased with several days of practice. Each symbol is the mean progress averaged over all eight targets on 1 d of one multiday OMP experiment. The shading of the symbols indicates the day within a given multiday experiment. In general, earlier days showed relatively few new patterns, and later days showed more new patterns and better progress.

Before we can conclude that new neural activity patterns emerge due to learning, we need to ensure that they do not emerge by chance or due to neural recording instabilities. We performed three key controls to address this. First, we show that the new activity patterns were specific to the OMP being learned, and not generally helpful through other OMP mappings. To test this, we mapped the neural activity patterns generated by the monkey after learning through 500 random OMP mappings to assess how much apparent learning would have occurred with mappings which the monkey never experienced. The apparent learning rarely exceeded the learning observed with the OMP mappings which the monkey did experience (SI Appendix, Fig. S2). This rules out the possibility that the animal learned a general strategy that would work for any OMP.

Second, we show that new patterns do not emerge without substantial learning pressure. That is, they do not appear by chance or because of neural recording instabilities. To assess this, we conducted control experiments in which the monkey used the same intuitive mapping for several days. This is a low learning pressure scenario because while control is good from the beginning of the experiment, there may still be incentive for the animal to increase his reward rate. The recording instabilities during these control experiments were indistinguishable from our multiday OMP learning experiments (SI Appendix, Fig. S3), and, importantly, they do not result in the same emergence of new patterns of neural activity (SI Appendix, Fig. S4B). We also mapped the neural activity recorded during multiday intuitive mapping experiments through 500 random OMP mappings, which the monkey never experienced (SI Appendix, Fig. S4C). This is a “no learning pressure” scenario because the monkey never received feedback about how his neural activity would have impacted behavior through these mappings. The day-to-day recording instabilities in a no learning pressure scenario did not manifest as learning (SI Appendix, Fig. S4D).

Third, we assessed the extent to which the new activity patterns moved outside of the speed limit during learning. If new neural activity patterns are formed by expanding the neural repertoire, the distance of the new patterns from the speed limit should increase with learning. We found that in multiday OMP learning, not only does the percentage of patterns that are new increase with learning (Fig. 2 and SI Appendix, Fig. S4 A and D), but also the distance of the new patterns from the speed limit increases (SI Appendix, Fig. S5). Taken together these controls confirm that the new neural activity patterns that emerge during multiday OMP learning are specific to the learned mapping, are directly responsible for behavioral improvements, and cannot be attributed to recording instabilities or other chance events.

We have shown that learning can proceed by the formation of new neural activity patterns that directly drive behavioral improvements. We wondered what characteristics make these new neural activity patterns different from the existing patterns. It might be that the new patterns are well explained by the preexisting correlation structure, which is captured by the intrinsic manifold. Such new patterns could arise from organized changes in neural firing rates that conform to the preexisting correlation structure. We characterize this as an inside-manifold change (Fig. 4A). Alternatively, it might be that the new patterns arise from changes in the correlations between the neurons. We characterize this as an outside-manifold change (Fig. 4B). To determine the extent to which the new patterns generated by the animal resulted from firing rate changes or correlation changes, we decomposed each neural activity pattern into an inside-manifold component and an outside-manifold component (SI Appendix, Fig. S6 A and B). This enabled us to separately determine the inside-manifold and outside-manifold contributions to progress (SI Appendix, Materials and Methods). The new neural activity patterns on the last day of a multiday learning experiment include patterns with both substantial inside- and outside-manifold contributions to progress (Fig. 4C and SI Appendix, Fig. S6 C–E). This means that animals learned to move the cursor to some targets by generating new neural activity patterns that were outside of the intuitive neural repertoire but within the manifold, whereas other targets were learned by generating new neural activity patterns that were both outside of the intuitive neural repertoire and outside of the intrinsic manifold. Thus, learning can occur by changing the correlation structure (SI Appendix, Fig. S6C, blue), and also by changing firing rates in a manner that preserves the correlation structure (SI Appendix, Fig. S6C, red). We conclude that the brain can overcome the neural constraints imposed by the intrinsic manifold that we previously observed during single-day learning (8), but doing so takes several days.

Fig. 4.

Monkeys can produce new neural activity patterns outside of the intrinsic manifold. There are two types of new neural activity patterns: (A) those that are outside the repertoire, but remain within the manifold, and (B) those that are outside the manifold. Either type can yield performance improvements. (C) Animals learn using both inside-manifold and outside-manifold strategies for a given OMP mapping. Each bar shows one target from one multiday OMP experiment. The overall learning, defined as change in progress from day 1 to the last day, is represented by the green dot. Targets are ordered based on the amount of learning. The inside-manifold contributions to that learning are shown in red. The outside-manifold contributions are shown in blue. For visual clarity, data presented here show only targets with behavioral improvement and only the helpful contributions are shown. See SI Appendix, Fig. S6 for a full presentation of these data.

Discussion

We found support for our hypothesis that populations of neurons can produce new patterns of activity to enable new behavioral capacities. Our BCI approach allowed us to establish a causal link from changes in neural activity patterns to changes in behavior. We encouraged new neural activity patterns to form by presenting the monkey with novel BCI mappings (OMPs). We detected the emergence of new patterns of neural activity after several days of practice with each OMP mapping. These patterns led directly to improved behavior. Some new patterns conformed to the preexisting correlation structure among the population of neurons, whereas other new patterns represented changes in the correlation structure. We expect that skill learning in general may proceed in part by the formation of new neural activity patterns appropriate for the learned behavior (19).

We can interpret changes in the brain at other levels of organization in the context of changes in neural populations. For example, learning has been associated with synaptic plasticity and the expansion of cortical maps. The time course of learning that we observed is consistent with findings that cortical synaptogenesis and motor map reorganization occur during late phases of skill learning (20). We posit that OMP learning involves synaptic plasticity. In fact, a function of synaptic plasticity could be to permit a network of neurons to generate new patterns of activity. This plasticity may occur among the neurons in M1 from which we record, or among neurons from which we do not record, but that drive the recorded population.

Learning has been associated with changes in tuning properties of individual neurons in previous BCI (6, 7) and motor skill learning (21) studies. We posit that the formation of new neural activity patterns during long-term BCI learning may provide a parsimonious explanation for the tuning curve changes reported in earlier studies. In particular, our results combined with earlier BCI studies (6–9, 15, 16, 18, 22–26) and motor learning studies (27, 28) suggest that fast and slow learning are driven by different neural mechanisms. Fast learning can be accomplished by reassociating preexisting patterns of neural activity with new behaviors (9). This would result in neural tuning changes that are coordinated across the population (6, 29). Slow learning, as reported here for OMP mappings, can involve the formation of new patterns of neural activity. This would correspond to neural tuning changes that are specific to individual neurons (29). Other slowly learned phenomena that may also entail the creation of new patterns of neural population activity include motor skill, or de novo (30), learning (31, 32), cognitive learning (33, 34), and stroke recovery (35, 36). In the future, it might be possible to facilitate learning in those contexts by directly guiding the formation of new neural activity patterns appropriate for the desired behavioral capacities, as we have done here.

Materials and Methods

We used a BCI paradigm in which the monkey moved a computer cursor from the center of a screen to one of eight peripheral targets by volitionally modulating the activity of a population of ∼90 neural units recorded with a multielectrode array chronically implanted in the arm region of primary motor cortex. Each experiment began by presenting the monkey with an “intuitive” mapping that relates neural activity to cursor velocities in a way that provides proficient control without requiring the animal to learn. Then, we induced learning by presenting the monkey with a novel mapping from neural activity to cursor velocity. The novel mappings encouraged the formation of new neural activity patterns. Subsequent analyses, demonstrate that the novel mappings are learned by the formation of new patterns of neural activity, and that those patterns directly drive the new behavior.