Structural constraints on learning in the neural network

Abstract

Recent research suggests the brain can learn almost any brain-computer interface (BCI) configuration; however, contrasting behavioral evidence from structural learning theory argues that previous experience facilitates, or impedes, future learning. A study by Sadtler and colleagues (Nature 512: 423–426, 2014) used BCI to demonstrate that neural network structural characteristics constrain learning, a finding that might also provide insight into how the brain responds to and recovers after injury.

in motor behavior, previous experience can facilitate or impede future learning (Hall et al. 1995). Structural learning theory has recently emerged in motor learning research as a hypothesis that learning is facilitated when a new motor task shares similar characteristics as a previously learned task (Braun et al. 2009, 2010); however, the underlying neural mechanisms are not clear. Recent studies involving brain-computer interface (BCI) paradigms may provide answers to these questions, because BCI allows observation of individual neuron activity as well as patterns of neural connectivity (Sadtler et al. 2014). In a research letter, Sadtler et al. argue that the neural network's natural patterns of comodulation set constraints on learning-facilitating behaviors that involve existing neural network relationships and impeding behaviors that require the development of new connections. They use BCI to explore if the neural network constrains learning, and their findings may shed light on the challenges of motor control recovery after neurological injury.

Behavioral research has demonstrated that the degree of difficulty to learn something new is experience-dependent (Wolpert et al. 2001). Studies on motor skill transfer show that a previously learned motor skill can influence an individual's ability to learn a future task, suggesting that the brain generalizes some information from previous experiences (Magill and Hall 1990). Understanding how the brain abstracts information about one learned task to a novel task is a problem of adaptive control addressed by structural learning theory (Braun et al. 2009, 2010; Krakauer and Mazzoni 2011). In structural learning theory, it is proposed that when learning a novel motor task, the brain identifies a group of control variables that impact task performance (Braun et al. 2009, 2010; Ranganathan et al. 2014). For instance, when learning to ride a bicycle, the rider develops a general strategy to successfully balance (Braun et al. 2009). The control variables involved in bicycle riding include characteristics of the bicycle (its weight and height), the environmental context (riding on a straight vs. curved road), and the rider (muscle systems involved in pedaling and balance). As the solution to riding the bicycle emerges through exploratory practice, the intervariable relationships to achieve the desired outcome amount to a “structure” that the rider learns. Structural learning theory argues that it is advantageous to define and control relationships between variables, rather than control the variables independently, because there are fewer relationships than there are variables. Thus searching for a successful solution among a subset of relationships would enable faster learning than searching for a solution among many control variables (Braun et al. 2010).

In structural learning theory, one hypothesis is that learning is facilitated when a new task shares similar structures as a previously learned task. For example, learning to ride a tandem bicycle or a motorcycle likely involves similar control variables and variable relationships as riding a bicycle. Rather than search for an entirely new control strategy specific to the task, it would be more efficient for the brain to explore within a subspace of control strategies identified through previous bicycle-riding experience. On the other hand, learning may be impeded when the structure of a new task is different from previously learned motor tasks, because exploration during new skill acquisition may remain within the control subspace of the previous task, as demonstrated by Ranganathan et al. (2014).

Although the application of structural learning theory to motor control is relatively recent, this theoretical construct has been adopted in other disciplines such as neuropsychology, engineering, and computer science as a way to address problems of system control. The element of structural learning theory that links across disciplines is the concept of dimensionality reduction. This term refers to a process or operation that converts a large number of variables within a high-dimensional space to a smaller number of variables, where each new variable represents a relationship among the control variables of the high-dimensional space (Cunningham and Yu 2014).

BCI is a paradigm in which dimensionality reduction may be particularly useful, because the high volume of neural activity could be reduced to a low-dimensional subset of comodulation patterns. Consider, for example, a three-neuron system that is used to control an object. Achieving a goal of moving the object to a target requires adjusting the firing rates of the three neurons. In this system, there are three variables. Sadtler et al. (2014) refer to this high-dimensional space as a “neural space” comprising the collective neural activity, and each dimension of the neural space represents the firing rate of one neuron (see Fig. 1). Each point in the neural space equally corresponds to a possible combination of the three neuron's activity. Moving the object requires exploring the neural space to find at what point the population activity will achieve the goal. If the firing rates of the three neurons are independent of each other, then the three-dimensional neural space is homogeneous and every point in the space equally represents a possible solution to reach the target.

Fig. 1.

A simplified representation of the intrinsic manifold, as described by Sadtler et al. (2014). Each point represents the firing rate of 3 neurons within the 3-dimensional neural space. The black plane represents the 2-dimensional intrinsic manifold, in which the firing rates of the neurons naturally comodulate during passive observation of cursor movement. The initial brain-computer interface mapping within the intrinsic manifold (yellow) was rotated so that it remained within the manifold (red) or outside of the manifold (blue).

If, however, there is a strong inherent correlation between neurons 1 and 2, then neurons 1 and 2 covary, and the neural space is reduced to a subset of all possible neural population activity in which neurons 1 and 2 covary. Sadtler et al. (2014) refer to a reduced space in which the two neurons correlate as an “intrinsic manifold.” Neural firing patterns that sustain the relationships between neurons 1 and 2 are considered “within manifold.” If the solution to reach the target requires neurons 1 and 2 not to covary, this firing pattern is considered “outside manifold.” Training on a novel task that required neurons 1 and 2 not to covary would, presumably, be more difficult than on a task in which they covaried. In their work, Sadtler et al. tested the hypothesis that learning within intrinsic manifold is easier than learning outside of manifold.

The experiment involved two monkeys with motor cortex (M1) implantation, using a closed-loop BCI system. To perform the dimensionality reduction, the authors first identified a neural space within M1 comprising ∼85 neurons. During the calibration phase of the study, the monkeys passively observed a center-out task, a commonly used task in which a cursor on a video screen moves from the center of a circle to one of several targets along the circumference. The neural activity in M1 during passive observation was recorded, and 10 relationships of neural firing among the neural space's 85 neurons were identified. This subset of relationships was the intrinsic manifold.

Following calibration, the monkeys were trained to control the cursor using a BCI mapping (i.e., how the cursor's movement would respond to the neural firing patterns) that corresponded to neural firing patterns occurring within the intrinsic manifold. After the first period of training each day, the BCI mapping was rotated, perturbing the previously learned solution. In one condition, the new rotation stayed within the intrinsic manifold, preserving the relationships between neurons. In the other condition, the rotation was outside manifold, and the solution required the monkeys to engage neurons within the neural space but using different relationships than those that defined the intrinsic manifold. Sadtler et al. (2014) hypothesized that the within-manifold perturbation would be easier to learn than the outside-manifold perturbation.

As the authors noted, a significant and novel aspect of their experiment was to use dimensionality reduction to identify the intrinsic manifold, which in turn, served as the foundation for the experimental manipulation. Consistent with their hypothesis, they demonstrated that the monkeys learned the within-manifold perturbation, but did not learn the outside-manifold perturbation, during a single-day training session. Furthermore, they ruled out several alternate explanations by ensuring that 1) the outside-manifold BCI mapping was the same degree of difficulty to learn as the within-manifold mapping, 2) the outside-manifold mapping required the same exploration through the neural space as the within-manifold mapping, 3) the rotations within and outside manifolds were the same, 4) the outside-manifold mapping's search space was of comparable size to that of the within-manifold mapping, and 5) physical movement did not occur and could not contribute to the tasks' learnability.

This study's practical contribution is to demonstrate that in BCI paradigms, dimensionality reduction reveals characteristics of neural network activity that can be used to inform BCI development. Sadtler et al.'s observation that both motor behavior and neural activity may be described in formal terms through dimensionality reduction is intriguing, and this study supports the notion that, similar to motor behavior, some BCI configurations are easier to learn than others, at least during a single-day training period. In sum, the authors present a rigorous and well-executed study that clearly demonstrates the impact that within- and outside-manifold perturbations can have on BCI learning.

Sadtler et al. (2014) conclude by drawing connections from their findings back to motor behavior, arguing that learning within- and outside-manifold perturbations may represent mechanisms that underlie learning in general. Although the authors provide compelling evidence in the context of BCI learning, it is not clear if the same neural network constraints observed in the study are constraints of cortical organization as a whole. There are at least two reasons why the rules may be different. One reason is that learning, even in a simplified laboratory paradigm such as a center-out task, likely involves more neural networks and manifolds than those captured by the BCI paradigm (Krakauer and Mazzoni 2011). The authors point out this limitation as well, noting that the dimensionality of the motor cortex may be much greater than what was represented in their report, and thus the degree to which this observation can be generalized is contingent on further research.

A second reason why these findings may not generalize to learning is that the intrinsic manifold may only exist when a population of neurons is constrained by BCI. The concepts of within- and outside-manifold networks may dissolve when the whole brain is considered as a unit, that is, unless an external factor, such as a BCI electrode, is introduced to the system. A further implication could be that introducing constraints impacts neural network connectivity.

From this perspective, the findings of Sadtler et al. (2014) may relate more to properties of a lesioned brain rather than a healthy one. For example, after cortical stroke, neural connections are disrupted due to tissue damage, and in BCI, normal neural relationships are disrupted because the BCI mapping imposes artificial limits regarding what neural activity contributes to cursor control. The disruption in connectivity in both cases may be the source of outside-manifold conditions. Thus the limits to learning that were observed in the outside-manifold perturbation might shed light on postinjury neural network control better than they reveal general principles of learning in the healthy brain.

To that end, one is left wondering how much practice it would take to learn outside-manifold perturbations, if learning them through practice is indeed possible. This issue is beyond the scope of Sadtler et al.'s article, but the answer may help in understanding how much practice is needed to drive neural network recovery after cortical injury. An avenue for further exploration could be to utilize BCI to model neurological diseases such as stroke and to develop BCI paradigms that impose network level constraints similar to those that occur with cortical lesions as a way to expose how the brain adapts to, and potentially recovers from, injury.

Brain-computer interface holds promise to deliver new ways for people to interact with machines, and to provide access for people with disabilities who are unable to move on their own. Additionally, as Sadtler et al. (2014) demonstrate, BCI can provide critical information about the neural network. Future work in rehabilitation research may benefit from studies in BCI to model neurologic disease in the healthy brain.

GRANTS

Research reported in this publication was supported by the Eunice Kennedy Shriver National Institute Of Child Health and Human Development of the National Institutes of Health under Award nos. T32-HD-064578 and R01-HD-065438.

DISCLAIMER

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

C.A.M. and C.W. conception and design of research; C.A.M. and C.W. prepared figures; C.A.M. drafted manuscript; C.A.M. and C.W. edited and revised manuscript; C.A.M. and C.W. approved final version of manuscript.