Carrot or stick in motor learning

Abstract

A study shows that reward and punishment have distinct influences on motor adaptation. Punishing mistakes accelerates adaptation, whereas rewarding good behavior improves retention.

We both love salsa dancing, but learning salsa is not easy. When one partner misses a step, the other may punish him with a frown, but when he masters a new move, her praise rewards him—or does it make him complacent? Carrot or stick: the manner by which reward and punishment affects motor learning is a long-standing question in education, sports, therapy and beyond. In this issue of Nature Neuroscience, Galea et al.¹ address this question using a simple reaching task in a perturbed visual environment.

The authors build on previous studies that have shown the crucial importance of reward on retention^2,3, but they now contrast the effect of reward with that of punishment by differentiating their effects on acquisition rate and retention. This study examined participants moving a cursor to targets displayed on a screen, steering with their hand movements hidden from view. To create a learning challenge, the cursor position was rotated by 30 degrees and participants had to practice to successfully reach the target. This exercise is similar to moving a computer mouse when you turn it upside down: a challenge that one masters with practice. The specific question of this paper was how money received for good performance (reward) or lost for bad performance (punishment) would affect the rate of learning and the retention of the acquired performance. The authors found that punishment accelerated the rate of adaptation, whereas reward improved retention of the new mapping.

This study is at the crossroads of at least three research disciplines that have examined the consequences of motivational and error feedback on motor performance: neuroscience, computational science and, of course, psychology (Fig. 1). These fields have approached this issue in different ways and each can inform and motivate future directions in motor control.

Figure 1.

Motor control is at the intersection of three fields: psychology, neuroscience and computational science. The metaphors for understanding human behavior differ across fields: neuroscience focuses on the brain, computational science stresses the robot analogy, and psychology addresses behavioral, cognitive and social processes.

In psychology, reward and punishment have long been recognized as instrumental for learning. As early as 1898, Edward Thorndike’s law of effect stated that if a response leads to a “satisfying state of affairs” it will be strengthened and, conversely, if it leads to unpleasant consequences it will be weakened⁴. The thesis that reward is a better motivator than punishment was also at the core of B.F. Skinner’s principle of reinforcement. Operant conditioning developed systematic reinforcement schedules to enhance learning and thereby shape behavior⁵. However, social psychologists have also reminded us that human nature is far more nuanced and more than a collection of systematically reinforced associations. Many studies have highlighted the mediating effects of emotions, such as threat, anxiety, pride or shame, on behavior. Invoking stereotypes, such as inferior performance of females in mathematics or athletics, lowers test performance. There are also inter-individual differences: some individuals thrive under competitive stress, whereas others choke under pressure⁶. Motivation, risk and reward, as well as individual differences, critically affect learning.

Psychology has therefore highlighted many facets of learning that research on motor control has thus far neglected⁷. Galea et al.¹ now present a simple approach for probing the motivational effect of two long-known factors affecting learning and retention: reward and punishment. Taking a closer look at the various psychological determinants and incorporating them into experiments and learning models may be an important direction for future motor control research.

Much research in neuroscience has focused on the brain areas and neurons involved in the various aspects of motor learning, including reward and punishment. For example, we know from imaging studies that many cortical and subcortical structures are affected by positive and negative reinforcement⁸. For example, some neurons in the ventral tegmental area can be tuned to reward and others to punishment⁹. With its focus on mechanism, neuroscience provides a knowledge base for formulating models and contributes to our understanding that reward and punishment have multifaceted effects on motor learning.

Neuroscience offers many opportunities for the behavioral study of motor learning, as it reveals targets for interventions and mechanisms to interpret behavior. In this spirit, the study by Galea et al.¹ attributes reward-based learning to the motor cortex and learning enhanced by negative reinforcement to the cerebellum. The interpretation of cerebellar learning from negative feedback, however, is very indirect, and it was recently shown that errors can overpower rewards¹⁰. Although Galea et al. provide an example, there is great potential for further connections between behavior and neuroscience. First, targeted interventions via transcranial magnetic stimulation can reveal the contribution of the respective brain area. Lesion studies can establish causality. For example, if lesioning the cerebellum largely abolishes fast eye movement adaptation, then that would be evidence of its causal role¹¹. Second, extending current mathematical models of plasticity in the nervous system promises to make precise testable predictions of behavior. It is important for motor control to take input from neuroscience and match the current simple learning models to the new insights from the nervous system.

In computational science, humans are typically viewed as problem solvers. For a given task, the human learns to compute the optimal solution, just as we would program a robot to perform this task. This perspective has led scientists to employ algorithms such as Kalman filters to describe processes underlying motor learning¹². Describing humans as optimal problem solvers implies that the outcome, such as winning the jackpot or paying penalties, should be irrelevant for motor learning. After all, reward and punishment contain, in principle, the same information. Consequently, modeling approaches have ignored, if not rejected, the influence of factors such as reward and punishment. Learning is regarded as an iterative process of correcting errors and optimizing cost functions, such as energy, effort or time spent. Computational motor control has produced a range of models that explain data, but psychological factors clearly cannot be ignored.

Computational science can contribute a broad set of modeling and data analysis techniques to motor control. The simple, highly standardized task of reaching in a remapped virtual environment affords precise manipulations, measurements and mathematical modeling. The manageable nature of the data facilitates differentiating the effects of positive and negative reinforcement during learning and retention. There is an opportunity to further disentangle how these effects interact with other known forms of motor learning. Can the findings be described by interacting explicit and implicit processes¹³? How does motivation affect learning over different time scales¹⁴? How can we construct models that apply across a broad set of experiments¹⁵? What is the effect of a given reward size on the stability of motor memories? Computational motor control is now poised to start exploring such additional determinants for motor learning.

Coming back to salsa dancing: should he praise her successful spin? Should she frown when he does not finish the turn in time? The study by Galea et al.¹ suggests an answer, although the simple reaching movement of this study is worlds away from the sophistication of a dance move. Studying a skill as complex as salsa is even more challenging than dancing it. Can motor control rise to the challenge?