Decision-making in sensorimotor control

Abstract

Skilled sensorimotor interactions with the world result from a series of decision-making processes that determine, based on information extracted during the unfolding sequence of events, which movements to make and when and how to make them. Despite this inherent link between decision-making and sensorimotor control, research into each of these two areas has largely evolved in isolation, and it is only relatively recently that researchers have begun investigating how they interact and, together, influence behaviour. Here we review recent behavioural, neurophysiological and computational research that highlights the role of decision-making processes in the selection, planning and control of goal-directed movements in humans and nonhuman primates.

Introduction

Real-world tasks typically involve a sequence of actions that are performed to achieve a high-level goal. Such tasks engage decision-making processes that determine which movement to make next and when to make it, how those movements that are selected are planned and controlled, and how movements and task goals are dynamically updated in response to changes in the world.

As an example of the inherent links between decision-making and action planning, consider a tennis player. Even at the level of planning and executing a single movement (such as a backhand aimed up the line), significant decision making is involved. Specifically, the player must decide how to coordinate and execute the shot to trade off reward (shot success) and costs (energy, injury). The first section of this Review will discuss how various costs and rewards shape the planning and execution of single movements associated with a single goal.

At another level of decision-making, the player must choose, often under time pressure, which of many possible movements they should perform. This involves selecting where to aim their shot (short or long, for example) and which type of shot to perform (such as a drop shot or lob). In the second section of this Review, we discuss how the brain plans and executes movements when presented with multiple potential targets or goals (and thus multiple potential movements), and how this is affected when there is limited time to choose between them.

As the player prepares to execute a given shot, having selected a movement goal, their opponent’s actions (such as running to the net) might force a revision of the goal. In the third section, we will discuss how information about choice options that arrives over time can result in modification of the goal and hence the ongoing movement.

Employing decision making at a strategic level, a player will often prepare a sequence of movements designed to ultimately win the point (they may serve wide, move to the net, and volley to the open court, for example). In our final section, we will discuss how decisions related to a sequence of movements tend to optimize performance across a task, and maximize the extraction of task-relevant information.

In this review we discuss how decision-making processes involved in the shaping, selection, revision and sequencing of movement operate to guide sensorimotor behaviour. Whereas traditional models and theories have viewed the processes of decision-making as being distinct from those involved in action planning and control, a key theme to emerge from this review is that decision making, in the context of action, dynamically interacts with the sensorimotor system at multiple levels. In exploring this theme, and integrating insights gained from behavioural, neurophysiological and computational approaches, we hope to provide insight into the bases on which sensorimotor decisions are formed and implemented, as well as constraints on biologically plausible models of decision-making.

Movements directed to a single goal

Building on the seminal work of Woodworth ¹, the generation of purposeful movement has traditionally been conceptualized as involving two distinct phases: a pre-movement planning phase, in which key parameters of the upcoming action are specified and readied for implementation, and a separate control phase, in which online corrective processes fine-tune the movement to ensure successful completion ^2–4. When researchers began developing quantitative models of goal-directed actions in the 1980s, the initial focus of this work was on movement planning. In particular, a number of models were developed to address what is known as the problem of redundancy — the fact that the goal of an action, such as to grasp a cup, can often be achieved with any number of different movement trajectories. One way of resolving this problem is to select motor commands that minimize some cost — a scalar measure that characterizes some attribute of a particular movement. In the context of target-directed reaching movements, a number of such costs have been proposed, including the variability of the final hand position (across repeated movements) ⁵ and the jerkiness (changes in acceleration) of the trajectory of the hand during the reach ⁶. Similarly, it has been shown that motor commands can be selected to optimize explicit reward. For example, when pointing towards a rewarding target surrounded by penalty areas, people choose an aim location that optimizes reward while taking into account natural movement variability ^7,8.

These early models focused on feedforward planning and often assumed that such planning involves specifying a desired trajectory (for example, a trajectory in which jerkiness is minimized and the hand moves smoothly in a straight line to the target) that the motor system attempts to generate. However, they tended to put little emphasis on the control processes through which ongoing sensory feedback is used to guide action. More recent accounts of goal-directed movement, such as optimal feedback control (OFC) models ^9,10,11, emphasize an important role for feedback control in target-directed reaching. According to such models, controlling a movement involves the selection of a ‘control policy’ that governs how sensory feedback will be used in real-time to generate motor commands. For example, the control policy will specify ‘feedback gains’ that determine how robustly the motor system will respond to mismatches between the current state of the arm and the final goal state. In addition to shaping how the arm is driven towards the goal, these gains determine how the system responds to errors that may arise due to natural variability in motor commands ¹² or external perturbations ¹³. The control policy thus determines how the movement will evolve as a function of the state of the motor system and how the motor system will handle errors.

It is important to note that the principle of optimization is still central to OFC models. The parameters of the control policy are selected to minimize a cost, which is typically defined as a combination of energy expenditure and inaccuracy¹¹. However, the traditional distinction between movement planning and movement control is blurred in these models because the control policy can specify, based on the initial state of the system and the goal state, the motor commands involved in initiating movement. According to OFC models, movement planning is concerned with the specification of feedback gains and movement control is the use of these feedback gains to drive movement.

A critical feature of OFC models is the concept of ‘minimum intervention’, whereby sensory feedback is used to correct movement errors that interfere with the goal of the action but not errors that are irrelevant to the goal. Intervening to correct for errors that do not threaten the goal is undesirable because such intervention will generally require increased effort and add noise into the system. The prediction of the minimum intervention principle also provides an opportunity to directly test desired trajectory models against OFC models. Imagine that your hand is bumped sideways when reaching towards either a small knob or a wide lever that opens a door. The OFC model predicts a stronger lateral correction of the hand movement when reaching towards the knob in comparison to the lever, since the goal of opening the door can still be achieved by contacting the lever at a different location than might have originally been planned. However, desired trajectory models predict equally strong corrections in both of these two contexts, since the putative aim is to follow a particular trajectory. Experimental tests along these lines have provided evidence for the minimum intervention principle, and hence support for OFC models (Fig. 1a) ^14,15.

Fig. 1. Corrective motor responses are tuned to task features.

a| In this task, participants were required to perform target-directed reaching movements using a robotic interface. On some trials, small mechanical perturbations were applied to the limb during the task. Traces show the paths of individual hand movements, recorded using the robotic interface¹⁵. Unperturbed movements to narrow or wide targets tend to be straight and to move to the closest point on the target. However, the application of mechanical loads immediately after movement onset disrupts execution of the planned movement. The resulting hand movement paths obey the principle of minimum intervention. That is, for a narrow target, the hand paths correct to enable the participant to reach the target. For a wide target, no correction is necessary and the hand just reaches to another point on the target. b-c| Visual attention influences the processing of target, but not hand-movement information ¹⁶. In this experiment, subjects performed a bimanual reach task, using a robotic interface to move two cursors to their respective targets. During the task, one of the cursors or one of the targets was displaced and the corrective reflex response was measured as the lateral force applied to the robotic interface by the participant. In addition, immediately after the onset of movement, a flash of light could draw attention either towards (perturbation attended) or away from (perturbation unattended) the side of the perturbation (b). There was no significant difference in the onset of the lateral corrective forces that occurred in response to cursor displacement when flashes were on the attended versus non attended sides (c, left panel); however, there was a significant delay in the corrective response to target displacement when visual attention was drawn away from the side of the perturbation (c, right panel). Vertical dashed lines represent the time window over which the corrective forces were averaged to obtain an estimate of the strength of reflex response (from 30 ms per- to 70 post-response onset). Gray horizontal line represents zero lateral force.. Panels b-c show that distracting visual information is more efficiently filtered during the extraction of hand compared to target information, suggesting a specialized mechanism that links representation of the hand in visual and motor systems. Part a is modified, with permission from REF164. Parts b and c are modified, with permission from REF 16.

The ability to respond quickly to perturbations is critical for skilled and dexterous motor behaviour and a recent focus of work in the field has been to examine the nature and timing of the sensorimotor mechanisms that lead to movement corrections based on visual^16,17, proprioceptive^15,18, and even tactile ¹⁹ feedback. A common approach used in this work to study the rapid updating of goal-directed limb movements is to examine how human participants respond to small mechanical perturbations of the limb ²⁰. These studies have shown that in addition to spinal-generated corrective responses (termed R1 responses), which reflect simple muscle stretch reflexes and can be observed in electromyogram (EMG) activity within 25-50 ms of the perturbation ²¹, mechanical perturbations also give rise to a second phase of EMG corrective responses, termed R2 responses. R2 responses occur 60 ms after the perturbation and, consistent with being transcortical in nature, can exhibit remarkable sophistication: they have been shown to reflect the physics of the limb and environment ^22–24 as well as features of the target, such as size ¹⁵. Evidence suggests that R2 responses are in continuous operation, as they occur even for very small disturbances in limb position that are close to the natural variability of limb motion ²⁵. In bimanual motor tasks, in which one hand is perturbed, rapid R2 responses can also be seen on the unperturbed hand, albeit with a small delay (~10 ms), indicating the fast integration and coordination of sensory feedback processing across the cortical hemispheres ^26,27.

When considering the types of sensory information used in rapid movement corrections, the role of visual feedback has received considerable attention. One way in which this has been typically studied has been to examine how the availability or removal of the ability to see the hand at certain times prior to or during target-directed reaching affects movement direction accuracy and the reach trajectory ^28–30. A general theme to emerge from this work is that early (initial) and late (terminal) components of a reach movement are differentially impacted by the removal of visual feedback, suggesting that these components are governed by distinct sub-processes ^31,32. Other studies have examined cases in which, during the reach movement, either the reach target is displaced ^6,33 or a visual representation of the hand’s position (such as a cursor) is altered ³⁴. Although both of these manipulations in humans result in rapid corrective responses (within ~160 ms ^35,36) the influence of the presence of distractors on the corrections made for each type of perturbation suggest that separate mechanisms are involved in processing the two types of visual information ^16,37 (Fig. 1b,c). Specifically, the fact that corrections to displacements of the hand cursor are relatively unaffected by distractors, in comparison to corrections to displacement of the target, may indicate that there is a dedicated allocation of attentional resources to visual feedback processing of the hand’s position ^16,37 This may also reflect, in part, the availability of proprioceptive feedback for the limb that can be used in estimating cursor position, but that is not similarly available for estimating target position.

Choosing between competing action goals

In many everyday actions, we must select a particular target from among multiple alternatives. According to traditional serial models of action planning ^38,39, we first select the target object and only then specify and prepare the corresponding goal-directed movement. However, this serial processing view has been challenged by both behavioural and neurophysiological evidence suggesting that these two processes — selection and specification — can operate continuously and in parallel ⁴⁰.

Some of the earliest evidence for this idea stems from behavioural findings showing that the movement trajectories of individuals reaching towards targets will, depending on the context, initially deviate toward non-target stimuli placed nearby ^41–44. This suggests that, in some cases, movement specification can precede target selection. Consistent with this idea, neurophysiological studies in humans and non-human primates have shown that competing reach targets elicit separate neural representations in sensorimotor brain areas prior to one of the targets being selected ^45–49. For example, it has been demonstrated that when there is uncertainty about which of two rules — each specifying a particular movement goal — would be applied to a single spatial cue, neurons in frontoparietal reach areas simultaneously represent both movement goals prior to rule specification (Fig. 2a) ^{50, 45}. Notably, this was despite the fact that the sequence of events during the task afforded the animals the possibility of first waiting for the rule to be specified and only then representing the single corresponding movement associated with that rule.

Fig. 2. Multiple potential actions can be specified before target selection.

a| Schematic depiction of an experiment measuring neural activity in the parietal cortex of a monkey taking part in a potential motor goal task ⁵⁰. In this task, a single ambiguous spatial cue was presented prior to a delay (memory) period. Animals were then given a contextual rule cue, which provided the instruction for them either to initiate a reach toward the position indicated by the preceding spatial cue (a direct motor goal) or to reach toward the opposite direction (an inferred motor goal). Importantly, the latter reach was made toward a location in which no object had been presented visually, ruling out the possibility that any activity related to this reach option could be due to some form of visual memory. The ‘move’ line indicates the animal’s movement onset time. The surface plot depicts the average activity of parietal reach region (PRR) neurons over the course of the experiment, aligned to the preferred direction of the neurons relative to the reach direction specified by the cue. Examination of activity during the memory period shows that PRR neurons simultaneously represent both motor goals (direct and inferred) when the rule associated with the spatial cue is not yet known. Similar results were observed in dorsal premotor cortex (PMd). Note that once the animals execute the rule-instructed reach, the neurons with preferred directions corresponding to the reach direction significantly increase their activity. b| Reach averaging in ‘go-before-you-know’ tasks is influenced by required movement speeds⁶⁹. In this task, participants performed reaches towards two competing targets, with the final reach target only being revealed after movement onset. Reach trajectories, from a single participant over multiple trials, for slow (top) and fast (bottom) movement speeds are shown. Spatial averaging, wherein initial movements are launched towards an intermediate (or averaged) spatial location, occurs only for slow movements, when time allows for corrective movements to be made. c| Spatial averaging reflects an average of corresponding movement, and not visual, directions⁷⁶. The top panel shows representative reach directions, from a single participant, for one target and two target trials (in which the two outermost targets were presented) in go-before-you-know tasks. Two target trials show standard spatial averaging behaviour.. The bottom panel shows that, when gradual, imperceptible mismatches between the hand position and the viewed cursor representing the hand position (i.e., visuomotor rotation of the hand cursor) are applied to the rightmost target (denoted by curved blue arrow) such that subjects must move their hands straight ahead in order to reach both the central and rightmost targets, individuals, in two-target trials, tended to reach in the direction that was the average of the movement paths associated with the two targets. This leftward shift in averaging behaviour from pre- (top panel) to post-adaptation (bottom panel) learning of the visuomotor rotation, suggests that averaging occurs at the level of motor representations.. Part a is modified, with permission from REF 50. Part b is modified, with permission, from REF 69. Part c is modified, with permission from REF 76.

One interpretation of these neurophysiological findings, which resonates with the influential notion of action affordances ⁵¹, is that the motor system, prior to target selection, prepares competing movement plans for potential targets ^40,52. However, it is also plausible (and difficult to rule out) that the activity of neurons in reaching-related areas instead represents purely spatial and/or memory-related information about multiple potential action targets (such as their directions ^45,53,54). Indeed, because in these tasks the final target is often cued prior to the requirement to execute a movement, it is not immediately obvious why the brain would go through the computational expense of automatically converting viewed or remembered potential targets into corresponding movement representations in the sensorimotor cortex.

Work on the oculomotor system has shown that movement-related neurons in oculomotor control structures, such as the superior colliculus and frontal eye fields, simultaneously encode competing targets for eye saccades ^55–60. Although a matter of ongoing debate, it has been suggested that this activity represents competing target-directed saccadic eye movements. Likewise, the extent to which competing reach movements may be specified in advance of target selection remains a matter of some debate. Indeed, there are reasons why this idea may be viewed as less plausible for reach movements than it is for eye movements.

First, the control of reaching movements (unlike oculomotor control) can require the parameterization of multiple and often redundant degrees of freedom. That is, for any given target-directed limb movement, there are often an abundance of possible motor solutions that can be used to achieve the same goal. This is not only true when considering the infinite number of different reach paths and the range of hand speeds along each path that could, in principle, be specified, but also because any one single joint motion can be achieved by different combinations of muscles ⁶¹. Furthermore, in addition to often having to account for external forces applied to the hand ^14,22, the limb control system must compensate for complex intersegmental dynamics. Together, this indicates that there are several additional complexities in planning limb movement, when compared to eye movements.

Second, the oculomotor and limb movement systems perform vastly different functions in everyday life. Limb movements occur relatively infrequently and are usually voluntary in nature, whereas we typically perform multiple saccadic eye movements per second when sampling the visual environment. It is perhaps not surprising then that the oculomotor system might, at any given moment, prepare multiple competing eye movements to salient visual items in one’s immediate surroundings. That the limb control system should follow the same rule is not as directly inferred from its everyday function ⁶². Indeed, there is evidence that, when performing coordinated eye and hand movements towards the same target location, the two effector systems are differentially impacted by the presence of a non-target (distractor) stimulus ⁶³.

A number of behavioural studies have sought to provide direct evidence in support of the view that the brain specifies competing reach movements in advance of target selection. Many have used tasks in which participants are simultaneously presented with multiple potential reach targets and, before knowing the final target location (which is cued after movement onset), are required to launch a reach movement toward the competing targets (termed ‘go-before-you-know’ tasks ^64–70). In such tasks, one might expect the initial movement to be influenced by competing motor plans. Indeed, it has been shown that people initially launch reaching movements towards an intermediate or midpoint location between the competing targets, which has been interpreted as arising from the specification, and simultaneous execution, of competing action plans ^64,70. However, it has been shown that launching movements toward an intermediate or averaged spatial location also minimizes the motor costs associated with the corrective movements required once the target has been cued ^71–74. When the strategic benefits of intermediate movements in go-before-you-know tasks are mitigated by reducing the time available to make in-flight corrective movements or by increasing the spatial separation between targets ^{71,75 69,71,75}(see Fig. 2b), such spatial averaging is largely abated. This would seem to argue instead that spatial averaging reflects an optimization based on task constraints and motor costs ^69,71,72. Nevertheless, these findings do not actually address how such optimized movements may themselves be computed or explain the representations on which they are based.

One obvious possibility is that approximately optimal averaging behaviour could be based on visual representations of the potential targets. That is, participants could prepare and execute a single movement in a direction that is the average of the visual directions to the competing targets. This would seem to provide a useful, and readily implementable, heuristic for optimizing movements in cases of target uncertainty. Alternatively, the visual targets could be transformed into motor representations of these targets (that is, reach directions or final hand positions) and these representations could be used when determining the optimal reach direction. To disentangle these two interpretations, a recent study ⁷⁶ applied target-specific, gradual visuomotor rotations to dissociate the visual direction of the potential targets from the direction of the movements required to reach the same targets, unbeknownst to participants (Fig. 2c). This dissociation revealed that movements executed towards multiple potential targets constitute a weighted average of the movement paths executed towards each target separately, and was not the result of visual averaging. As such, these findings provide robust evidence that spatial averaging results from a direct visual-to-motor mapping of target locations onto corresponding motor representations of those targets, a transformation that may provide the basis for the computation and launching of optimal initial movement directions.

A challenge for future neurophysiological work will be to determine whether, in go-before-you-know tasks, competing targets elicit separate neural representations in reach-related brain areas (as in REFs ^45,50) or a single representation, corresponding only to the initial movement trajectory. In addition, it will be important for future behavioural work to investigate cases in which averaging behaviour should, and should not, be expected. Although the work discussed above provides evidence that reach parameters such as direction or final hand position are used to compute optimal initial movements, there is also evidence that the dynamics associated with competing reach movements, such as grip force, are not averaged ⁷⁷. Furthermore, given that spatial averaging can be influenced by a wide array of cognitive factors (such as reward history ⁷⁸, task set and attention ⁷⁹, Box 1), how such top-down cognitive processes and biases modify representations of the competing targets remains an open and active area of investigation ^50,80,81.

Box1. ‘Cognitive leaking’ into movement control.

When required to select one of several potential reach targets, the kinematics (trajectory) of the hand can reveal aspects of the cognitive and decision-making process underlying target selection ⁹⁸,¹⁴⁹–¹⁵⁴. In particular, when subjects are encouraged to initiate a movement quickly, their initial movement direction and subsequent movement adjustments can reveal different cognitive influences on motor planning. For example, it has been shown ⁴² that reaches made toward a target object in the presence of a distractor (non-target) object will sometimes deviate toward the distractor (see also REF 43). This suggests that distractor objects initially compete for action selection and are later suppressed ⁴⁴. A general finding of this work is that such initial deviations toward distractor stimuli occur only when they share characteristics and/or features with the target ⁷⁹,¹⁵⁵, providing support for the idea that planning, and not visual stimulation alone, is responsible for driving such deviations.

In situations in which there are two or more competing targets, studies have shown that hand trajectories can be biased by both bottom-up (target luminance ¹⁵⁶) and top-down (the presence of numerical symbols ¹⁵⁷–¹⁶⁰) influences. A recent study showed that reach trajectories were influenced by the rewards associated with two simultaneously presented targets (see the figure, part a). Reach trajectories were straighter towards the selected target when the selected and unselected targets were associated with positive (+5 points) and neutral (0 points) rewards (red traces), respectively, than when they were associated with neutral (0 points) and negative (-5 points) rewards (blue traces; solid and dashed lines represent reaches made to the left and right targets, respectively), suggesting that gains are processed more quickly than losses ⁷⁸. This suggests that movements generated in the presence of competing targets are influenced not only by motor representations of these targets (which could be used to determine motor costs) but also by higher-level representations that include reward valence.

In principle, such movement biases could be exploited by observers to make inferences about the task parameters and intentions that govern a person’s movements. Indeed, it has been shown that someone watching a video clip of an actor who chooses which of two potential targets to reach towards is faster to indicate which direction they think the actor is reaching than someone watching an actor who is being told where to move¹⁶¹. This is because the actor generates subtle preparatory actions when deciding between options which are absent or reduced when no decision is required (see the figure, part b¹⁶¹). This suggests that hand trajectories can provide a read-out of an evolving competition between motor goals, and that this can be readily exploited by observers to predict action outcomes ¹⁶²,¹⁶³.Part a adapted, with permission, from ⁷⁹. Part b adapted, with permission from ¹⁶¹

Directly mapping potential visual targets onto associated motor representations might provide a mechanism through which movement-related costs and constraints can be incorporated when making decisions about action selection. Consistent with this idea, recent work shows that when humans make free choices between two potential reaching movements, they tend to choose the movement that has the lowest movement-related cost ^82,83. A more recent study shows that the prediction of the effort associated with candidate movements is computed very quickly and can influence decisions at the level of motor cortex within 200 ms ⁸⁴. This suggests that the decision-making process can rapidly access knowledge about the future biomechanical costs of both movements in order to compare these and select the lowest cost option. Other recent work has explored the subjective (internal effort) and objective (energy expenditure) cost functions utilized in choosing between effortful reaching movements ^85,86 and how free choices between reach targets are biased by prior target predictability and expected value ⁸⁷. Together, this work indicates that decision-making related to action selection is governed by many of the same underlying optimization principles that are utilized in the control of actions, suggesting significant overlap in the neural processes supporting these two processes.

The above findings suggest that in both go-before-you-know situations and free choice scenarios at least some motor parameters associated with competing targets are specified prior to movement selection and initiation. However, these results do not provide direct support for the idea that individuals specify competing movements in the much more naturalistic case in which target cuing occurs before movement execution (that is, ‘go-after-you-know tasks’). Direct evidence for this idea instead comes from a recent study ⁸⁸ in which participants, prior to target cuing, were presented with two potential targets, one of which could be reached using two different trajectories (that is, an ambiguous target). It was found that the reach movement generated when the ambiguous target was cued often borrowed kinematic components of the movement that would have been required for the non-cued, competing target (the non-ambiguous target). This interaction, which resulted in faster reaction and movement times, could only arise if multiple potential movements were specified in advance of target cuing. Follow-up work further demonstrated that this movement ‘co-optimization’ effect can also be observed across sequentially presented potential targets ⁸⁹, suggesting that individuals successively prepare actions for each potential target as it appears in the sequence. This is noteworthy as it may shed insight into the mechanisms through which the sensorimotor system operates in natural, everyday environments, in which the available action options change from one moment to the next as we move in the world.

Revising movement

In response to changes in the world or our evaluation of the values of different action options, the brain continuously adjusts and refines its goals in order to achieve the desired outcome. One strategy used by the sensorimotor system, which facilitates rapid decision-making during unfolding actions, is the specification of sophisticated contingency plans in situations in which the goal of an action may be threatened. An example is provided by a recent study ⁹⁰ in which participants performed reaching movements with obstacles located either side of a direct path between the hand's starting location and the target (Fig. 3a). On some trials, a mechanical load was briefly applied to the limb early in the movement, so that the hand was perturbed towards one of the obstacles. For small perturbations, the corrective response counteracted the perturbation so that the hand passed between the obstacles, whereas for large perturbations the response was in the direction of the perturbation so that the hand passed around the obstacle. For intermediate perturbations, the hand would sometimes pass between, and sometimes, around the obstacle and this choice was evident in muscle activity as little as 60 ms after the perturbation. Thus, the motor system can switch, with remarkable speed, from one motor plan to another in order to accomplish the task goal.

Fig. 3. Rapid switching between sensorimotor decisions based on context.

a| Obstacle avoidance behaviour reveals the preparation of contingency plans during reaching⁹⁰ [. Reaching movements were made between a starting location and a target, with obstacles on either side of the straight line path to the target. When small mechanical perturbations were applied, the subjects corrected their reach to pass between the targets (lines show paths on individual trials). When large perturbations were applied, the subject corrected their reach in the direction of the load so as to pass around the left obstacle. When medium perturbations were applied, on some trials the subject chose to pass around the obstacles (red lines) and on other trials they chose to pass between the obstacles (blue lines). Traces at the bottom show electromyography (EMG) responses recorded from the lateral triceps and averaged across subjects during performance of the reaching task. The decision to avoid the obstacle to the left versus right can be observed as a difference in muscle activity at ~60 ms after application of the perturbation (indicated by the asterisk). This shows that, for a single reach target, the motor system can flexibly switch between motor plans with remarkable speed.. The stretch response epochs of muscle activity are as follows: R1 = 20-45 ms; R2 = 45-75 ms; R3 = 75-105 ms; and early voluntary (EV) = 105-135 ms. These epochs range from purely spinal-generated corrective responses (R1) to those involving increasing cortical involvement (R2-EV)²⁰. b| Target selection behaviour reveals rapid switching between multiple spatial goals⁹⁰. Black paths show unperturbed movements when a participant reaches to either a single target (left) or two possible targets (right). When perturbed leftward, a rapid correction is seen to a single target (blue traces). However, in the presence of two targets the participant on most trials reaches for the second target (red traces). Bottom, EMG responses from the lateral triceps, averaged across subjects during the performance of the target selection task. Differences in muscle activity corresponding to redirected movements to the second target versus the single target were observed ~75 ms after the perturbation. When compared with panel a, which shows the decision time required to implement a new plan for the same spatial goal, this result shows that there is only an additional 15 ms of decision time required to switch to a new goal. This marginal time cost suggests that the motor system separately maintains contingency plans alongside of the executed action in the event that the task goal becomes threatened. Modified, with permission, from REF 90.

The same study also examined a situation in which participants were presented with two targets and told they could reach for either target (Fig. 3b). In the absence of a perturbation, participants always selected the closest target. However, when a perturbation was applied that pushed the hand toward the alternative target, participants almost always switched their movement goal to this other target. The muscle activity in the switch trials could be distinguished from muscle activity in single target trials (in which the same perturbation was applied) within ~75 ms of the perturbation. Thus, there is a 15 ms cost associated with the decision to switch to a new goal rather than execute a new plan for the same goal. The impressive speed of this target updating process suggests that the re-routed movement was not planned de novo, but rather specified in advance of movement as a contingency plan and maintained in parallel during the unfolding action to be used in the event that the movement goal became compromised. As discussed above, the notion that the brain specifies parallel motor goals would allow rapid movement revisions in response to unexpected changes in the environment and is broadly consistent with the idea that the brain can maintain multiple motor representations or movement plans ^{52,73,91–93}.

To study in more detail the evolution of decision-making processes and their links to sensorimotor systems, perceptual decision making tasks have also been used. Typically, individuals are required to discriminate between two directions of motion in a noisy random dot display and to indicate their decision with an eye movement. Such tasks have led to a quantitative understanding of the mechanisms that evaluate sensory evidence and reach a decision ^94–96. Several formalisms propose that a representation of noisy evidence is accumulated in favor of each choice until it reaches a threshold, leading to that decision ⁹⁴. Such formalisms have well-described neural correlates and can explain both the accuracy and speed of decision making ⁹⁷. Although perceptual decision making has primarily been studied with saccadic eye movements and has been generally viewed as a process that is completed before actions are specified, more recent studies in humans performing reaching movements have revealed a much more continuous flow from the decision process to the motor system ⁹⁸.

In decision making, the decision variable is the accumulated evidence in favor of one decision over another. In sensorimotor control, a key variable is the strength of feedback gains, which are thought to be set by the control policy governing an action. To examine the relationship between decision making and motor control processes, a recent study asked participants to indicate the direction of motion in a random dot display by making either an elbow flexion or extension movement ⁹⁹ and manipulating the difficulty of the task. The cue to respond was a rapid extension of the elbow generated by a robotic interface, which resulted in flexion stretch reflexes (to counteract the elbow extension) whose magnitude could be assessed through EMG. By fitting an accumulate to bound (drift-diffusion) model to the accuracy of the participants’ responses it was possible to estimate the accumulated evidence in favor of the choice (the decision variable) on each trial. This allowed the researchers to show that the magnitude (the gain) of the medium and long latency flexion stretch reflexes (R2 and R3 responses) increased almost linearly with the accumulated evidence in favor of a flexion movement. This suggests that the motor system receives a continuous flow of information from the decision process, which it uses to preset the gain of the flexion reflex so as to reach the appropriate target more quickly when cued. This result complements findings in eye movements in non-human primates ¹⁰⁰, and suggests that the brain does not wait for a decision to be completed before recruiting the motor system, but rather continuously relays information linked to a probable motor response.

Decision making and sensorimotor control processes are further entwined by the time delays that are present in the sensorimotor system. For example, there is a sensory delay between the presentation of a stimulus and the accumulation of evidence in brain areas involved in eye movements, such as the lateral intraparietal (LIP) area ^97,101. In addition, even after the decision variable crosses a threshold leading to a decision, there is a delay in generating the appropriate motor response ^97,101. The sum of these delays can be as long as 450 ms for a manual response. This means that whenever we initiate an action based on a stream of perceptual information — even if the onset of the action terminates the perceptual stream (in the case of an eye movement away from the stimulus, for example) — there will be additional information in the processing pipeline that cannot be used for the initial decision but could still be processed. In tasks in which the response is a ballistic eye movement, such processing is hard to appreciate. However, studies of reaching movements have shown that such post-initiation processing can have significant effects. In a perceptual decision making task in which the required response was a reach toward one of two targets, specified by the stimulus, movements sometimes started towards one target before veering towards the other, even when the stimulus was extinguished on initiation of the movement¹⁰². Such ‘change of mind’ trials can be explained by the accumulation process continuing after movement initiation, with the evidence changing in favour of the other target. Using such a model it was possible to explain the frequency of trials in which subjects changed their mind to either correct an error or spoil a good start, as a function of trial difficulty ¹⁰². The neural representation of such changes of mind has recently been described in the single-trial neural population activity of the motor cortex ¹⁰³ and prefrontal cortex ¹⁰⁴ of non-human primates, wherein the neural state initially reflects one choice before changing to reflect the final choice (Fig 4a,b).

Fig. 4. Evidence for changes of mind.

a| The schematic depicts a decision-maze task, in which monkeys were presented with two targets surrounded by barriers that could change configuration during a trial¹⁰³. Neural population activity was recorded using 96-electrode arrays in the dorsal premotor cortex (PMd) and primary motor cortex (M1). b| In switch trials, initially only one target was attainable, but the barrier configuration could be switched so as to make a second, previously unattainable target, the easiest target to attain. Trials in which the monkey either did not, or did, take advantage of the switch are shown in the left and right panels, respectively. A decoding algorithm for target choice (left or right) was trained using neural population activity from forced-choice trials, in which only one target was attainable, and used to predict target choice in switch trials over time from the activity recorded during switch trial performance. The traces depict decoded final target choice over time for single trials in which the left or right target was ultimately chosen (blue traces or red traces, respectively). The red and blue dots superimposed on the traces indicate the time of the barrier switch. These findings not only show that free choice trials are neurally represented in a similar fashion to forced-choice trials, but that, on the barrier switch trials, the neural activity would sometimes initially indicate one choice, before switching to the opposite choice, consistent with a change of mind. c| Trajectory revision in response to post initiation processing¹¹⁶. In this study, human subjects watched a noisy random dot display, wherein the direction coherence of the dots was varied, and had to indicate both the net direction of motion of the dots (left vs. right) and how confident they were in their choice (high vs. low) by reaching from a central location to one of four choice targets. Although the random dot display was extinguished on movement initiation, a small percentage of trials showed changes in reach paths (traces) indicative of changes of decision (top) or changes of confidence (bottom; only trials in which a change of decision/confidence are shown)d| A drift diffusion model can account for initial decisions, confidence and reaction times, as well as for changes of decisions and confidence. In this model, noisy momentary evidence of the direction of dot motion accumulates in two processes providing evidence for rightward (green trace) and leftward motion (red trace), respectively. The first process to cross a decision bound determines the initial choice (rightward in this case, top plot). There is a correspondence between the level of the accumulated evidence, the elapsed time and the confidence (log odds, bottom plot) that a rightward decision would be correct. In the model, the boundaries between confidence/choice combinations can change after the initial decision (vertical dashed line) to capture the physical cost of altering the reach for a change of mind. In the example trial shown, the boundary between low and high confidence for rightward motion changes after the initial decision, requiring greater evidence to be accumulated (black trace) in favour of high confidence before the participant changes their confidence decision during movement, as in this example. Non-decision time refers to the sum of sensory and motor delays, and corresponds to the unused information that can be processed after commitment to an initial choice (post-initiation processing). Parts a and b are adapted, with permission, from REF 103 Parts c and d are adapted, with permission, from REF 116.

Although this recent work demonstrates that incoming sensory information can influence and update the motor responses used for perceptual reporting, it has also been recently shown that the features of the motor responses themselves can influence perceptual decisions. One study ¹⁰⁵ showed that different levels of physical effort associated with reporting two directions of dot motion with the hand could bias perceptual reports towards the less effortful option. Strikingly, these biasing effects were effector-independent, transferring (at least initially) to other response modalities that did not have increased effort, such as a vocal response. It has also been demonstrated that increasing the physical effort associated with a target-directed motor response results in a corresponding reduction in the frequency of changes of mind toward those targets, suggesting that subjects trade off accuracy and effort ^106,107. Taken together, this recent work suggests that in perceptual decision-making tasks, the motor system, rather than acting as an impartial observer and conveyer of an upstream perceptual decision, can influence how sensory inputs are transformed into decision variables. This general notion fits within the broader context of psychological studies on embodied cognition which show, for example, that the perceived steepness of hills and distances of landmarks, estimates of jumping height, and even selection of target versus non-target objects, can all be influenced by the presumed efforts and limitations of the motor system ^108–111.

The role of confidence — the subjective degree of certainty that a correct choice has been made — has received a great deal of attention in recent work on decision-making ^112–114. This is because confidence can be influenced by factors such as decision time ¹¹⁵ and plays an important role in optimizing choices. For example, subjects in post-decision wagering tasks tend to choose a small but guaranteed reward in regimes in which they are likely to have low confidence, compared to their choices in regimes in which confidence is high ¹¹². One recent study showed that initial confidence, choice and reaction time (as well as changes of confidence and choice) can be coherently explained by a simple race model (Fig 4c,d) ¹¹⁶.

Confidence can also have a substantial effect on sequences of movements. When we make a sequence of choices we often need to get them all correct to achieve a goal; however, we don't receive feedback on whether individual decisions are correct. It has recently been shown that the confidence in the first of two decisions affects the way the decision making process is set up for a second decision ¹¹⁷. If both choices must be correct in order for subjects to receive a reward, subjects take longer (and are therefore more accurate) to make the second decision if they have high confidence in the first decision. That is, participants invest more time on the second decision when they have high confidence that the first decision was correct. In a drift diffusion model developed to capture this behaviour, the height of the threshold on the decision variable for the second decision increased linearly with confidence in the first decision ¹¹⁷. This suggests that confidence has a role in setting the parameters for future decisions and that decision criteria can be adjusted rapidly between decisions so as to optimize performance.

Sequencing movements

Real-world action tasks typically involve a sequence of movements where the final state associated with one movement in the sequence sets the initial state for the next, and so on ¹¹⁸. Researchers interested in the sequencing of movements have often used tasks in which participants generate a series of finger presses. These include the serial reaction time task, in which participants have to respond to visual stimuli using a finger press at a prescribed pace, and the discrete sequence production task, in which participants execute a known sequence of finger presses as fast as possible, either from memory or supported by sequential cues ¹¹⁹. However, unlike many natural action tasks, the movements in these paradigms are not only fixed, they are largely independent; that is, the initial state of a given movement does not depend on the final state of the previous movement. By contrast, in many real-world tasks individuals can choose the sequence of actions that they perform and each of these actions can be executed in different ways (with different kinematics or even different effectors). Thus, real-world tasks involve a decision making process about which actions to perform and when and how to perform them.

In general, the efficient performance of tasks involving action sequencing requires future constituent actions to be considered when planning the current action. In principle, such ‘looking ahead’ would enable the motor system to better optimize costs and rewards across the entire task. This ability has often been examined in the context of ‘travelling salesperson’ problems, in which participants attempt to choose the shortest possible path that connects a fixed set of targets ¹²⁰. Behavioural studies have shown that, when given enough time, individuals are often capable of coming up with near optimal solutions ^121,122. However, a key question is how well people optimize the sequencing of movement components in action tasks in which there are substantial movement-related costs, tight time restrictions, and different task constraints and rewards.

Recent work ¹²³ has investigated decision-making in a movement foraging task in which individuals could choose the order in which they ‘harvested’ multiple targets that varied in size, value and location across a workspace, either by moving a hand-held handle to targets (hand task) or by briefly fixating each target (eye task; Fig. 5a). The short trial duration (3.25 s) meant that, in both tasks, participants could only harvest around half of the targets, placing a premium on rapid and efficient decision-making about which targets to harvest. The foraging data were analysed with a probabilistic model that was inspired by optimal foraging theory ^124,125, which predicts target-by-target harvesting probabilities based on the rate of reward, costs associated with target distance and size, and decision noise. This model showed that, in both tasks, participants rapidly and naturally selected near-optimal harvesting paths that maximized reward. Whereas target value and distance influenced choice behaviour in both the hand and eye tasks, the relative influence of distance was stronger for the hand task. A key feature of the model was that it can incorporate a number of future successive harvests by employing temporal discounting: that is, it can ‘look ahead’. Using the model it was possible for the researchers to determine, for each participant, the number of targets that they considered ahead when using the eye or arm (that is, the number that best accounted for their behaviour). Whereas eye movement decisions were typically made in isolation of potential future targets, hand movement decisions considered multiple future targets in advance (Fig. 5b), a process that presumably involves attending to future target locations and their properties ¹²⁶.

Fig. 5. Optimized sensorimotor decisions for sequences of actions.

a| The schematic depicts hand (left) and eye (right) foraging tasks. In these tasks, individuals choose the order in which to ‘harvest’ multiple targets, either by moving a hand-held handle to targets (hand foraging task) or by briefly fixating each target (eye foraging task).. In the examples shown, smaller targets yield higher rewards, grey targets indicate those targets that have been successfully harvested, black traces indicate the paths used to harvest those targets and the numbers indicate the corresponding order of harvests¹²³. b| Histogram shows the number of subjects in the hand (black bars) and eye (grey bars) task whose best fitting model incorporated a given number of look-ahead targets. This model, applied on an individual basis, allowed an estimation of the number of targets that each participant considered in advance of their next movement (that is, looked-ahead to), and showed that while hand movement decisions considered multiple future targets in advance, eye movement decisions were largely made in isolation of potential future targets. c-e| Active sensing maximizes visual discrimination¹³⁰. In this task, subjects had to determine whether an image was patchy or stripy (c). A gaze contingent display was used in which the image was initially overlaid by a mask. In active trials, for each fixated location, a small aperture of the image was revealed (that is, the location revealed was actively selected through an eye movement). In passive trials, the locations that were revealed were either randomly chosen (random trials) or determined by a Bayesian active sensor model (ideal BAS trials), which determined the optimal (that is, most informative) location to reveal. (d). Following a number of fixations (25 are shown here), the subject indicated whether they thought the underlying image was patchy or stripy. Plot in part e shows, for active (red trace), passive (blue trace) and BAS trials (black trace), the proportion of correct choices as a function of number of locations revealed. Performance in active trials was superior to passive random trials but not as good as in the passive ideal BAS trials (e). An observer model was able to account for the performance (lines ± shaded s.e.). f| Performance of the BAS model for a single trial with a stripy underlying image (left panel). The maps (four right-hand panels) show the expected ‘informativeness’ (BAS score) as a function of the location of the next potential fixation, at different points in the trial (defined by fixation numbers 1, 4, 7, and 14). Histogram insets show the evolving probability assigned by the model to the two categories, which by saccade 14 correctly favors stripy (S), rather than patchy (P). Note that although each fixation can be spatially distant from the optimal location (, they are still high in terms of the information gain percentile. Panels a and b modified, with permission, from REF 123. Panels c-f modified, with permission, from ¹³⁰.

The above findings indicate that the motor system can rapidly and flexibly adjust its sequencing decisions based on the motor effectors used and their associated movement-related costs (including biomechanical effort, movement and time). Notably, although it has been shown that, in navigational planning in rodents, the hippocampus generates sequences of neural events encoding spatial trajectories from the current location to the known goal location ¹²⁷, the neural correlates governing route planning through reachable space remains unknown. An exciting challenge for future work will be to explore how movement-related costs, which are presumably computed in the motor system ⁸⁴, may interface with the brain’s navigational systems located in the medial temporal lobe ¹²⁸ and ultimately be factored into such decision processes.

The ability to sequence behaviours intelligently is not only important for structuring actions: it is also crucial for extracting the sensory information about the world that ultimately shapes the actions we make. For example, the efficient sequencing of eye movements when sampling the visual environment should take into account the information that has been garnered through each previous fixation. Consistent with this idea, it has been shown that individuals, when searching for a target location among distractors, exhibit sequential eye movement patterns in which each eye movement minimizes the uncertainty of the target location over the visual scene ¹²⁹. This active sensing strategy, which qualitatively approximates an optimal extraction of task-relevant information ¹²⁹, suggests that the oculomotor system selects, from one saccade to the next, eye movements that sample sensory information in a way that maximizes task performance.

Whereas the goal of visual search tasks can typically be obtained by fixating a single location (the target), in many everyday tasks fixations at multiple locations are required to extract the information needed to accomplish the task goal. Tasks such as categorization often involve accumulating information across multiple separate fixations and integrating this information with prior knowledge about the stimulus being viewed. Recent work ¹³⁰ directly examined the efficiency of information extraction for individual fixations by employing a visual categorization task in which, for each fixated location, a small aperture of a masked image was revealed (Fig. 5c-d). Participants’ eye movements in this task were compared with those generated by a Bayes-optimal algorithm seeking to maximize, with each individual eye movement, information relevant to stimulus categorization (Fig. 5e-f). Specifically, the active sensing strategy employed by the algorithm involved computing, from the information already acquired about the scene via eye movements and knowledge of the statistical structure of patterns, the location in the scene that, when fixated, was likely to lead to the best reduction in categorization error. Notably, the authors showed that, although participants’ scan paths in this task were not quite optimal (approximately 70% efficient compared to the Bayes-optimal algorithm), their resulting discrimination accuracy was far better than it was when the image locations revealed were randomly chosen. This result shows that the sensorimotor system integrates information across multiple fixated locations in order to select eye movements that enhance information extraction.

Conclusions and future directions

Interactions between decision-making and motor control occur at multiple levels in the planning and control of action. In this review, we have discussed four key aspects of decision making related to sensorimotor control: how the brain selects a particular movement when reaching to a single target, how it represents and selects between competing movement goals, how it flexibly revises ongoing movements and movement goals and how it chooses sequences of movements. We predict three overarching challenges for the future of the field.

Interactions between cognitive and motor systems

One major challenge for future work in the field of motor control is to better understand how sensorimotor systems interact with cognitive systems. Although tremendous strides have been made in recent years by investigating the interaction between decision-making processes and motor control, other interactions — for example, between memory systems and motor control — remain rich areas for future investigation. Recent work on motor learning, for example, has emphasized the role of explicit or cognitive strategies in adapting to visuomotor rotations ^131–133; however, our current understanding of how different memory systems support the planning and control of action tasks remains quite limited. Real-world action tasks, such as making tea or cooking a meal, involve interacting with multiple objects in our environment, which are lifted, moved about in space and then lifted again. Such tasks involve declarative memories about object properties, spatial memories about object locations and episodic memories about dynamic changes in the environment that are brought about through action. Thus, a full understanding of the planning and control of such tasks will require elucidating how these different memory systems interact with sensorimotor processes.

Decision making in real-world situations

We now have a detailed understanding of the interaction between decision making and motor control in a narrow range of tasks. Although these tasks are amenable to analysis and modelling, they do not capture the full complexity of real-world decision making in the context of action. In our daily lives we have to make high-level decisions about which tasks to perform and, at an even higher level, which tasks to learn. In terms of the latter, we frequently make decisions about whether to invest in learning a new skill, ranging from trying out a new tennis grip to learning to speak a new language. Such decisions are based on an estimation of the time and effort involved, the motor skill level we will eventually attain, the success we will have and the expected payoff that such learning will provide. Such decisions presumably require some knowledge of the capacity and constraints of our own motor systems and a prediction of future performance. However, it remains poorly understood how such factors influence decision-making processes at this higher-level.

From computational mechanisms to neuronal implementation

Although substantial progress has been made in understanding the computational mechanisms underlying decision formation (such as the integration of sensory evidence leading to a decision) and how these correlate with, and are causally related to, neuronal responses in multiple brain regions ^{94,100,134–138}, we still have a fairly sparse view of the whole-brain neural circuits underlying decision-making related to sensorimotor control. Recent work in human neuroimaging, though lacking the spatial and temporal resolution of neuronal recording techniques, has begun making some inroads into understanding how, at the whole-brain level, sensorimotor and cognitive networks in the brain interact and together contribute to the planning and control of action tasks ^139–142. However, in comparison to the neuroimaging areas of perception, memory and language ^143–148, this is a relatively poorly investigated topic, a gap owing, in part, to the inherent challenges and difficulties in studying motor behavior in the MRI scanner environment (such as motion-related artifacts in MRI signal; limited workspace). Developments in this particular research area may help reveal how decision-related computations are instantiated across distributed brain areas, which will provide future sites for neuronal recordings in non-human primates, as well as important constraints for biologically plausible computational theories.

In summary, over the last decade, there has been considerable progress in our understanding of the bidirectional interplay between decision making and sensorimotor control. The exciting challenges ahead lie in better understanding decision-making as an evolving and continuous process that adjusts and refines ongoing actions, as well as the neuronal implementation of this process. The success of this more integrated approach to studying the relationship between cognition and motor behaviour will be measured by how well models are able to explain more naturalistic behavioural tasks, deficits observed in neurological or behavioural disorders, and inform the development of more complex robotics and recovery of function options for individuals.

Glossary.

Movement trajectories: A series of arm configurations over time.

Feedforward planning: The specification of specific parameters characterizing an action (such as direction, speed or smoothness) in advance of movement.

Feedback control:The use of error signals, such as the difference between actual and desired movement, in generating a corrective action that minimizes these errors.

Noise: Random or unpredictable fluctuations and disturbances of neural, neuromuscular or environmental origin.

Sub-processes: Processes that are components of a larger process.

Action affordances: Representations of the actions that objects and stimuli in the environment afford at the level of the sensorimotor system.

Saccades: Rapid movements of the eyes that change fixation from one point to another.

Intersegmental dynamics: the interaction torques that arise when the motion of one individual arm segment propagates to the adjacent segments.

Motor representations: The coding of a stimulus and/or associated action in the motor system.

Random dot display: A visual display of moving dots frequently used in perceptual decision-making experiments. Determining the net direction of the dots can be made difficult for the observer by varying the number of dots that are moving in the same direction (coherence) compared to the number of dots that move in random (non-coherent) directions.

Accumulate to bound (drift-diffusion) model: A well defined model in which evidence is accumulated for one or other choice options at each time step, and integrated until a predetermined decision threshold is reached.

Embodied cognition: The theory that many features of cognition are shaped and constrained by the body of the individual.

Wagering tasks: A set of gambling tasks used in psychology to assess an observer’s belief about the outcome of an event or fact.

Race model: A well defined model in which evidence for each choice option is accumulated separately. A decision is made either when one of the accumulators reaches a predetermined threshold or, when the decision is forced, the decision associated with the accumulator with the highest evidence is selected.

Temporal discounting: The discounting of the value of items or rewards as a function of time, with their value being deemed greater the closer they approach the present time.

Active sensing: An active strategy through which the body’s sensors are directed so as to maximally extract goal-relevant information.