Abandoning and modifying one action plan for alternatives

Abstract

Visual scenes are often complex and crowded with many different objects. To interact effectively, we must choose one object at a time as a goal for action. Certain external cues can act as a stop signal, quickly cancelling an ongoing action. Less recognized are internal signals. These can come from recent experience, anticipated action outcomes, cognitive states, and when attention is captured by a salient object. These signals elevate one action plan over alternatives and can quickly modify an initial choice. Here, we focus on these internal processes responsible for selecting, abandoning and modifying action plans. We first highlight how the brain resolves competition among multiple action plans. Critical is the existence of parallel motor planning processes, which allow efficient and timely changes. Then, we discuss how the action system interplays with perception, attention and memory processes to bias action selection and suppress or modify erroneous selections. Subsequently, we show how tracking the continuous modification of action trajectories can provide a tool to read out changes in internal cognitive states. Taken together, we shed light on a broader view that sensorimotor networks can continuously modify actions through simultaneous evaluation of alternative activities in concert with widely distributed perceptual and cognitive networks.

This article is part of the themed issue ‘Movement suppression: brain mechanisms for stopping and stillness’.

1. Introduction

While we are making hurried decisions in real life, ongoing internal conflicts are often revealed through our actions. Thirsty and running to catch a train, we see a soda machine. Our finger first moves to the regular coke button but then swerves to the diet coke, reflecting our internal conflict between taste and calories. Some decisions and associated actions are quickly reversible. Extensive work has been done to understand how the inhibitory control system is engaged to countermand or modify ongoing action when confronted with externally driven cues such as a stop signal or sudden displacement of a target location [1–5]. However, changes of action plans can be also led by dynamics of internal competition without much of external perturbation. For instance, attention just caught by a salient object, recent and anticipated experiences during action, and current internal representations, can all lead to bias and erroneous initial actions, resulting in the necessity of quick changes. In this article, we focus on processes involved in these internally driven action changes.

Such interactions with a complex external world are likely to require seamless coordination among multiple brain systems. These systems allocate attention, select a target and control eye and hand movements. Historically, however, there has been relatively little interaction between work in motor control and higher-order cognitive and decision domains. A traditional framework of voluntary behaviour assumes a functional architecture of serial information processing stages categorized as perceptual, cognitive and motor control modules. Consequently, the role of the motor system is viewed as simply implementing the course of a single motor programme commanded by the cognitive system [6]. By contrast, a growing number of behavioural and neural studies in humans and non-human primates show the active role of sensorimotor networks for resolving action competition [7,8]. Thus, we begin our discussion by acknowledging this ongoing progress and exploring how sensorimotor networks in the brain resolve competition between action plans [9–11]. Furthermore, we describe how the concurrent processing of competing action plans can provide a framework for rapid and efficient action correction.

We then extend our discussion to consider how action planning and modification processes continuously interact with multiple other cognitive systems such as attention and memory. Surprisingly, we show how irrelevant yet salient information is handled differently in action selection when compared with more conventional perceptual selection. We also present how implicit memory of recent perceptual, cognitive or motor experience, as well as anticipation of action outcomes, can bias action selection and influence its subsequent modification. Finally, we describe how changes in movement trajectories can provide a unique tool to capture changes of internal cognitive states. Throughout the discussion, we focus on mechanisms and outcomes involved in correction or modification of reaching movements, as we argue that this is a model system for a wide variety of behaviours. Taken together, we aim to unify traditional and recent views and promote an alternative, broader framework describing how sensorimotor circuits operate flexibly in complex visual environments in concert with perceptual and cognitive systems [7,8,12].

2. How does the brain resolve a competition of action plans in complex environments?

Selecting one object among multiple alternatives and planning appropriate actions reflects fundamental decision-making processes that are part of our everyday purposeful behaviour. Recent neurophysiological evidence has demonstrated that neural correlates of decisions, particularly those resulting in action, are found within the same sensorimotor circuits that are responsible for planning and controlling the associated actions (i.e. effector-specific decision). For instance, neural activity related to reach target selection has been observed in the dorsal premotor area (PMd) as well as in the parietal reach region, which exhibits reach-related planning and execution signals [13–16]. Similarly, activity related to target selection for eye movement has been identified in the superior colliculus (SC), frontal eye field (FEF), lateral intraparietal area and supplementary eye fields, in which signals related to eye movement execution are generated [17–21]. Furthermore, recent work [22,23] has demonstrated that the involvement of sensorimotor areas can be extended to effector-independent decisions, assumed to be originated from higher-level areas such as the dorsolateral prefrontal cortex. For instance, a temporary inactivation in primate SC, representing a competition among eye-movement goals, also causes dramatic target selection deficits for reaching movements [22]. Thus, the SC can contribute to a general-purpose priority map, which, in turn, influences target selection for other actions, such as reaches. These accumulated neurophysiological data have consistently indicated that sensorimotor circuits do not passively reflect the result of completed cognitive processes; rather, they are critically linked to the dynamic decision-making process itself [7,8].

Recent neural and behavioural studies have also demonstrated that these sensorimotor circuits can process multiple competing action plans in parallel, enhancing the efficiency of action modification. The evidence of parallel processing has been observed in reach-related areas such as M1, PMd and area 5 in the parietal cortex [24–26], and saccade-related areas such as SC and FEF, respectively [27–29]. For instance, Cisek & Kalaska [13] demonstrated that when two potential reach targets are present, the population activity in PMd can simultaneously encode the two competing movement goals during a delay period even before a correct target is indicated. Then, as the final choice is made followed by the movement, PMd activity associated with one response diminishes, and the other increases. In addition, it has also been shown that during a two-choice reaching task, where sensory information about the correct choice was changing with an urgent response deadline, the activity in PMd and M1 tracked the state of dynamic information, combined it with an urgency signal and committed decision [30]. Similarly, McPeek & Keller [27] observed the concurrent processing of saccades in SC. They trained monkeys to make a saccade to the odd-coloured target among homogeneous coloured distractors (figure 1), while recording neural activities in SC. They observed that when monkeys make an erroneous saccade to a distractor, they are capable of executing a second saccade to the target after a very brief inter-saccadic interval. During these two-saccade responses, SC is capable of simultaneously maintaining the priority of a second visual goal and completing the preparatory process concurrently with the execution of an initial saccade. It provides evidence that the two-saccade goals are simultaneously represented on a common motor map. Therefore, these studies show that a decision among competing action plans or goals can simultaneously take place within the sensorimotor system.

Figure 1.

A schematic display of a colour-oddity task and effect of distractors on target selection. In this task, an odd-coloured target is presented with two distractors. The colours of the target and distractors are randomly switched in each trial. Examples of movement trajectories are from Moher & Song [31], in which participants search for and reach towards the odd-coloured target with their index finger. These trajectories are plotted in two-dimensional space for clarity though they are three dimensional. Curved reach trajectories, i.e. partial errors, are coloured in blue contrasted with those of direct movements in black. Similar patterns of trajectories are also observed in other studies requiring reach or saccade towards the oddity target among distractors (e.g. [27,32]).

Converging evidence that the brain simultaneously processes information about multiple potential actions has been behaviourally manifested in curved movement trajectories when participants are required to launch a reach or a saccade towards a target in the presence of competing stimuli [31–35]. For example, Song & Nakayama [32] further examined the temporal evolution of competing two-reach plans by analysing curved trajectories in a colour-oddity task (figure 1). They observed that when the colours of the target and distractors are randomly switched from one trial to the next, causing significant competition, in some trials, reaches are directed to the target (figure 1, black trajectories), and in other trials, reaches are initially directed towards a distractor and are corrected in mid-flight, showing highly curved trajectories (figure 1, blue trajectories). They demonstrated that corrective movements occur very quickly after the onset of initial movement, which is significantly faster compared with a sequentially planned corrective movement. This result indicates that a corrective new target is selected even before the initial incorrect movement is executed. Interestingly, they also observed that trials with highly curved trajectories are no less efficient in terms of accuracy or total time. The extra time taken up in movement duration is offset by shorter initial latencies. Again, concurrent planning provides an explanation as to why misdirected reaches, hastily initiated, can be corrected with minimal loss in overall efficiency.

Furthermore, when participants are required to initiate a reach or make saccade before knowing which of several potential targets will become an action goal, recent studies have shown that parallel planning is manifested as spatially averaged saccades landing midway between the location of the two targets or initial reach movements towards the midpoint of the target distribution [36,37]. In addition to spatial averaging of reach paths, recent studies have also demonstrated that averaging can be extended to the hand orientation or the specification of a sensory motor control policy [37,38].

To sum up, these neurophysiological and behavioural findings support competition among simultaneously represented potential action plans in sensorimotor network and suggest a continuous flow and blending between brain systems. Furthermore, this parallel encoding provides a potential neural basis for the ability of individuals to quickly abandon initial decisions and modify actions efficiently when necessary. Thus, it negates the idea that sensorimotor control is the subordinate of cognition.

3. How does the salience of irrelevant objects affect modification of action plans?

In a complex environment including multiple objects, behavioural goals guide the selection process and help us to find a relevant object [39,40]. However, perceptually salient objects such as flashing billboards can automatically divert attention, competing strongly for selection even though they are not meaningful to a person's goal. In perceptual processing experiments, the more perceptually salient stimuli typically attract greater attention [41–43]. The inability to effectively suppress such erroneous attentional capture can thus delay target selection, which can hamper effective decisions for survival. Thus, the role of suppression in the selection process has been examined alongside the more traditionally studied activation of task-relevant features, objects and locations [44–47]. For instance, recent studies have demonstrated that when perceptually salient distractors occur with high frequency or are present in the immediately preceding trial, they may be suppressed, reducing attentional and oculomotor capture [44,46,48].

Moher et al. [49] further examined the impact of salient distractors on action planning and discovered something unexpected: suppression for goal-directed action, which differs from what has been seen in perceptual studies. They asked participants to reach for a shape-defined singleton target while trying to ignore randomly appearing physically salient colour distractors. They also manipulated physical salience by varying the colour of the distractors (figure 2a); all objects appeared in red, except for colour singleton distractors, which appeared either in equiluminant pink (low feature contrast, LFC, weak physical salience) or blue (high feature contrast, HFC, strong physical salience). Then, opposite to what has been established in prior perception studies [42,43], they found that the more physically salient distractor caused less deviation in hand movement trajectories (figure 2b). An initial action plan attracted towards a more salient distractor is more effectively suppressed than towards a less salient distractor [50–52]. It is shown by the greater distractor attraction scores, a measure of how far hand movements deviated towards the salient distractor, from pink LFC distractors than blue HFC (figure 2c). In figure 2d, the signed initial trajectory angle (ITA), in which a positive number indicates the hand is deviated towards the location of the distractor, was also higher for LFC than HFC distractors. Furthermore, salience-driven suppression was more pronounced in early initiation latency. It suggests that instead of slow-acting suppression observed in previous studies, salience-driven suppression is rapid. To summarize, it appears that the salient distractor triggered a suppression mechanism, reducing distractor interference relative to the weakly salient distractor in an integrated attention–action system.

Figure 2.

Salience-driven suppression for goal-directed reaching from Moher et al. [49]. (a) A sample sequence of trials. The task was to search for a shape-defined singleton (e.g. a diamond among circles or a circle among diamonds). One of the non-unique shapes was coloured either pink or blue, with equal probability, on 50% of all trials. (b) Average trajectory across all subjects for a target located in the lower right corner with no distract (black line) or a distractor in the upper left corner that is low feature contrast (LFC; pink line) or high feature contrast (HFC; blue line). (c) Distractor attraction scores calculated across the entire resampled movement, averaged across all subjects. The pink line shows scores for the LFC distractor, and the blue line shows scores for the HFC distractor. Positive scores indicate hand position that is pulled towards the location of the colour distractor on distractor present trials. (d) Signed initial trajectory angle (ITA) across four quartiles of initiation latency, from shortest to fastest, for both LFC and HFC distractors. All error bars reflect s.e.m.

To determine whether this rapid salience-triggered suppression is specific to goal-directed action, Moher et al. [49] also created a perception-based version of the task. In this task, participants indicated the orientation of a line (vertical or horizontal) inside the unique shape target with keypress, while trying to ignore physically salient HFC or LFC distractors. They demonstrated that perceptual interference was greater from the HFC than the LFC distractors, reflected in longer response times. Similar results were obtained in an otherwise identical perceptual task that required a localization judgement of the target. It is, thus, the opposite direction of the reaching behaviour. This novel finding reveals a clear dissociation in how physical salience affects performance depending on whether participants are required to make a movement towards their target or simply to report about it.

Furthermore, this salience-driven suppression for action can be also extended to a value-driven capture paradigm [49]. In this paradigm, during the training phase (figure 3a), in which participants reached to a target circle (unpredictably red or green) on every trial, one target colour was associated with high monetary reward (e.g. 10¢), the other with low reward (2¢). Then, in a subsequent test phase (figure 3b), while either the high- (previously associated with high reward) or low-value (previously associated with low reward) colour appeared as an irrelevant distractor, participants were required to reach to the unique shape or perform perception-based version of the task on each trial. Consistent with a physical salience-triggered suppression (figure 2), they also found that reward-driven salience also triggers suppression in goal-directed action only: while highly salient stimuli interfere strongly with perceptual processing, increased associated value attenuates action-related interference. To summarize, salience-triggered suppression occurs only when goal-directed actions towards specific objects are required, so that weakly salient distractors make it harder to suppress initial erroneous motor plans and delay modification of movements towards a relevant target [49].

Figure 3.

Schematic displays for the training and test phases in the value-driven suppression paradigm from Moher et al. [49]. Participants were required to reach to the red or green target during the training phase (a) and to the unique shape during the test phase (b).

4. How do recent history and future anticipation bias a competition of action plans?

Actions do not often occur in isolation, but occur embedded within behavioural streams of varying duration. For instance, a person often selects several apples one-by-one at a grocery store, and then moves to a neighbouring bin to select an orange. Previous studies have demonstrated that target selection for actions does not depend only on current needs, but also depends critically on recent events, particularly for actions repeated with the same characteristics [35,53,54]. Thus, in addition to current goals, selection history could affect the current selection process. For instance, when humans and non-human primates performed a colour-oddity visual task (figure 1), both saccades and reaches to the target are facilitated when the colour of the target remains the same, and as the number of consecutive same-colour target repetitions increases, reaches towards the target are initiated and completed increasingly quickly. By contrast, when the target colour switches from trial to trial, initial saccades and reaches are directed towards a distractor more often, and subsequently corrected in-flight to the target (i.e. more curved trajectories) [31,35,55]. This priming of popout shows that the repeated perceptual history can significantly enhance the efficiency of target selection for actions, which in turn, reduces occurrences of a redirected movement from a distractor to the target. Neural mechanisms encoding the history of saccade target selection have been examined in FEF. FEF neurons appear to discriminate a saccade target from distractors faster and more accurately if the prior and current trials share the same target features such as colour or shape [56]. These results suggest that neural substrates involved in saccade target selection can also accumulate information of selected saccade target over time.

While priming of popout reflects both enhancement of the repeated target features and suppression of the repeated distractor features, Goolsby et al. [57] isolated the repetition effect of a purely inhibitory component. They asked participants to perform a similar discrimination task (figure 1). However, in some trials, all objects were homogeneously coloured, and thus no response was required. Responses on the next trial were slower when the target matched the colour of the homogeneous items from the previous target-absent trial, suggesting that the previewed feature is inhibited so that selection is automatically biased away from it. Conversely, responses were faster if the distractors matched the previewed colour. This distractor-previewing effect has been also extended to saccades and reach target selection [58,59]. Furthermore, target selection biases observed in priming of popout or distractor-previewing effect can transfer across different types of actions (i.e. from an eye movement to a hand movement) [59,60]. These results suggest that selection biases are represented in memory largely independently from their associated actions.

Such history effects on action target selection are not limited to repetition of low-level physical stimulus properties. They are also seen for other more abstract factors, such as trial difficulty. In both humans and non-human primates [53,54], when relatively easy single target trials and more difficult search trials are randomly mixed within a block, the latency of reach movement initiation in a given trial depends on the difficulty of the previous trials. During a sequence of easier trials, initiation latencies become shorter. By contrast, during a sequence of more difficult search trials, latencies increase. Interestingly, the occurrence of curved reach trajectories reflecting redirected target selection process is also reduced when the same difficulty levels are repeated. Implicit short-term memory has been proposed as the mechanism underlying these perceptual and cognitive repetition effects [53,61].

Furthermore, history of specific action outputs may be also bound together in memory with stimulus properties and can shape a subsequent competition of action plans [62]. For instance, Jax & Rosenbaum [63] showed that once participants modify their reach trajectories to avoid an obstacle while reaching to a target, they show a ‘hand-path priming’ effect by using a similar modified movement trajectory on a subsequent trial with the same target even when the obstacle has been removed. Moher & Song [31] also observed that following ‘partial error’ responses, in which participants initially reach to the wrong target object before changing their mind, correcting mid-flight (figure 1, blue), they are more likely to commit partial errors responses on a subsequent trial when the target feature is repeated on consecutive trials. Thus, these results suggest that the perceptual-motor system creates a template from recently executed motor plans in order to reuse a similar plan for subsequent movements, rather than generating entirely new motor plans for every event.

In addition to recent history, anticipated outcomes of future actions can also affect the degree of a decision competition among potential action plans. Cos et al. [64] showed that anticipated biomechanical costs associated with different movement outcomes contribute to decision-making processes. When one possible motor response carried a higher energy cost than an alternative motor response, participants were biased to select the less costly response. Furthermore, a series of recent studies [65,66] have also demonstrated that when participants are required to report their perceptual decision by reach to one of the two choice targets, the frequency of changing their decision is affected by physical effort associated with a modification of planned action. For instance, as choice targets are spaced farther apart, increasing the effort required in redirecting reaching movements from one to the other, there is a reduction in the frequency of decision changes. It appears that when revision of an initial choice would incur higher energy and time costs, due to increased physical distance between competing responses, participants adjusted their perceptual decision thresholds. All these studies commonly suggested that when multiple action options compete for selection, more efficient movement options bias decisions themselves. To summarize, the degree to which multiple action plans compete for selection and modification is not determined in isolation, but rather is affected by both recent experience of perceptual, cognitive and action processes and anticipated future action outcomes.

5. How do changes in action plans reflect changes in internal dynamic states?

Recent studies on human perception and cognition have begun to take advantage of the linkage between cognitive and motor processes and the concurrent motor processing in sensorimotor areas by using continuous movement to understand progress in internal competition among multiple cognitive states and decisions [8]. For instance, to understand how we commit to a decision and revise it by accumulating perceptual evidence, recent studies have used self-correction of goal-directed reaching movements as a proxy of changes of perceptual decision-making [65,67,68]. Resulaj et al. [67] asked participants to decide the perceived direction of motion of a central random-dot display by moving a handle from the central home position to one of two choice targets (leftward versus rightward). Despite the participants receiving no more perceptual information as soon as they initiated their movement, in some trials, they changed their mind to revise the action, producing curved trajectories. This result suggests that perceptual evidence is accumulated over time until it reaches a criterion level for the initial decision, and that the brain continues to exploit information in the processing pipeline, determining whether to reverse the initial decision. It is also consistent with what has been observed in the oculomotor systems [69–71]. By modifying a choice-reaching task used in Resulaj et al. [67], van den Berg et al. [68] further examined whether the same mechanism that supports a change of mind also supports a change in confidence. In this task, they asked participants to indicate simultaneously the direction motion (leftward versus rightward targets) and confidence level (top versus bottom targets for high and low confidence, respectively) by moving a handle to one of four choice targets. By analysing the direction of reach-trajectory curves, they could read out whether participants change their mind or confidence. On a small fraction of trials, participants changed their initial decision about the motion direction and, more frequently, about their confidence about a decision after they had made it, as if they had continued to think about it. They suggested that changes of confidence arising through post-decision processing can explain previously reported dissociation between confidence and decision by over- or underestimating confidence about choice [72,73].

The application of reaching movements as a tool to study cognition is not limited to understanding dynamics in perceptual decision-making. For instance, a previous study revealed the spatial representation of numbers by systematic trajectory shifts (figure 4) [74]. In this study, participants had to determine whether a single Arabic numeral presented on the centre square was less than, greater than or the same as the number five. They reached to one of the three squares on the screen: less than (left), greater than (right) or equal to (centre). As shown in figure 4, reach trajectories are systematically shifted in position according to numerical differences between the target and the number five; the greater the numeric deviation, the greater the deviation of the trajectory from the standard trajectory. This provides converging evidence that when the numeric magnitude of the target is encoded, the proximity and orders between numbers are spatially represented along a hypothesized mental number line. In addition to spatial number representations [74,75], studies on linguistic processes [76,77], attention [78–80], perceptual processes [81,82], statistical learning [83], decision-making [84–86], cognitive control [87] and conscious awareness [88,89] have also taken advantage of continuous modification of action. Taken together, these studies reveal that pre-programmed competing motor plans can be manifested in early reach movement trajectories and reveal real-time flow of earlier cognitive states into motor output.

Figure 4.

Spatial number representation revealed by systematic reach-trajectory shifts from Song & Nakayama [74]. Participants are shown a single-digit Arabic numeral in a centre square and asked to compare its value with the standard, 5. They then reach for and touch one of three squares on the screen: the left for ‘less than’, the centre one for ‘equal to’ or the right one for ‘greater than’. The panels in this figure depict examples in which the value of the target is ‘equal to’ (a) or ‘less than’ (b–e) the standard. The lower panels demonstrate gradual shifts of reach trajectories towards the centre square as the difference in value between the target and the standard decreases. These trajectories are three dimensional, but for clarity, the most relevant x (left–right) and y (upward–downward) dimensions are plotted.

6. Closing remarks

In the real world, people not only have to find target objects, and shift their eye gaze towards relevant information, but also reach to those objects to manipulate them in ways that will help them achieve their goals. Even the most advanced perceptual system is not biologically useful unless it can be coupled with action. Visually guided goal-directed actions are often executed in complex and crowded visual scenes, where several different objects compete for attention and action. Thus, arriving at a decision to act requires precise coordination between several brain systems to orient attention, selecting a single goal from multiple possibilities and perform visually guided actions such as eye and hand movements.

In some cases, decisions are irreversible, but in others we can alter our initial choice by changing our mind. These changes of our mind could occur because new information (external signals) is acquired or because alternative considerations (intrinsic signals) arise. Here, we have shown how intrinsic signals can lead us to abandon one action plan and take on another. As we discussed, because the sensorimotor system generates multiple competing plans in parallel before actions are initiated by integrating signals from perception, cognition and motor domains, this concurrent processing enables us to efficiently resolve competition and select one appropriate action rapidly (figure 1). Furthermore, we have demonstrated evidence supporting continuous information flow and interaction between action and cognitive processes in studies dealing with the wide range of perception, cognition topics [8]. In these studies, the detailed analysis of continuous changes of reaching trajectories provides new information of how multiple competing cognitive representations evolve and what their intermediate products are (figure 4). These discoveries could not be explained by a traditional model assuming that action specification simply follows decision-making occurring at an early stage in a ‘cognitive’ system.

We have also highlighted that interaction among multiple systems can be complex. For instance, we have demonstrated that the same external information such as perceptual salience or the associated value of a stimulus can trigger a suppression mechanism for action but not for perception [49]: higher contrast or high-valued stimuli were more distracting in perceptual tasks, whereas they were less disruptive when quick motor actions were required (figure 2). At first glance this appears to be striking and puzzling insofar as one would expect that information influencing conscious vision would similarly influence visually guided actions. While we have not yet fully understood an underlying mechanism for these unexpected results, we conjecture that it could be because the costs of being distracted by salient stimuli are much higher when there is time pressure and the necessity of maintaining an action course, so the brain employs an adaptive tactic to tune out stronger distractions more than weaker ones only when goal-directed action is involved. This distinction between perception and goal-directed action underscores the value and necessity of combining visually guided actions with traditional psychophysical approaches to fully understand how we integrate perception and action to accomplish behavioural goals and resolve competing internal processes in complex visual environments. Taken together, investigations using combined cognition and action provide exceptional research opportunities that will enhance our understanding of a wide range of brain mechanisms—common to humans and other animals—that provide seamless coordination of behaviour in the complex world.

Competing interests

We have no competing interests.

Funding

This work is supported by NIGMS-NIH (P20GM103645) and NSF CAREER award (BCS 1555006) to J.H.S.