Interfacing to the brain’s motor decisions

Abstract

It has been long known that neural activity, recorded with electrophysiological methods, contains rich information about a subject’s motor intentions, sensory experiences, allocation of attention, action planning, and even abstract thoughts. All these functions have been the subject of neurophysiological investigations, with the goal of understanding how neuronal activity represents behavioral parameters, sensory inputs, and cognitive functions. The field of brain-machine interfaces (BMIs) strives for a somewhat different goal: it endeavors to extract information from neural modulations to create a communication link between the brain and external devices. Although many remarkable successes have been already achieved in the BMI field, questions remain regarding the possibility of decoding high-order neural representations, such as decision making. Could BMIs be employed to decode the neural representations of decisions underlying goal-directed actions? In this review we lay out a framework that describes the computations underlying goal-directed actions as a multistep process performed by multiple cortical and subcortical areas. We then discuss how BMIs could connect to different decision-making steps and decode the neural processing ongoing before movements are initiated. Such decision-making BMIs could operate as a system with prediction that offers many advantages, such as shorter reaction time, better error processing, and improved unsupervised learning. To present the current state of the art, we review several recent BMIs incorporating decision-making components.

Brain-Machine Interface Types and the Possibility of Incorporating Decision Making

brain-machine interfaces (BMIs) establish direct communication between brain regions and assistive appliances, such as artificial body parts and computers (Andersen et al. 2004; Blabe et al. 2015; Carmena and Cohen 2012; Donoghue et al. 2007; Gilja et al. 2011; Hatsopoulos and Donoghue 2009; Lebedev 2014a; Lebedev and Nicolelis 2006; McFarland et al. 2006; Nicolelis 2011; Nicolelis and Lebedev 2009; Schwartz et al. 2006; Wolpaw et al. 2002). BMIs are expected to provide novel treatment strategies for various neurological conditions, including sensory, motor, and cognitive disabilities (Blabe et al. 2015; Burns et al. 2014; Dobkin 2007). For example, a BMI intended for spinal cord injury bypasses the site of neural damage and establishes a direct link between cortical motor areas and an external prosthetic device (Birch et al. 2002; Bouton et al. 2016; Collinger et al. 2013; Hochberg et al. 2006; Hochberg et al. 2012; Ikegami et al. 2011; Jarosiewicz et al. 2015; Lebedev and Nicolelis 2006; Wolpaw et al. 2002). In addition to clinical applications, BMIs are also increasingly used by healthy people, for example, as gaming devices (Blankertz et al. 2010; Holz et al. 2013).

The following types of BMIs have been described in the literature: 1) motor BMIs that extract motor commands from neural signals and send them to external actuators like prosthetic arms (Collinger et al. 2013; Hochberg et al. 2012; Nicolelis 2001), 2) sensory BMIs that evoke artificial sensations using stimulation of sensory brain areas (Bensmaia and Miller 2014; Fitzsimmons et al. 2007; Flesher et al. 2016), 3) bidirectional or sensorimotor BMIs that combine motor and sensory information streams (Chapin and Moxon 2000; Friehs et al. 2004; Klaes et al. 2014; Lebedev 2014a; O’Doherty et al. 2011), and 4) cognitive BMIs, an umbrella term for systems that extend beyond motor and sensory functions and tackle higher order domains, such as executive control, imagery, intention, decisions, forward modeling, attention, and learning (Andersen et al. 2004; Andersen et al. 2010; Musallam et al. 2004a; Pesaran et al. 2006). This classification is useful but oversimplified, because the brain does not clearly segregate functions into motor vs. sensory or simple vs. complex. Even in the “purest” primary motor and primary sensory cortical areas, sensory and motor functions are intermixed (Fromm and Evarts 1982; Jones 1986; Lebedev and Wise 2002; Petreanu et al. 2012; Porter 1996), which makes the term “sensorimotor” more appropriate (Evarts 1973; Lilly 1956). We argue below that motor decision making is principally a sensorimotor process that engages neural circuitry of multiple cortical and subcortical areas. We also argue that using BMIs to extract neural information during different stages of decision making could be beneficial for both practical neuroprosthetic applications that reproduce naturalistic behaviors (Collinger et al. 2013; Gilja et al. 2012; Jarosiewicz et al. 2015) and basic research on neurophysiological mechanisms of decision making.

In this review, we first introduce a model describing the neural processing steps involved in generating motor decisions. Next, we suggest several ways in which BMIs could link to these mechanisms. A hypothetical BMI for decision making is illustrated in Fig. 1. A monkey is seated in front of a computer screen, where a cursor and several visual targets are displayed. The monkey applies a rule (e.g., “prefer circles to triangles, triangles to squares, and squares to circle”) to select one of the targets and place the cursor over it. The monkey moves the cursor manually, with a handheld joystick or using the BMI. When the BMI is engaged, a decoding algorithm analyzes the neural activity representing the process of decision making and, after the monkey’s decision is decoded, initiates the cursor movement. Depending on the brain areas recorded, the decoder could extract different variables, for example, target shape (e.g., circle vs. triangle), movement direction (e.g., right vs. left), time of movement onset, etc. The BMI could also adjust the ongoing cursor movement on the basis of brain signals. Since the decoding starts before the overt behavioral response, this BMI can perform predictions (Adams et al. 2013; Shadmehr et al. 2010), which offers such advantages as lower response latency and better error processing. Additionally, a decision-making BMI could enact inhibitory control, that is, the capability to stop a prepared action before it is executed (Mirabella 2012). BMIs with inhibitory control could adapt to unexpected changes in either the external environment or the mind (Mirabella 2014; Schall 2001; Schall and Godlove 2012; Verbruggen and Logan 2008). We propose below that inhibitory control is an integral component of motor behavior. Accordingly, BMIs with decision decoding and inhibitory control could reproduce goal-directed behaviors in a more natural way compared with BMIs that process neural activity in less sophisticated ways, for example, extract arm coordinates continuously (Carmena et al. 2003; Lebedev et al. 2005; Serruya et al. 2002; Taylor et al. 2002).

Fig. 1.

Schematics of a hypothetical decision-making BMI. A monkey is seated in front of a computer screen that displays a cursor and 2 targets. The monkey uses a rule (for example, “prefer circles to triangles”) to select the target and then places the cursor over the target to receive a reward. Initially, the monkey performs the task manually, using a handheld joystick to move the cursor. At the same time, neuronal ensemble activity is recorded in the monkey’s brain with the use of chronically implanted microelectrode arrays, and a decoding algorithm is trained to extract various components of the decision-making process from the neuronal activity. Depending on the brain areas implanted, the decoder could extract representation of the selected object, prepared movement direction, executed movement direction, and the decision to inhibit movement initiation. It is suggested that decoding these signals before movement onset can improve BMI accuracy and versatility.

Although in this review we focus mostly on the nonhuman primate literature, we foresee that decision-making BMIs in near future will become widespread in clinical and consumer applications for humans. The rise in the number of human studies on invasive BMIs (Aflalo et al. 2015; Bouton et al. 2016; Collinger et al. 2013; Hochberg et al. 2006; Hochberg et al. 2012; Jarosiewicz et al. 2015) is particularly promising in this regard because invasive recordings and relevant BMI applications could facilitate greatly the neurophysiological investigations of human decision making, which are much more sophisticated compared with those of nonhuman primates.

In addition to envisioning advances in the BMIs intended for practical purposes, we suggest that BMI research could advance basic research on neural mechanisms of decisions. We adhere to Richard’s Feynman’s motto, “What I cannot create, I do not understand.” Indeed, BMIs can both practically test theoretical constructs regarding neuronal processing and instigate new discoveries. Additionally, BMIs introduce useful tools for experimental design that expand the range of testable hypotheses, such as the capacity for manipulating behaviors and neural signals in a controlled way and making them interdependent (Golub et al. 2016). With BMI approach, events in a decision-making task could be triggered by decoded brain activity (Schultze-Kraft et al. 2016), and relevant neural signals that are normally hidden from our consciousness could be made available to a subject as neurofeedback (Vernon 2005). Moreover, BMI provoke brain plasticity (Grosse-Wentrup et al. 2011; Lebedev et al. 2005), a useful manipulation that could provide valuable insights on learning to make decisions.

Motor Decision Making: Goal-Directed and Volitional Actions

Decision making, i.e., selecting a course of action from several possible alternatives that usually have qualitative, spatial, temporal, and conditional characteristics (what, where, when, and whether) is a common undertaking for living organisms. A comprehensive classification of biological decisions is somewhat difficult because of the many forms of behaviors involved, ranging from simple actions, such as reaching for food, to making complex choices in social and/or technological environment, such as planning a future career. Most generally, the choice is made in accordance with its probability of leading to the best outcome in terms of biological fitness (Kim and Lee 2011). Hence, the ability to accurately predict future outcomes is critical for all forms of decision making. To make an optimal decision that will increase the probability of surviving and reproducing, the brain should confidently estimate the consequence of each choice. Since we live in the world where events cannot be fully predicted, most decisions are made based on limited information about their consequences. Therefore, decisions bear a certain degree of uncertainty. Complex networks of brain structures, including cortical and subcortical regions, minimize this uncertainty by processing current available information and accumulated experience (e.g., see Barraclough et al. 2004b; Platt and Glimcher 1999). In many cases, uncertainty cannot be completely removed, and there are also instances in which preplanned actions must be aborted to avoid catastrophic consequences. For instance, when one is about to dive in the Australian reef but suddenly sees the fin of a white shark, the prepared action must be cancelled. Even in less extraordinary circumstances, inhibitory control of a pending action is essential (Mirabella 2014).

Here, we focus on decisions that underlie goal-directed motor actions. Decoding such motor decisions could be a useful addition to classical motor BMIs that connect the brain directly to external actuators (Donoghue 2002; Lebedev and Nicolelis 2006; Schwartz 2007). Although motor decisions are often described as voluntary activities, a distinction should be made between voluntary and goal-directed actions, which differ by the degree to which top-down commands vs. bottom-up sensory inputs are involved (Scott 2016). Voluntary movements, which are heavily dependent on top-down commands, pose an enigma, because we still do not have a clear understanding of what makes an action volitional. It is debatable what exactly volition is, and how and when it is generated by the brain. According to Libet et al. (1983), humans consciously experience intention to act 200–500 ms after the movement plan has already been initiated by the brain (e.g., see Fried et al. 2011), which suggests that conscious experience of volition is not a cause of motor activity but its by-product, even though subjects perceive their actions as freely chosen (Hallett 2007). To explain functional significance of the subjective experience of volition, Libet (1993) proposed that such an experience may play a role in the buildup of our free will, because it would allow us to withhold upcoming but undesired actions (see Haggard 2008; Mirabella 2007 for reviews). However, the issue is far more complicated than that. Animals with arguably more primitive subjective states than humans can suppress pending movements, for instance, monkeys (Mirabella et al. 2011) and rats (Eagle et al. 2008). Furthermore, humans can cancel actions unintentionally and unconsciously (see van Gaal et al. 2008). All in all, a wide range of planned movements can be stopped without resort to consciousness. It would be fair to conclude that the functional role and neural underpinnings of willingness still represent a black hole in our knowledge.

Goal-directed actions are not restricted to volitional behaviors because they can be based on sensory inputs rather than conscious intentions (Scott 2016). Well-learned, goal-directed movements automatically link the incoming sensory information to motor actions, whereas awareness of this neural processing may remain minimal. Automatic and rapid motor routines associated with common interactions with handleable objects are referred to as stimulus-driven action affordances (Gibson 2014). In the neuropsychological disorder called utilization behavior, such routines are performed completely involuntarily: patients automatically produce stimulus-driven motor responses even though they do not want these actions to occur (Humphreys and Riddoch 2000). In healthy people, unwanted stimulus-driven action affordances are prevented by inhibitory control. For example, the sight of a glass of water would trigger reaching and grasping in a healthy individual only if that individual is thirsty and this action is socially appropriate.

Thus goal-directed actions may or may not involve awareness, and they can be studied in both animals and in humans. We will focus on the conceptual framework of goal-directed actions and set aside the issues of awareness and volition.

Multistep Decision Model

Goal-directed actions are generated by a multistep decision process mediated by multiple brain regions (see Mirabella 2014). Figure 2 shows a framework of the salient features underlying this process. The core novelty of this model is that it predicts that evaluation of actions can occur at all processing stages, from the early planning to the eventual execution. Additionally, the model suggests that action generation is intrinsically linked to the mechanisms of its suppression during at least two stages: at the very beginning of the decision process and just before executing the action. For the sake of clarity, the model will be described as a sequence of neural computations. However, some of these computations can overlap in time. Additionally, the model component named “monitoring system” can modify each processing step.

Fig. 2.

Multistep decision model underlying the genesis of goal-directed actions. The model entails a set of decision processes leading to either the execution or the inhibition of an action according to their expected outcome. Clearly, the model does not have either a serial or a parallel structure, i.e., some processes must occur before others (e.g., the first decision aimed at evaluating whether acting is worthwhile must occur before goal selection), but other processes might occur in parallel (e.g., the monitoring system, whose role is to compute predictions about outcomes, is active during all the steps). CCBY 4.0, http://journal.frontiersin.org/article/10.3389/fnsys.2014.00206/full.

The earliest processing step is the “should I stay or should I go” decision that depends on motivation. The brain evaluates whether there is a need to change the current state to pursue a desire (e.g., a desire to drink a cocktail) or the state should remain unchanged because of the constraints to the desire (e.g., driving back home from the party). This evaluation is continuously performed by the brain to select only useful actions from numerous stimulus-driven affordances triggered by simple observation of affordable objects. Processing such affordances has been linked to supplementary motor area (SMA) (Grèzes and Decety 2002) and pre-SMA (Richard Ridderinkhof et al. 2011). Brain activity elicited by affordances may or may not trigger a chain of decisions leading to action execution. Although affordances increase the motivation to act, they need to be coupled with internal states, such as arousal, for an action to be executed. Accordingly, brain regions involved in arousal, in particular the dopaminergic system and the basal ganglia (Turner and Desmurget 2010), contribute to affordance selection.

If the initial processing has deemed an action worth pursuing, action preparation proceeds to the goal selection step, which should bring the best outcome from several available alternatives. Goal selection has two phases. In the first phase, values are assigned to the available options. In the second phase, behavioral context is considered to adjust these values. Likely at this point, the neural representations of possible goals begin to compete through mutual inhibition (Fig. 2). The monitoring system determines the winner of this competition. This system is formed by a network of areas that selects the best motor strategy on the basis of the available sensory data, learned behaviors, and the outcomes of previous actions. Depending on the task being performed, the monitoring can engage different networks. Experimental evidence suggests that there are some key nodes in the monitoring system. Thus it has been shown that the orbitofrontal cortex (OFC) plays a central role in linking stimuli to their values (Padoa-Schioppa and Assad 2006; Padoa-Schioppa and Assad 2008). In addition to OFC, the posterior parietal cortex (PPC) assesses available environmental resources (Genovesio et al. 2014). Lateral prefrontal cortex is another key node (Barraclough et al. 2004a; Genovesio et al. 2005; Wise 2008) that mediates flexible behaviors using information about available options, current context, and the outcomes of previous actions obtained through the interactions with the anterior cingulate cortex (ACC) (Brown and Braver 2005; Sheth et al. 2012), the dopaminergic system (Bray and O’Doherty 2007; Montague et al. 2004; Schultz et al. 1997), the pre-SMA (Bonini et al. 2014), and the frontal pole cortex (Tsujimoto et al. 2010; 2011). Attention plays an important role in goal selection. Control of attention is performed by the frontoparietal network that mediates preferential processing of relevant information (for a review see Corbetta and Shulman 2002; e.g., Lebedev and Wise 2001; Reynolds and Chelazzi 2004).

Surprisingly, even classical sensory areas can contribute to goal selection not just by providing sensory signals to higher level regions but by processing decision themselves. For example, the activity of single neurons in macaque area V4 reflects the selection of elemental object features (e.g., color or orientation of bars) and their translation into the definition of animal’s goal (Mirabella et al. 2007).

Following goal selection, the next step serves to select the action that is more likely to lead to attaining the selected goal. Given that one goal can be pursued in many ways, multiple potential actions start to compete, as shown by Cisek and Kalaska’s (2005) single-unit recordings from the dorsal premotor cortex (PMd) in monkeys performing a task in which two spatial cues indicated the potential directions for reaching movements. A nonspatial cue then narrowed the choice to one action. At this point, PMd neuronal patterns shifted from simultaneously representing both potential targets to preferentially representing the final action. These findings suggest that PMd neurons represent multiple competing actions until the winner is determined through the process of biased competition (Cisek 2006; Cisek and Kalaska 2010). Similar representation of multiple movement directions has been shown in PPC neurons (Calton et al. 2002; Cui and Andersen 2007; Klaes et al. 2011).

As during the goal selection phase, action selection can be biased by the monitoring system at any stage of action implementation (Fig. 2). For example, the monitoring system can delay action execution by inhibiting the conversion of motor preparatory activity into the motor execution output. Kaufman et al. (2014) suggested that this inhibition is enacted by a population mechanism that effectively confines the preparatory activity to PMd. The model explained that during preparation of a movement, the contributions from single PMd neurons cancel out at the population level, which prevents movement onset. However, the actual mechanism appears to be more complicated than that, because delay-period activity related to motor preparation is not confined to the higher order cortical areas and occurs even in the spinal cord (Prut and Fetz 1999).

Even after it has been decided that the selected action will achieve the desired goal, the selection continues to be scrutinized until it is finally executed. This late “should I stay or should I go decision” is necessary, because during the interval between the time when an action is selected and the moment when the motor output is about to be generated, the context could suddenly change, altering the action’s value and thus requiring its suppression. The neural representation of pending action suppression has been studied using the stop-signal (or countermanding) paradigm (Logan and Cowan 1984). This paradigm probes a subject's ability to withhold a movement triggered by a go signal after an infrequent stop signal is presented at a variable delay (Fig. 3).

Fig. 3.

Temporal sequence of trials of stop signal task. All trials begin with the presentation of a central stimulus. Subjects are required to touch it for a variable time. Thereafter, the central stimulus disappears and simultaneously a target appears to the right (go-signal). In the no-stop trials, subjects have to perform a speeded reaching movement toward the peripheral target. Differently, in stop trials (33% of the total trials), after variable delays (known as stop-signal delays, SSDs), the central stimulus reappears. To perform correctly, subjects have to inhibit the pending movement by keeping the arm on the central stimulus (stop-success trial). Otherwise, if subjects execute the reaching movement despite the stop-signal presentation, the trial is scored as an error (stop-failure trial). CCBY 4.0, http://journal.frontiersin.org/article/10.3389/fneng.2012.00012/full.

The countermanding task assesses two types of inhibition: the reactive inhibition that represents the ability of a subject to react to the stop signal and is measured by the stop-signal reaction time (SSRT) (Band et al. 2003; Boucher et al. 2007; Logan and Cowan 1984) and the proactive inhibition that occurs when subjects adjust their response strategies because they are aware of the possible occurrence of the stop signal (Mirabella et al. 2006; Mirabella et al. 2008; Mirabella et al. 2013; Verbruggen and Logan 2008; Zandbelt and Vink 2010).

A large network of brain regions is involved in action inhibition, including both cortical and subcortical structures. The inferior frontal gyrus (IFG), especially the one in the right hemisphere, plays a key role in inhibiting actions (Aron et al. 2014). Another critical node is pre-SMA (Aron et al. 2007; Jahfari et al. 2012; Li et al. 2008; Scangos and Stuphorn 2010a; Zandbelt and Vink 2010). Two basal ganglia are believed to be part of the inhibitory network: the subthalamic nucleus (STN) (Aron and Poldrack 2006; Mirabella et al. 2012; van den Wildenberg et al. 2006) and the striatum (Li et al. 2008; Zandbelt and Vink 2010), even though their precise role is still hotly debated. Quite surprisingly, it was recently discovered that the PMd (Mattia et al. 2012, Mattia et al. 2013; Mirabella et al. 2011) and the primary motor cortex (M1) (Coxon et al. 2006; Mattia et al. 2012; Swann et al. 2009) also play a role in inhibiting pending actions. For instance, Mattia et al. (2012), by recording event-related potentials (ERPs) from the lateral surface of the frontotemporal lobes of epileptic patients performing a countermanding task, found that an ERP complex was selectively expressed before the end of the SSRT in M1 and premotor cortex (Fig. 4).

Fig. 4.

Distribution of stop-event-related potentials (ERPs) in successful-stop (SS) trials. A: average stop-ERPs (solid red curves) of SS trials centered on stop-signal appearance corresponding to the selected channels for an example epileptic patient. Gray areas represent time intervals at which the stop-ERP was significantly different from 0 (Wilcoxon signed-rank test, P < 0.01). Brodmann's areas (BAs) over which electrodes were positioned are indicated. Colored areas identify electrodes placed over the primary motor cortex (red; BA4), premotor cortex (yellow; BA6), and dorsolateral prefrontal cortex (green; BA9). B: histogram of the stop-ERP sizes shown in A. Stop-ERP sizes were computed as the integral of absolute values of stop-ERP voltage deflections in the interval periods marked by gray areas within SSRT. Dashed line represents the threshold value for selecting channels with large enough stop-ERPs used for population analyses. C: number of channels showing large enough average stop-ERPs across 5 patients (n = 39) grouped by BA. Blue bar (others) represents those areas where channels were not selected more than twice across all patients. D: box plot of stop-ERP onsets measured with respect to the end of SSRT across all selected channels in all patients. Stop-ERP onset was defined as the first time that an electrode voltage was significantly different from 0. Average onset times are indicated by diamonds. Bars indicate the first and the third quartile. Vertical lines indicate the extreme time lags in the channel group. CCBY 4.0, http://journal.frontiersin.org/article/10.3389/fnsys.2014.00206/full.

Taken together, these studies demonstrate a considerable overlap between the brain region that plan actions and the regions that perform action suppression. There is also an overlap between the networks engaged in proactive controls and those activated during reactive control (van Belle et al. 2014). Overall, these findings show that a wide network of brain areas is involved in motor decision making and inhibitory control.

Notably, motor decision making can continue even after a limb movement is initiated. Indeed, limb movements, unlike saccades, are not ballistic movements (De Jong et al. 1990; Scangos and Stuphorn 2010b). Consequently, they can be stopped or modified at any point along their path (e.g., see Mirabella et al. 2008; Mirabella et al. 2013).

Recording and Decoding Approaches for a Decision-Making BMI

Neural computations described in the previous section are highly distributed and complex. Therefore, BMI decoding of these neural patterns is a challenging task. We will focus mostly on intracranial BMIs based on single-unit or multiunit recordings from ensembles of brain neurons (Andersen et al. 2014; Lebedev and Nicolelis 2006; Nicolelis and Lebedev 2009; Schwartz 2007; Schwarz et al. 2014); noninvasive approaches will be covered in less detail. There are many brain areas that could provide sources of neuronal activity for a decision-making BMI. Traditional, well-studied areas linked to decision making include lateral intraparietal cortex (Shadlen and Newsome 1996), dorsolateral prefrontal cortex (Kim and Shadlen 1999) and frontal eye fields (Hanes and Schall 1996), and PMd (Mirabella et al. 2011). Since it is already known that neurons in these areas modulate their firing during decision-making steps, it is highly likely that a BMI connected to the same sites could extract useful and relevant information.

Being able to decode the interplay between motor preparation and inhibitory prevention of premature movement initiation is of particular interest for BMI applications striving to enact the capacity to delay movements until the decision is fully formed and to be able to stop a prepared movement if the behavioral context changes (Mirabella 2012). Key areas for such an inhibitory-control BMI are PMd (Mattia et al. 2013; Mirabella et al. 2011), M1 (Mattia et al. 2012), parietal reach region (PRR) (Musallam et al. 2004a), and the basal ganglia, particularly STN that plays a role in both reactive (Mirabella et al. 2012; van den Wildenberg et al. 2006) and proactive inhibition (Mirabella et al. 2013). STN is a common site for deep brain stimulation for the treatment of advanced form of Parkinson’s disease (Benabid et al. 2009), so these patients could be subjects for testing STN-based BMIs.

BMI experiments could help to clarify the functional role of specific brain areas involved in decision making. Ideally, to fully examine the contributions of different regions, a BMI should simultaneously sample activity from multiple areas (Nicolelis and Lebedev 2009). Increasing the number of areas being recorded has been shown to improve BMI accuracy (Lebedev 2014b; Lebedev and Nicolelis 2006). Recording large neuronal samples is generally handy for BMI accuracy (Batista et al. 2008; Fitzsimmons et al. 2009; Ganguly and Carmena 2009; Golub et al. 2014; Lebedev and Nicolelis 2006; Santhanam et al. 2006), so the same should hold true for decision-making BMIs. Indeed, although single neurons are on average very well tuned to decision-making tasks (Hanes and Schall 1996; Mirabella et al. 2007; Shadlen and Newsome 1996), there is a substantial trial-to-trial variability in the single-cell discharges, which would hinder an attempt to decode information from the activity of one or a few neurons. The signal-to-noise ratio improves when larger neuronal populations are recorded. Additionally, large neuronal samples are needed for simultaneous decoding of multiple behavioral variables (Fitzsimmons et al. 2009).

In practice, a limited number of areas can be sampled in both animal and human experiments. In humans, because of the safety requirements, typically only one area can be sampled (Aflalo et al. 2015; Bouton et al. 2016; Collinger et al. 2013; Hochberg et al. 2006; Hochberg et al. 2012). Such limited neuronal samples can still be useful for decoding multiple parameters because even a single brain area typically participates in the encoding of more than one function (Nicolelis and Lebedev 2009). For example, overlapping populations of PMd neurons become active when multiple potential actions are represented (Cisek and Kalaska, 2005), as well as when an action is executed or inhibited (Mattia et al. 2013; Mirabella et al. 2011). These PMd activity patterns are modulated by selective spatial attention, yet another behavioral variable (Lebedev and Wise 2001). Accordingly, multiple variables could be extracted from PMd alone by using an implant that samples a sufficiently large neuronal population.

Previous motor BMIs utilized implants placed in M1 (Carmena et al. 2003; Ganguly and Carmena 2009; Gilja et al. 2012; Lebedev et al. 2005; Sadtler et al. 2014; Serruya et al. 2002; Taylor et al. 2002; Velliste et al. 2008; Wessberg et al. 2000), premotor (Carmena et al. 2003; Lebedev et al. 2008; Schwartz and Moran 1999; Stavisky et al. 2015), and posterior parietal regions (Carmena et al. 2003; Mulliken et al. 2008; Philip et al. 2013), the regions where neurons modulate their activity during ongoing movements. Experimental BMI runs typically consisted of two phases: 1) a training period before the BMI usage needed to set the decoder initial parameters, and 2) BMI control. During the training period, neural recordings are conducted for ~5–10 min while subjects perform overt limb movements (Carmena et al. 2003; Lebedev et al. 2005; Serruya et al. 2002; Taylor et al. 2002) or passively observe the movements of a computer cursor or an artificial limb (Hochberg et al. 2006; Ifft et al. 2013; Tkach et al. 2008). On the basis of these recordings, the relationship is evaluated between the neuronal patterns and kinematic parameters of interest, and the decoder is set to predict the kinematics from neuronal signals.

The same basic approach would be appropriate for training a decision-making decoder: an animal would first perform the task manually and then switch to BMI control. The training period should include samples of behaviors that the BMI control is required to reproduce, for example, stopping prepared movements if the BMI incorporates inhibitory control. Figure 1 gives an example of a decision-making task where a monkey initially reports its decisions manually, using a joystick to place the cursor on a screen target. The task may incorporate a requirement that the joystick should stay still in the no-go trials. Next, after the BMI decoder is trained, the animal starts to perform the task without the joystick, by simply thinking about the desired action. In this implementation, the decoder evaluates various components of decision making until it determines the go or no-go decision with the required degree of certainty. Note that this decision-making BMI does not require the user to control cursor position continuously. Once the decision is detected, the BMI can automatically move the cursor to the target without any contribution from the monkey to the ongoing movement. Alternatively, the monkey could be allowed to correct the movement (by brain signals or, in some experiments, manually) after it has started.

We are not going into the details of decoding algorithm appropriate for decision-making BMIs. These algorithms are not principally different from those covered in previous reviews (Bashashati et al. 2007; Li 2014; Schwartz et al. 2001).

Potential Advantages of Decision-Making BMIs

Decision-making BMIs offer several advantages over traditional motor BMI that perform continuous decoding of movement parameters (Carmena et al. 2003; Ganguly and Carmena 2009; Lebedev et al. 2005; Taylor et al. 2002; Wessberg et al. 2000; Wu et al. 2004) under an assumption that motor cortical neurons normally represent limb kinematic parameters (Georgopoulos et al. 1982; Georgopoulos et al. 1988; Moran and Schwartz 1999). This assumption does not consider the dependency of cortical activity on behavioral context. For example, because of the differences between cortical patterns at rest and during movements, it is difficult to reproduce with BMIs the motor behaviors that consist of movements intermingled with periods of stationary posture (Golub et al. 2014; Velliste et al. 2014). Decision-making BMIs could overcome this difficulty by parsing such behaviors into periods of immobility, when decisions are being made, and periods of motor execution (Hasegawa et al. 2006; Lebedev et al. 2008).

The major advantage of decision-making BMIs is that, by obtaining information about values, goals, and motor preparation before movement onset, they act as predictive systems, also called systems with forward or internal modeling (Conant and Ashby 1970; Francis and Wonham 1976). Importantly, the brain itself normally operates as a predictive system. Predictive neural processing is very effective for motor control because it enables the brain with the capacity to utilize the internal model of body biomechanics for error correction and minimization of processing delays (Adams et al. 2013; Kawato 1999; Shadmehr et al. 2010; Wolpert et al. 1995; Wolpert and Kawato 1998).

It seems reasonable to suggest that BMI performance would benefit from the same principles (Cui 2016). Indeed, an advanced knowledge of motor intent could prepare a BMI-controlled external device for an upcoming action so that the device could produce movements with minimal processing and execution delays. Moreover, advanced knowledge of an action goal could be used to reduce execution errors after the movement starts. In the most trivial scenario, the goal of movement is set by a computer that runs the experimental task, so the ideal trajectory is known in advance. In several human BMI experiments, the ideal trajectory was used to assist the subjects’ training: corrections were issued to the neural commands by bring the arm closer to the ideal trajectory (Collinger et al. 2013; Hochberg et al. 2012). Following several weeks of such assisted control, the subjects could eventually perform without any assistance. In a monkey study, ideal trajectories were used for coadaptive learning (Taylor et al. 2002), one of many possible supervised learning algorithms (Mohri et al. 2012). Here again, the ideal trajectories were imposed by the computer. A less trivial scenario involves the derivation of the ideal trajectory from the brain activity, for example, from an instructed-delay period preceding movement onset (Hwang and Andersen 2009; Musallam et al. 2004a). In this case, after the trajectory is determined, the BMI can generate the movement without any additional contribution from the user. A more sophisticated BMI would integrate the motor goal decoded before movement onset with continuous decoding of motor commands during movement execution. Such a BMI was demonstrated in monkeys (Shanechi et al. 2013). In this study, target location was decoded from the activity of PMd and SMA neurons recorded during an instructed-delay period. Next, cursor trajectory was generated by an optimal feedback controller that utilized both the decoded target location and the ongoing activity of cortical neurons.

An additional benefit of decision-making BMIs is related to their capacity to decode abstract representation of actions, such as representing an object to be grasped (Murata et al. 1997) instead of controlling every detail of the reach and grasp movement, simultaneous tracking of several potential motor plans (Cisek and Kalaska 2002), decoding behavioral context (Velliste et al. 2014), and being able to cancel or modify a movement before it starts to be executed (Mirabella 2012). Decoding of such motor representations could be utilized in a shared-control design, where the user issues a high-level command and a robotic controller handles low-level details of an action (Tonin et al. 2010). Additionally, a passive BMI design was proposed, where an external device adapts to the user by monitoring the user’s cognitive neural activity (Zander and Kothe 2011). The BMI is called passive because the user is not required to actively control the device with the BMI output.

Overall, we envision a next-generation of BMIs with the capacity to decode motor decisions in advance of action initiation. The development of such systems will facilitate translation of neuroprosthetic systems from the laboratory settings to the real world. Several examples of BMIs with decision-making components are presented below.

Decoding Go vs. No-Go Decisions, and Potential Motor Targets

One of the earliest studies on decoding decisions was conducted by Hasegawa et al. (2006). In this study, monkeys were trained on an oculomotor go/no-go task, and neuronal activity was recorded in superior colliculus (SC). A binary regression method was employed to predict the go/no-go decision from the activity of single SC neurons with 80% accuracy. The prediction was issued on average 77 ms before saccade onset. In the next study by this group, a prediction of visual target location was added to the go/no-go decoding (Hasegawa et al. 2009). The authors suggested that such predictive decoding could increase the speed of BMI operations.

Santhanam et al. (2006) demonstrated similar decoding of target locations for an arm reaching task. In these experiments, monkeys watched potential targets for an arm reaching movement quickly flashed on a screen. Neuronal population activity was recorded in PMd and converted into a sequence of predicted target locations. The authors argued that since the observed PMd responses to the targets represented prepared movements, such rapid decoding could be utilized to achieve a high-performance BMI control (6.5 bits/s). Such a BMI would save time by moving the cursor to the target as soon as target location is decoded from the PMd preparatory activity.

Note that these decoding approaches belong to the class of exogenous BMIs, i.e., BMIs that require an external stimulus to operate. Exogenous BMIs have been implemented using noninvasive recordings, as well. For example, P300 (Donchin et al. 2000; Fazel-Rezai et al. 2012) and steady-state visual evoked potential (SSVEP) (Bakardjian et al. 2010; Ortner et al. 2011) BMIs decode electroencephalographic (EEG) responses to visual stimuli.

Decoding Action Timing

For an action to be successful, it is important to correctly decide not only what to do but also when to initiate the selected movement. Deciding when to start an action is often described as interval timing in the literature (Buhusi and Meck 2005; Ivry and Spencer 2004; Meck and Malapani 2004; Merchant and de Lafuente 2014). This temporal programming is supervised by two neural systems (Buhusi and Meck 2005). One system involves the cerebellum and operates in the millisecond range. The other involves cortico-striato-thalamic networks and works on the seconds-to-minutes timescale (Hinton and Meck 2004; Matell et al. 2003; Meck 2005; Rao et al. 2001).

Neurophysiological studies in monkeys have reported that many cortical and subcortical neurons exhibit ramping activity during delay intervals of instructed-delay tasks (Durstewitz 2004). This activity pattern can represent motor programming (Romo and Schultz 1987; Weinrich and Wise 1982), stimulus anticipation (Mauritz and Wise 1986), sensorimotor transformation (Crammond and Kalaska 1994), orientation of selective spatial attention (Lebedev et al. 2004; Lebedev and Wise 2001), working memory (Fuster 2001; Goldman-Rakic 1995), timing of motor acts (Niki and Watanabe 1979), anticipation of reward (Roesch and Olson 2005; Schultz 2006), and decision making (Kim and Shadlen 1999; Schall 2001; Sugrue et al. 2005).

Lebedev et al. (2008) decoded temporal intervals from M1 and PMd activity. In this experiment, monkeys produced hand movements at regular time intervals (2.5–4.5 s). This behavior was entirely self-initiated, i.e., no external stimuli were used. Monkeys started each trial by touching a button. They continued to touch the button for the required duration and then lifted the hand. For each moment in time, the interval that passed from the touch onset and the interval till the button release were decoded from cortical activity using a linear decoder. These results are consistent with the processing described by our model, where timing signals are generated and sent to the sensorimotor cortex by the monitoring system. During the self-timed delay, the monitoring system assures that an action is withheld.

Decoding of self-timed intervals represents an endogenous BMI, i.e., the BMI that relies on motor intents generated by the subject. Endogenous BMIs do not require an external stimulus to operate. In the noninvasive domain, endogenous BMIs often employ motor imagery (Halder et al. 2011). Recently, neural correlates of motor imagery were demonstrated using invasive recordings from PPC in a tetraplegic patient (Aflalo et al. 2015). Individual PPC neurons and their populations clearly responded to imagined trajectories, motor goals, and different types of movement. Moreover, the patient learned to control a robotic arm with these signals.

Decoding Value

As described in the model of decision making (Fig. 2), a key step in defining the goal is the computation of expected outcome of an action. In BMI literature, this type of neural computation is considered a cognitive process, in contrast to the lower level motor or sensory processing occurring during movement execution. Accordingly, the term “cognitive neural prosthesis” (CNP) has been proposed to describe a BMI that decodes the cognitive state of a subject, including parameters that represent executive functions, decision making, and attention (Andersen et al. 2004; Andersen et al. 2010).

Cognitive planning of an arm movement starts in multiple areas in the frontal and parietal cortices several hundreds of milliseconds before the actual movement. This frontoparietal activity carries information about expected values of different actions, in addition to spatial and temporal parameters of upcoming movements. Musallam et al. (2004b) developed a CNP that decoded both expected rewards and the intended arm trajectories from neuronal activity sampled in parietal reach region (PRR). In these experiments, monkeys prepared an arm movement but withheld it until a trigger stimulus was presented. Additionally, characteristics of reward (type, size, or probability) were signaled to the monkeys as the size of a cue presented in the beginning of each trial. It was found that the expectation of a preferred reward increased neuronal responses and enhanced spatial tuning. Using this property of neuronal responses, the authors implemented neural decoders that predicted the properties of expected rewards and target location simultaneously. Thus this study showed that expected value, an essential component in decision making, can be decoded from PRR activity using a BMI.

Recently, Marsh et al. (2015) showed that neuronal activity in monkey M1 represents reward expectation during an arm-reaching task and that this representation persists even when animals passively observe cursor movements. The authors proposed that it would be possible to use M1 reward-related activity to implement a BMI that adjusts its performance using a reinforcement-learning scheme. The presence of reward-related activity in monkey M1 and PMd was confirmed by another group (Ramkumar et al. 2016). Moreover, a reinforcement-learning BMI, called symbiotic BMI, was implemented in a rodent study, where motor intents were decoded from M1 activity while nucleus accumbens activity provided information about rewards (Mahmoudi and Sanchez 2011). This BMI employed an actor-critic design that allowed the system to adapt in an unsupervised way. Note the resemblance of the block diagram of this design (Fig. 5) to the schematics of our decision-making model (Fig. 2).

Fig. 5.

Symbiotic BMI controller based on reward-related activity of rat nucleus accumbens (NAcc). The controller includes two components: 1) the actor, driven by M1 activity, and 2) the critic, based on the recordings from NAcc. The critic evaluates the actions of the actor by using the value estimator and adapts the actor’s parameters to increase the probability of reward. CCBY 4.0, http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0014760.

Decoding a Change in Motor Plan

Several studies have reported neural decoding for a task that involved changes in motor plan (Ifft et al. 2012; Kaufman et al. 2015; Kiani et al. 2014; Resulaj et al. 2009) and inhibitory control (Velliste et al. 2014). For example, Ifft et al. (2012) utilized BMI approach to detect cancellation of a motor plan and its replacement by a different plan (Fig. 6). Activity of M1 and S1 neurons was recorded in two rhesus monkeys implanted with multielectrode arrays. The monkeys acquired screen targets with a cursor that they moved with a joystick. In the beginning of each trial, they placed the cursor at screen center. Next, a visual target appeared at a peripheral location. In some trials the target stayed at that location, and in the others it was replaced in 50–250 ms by a different target at a new location. In the latter trials, the monkeys managed to inhibit movements to the initial target and successfully acquired the second target. In an offline analysis, the locations of both the first and second targets were decoded from cortical activity using a linear discriminant analysis (LDA) classifier. This analysis showed that cortical activity simultaneously represented the first and second targets.

Fig. 6.

Decoding a change in motor decision. A: experimental setup where a monkey sits in front of a computer screen and uses a joystick to move a cursor. B: 4 potential locations of screen targets. C: task sequence. The cursor was first placed over the central target. An initial target then appeared on the boundary ring. This target persisted in one-fourth of trials, and it jumped to a new location in the remaining trials. D: time records for the joystick coordinates and target onsets and offsets. CCBY 4.0, http://journal.frontiersin.org/article/10.3389/fneng.2012.00016/full.

These findings suggest that, in agreement with our decision-making model, neural signals carrying information about contextual changes are represented in cortical activity and could be decoded by a BMI. In the model, these signals are generated by the monitoring system. Therefore, the study of Ifft et al. (2012) provides insights on the population representation of decision monitoring. We suggest that the capacity of inhibiting a prepared action and replacing it by a new one could improve the performance of practical BMIs in the future.

Evaluation of Artificial Sensory Evidence

Processing of sensory information is essential for motor control and plays a key role in in goal-directed behaviors (Hernández et al. 2010; Mirabella et al. 2007; Shadlen and Newsome 1996; Wise and Murray 2000). BMIs bring a new development to this domain because they can deliver artificial sensory information to the brain using electrical stimulation of sensory areas (Boi et al. 2015; Dadarlat et al. 2015; Fagg et al. 2009; Lebedev and Nicolelis 2006; Lebedev et al. 2011; London et al. 2008; Nicolelis and Lebedev 2009; Rothschild 2010; Vato et al. 2014).

O’Doherty and colleagues developed an interface, called brain-machine-brain interface (BMBI), that simultaneously decoded motor commands from M1 and delivered artificial tactile sensations to S1 using intracortical microstimulation (O’Doherty et al. 2009; O’Doherty et al. 2012). Monkeys learned to use this BMBI to perform active tactile exploration with a virtual hand. The virtual hand scanned targets on a computer screen that were visually identical but were associated with different artificial textures. The monkeys felt the texture when they placed the virtual hand over a target and received a temporal pattern of microstimulation in S1. The animals learned to quickly explore the virtual target until the one with a specific artificial texture was found. They held the artificial hand over that target for a required hold period to obtain a reward. This behavior was clearly based on a behavioral strategy that involved multiple decisions related to exploration of each target and choosing the next target to explore. It was found that monkeys spent ~500 ms exploring each target and were much more likely to revisit the target associated with the rewarded artificial texture than the unrewarded ones.

Another BMBI was demonstrated by Klaes et al. (2014). In this study conducted in monkeys, control commands that moved a virtual arm were derived from PPC recordings, and artificial tactile sensations were implemented using intracortical microstimulation of S1. Aided with this BMBI, monkeys learned to locate a hidden from view virtual object and to discriminate virtual objects with different artificial textures.

These results show that artificial sensations produced by intracortical microstimulation are suitable for guiding goal-directed actions. Moreover, these goal-directed actions can be fully generated with a BMBI, where both motor control loop and sensory feedback loop are handled by the interface. This information flow matches our description of planning and modifying goal-directed movement (e.g., see Mirabella et al. 2008; Mirabella et al. 2013), even though sensory inputs and motor outputs travel via artificial routs. It would be of interest to test in the future if a BMBI could implement an efference copy (Crapse and Sommer 2008) for artificial sensations, that is, forward modeling and predictive processing of sensory information.

Shared Decision Making: Brain-to-Brain Interface

Finally, we will mention the brain-to-brain interface (BTBI) approach as an example of how neural activity underlying decisions could be shared by multiple subjects. BTBIs utilize neuronal activity recorded in several brains simultaneously to enact goal-directed behaviors. Using multiple brains improves an overall performance and/or helps to solve difficult tasks. In one such demonstration, several monkeys controlled a virtual arm with their brain activity cooperatively (Ramakrishnan et al. 2015). In the other two studies conducted in rats, BTBIs enabled information exchange between different brains (Pais-Vieira et al. 2013; Pais-Vieira et al. 2015). Signals were transmitted from one rat, called the encoder, to the other rat, called the decoder (Pais-Vieira et al. 2013). While the encoder rat performed a sensory discrimination task, its brain activity representing behavioral choice was delivered to the decoder’s S1 using intracortical microstimulation. The decoder rat learned to reproduce the performance of the encoder rat. A similar experimental design (Pais-Vieira et al. 2015) was used to connect several rat brains into a net that performed computer-like operations, such as pattern recognition and dynamic memory allocation.

Although these tasks were not focused on decision making, BTBI design clearly could be used to construct decision-making schemes, where a decision is made collectively by several participants analyzing the same sensory information or its different components. One example of such a scheme recently emerged in EEG-based BMIs (Stocco et al. 2015). This study implemented a BTBI where one subject could guess what was on the mind of the other subjects by using an interactive question-and-answer game. EEG patterns of the subject, called the respondent, were decoded and delivered to the visual cortex of the subject, called the inquirer, using transcranial magnetic stimulation (TMS). TMS pulses evoked sensations of light flashes.

Although current BTBIs are relatively simple, more sophisticated decoding of decision-making will probably be added to such systems in the future, particularly when multichannel communication protocols start to be used. We would not even exclude the possibility that making one human read the mind of the other using a BTBI could one day yield knowledge that mathematical analysis of neural patterns cannot provide. Indeed, a brain could be better off listening to the other brain and deciphering the message by using an immense computational power of neural circuitry adopted for this kind of signals.

Concluding Remarks

Notwithstanding an impressive growth of BMI research, there are still many unexplored issues, particularly regarding the decoding of higher order neural representations. In this review, we have focused on the possibility that BMIs could be applied to decoding the process of decision making that underlies goal-directed motor behavior. We propose that enhancing the existing motor BMIs with these new features will bring them closer to the eventual goals of restoring naturalistic behaviors (Aflalo et al. 2015; Gilja et al. 2012) and providing clinical solutions to patients suffering from motor and sensory disabilities. Our suggestion is supported by several studies that demonstrated successful decoding of decision-making components using a BMI approach.

In addition to the clinically oriented direction of decision-making BMIs, we envision that such BMIs will become useful tools for studying neural mechanisms of decision making and testing models of these mechanisms, such as the model proposed in this article. The ability to decode various decision-making variables provides experimentalists with a possibility of making task events contingent on these variables. Additionally, decision-making BMIs could provide the subjects with neurofeedback, informing them about their own neural processing. This approach could be used to induce neuronal plasticity and possibly could even clarify the role of consciousness and volition in making decisions. Although these ideas are presently speculative, their viability is supported by a recent report of Schultze-Kraft et al. (2016), where subjects’ ability to exert inhibitory control was challenged by a BMI that extracted their intention to move from EEG readiness potentials.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

G.M. and M.A.L. interpreted results of experiments; G.M. and M.A.L. prepared figures; G.M. and M.A.L. drafted manuscript; G.M. and M.A.L. edited and revised manuscript; G.M. and M.A.L. approved final version of manuscript.