Brain–computer interfaces for dissecting cognitive processes underlying sensorimotor control

Abstract

Sensorimotor control engages cognitive processes such as prediction, learning, and multisensory integration. Understanding the neural mechanisms underlying these cognitive processes with arm reaching is challenging because we currently record only a fraction of the relevant neurons, the arm has nonlinear dynamics, and multiple modalities of sensory feedback contribute to control. A brain–computer interface (BCI) is a well-defined sensorimotor loop with key simplifying advantages that address each of these challenges, while engaging similar cognitive processes. As a result, BCI is becoming recognized as a powerful tool for basic scientific studies of sensorimotor control. Here, we describe the benefits of BCI for basic scientific inquiries and review recent BCI studies that have uncovered new insights into the neural mechanisms underlying sensorimotor control.

Introduction

Successful sensorimotor control requires the coordination of multiple cognitive processes. On a moment-by-moment basis, the brain integrates various sources of sensory information [1–3], selects and plans upcoming movements [4–6], internally predicts the consequences of motor commands [7], and adapts to compensate for changes in the body and environment [8,9]. Addressing the complexity of these interconnected processes poses a formidable challenge for neuroscientists seeking a more complete understanding of the neural mechanisms underlying sensorimotor control.

A common paradigm for studying sensorimotor control is arm reaching (Figure 1, left). Even the simplest of arm movements emerge from a complex set of neural, muscular and skeletal systems. Movement generation involves multiple distinct cortical areas that project to the spinal cord, numerous striatal and cerebellar loops, and several brainstem and thalamic nuclei [10]. However, in typical experiments, we can monitor only a tiny fraction of the hundreds of thousands of neurons that project to motoneuron pools, and it is often unknown whether the recorded neurons project to the spinal cord or affect behavior only indirectly. As a result, it is difficult to causally attribute behavioral changes to specific changes in the recorded neural activity. Furthermore, the arm is a multi-jointed structure actuated by dozens of muscles [11], and because of these complexities, the arm’s nonlinear dynamics are not typically measured in studies of arm reaching. In addition, sensory feedback about the arm movement is carried through multiple sensory modalities, including vision and proprioception, that have different latencies and need to be combined [12]. Although visual feedback about the arm can be readily manipulated [8,12–14], proprioceptive feedback cannot be decoupled from movement as easily [15].

Figure 1.

Conceptual illustration of the sensorimotor control loop for arm reaching (left) and BCI (right) movements.

How can we obtain a more complete understanding of the cognitive processes underlying sensorimotor control in light of this daunting complexity? Perhaps we could gain traction if we could simultaneously record neural activity from multiple brain areas in the motor system, including all neurons that directly drive movement; if we could identify and reversibly reprogram the precise mathematical relationship between neural activity and movement; and if we could independently alter different modalities of sensory feedback.

In this review, we describe how brain–computer interfaces (BCIs) provide a simplified, well-defined, and easily manipulated experimental paradigm that facilitates the basic scientific investigation of the cognitive processes engaged during sensorimotor control (Figure 1, right). A BCI creates a direct mapping between recorded neural activity and the movement of a device, such as a computer cursor (or robotic limb) [16,17••,18–22,23••] and substantially simplifies the complexities described above (Table 1). Although neurons throughout the brain can indirectly influence movement, only the activities of experimenter-chosen output neurons directly drive BCI cursor movements. Thus, all aspects of behavior must be expressed in the activity of these recorded output neurons, and it is possible to causally interpret the role of each neuron in behavior since the mapping between neural activity and cursor movement is completely known to and specified by the experimenter. Cursor dynamics can be defined by the experimenter to be linear, and they can be altered as desired. Furthermore, we can flexibly manipulate the sensory feedback because proprioceptive feedback is not hard-wired to cursor movement. As a result of these simplifications, we and others have begun to leverage BCI as a powerful experimental paradigm for addressing basic scientific questions about sensorimotor control. Although similar concepts apply to EEG-based (e.g. [24]) and ECoG-based (e.g. [25]) BCI, we focus on intracortical BCI in this review. Previous reviews have described use of BCI for addressing scientific questions [26–28]. Here, we focus on the key simplifications offered by BCI and describe the scientific insights that have emerged by leveraging each simplification.

Table 1.

Comparison of BCI control to arm reaching. Bold items indicate entries that make BCI a simplified, well-defined and easily manipulated system for studying sensorimotor control

	Arm reaching	BCI
Effector	Arm	Cursor or robotic limb
# of non-output neurons	Millions	Millions
# of output neurons	Thousands (only a subset are recorded)	Tens-to-hundreds (all are recorded)
Neuron-to-movement mapping	Unknown	Known
Effector dynamics	Difficult to measure, nonlinear	Known, can be linear
Sensory feedback	Tied to arm	Flexibly manipulable

Monitoring all neurons that directly drive movement

In arm reaching studies, it is typically not possible to attribute every aspect of behavior to specific features of the recorded neural activity because there are unrecorded neurons that can directly drive behavior. In contrast, for BCI, all neurons that directly drive behavior are recorded, by construction. Thus, it is possible to causally attribute changes in behavior to the activity of specific neurons.

This property of BCI is particularly well exemplified by single-unit operant conditioning studies. In operant conditioning, the subject’s task is to learn to volitionally modulate neural activity to specified levels using real-time visual [29,30] or auditory [31••] feedback. The visual or auditory feedback represents the BCI ‘behavior’. Operant conditioning is particularly valuable for studying (internal) cognitive processes because it allows the experimenter to manipulate neural activity that has, under ordinary circumstances, only an indirect relationship to externally measurable variables. This approach was pioneered by Fetz [30] in the motor cortex, and later adopted by a large body of studies [32–37]. More recently, studies have demonstrated the importance of operant conditioning for studying the neural substrates of cognitive processes, including spatial [31••] and object-based [38] attention, that are involved in sensorimotor control. In particular, Schafer and Moore [31••] found that volitional changes in frontal eye field (FEF) activity are associated with selective visual attention.

Distinguishing between output and non-output neurons

Beyond monitoring all neurons that directly drive behavior in a BCI (output neurons), it is also possible to simultaneously monitor additional neurons that are not explicitly mapped to behavior (termed non-output neurons). This is advantageous for investigating whether and how the activity of output neurons of sensorimotor control differs from that of non-output neurons. For example, it may be that the non-output neurons support internal cognitive processes that enable output neurons to produce activity that is suitable for driving behavior. In arm reaching studies, it can be a daunting task to identify which motor cortical (M1) neurons project directly to motoneurons that innervate muscles [39–41]. In contrast, in BCI, the experimenter simply designates which recorded neurons are output versus non-output and examines neural activity as the subject learns to use the BCI over time.

This property of BCI has been exploited by comparing the activity of output and non-output neurons in M1. Output neurons have been observed to show stronger task-related modulation than non-output neurons [42•,43,44,45•]. This finding does not imply that the non-output neurons stay silent or at baseline activity levels. Rather, the non-output neurons typically change their activity together with the output neurons [46], albeit with weaker task-related modulation. It is also possible to record non-output neurons outside of M1. Carmena and colleagues recorded non-output neurons in the striatum and found that corticostriatal plasticity is necessary for BCI learning [47] and is specific to the output neurons in M1 [48••].

Defining a simple mapping

In arm reaching studies, we typically do not know the exact mapping between neural activity recorded in the brain and arm movement. Even if we could identify the mapping, it would likely involve non-linear dynamics due to spinal [49] and muscle [11] properties, making it difficult to analyze. With BCI, the experimenter specifies exactly how the activity of the output neurons maps to cursor movement. The mapping can be chosen to define linear dynamics, for example:

$${\mathbf{x}}_t=\mathbf{A}{\mathbf{x}}_{t-1}+\mathbf{B}{\mathbf{u}}_t$$ (1)

where x is the cursor movement (e.g. velocity), u is the activity of the output neurons, and A and B are the mapping parameters. Commonly used BCI mappings, including the Kalman filter, optimal linear estimator and population vector algorithm, can be expressed in this form.

Studies have exploited the linearity of the BCI mapping to elucidate the neural mechanisms of cognitive processes underlying sensorimotor control. It is widely believed that we form internal models of our effectors, which enable internal prediction to compensate for sensory feedback delays and selection of appropriate neural activity patterns [8]. We found that cursor movement errors can be explained by a mismatch between the internal model and the BCI mapping [50••]. By using a linear BCI mapping, we could focus on the family of linear internal models, which facilitated the identification of internal models from neural activity. Another convenient property of a linear BCI mapping is that one can exactly identify its null space — the space of neural activity changes that do not affect movement [51]. Analyses of neural variability in this null space have led to insights into sensorimotor learning [45•] and error correction [52].

Changing the BCI mapping

A key feature of sensorimotor control is the ability to learn, adapt, and refine motor skills over time. This ability is mediated by a suite of learning processes that operate in parallel to maintain proficient control [14]. Our understanding of these learning processes has relied largely on studies of arm movements [8], hand movements [53], and muscle activity [54–56]. Disentangling the neural basis of these processes has proven challenging [57–59], primarily because it is difficult to interpret the behavioral relevance of adaptive changes in neural activity if the causal role of those neurons in generating behavior is not known [60].

BCI can help to overcome this difficulty. Just as the experimenter can define the BCI mapping from neural activity to behavior, he or she can also redefine the mapping as desired throughout an experiment and observe whether the subject is able to learn to use the new mapping. Importantly, learning-related changes in behavior can then be causally attributed to specific changes in the recorded neural activity. The mapping can be redefined by shuffling its weights [16] or by altering the weights in a structured fashion [46,61,62,23••]. These studies have found that the brain is able to attribute behavioral errors to individual neurons responsible for the errors [61,62], and that the underlying network structure shapes learning [23••]. The latter study [23••] suggests that cognitive tasks which require novel population activity patterns (i.e. additional dimensions of the population activity [63]) can be hard to learn.

Flexibly manipulating sensory feedback

Executing a reaching movement requires estimating the current state of the arm by combining visual, proprioceptive, and other sensory information. Understanding multisensory integration is challenging because each sensory modality has a different coordinate frame and processing delay, and the subject weights the different modalities based on their reliability [2]. To understand how different sensory modalities contribute to sensorimotor control, one would like to be able to manipulate or turn off each sensory modality independently. Visual feedback is relatively easy to manipulate [8,12–14]. Proprioception, however, is tied to the arm, making it difficult to manipulate or turn off in a way that still allows the subject to perform the task [15].

With BCI, the experimenter can independently manipulate visual and proprioceptive feedback, thereby facilitating the study of multisensory integration during sensorimotor control. Because BCI movements are not tied to arm movements, it is possible to perform BCI control with the arm stationary, moving in concert with the cursor, or moving independently of it [64,65]. This property of BCI was leveraged to study the influence of proprioceptive feedback in M1 [17••]. The study found that BCI control was more accurate when the arm was moving in concert with the cursor, thereby demonstrating the importance of informative proprioceptive feedback for M1. In a related study, Dadarlat et al. [66•] recently showed that microstimulation of primary somatosensory cortex can induce artificial proprioception, and that the sensorimotor system can optimally integrate this information with noisy visual feedback.

Conclusions

Science proceeds through the use of model systems: to understand an important but complex aspect of the natural world, we carefully select reduced systems that are experimentally tractable and that bear an important relation to the full system. Insights can emerge from the study of the simpler system that apply directly in the complex system, or at least help to crack open new avenues to understanding the more daunting problem [67]. An important question is whether BCI is an appropriate reduced system for studying sensorimotor control. BCI cursor control performance is approaching arm reaching performance [20], thereby enabling well-controlled scientific studies. In addition, well-established cognitive processes present during arm reaching, such as internal forward prediction [50••], learning [16,23••], and multisensory integration [17••], are also engaged during BCI control. Nevertheless, it remains an open question whether the neural mechanisms underlying these cognitive processes are conserved between arm reaching and BCI control. Current findings suggest that there is at least some overlap in the brain areas involved in native motor control and BCI control. Coordination between the cortex and the basal ganglia appears to be required for motor learning in both native motor [68,69] and BCI [47,48••] control. Although the role of cerebellar circuits in BCI control has not yet been explicitly demonstrated, BCI subjects adapt to visuomotor rotations [62] and form internal models [50••], both of which are believed to be primarily cerebellar-dependent processes [8]. Because of the similarity of the cognitive processes and brain areas involved in native motor and BCI control, we view BCI as a stepping stone toward understanding the full native sensorimotor control system. The BCI paradigm, being a reduced system, offers vastly improved accessibility and manipulability, without simplifying away the complexities of brain processing that we wish to study.

As with any reduced system, BCI does have certain limitations for studying the full system. There are aspects of sensorimotor control that are difficult to study with a cortical BCI because they occur ‘downstream’ of the particular set of neurons used for BCI control (e.g. spinal cord mechanisms). Also, it is an open question how the absence of some natural sensory feedback mechanisms (e.g. touch) shapes the cognitive processes being studied during BCI control. Despite its limitations, BCI studies have made important headway toward elucidating the neural mechanisms of attention, prediction, learning, and multisensory integration in the context of sensorimotor control. Sensorimotor processing is just one manifestation of cognition, and the BCI approach may eventually be harnessed to yield insight into the many forms of cognition of which we human beings are capable.

• Brain–computer interfaces (BCIs) facilitate studies of sensorimotor control.

• BCI can provide insights into prediction, learning, and multisensory integration.

• Components of a BCI are simple, well-defined, and easily manipulable.

• Similar cognitive processes underlie arm reaching and BCI control.