Observation-Based Learning for Brain-Machine Interfaces

Summary

Canonically, “mirror neurons” are cells in area F5 of the ventral premotor cortex that are active during both observation and execution of goal-directed movements. Recently, cells with similar properties have been observed in a number of other areas in the motor system, including the primary motor cortex. Mirror neurons are a part of a system whose function is thought to involve prediction and interpretation of the sensory consequences of our own actions as well as the actions of others. Mirror-like responses are relevant to the development of brain-machine interfaces (BMI) because they provide a robust way to map neural activity to behavior, and because they represent high-level information about goals and intentions that may have utility in future BMI applications.

Introduction

Mirror neurons are a unique class of neurons that are characterized by the strong similarity in their activity during observation of an action and execution of that same action. The discovery of mirror neurons by di Pellegrino and Rizzolatti [1**] inspired a large field of research that has not only contributed to our understanding of the cortical motor system but has also provided insights into the neural basis of action perception [2,3**,4*] and has even suggested possible mechanisms underlying the perception of others’ intentions or even emotional states [5]. They may also be relevant to neuropsychiatric conditions such as autism and schizophrenia [6-8]. Numerous reviews detail the experimental evidence for mirror-like activity in animals and humans [9-13*]. Here, we review a number of recent reports of mirror-like activity primarily at the single neuron level and discuss their significance in the context of brain machine interface (BMI) development.

Discovery and characteristics of mirror neurons

Neurons with mirror-like properties were first discovered in the ventral premotor cortex (PMv, area F5c of the inferior frontal cortex) and in the rostral inferior parietal cortex (parietal mirror neurons; located in PF/PFG complex of the IPL) of the monkey [1**, 4*]. The first published account of mirror neurons describes their discovery as accidental. Di Pellegrino and colleagues were analyzing responses of F5 neurons related to monkey’s reaching for and grasping of food or objects of various geometric shapes and sizes. Following initial recordings and in between experimental trials, the researchers witnessed the activation of a relatively large proportion of F5 neurons as the monkey motionlessly observed experimenters picking up, moving, and replacing the food and target objects in the experimental setup [1**]. Subsequent research revealed that mirror neurons are active during observation of object-oriented actions such as hand grasping, holding, tearing and manipulating as well as mouth actions that are related to either communication or ingestion [14-16].

A key feature of a mirror neuron is the congruence between its neural activity during observation of an action and execution of that same action. Neurons that respond to observation of action had in fact been identified earlier in the superior temporal sulcus (area STS) [17, 18] but these neurons do not have the motor properties exhibited by those in area F5. Another defining characteristic is that the observed action must be a transitive (goal or object-directed) one. The observation of an aimless movement will not trigger the mirror response [10]. Mirror neurons are also subdivided into two classes that are characterized by the degree of response similarity between an observed and executed action. Strictly congruent mirror neurons respond when both the observed action and the execution of that action are identical in terms of both the goal and the way in which the goal is achieved. Broadly congruent mirror neurons only require some similarity between the observed and executed action. Approximately one third of the identified mirror neurons in F5 are classified as strictly congruent and the remaining two thirds are classified as broadly congruent [15].

More recent studies have demanded a broader definition of mirror neurons to include sensory modalities other than vision and to consider the ultimate intentions of an action. Kohler and colleagues showed that some mirror neurons in F5 of monkeys are active not only during observation of an action, but are also modulated by the sounds associated with that action. Some of these neurons require both the vision and the sound of the movement to be active while for others just the sound alone is sufficient to elicit the mirror response [9, 19]. There is evidence of an auditory mirror system also present in humans [5]. Gazzola and colleagues demonstrated that in the same human subject, parts of the IPL and the premotor cortex are active during execution of action as well as during the presence of sounds associated with that action [5].

It is also clear that the intention - not just the immediate sensory characteristics - of an action are critical in eliciting mirror-like responses. For example, neurons in the inferior parietal lobule modulate differently to identical actions with different outcomes [4*]. Fogassi and colleagues trained monkeys to reach for and grasp either an object or a piece of food in an identical fashion. If the object was food, then the monkeys were allowed to eat the piece of food; if the object was not food, they placed it into a container. The monkeys then observed the experimenters perform the same actions. A third of the recorded parietal mirror neurons fired similarly for both food and non food object directed movements. Interestingly, however, the remaining two thirds discharged differently for food-directed grasps. Controls were put in place to rule out the possibility that the preferential neural activity was due to differences in visual properties of the objects that monkeys grasped, force exerted during the grasp or differences in movement kinematics during object grasping [4*].

The mirror-like responses of neurons in areas F5 and the IPL are perhaps not unexpected given the roles of these two cortical areas in sensorimotor integration required for reaching and grasping. Area F5 is directly and mutually connected to area AIP in the inferior intraparietal sulcus; together they form a ventral network that is believed to play a role in the visuomotor transformations involved in prehension [20-26]. Moreover, neurons in PMv and IPL have been shown to possess so-called “near field” visual receptive fields that surround the body part that these neurons presumably control [27-29].

Mirror-like responses in a distributed motor network

Recent evidence suggests that mirror-like activity is distributed throughout the cortical motor system. Neurons with mirror-like responses have been found in both the monkey dorsal premotor cortex (PMd) [30*], and primary motor cortex (MI) [31*]. Interestingly, in both these studies, mirror-like responses were evoked by abstract, video-based movements instead of actual goal-directed movements performed by conspecifics or human experimenters. To study mirror responses in PMd, Cisek and colleagues trained their monkeys to either perform or observe a center-out reaching task on a computer monitor positioned in front of them. In this task, monkeys were briefly shown two colored targets, which were then extinguished. A color cue indicated which of the extinguished targets the monkeys had to reach for, and the actual movement was cued on the presentation of eight radially spaced targets appearing in the same, non-instructed color. The monkey had to remember where the instructed target was located and then make a reach movement guiding a visual cursor to that location. During observation of the task, the sequence of events was the same while an experimenter, being out of monkeys’ sight, controlled the observed cursor, and the monkeys remained motionless. The researchers found that PMd neurons displayed similar activity patterns during execution and observation of the task. Moreover, the onset of this activity during the observation phase took place during the instruction period while both the monkey and cursor were stationary, indicating that PMd neurons anticipated the direction of the impending, observed movement [30*].

More recently, mirror-like responses have also been observed in the primary motor cortex (MI) [31*]. In their study, Tkach and colleagues trained the animals to perform a task in which they had to guide a visual cursor to a sequence of targets appearing at random locations in the video workspace. While the monkeys performed the task, the target locations as well as the trajectories generated by the animals were recorded. During the observation phase, the monkeys were trained to relax their arms in a steady posture while they observed video playback of their previous movements. Experimenters varied the visual components of the playback by making either the target or the cursor invisible, as well as by showing playback of artificial, computer-generated cursor movements. They found that MI neurons exhibited movement-related neural activity during task observation that was congruent with the neural activity during task execution. The presence of both visual components — the target and the moving cursor — elicited the most congruent response. Congruent neural activity was also present during the observation of artificially-generated movements, but the timing of this activity was altered in a manner that reflected the differences in timing between the artificial and natural cursor movement. Tkach and colleagues interpreted their results as reflecting the monkey’s internal generation of motor commands that, if executed, would produce a movement similar to the one observed. There is support for this interpretation from evidence in studies of the human mirror neuron system (hMNS) as well [32-34].

A variety of theoretical work supports the hypothesis that mirror-like activity originates within the same distributed cortical network that allows us to recognize and predict the sensory consequences of our actions. So in this sense, it is perhaps not surprising that mirror-like responses have been observed throughout the frontal motor system. The underlying premise of this hypothesis is that the observation of a goal-directed behavior recruits the same predictive internal models of movement generation and sensory feedback that allow us to plan and correct errors in our own movements. Inverse models (which map kinematic trajectories to motor commands) have been hypothesized to play a role in the planning and execution of movement, while forward models (from motor commands to kinematic trajectories) have been proposed to provide feedback signals that expedite movement error estimation [35, 36]. According to one theory, during observation of an external agent’s goal-directed action, the observer’s internal model is recruited first in the inverse direction producing an estimate of the motor commands responsible for the observed movement and then fed back through the forward model. The error between the predicted and observed movement is then used to update the internal motor command until the error is minimized [37*]. Thus, this hypothesis predicts that mirror-like responses originate in a process akin to internal movement simulation, occurring over a widely-distributed cortical network. Recent observation of mirror-like activity in motor cortex outside of canonical “mirror areas” may support this view.

Mirror neurons in the context of BMIs

Mirror responses are particular interesting in the context of brain-machine interfaces. In the development of a motor neural prosthetic device, a critical step is creating a mapping between neural activity and movement; for example, a linear model that relates the neural firing rates of a cortical population to some set of kinematic parameters, such as end-effector position or velocity [38-41]. This mapping then makes it possible to “decode” the intended movement from the observed neural activity. This procedure works well when the subject is capable of producing the overt movement to which neural activity will be mapped. Mirror responses would potentially make it possible to create the mapping between neural activity and intended actions in the absence of overt execution of those actions. This ability is obviously crucial, since the most likely candidates for BMI motor prostheses lack the capacity to produce the overt movement.

One solution to this problem stems from evidence that imagery of an action produces action-related activation in the human motor cortical areas [42, 43]. In fact, human patients using BMIs in clinical trials [44] were asked to imagine the movements to which their neural activity was mapped. While this worked, the capacity to imagine a particular movement varied among these patients (Maryam Saleh, personal communication). Moreover, it has been proposed that there are at least two different forms of motor imagery: visual and kinetic imagery [45]. While asking someone to visually imagine him- or herself performing a task is simple to communicate (i.e. “third-person” visual imagery), it is more challenging to instruct someone to mentally rehearse a movement (i.e. “first-person” kinetic imagery) [45], and there is some evidence that kinetic imagery may be more effective in activating the primary motor cortex [46]. In contrast, the mirror neuron system appears to faithfully encode observed goal-directed movement without the need for an explicit, voluntary mental effort on the subject’s part, as long as they are paying attention [47]. Since MI is often the cortical site selected for current BMI applications, the presence of mirror-like activity in MI [31*] is of particular importance because it removes the need to implant other cortical areas, such as F5, in order to gain access to the mirror responses. Interestingly, analysis of mirror responses in MI revealed a subset of neurons that show much greater movement-related activity during observation of movement as compared to actual execution of movement [31]. Specifically, their activity conveyed more information about the velocity and direction of observed movements than performed movements (Figure 1). This result suggests that mirror activity in the motor system is more than weak resonance within sensorimotor circuits and that there are mirror neurons that function in a greater capacity during observation than during execution of movement.

Figure 1.

Three examples of cells showing greater cursor movement-related activity during action observation (orange traces) than during action execution (dark blue traces). (A) Amount of mutual information (in bits) these cells convey about cursor velocity computed at various time lags between the activity of the cell and the cursor velocity. Positive lags indicate that neural activity precedes the measured velocity while negative lags indicate that the neural activity followed the measured velocity. (B) Mean firing rates (50 ms bin size) of the same cells as in (A) during instantaneous cursor movement in one of 16 discretized directions of the workspace, fit to a cosine function. The correlation coefficients (CC) of the fit function to the data are shown in the upper left of each graph.

The ability to utilize mirror-like responses in the motor cortex for a BMI application has been demonstrated in primates performing three-dimensional movements [48*, 49]. To address the challenge of constructing a mapping between action and neural activity when overt action is not possible, the researchers recorded the neural activity of a relaxed monkey while the monkey observed a small set of computer-generated movements in the same 3D environment. The desired features of the neural signal — in their case, directional tuning - were extracted and were mapped to the computer generated trajectories that the monkeys observed. These features were then used as initial parameters of the decoding model and were iteratively improved as the animal attempted neural control. This approach has recently been used to train a decoder that controlled a robot to reach and grasp for food via motor cortical activation by initially having the monkey passively observe the robot reach and grasp [50].

Other types of BMIs, such as those that rely on the EEG signal for control may also be able to utilize the mirror neuron system. Since the EEG signal correlates with both actual and imagined movements [51], current EEG decoders primarily rely on imagery to construct the decoder model. Mirror responses may provide an alternative means of relating neural activity to movements, since mirror responses have been demonstrated in EEG recordings [52].

The mirror neuron phenomenon offers compelling further opportunities for future BMI research. Most BMI systems focus on decoding the moment-by-moment, continuous kinematic trajectory of the hand without taking into consideration the target, nature and value of the goal, but these features are beginning to attract more attention in the development of BMIs. Kemere and colleagues demonstrated substantial improvement in decoding performance when goal location information was incorporated into their trajectory decoder [53]. These authors developed a decoder that first predicts the location of a target to be reached and then estimates the most likely “canonical” trajectory to reach that target. If mirror-like responses in the motor cortical system partially reflect target location, this information could be incorporated into the on-line, trajectory decoder to improve BMI performance. Moreover, the expected value of a goal as measured by the reward type and quantity associated with the goal has also been decoded from populations of neurons in PMd and posterior parietal cortex (area MIP) [54]. As with target location, if mirror responses in the motor system reflect goal value, this could be used to improve BMI performance.

Conclusions

Recently, a number of studies have provided evidence for mirror-like responses outside of canonical mirror areas. These findings are consistent with the view that mirror responses may be a reflection of activity in a larger distributed sensorimotor network involved in predicting and interpreting the sensory consequences of motor commands. The presence of mirror-like responses in areas like primary motor cortex may have useful applications for training neural prosthetic decoding algorithms in the absence of overt movements. Furthermore, the dependence of mirror responses on the context and intentions of movements suggests that in addition to lower-level representations of trajectory formation, high-level representations of intentions and value may be available in the motor system for BMI applications. Making use of information about movement intentions, which is likely distributed across multiple brain regions, may make possible to construct more robust and efficient mappings between neural activity and behavior, as well as facilitate development of decoders that move beyond the prediction of low-level kinematic representations.