Towards an Empirically Grounded Predictive Coding Account of Action Understanding

Recent work in cognitive and systems neuroscience suggests that the brain is a prediction machine (Clark, 2013), continually attempting to predict the external causes of sensory information. This idea is formulated in the predictive coding framework, a modern theory of brain function (Friston, 2010), which proposes that incoming signals are continuously compared with internal predictions at all levels of the cortical processing hierarchy. This process is hypothetically instantiated in two types of neurons (Summerfield and Egner, 2009): representation units, which encode the predictions based on prior information, and error units, which compare the incoming signals with the predictions conveyed via the representation units. When there are discrepancies between the predictions and input signals, the error units produce a prediction error signal, which is used to update the generative model at the next level of the cortical hierarchy.

Predictive coding has been influential in constructing computational principles of many domains of cognition (Summerfield et al., 2006; Egner et al., 2010), one of which is action understanding (Kilner et al., 2007a, b). Action understanding is an important skill for the survival of many species, and it is considered to be a building block of several high-level social cognitive skills in primates, such as communication, imitation, intention understanding, and empathy (Blake and Shiffrar, 2007). Neurophysiological and neuroimaging studies in primates over the last two decades have identified a dedicated brain network, known as the mirror neuron system (MNS), that is thought to underlie action understanding (Rizzolatti and Craighero, 2004). This system, in its classic formulation (Fig. 1a), consists of three nodes: the posterior superior temporal sulcus (pSTS), which serves as the visual input to the system by getting visual information from the early visual cortex; and two regions that contain neurons that discharge during both action execution and action observation, called mirror neurons: the inferior parietal (IPC) and ventral premotor cortices (vPMC). According to Kilner et al. (2007a, b), the MNS is a predictive system, following the principles of predictive coding. In contrast to the classic formulation of the MNS, which sees action understanding strictly as a feedforward process, Kilner and colleagues (2007a, b) propose that visual action information is processed throughout the MNS by means of the reciprocal connections (i.e., both feedforward and feedback) between the pSTS and IPC, and IPC and the vPMC (Fig. 1a). In this theory, incoming information is compared with predictions at each level of the MNS.

Figure 1.

Action understanding in the predictive coding framework. a, Sensory information is propagated through the early visual cortex (EVC; green circle), pSTS (orange circle), parietal cortex (yellow circle), and premotor cortex (pink circle) via feedforward connections (blue arrows), whereas prior information is propagated in the opposite order via feedback connections (red arrows). Possible sources of prior information are depicted with large arrows oriented to vPMC: the purple arrow represents a prefrontal source for prior information, which was suggested by Maranesi et al. (2014), and the light green arrow represents a thalamocortical source that may contribute. b, Depiction of representation units and error units at all levels of the mirror neuron system. Sensory information is fed into the system from the early visual cortex and propagated through feedforward connections (blue arrows). Predictions, originating from prior information, are propagated via feedback connections (red arrows). Prior information originates either from prefrontal cortex (purple arrow) or early visual cortex mediated via thalamic projections (green arrow). Bidirectional red arrows between the error units and representation units indicate the computation of prediction error signal. Prediction error signals are used to update the predictions, which are then fed to the error units of the next level of the cortical hierarchy (blue arrows).

A handful of experimental studies in humans have provided empirical support for the predictive coding account of action understanding. Kilner et al. (2004), using event-related brain potentials, found that during action observation, the human brain generated a motor-preparation-like negative potential when the action was in a predictable context; no such potential was found when observation occurred within an unpredictable context. In another study, using an fMRI-adaptation paradigm, Saygin et al. (2012) found that the parietal node of the MNS showed more adaptation to unpredictable actions than to predictable ones. The authors interpreted the differential adaptation in the parietal cortex for unpredictable actions as reflecting prediction error signals generated by a mismatch between predictions sent from the vPMC and actual sensory input from the pSTS.

Although human neuroimaging studies—which are confined to investigating the macroscopic level of brain organization—can be informative about cortical function, empirical support for the predictive coding account of action understanding at the level of single neurons is lacking. A recent study by Maranesi et al. (2014) attempts to provide this support. To do so, Maranesi and colleagues (2014) trained monkeys to perform a go/no-go visuomotor paradigm with a predictive component. The activity of neurons in the vPMC was recorded throughout the task. At the beginning of each trial, an auditory cue indicated whether an observed hand would reach to grasp an object (Action condition) or remain still (Inaction condition). The offset of the auditory cue served as the go or no-go signal, upon which the hand either performed or withheld an action, respectively. Importantly, the auditory cue was always predictive of the trial type, enabling the monkey to predict with 100% certainty the upcoming behavior of the observed hand.

Maranesi and colleagues (2014) identified two types of mirror neurons in the premotor cortex: Action mirror neurons and Inaction mirror neurons, replicating a previous finding of theirs (Bonini et al., 2014). Furthermore, within each neuron type, they identified two classes of mirror neurons: reactive and predictive. Reactive neurons of both neuron types discharged after the critical go or no-go signal, which indicated the start of an executed action (Action-type) or a withheld action (Inaction-type), respectively. Conversely, predictive neurons discharged while the cue was still sounding, before the critical signal and the required action was initiated or withheld: predictive Action mirror neurons began to discharge ∼100 ms before the go-signal, whereas predictive Inaction mirror neurons discharged ∼480 ms before both signal types. These results demonstrate the existence of mirror neurons that can predict upcoming actions. Furthermore, they suggest that the predictive function of these neurons varies. Some predictive neurons predict only actions (Action-type), whereas others predict both actions and nonaction-related events (Inaction-type).

These findings provide significant empirical support for the predictive coding framework of action understanding. However, to fully fit the MNS into this framework (Kilner et al., 2007a,b), mirror neurons must be defined in terms of the theoretical constructs of predictive coding, namely representation and error units, and the origin of the prior information must be identified (Fig. 1b). Doing so, however, is a complicated endeavor and is highly dependent upon the specific paradigm chosen for the task. It is likely that the predictive mirror neurons (both Action and Inaction) represent the system's representation units, because they began to discharge before the go/no-go signal, presumably driven by the input of prior information about the meaning of the tone (Fig. 1a,b, purple and green arrows). Seen from this view, the increased activity of these neurons after the go/no-go signal is due to updates from error units (Fig. 1b, bidirectional red arrows). The data further suggest that representation units within the same brain region can serve different functions, a finding that adds novel complexity to models of predictive coding. The predictive Action mirror neurons appear domain-specific, discharging before and after the go/no-go signal during go trials only. The Inaction mirror neurons, in contrast, are more domain general, discharging during both go and no-go trials. In further contrast to the predictive Action mirror neurons, these neurons appear to encode prior information for the context in general and upcoming actions in particular.

Unlike representation units, the discharges of error units are not driven by sensory signals per se but instead by the difference between them and the predictions; the greater the discrepancy, the higher the firing rate of error units. These prediction error signals are a critical component of the predictive coding framework (Summerfield and Egner, 2009; Friston, 2010), and thus mirror neurons that behave like error units are necessary for a predictive coding theory of action understanding. Unfortunately, the paradigm used by Maranesi and colleagues (2014) precluded the ability to definitively label any of their mirror neurons as error units because there was no way to measure any errors in predictions. However, reactive mirror neurons may be potential candidates because they were involved in processing the sensory/incoming signals. One clever strategy to measure this signal and hence identify error units would be to include conditions in which predictions are violated; for example, an auditory cue signaling a no-go trial followed by an action instead of a withheld action. This paradigm would allow researchers to directly measure error signals; the neurons with greater activation when predictions are violated versus fulfilled would fall into the category of error units. Just such a signal was found in the inferotemporal cortex in a recent neurophysiology study investigating prediction and visual object recognition (Meyer and Olson, 2011), providing hope that similar responses may exist for action understanding. Future neurophysiological studies should consider such experimental designs to directly ground the theoretical constructs of the predictive coding account.

One question that remains unanswered is where the prior information is generated in the predictive coding account of action understanding. Maranesi et al. (2014) suggest that the prior information that is exerted on the vPMC originates from prefrontal cortex (Fig. 1a,b, purple arrows). Although this is a possibility given the established role of the prefrontal cortex in encoding prior information (Summerfield et al., 2006; Vilares et al., 2012), the timing of mirror neuron firing in Maranesi et al. (2013) suggests another intriguing possibility. They found that mirror neurons in the vPMC became active as quickly as 60 ms after the onset of an actor's movement. Interestingly, the speed of this activation is substantially faster than the known temporal profile of biological motion neurons in the pSTS (∼100–150 ms; Barraclough et al., 2005), which is classically thought to be the input into the MNS (Rizzolatti and Craighero, 2004). This suggests that the rapid activation of mirror neurons in the vPMC may reflect an initial guess about the specific action being perceived (Bar, 2003), which is used as the initial prediction to send down the cortical hierarchy (Bar et al., 2006; Summerfield et al., 2006; Kveraga et al., 2007). We propose that, in at least some instances of action observation, the prior information used as the input for the initial guess originates very early in visual processing (Fig. 1a, green arrow), perhaps in early visual cortex mediated by thalamocortical projections connecting the medial pulvinar with the vPMC (Cappe et al., 2009). This alternate source of input into the MNS is a novel addition to models of action understanding and should be explored with future work.

In conclusion, Maranesi et al. (2014) provide direct evidence for predictive activity of mirror neurons and therefore is a foundational step in supporting the predictive coding account of action understanding. However, several aspects of their findings—highlighted by the authors in the original paper and by us here—add novel complexity to this account. Specifically, the model proposed by Kilner et al. (2007a,b) should be updated to account for the different sources of prior information (early visual cortex mediated by thalamic connections or prefrontal cortex) and potentially different types of representation units (domain-specific and general). In addition, future work must fully ground mirror neurons within the computational principles of predictive coding. This means that studies on prediction and action understanding must include prediction violations in their experimental paradigm if we are to ever hope to identify error units. Undoubtedly, these studies will enable us to better understand the neural basis and computational principles of action understanding, one of the most important social skills primates have.