Brain negativity as an indicator of predictive error processing: the contribution of visual action effect monitoring

In this study, we are separating different kinds of possible contributors to an electroencephalogram (EEG) error correlate (Ne/ERN) in a throwing task. We tested the influence of action effect monitoring on the Ne/ERN amplitude in the EEG. We used a task that allows us to restrict movement correction and action effect monitoring and to control the onset of result feedback. We ascribe the Ne/ERN to predictive error processing where a conscious feeling of failure is not a prerequisite.

Abstract

The error (related) negativity (Ne/ERN) is an event-related potential in the electroencephalogram (EEG) correlating with error processing. Its conditions of appearance before terminal external error information suggest that the Ne/ERN is indicative of predictive processes in the evaluation of errors. The aim of the present study was to specifically examine the Ne/ERN in a complex motor task and to particularly rule out other explaining sources of the Ne/ERN aside from error prediction processes. To this end, we focused on the dependency of the Ne/ERN on visual monitoring about the action outcome after movement termination but before result feedback (action effect monitoring). Participants performed a semi-virtual throwing task by using a manipulandum to throw a virtual ball displayed on a computer screen to hit a target object. Visual feedback about the ball flying to the target was masked to prevent action effect monitoring. Participants received a static feedback about the action outcome (850 ms) after each trial. We found a significant negative deflection in the average EEG curves of the error trials peaking at ~250 ms after ball release, i.e., before error feedback. Furthermore, this Ne/ERN signal did not depend on visual ball-flight monitoring after release. We conclude that the Ne/ERN has the potential to indicate error prediction in motor tasks and that it exists even in the absence of action effect monitoring.

NEW & NOTEWORTHY In this study, we are separating different kinds of possible contributors to an electroencephalogram (EEG) error correlate (Ne/ERN) in a throwing task. We tested the influence of action effect monitoring on the Ne/ERN amplitude in the EEG. We used a task that allows us to restrict movement correction and action effect monitoring and to control the onset of result feedback. We ascribe the Ne/ERN to predictive error processing where a conscious feeling of failure is not a prerequisite.

as humans we are constantly interacting with our environment, be it in social contexts or in handling objects. However, our actions are not always adequate and productive. We make errors, but we can learn from them, adapt, and progress if we are capable of detecting and evaluating them. In other words, we need to perceive that we have made an error, determine what has caused it, and modify our behavior accordingly. Such error processing takes place within complex perception-action loops (e.g., Fuster 2004; Haruno et al. 2001). On the basis of computational studies, it has been proposed that the central nervous system (CNS) internally represents aspects of these perception-action loops in planning, control, and learning by way of internal models (Kawato 1999; Wolpert et al. 1995; Wolpert 1997). These models are conceived to either mimic the input-output relationships of the controlled object (including one’s own body) or their inverses (Kawato 1999). Thus they come in two varieties: Inverse models receive intended action effects as an input and, in conjunction with information about the current state of the sensorimotor system and the environment, specify the motor commands necessary to actually produce the sensory consequences associated with the desired action effects. Forward models, on the other hand, receive an efference copy of the issued motor commands and, along with information about the current state of the system and the action context, generate predictions about the expected sensory consequences. Predictions generated by the forward model can, among other applications, be used to detect performance errors by determining the deviation of expected sensory consequences from the intended sensory consequences before actual sensory feedback becomes available or even in the complete absence of sensory feedback. The present study aims to monitor such predictive error processing activities by analyzing event-related potentials (ERP) in the electroencephalogram (EEG).

The error negativity (Ne; Falkenstein et al. 1991) or error-related negativity (ERN; Gehring et al. 1993) is a negative deflection in frontocentral regions typically occurring shortly after the onset of an erroneous movement. Because the Ne/ERN can be observed before the availability of external feedback about the movement outcome (knowledge of results, KR), it is reasonable to assume that the Ne/ERN reflects predictive error processing based on a prediction about the future outcome of an action. In other words, the Ne/ERN might indicate a predicted mismatch between a desired action outcome and the future outcome. Importantly, when we speak of prediction in this context, we do not presume that such an evaluation of the action outcome necessarily has to be consciously accessible by the subject. The Ne/ERN signal emerges at a time when a conscious reflection of the action might not yet be processed. Still, the signal is a neural predictor for the actual action outcome.

The Ne/ERN has been shown to be elicited when the expectation to reach a particular target is not fulfilled (Holroyd et al. 2003; Yasuda et al. 2004). The vast majority of studies on the Ne/ERN uses tasks involving so-called outcome or high-level errors (Krigolson and Holroyd 2007a). In tasks such as stimulus-response mappings (e.g., Falkenstein et al. 1991; Gehring et al. 1993; Holroyd and Coles 2002; Nieuwenhuis et al. 2003), Go/NoGo paradigms (e.g., Falkenstein et al. 2000; Vocat et al. 2008), or piano key tone mappings (e.g., Lutz et al. 2013), where participants are mainly asked to press a button, motor demands are minimal and motor execution is less relevant. Predictions take place at a cognitive level and concern the final goal state of an action. In contrast, error processing could also be focused on the course of the action itself (i.e., the dynamics and kinematics of the movement) when the movement execution is evaluated. Generally, because such errors are less strictly related to the effect than high-level errors, Krigolson and Holroyd (2006, 2007a) called them low-level errors and localized the processing of these errors in posterior parietal regions. In real-life, however, and also in many research paradigms, both error types are often mixed. That is, kinematics and dynamics of the movement are relevant with respect to reaching the action goal or at least contribute to it. It is therefore suggested that an error prediction based on internal forward models within a posterior low-level error system might serve as an input signal to a frontocentral high-level performance monitoring system. Once the low-level error system detects a mismatch between desired and actual movement kinematics/dynamics so that high-level goal attainment becomes unlikely, a high-level error signal is generated (indicating a predicted mismatch between desired and future action outcome; Krigolson and Holroyd 2007a). In the last few years, several studies have addressed such tasks, i.e., allowing high- and low-level errors. For example, participants had to reach a target (Anguera et al. 2009; Krigolson and Holroyd 2007a; Krigolson et al. 2008; MacLean et al. 2015; Torrecillos et al. 2014; Vocat et al. 2011), hit a target in a throwing task (Maurer et al. 2015), avoid a collision with a nontarget in a tracking task (Krigolson and Holroyd 2006, 2007b), or produce different force levels with their index finger (de Bruijn et al. 2003). A low-level error in these tasks (i.e., a deviation from the desired movement path) supposedly leads to an anticipation about the failure to achieve the action goal (reach target, avoid collision, produce required force level), which is then again represented as high-level error. As a result, these studies observed ERPs predictive to errors and before KR about the movement outcome. However, the latency was somewhat prolonged compared with choice reaction time tasks (100-300 ms after movement onset). Because of the complexity of the task, a higher amount of information has to be processed to generate a prediction, which is then reflected by a later ERP in the EEG signal.

One might ask whether the denotation “Ne/ERN,” as it is described in the early studies on the neural correlates of error prediction (e.g., Falkenstein et al. 1991; Gehring et al. 1993), can be transferred to studies investigating error prediction in complex motor tasks (e.g., Anguera et al. 2009; Torrecillos et al. 2014; Maurer et al. 2015). In this article, we refer to the Ne/ERN as a neural correlate of error prediction. In detail, this prediction is referred to an upcoming high-level error before KR (i.e., the terminal feedback of a high-level error).

In conclusion, the empirical evidences for neural correlates of predictive processes in error perception (such as Ne/ERN) are quite convincing in simple motor tasks with “high-level” errors. Most studies examining low-level kinematic errors have assessed the neural correlates of error prediction while the movement was still in progress and/or corrective submovements could still be initiated. In addition, visual monitoring of the action effect (e.g., visually tracing the trajectory of the cursor or the manipulated object) was often in progress during the time of error prediction (shortly after movement onset). Monitoring the ongoing action and controlling corrective submovements requires a certain amount of processing activity in the brain, which might also be part of the measured EEG signal. Thus there are at least three possible factors that may contribute to a frontocentral negative ERP at this moment: 1) error prediction processes, 2) movement corrections, and 3) action effect monitoring. As a consequence, a separation of brain activity related to predictive error processing and those related to movement corrections and action effect monitoring is necessary to make generalizable conclusions about the role of the Ne/ERN.

To study error prediction in motor tasks, as well as to disentangle the three possible contributing factors to error-related brain negativity, a task is needed where clear predictions about the outcome in terms of success or failure can be derived on the basis of movement execution (requirement a), no online corrections are possible (requirement b), and no action effect monitoring is possible (requirement c). A further requirement (d) is related to the availability of feedback regarding the action outcome (knowledge of results), which has to be delayed with respect to movement termination.

The last requirement (d) is important because it allows a differentiation between low-level and high-level errors. As described earlier, a low-level error is defined as a mismatch between desired movement kinematics or dynamics (e.g., a movement path) and the actually executed movement kinematics/dynamics. A high-level error is conceived as a mismatch between a desired action result and the predicted action result, assuming that this prediction is based on information related to movement execution (efference copy signals as well as concurrent sensory signals). If there was no delay between movement termination and feedback, there would be no need for a prediction, because the actual result would be available instantaneously. In this case, a detected mismatch would be the result of a comparison between the desired action result and the actual action result. This mismatch is also represented by an ERP signal, namely, the feedback-related negativity (FRN; Miltner et al. 1997). In summary, the Ne/ERN is one of the ERPs related to error prediction and is suggested to represent a mismatch between a desired action outcome or result and a predicted action outcome (hence, it indicates a prediction of an outcome error), whereas the FRN presumably represents a mismatch between the desired action outcome or result and the actual action outcome. However, note that both indicate a detection of an outcome error (Krigolson and Holroyd 2007a).

In a recent study, Maurer et al. (2015) provided a task that in general is capable of fulfilling all four requirements mentioned above (a, b, c, d). They used a virtual ballistic throwing task where participants had to hit a target with a ball on a computer screen (see Fig. 1 and materials and methods for more details). The task outcome is clearly differentiable in target hits and errors (requirement a). The throwing movement is fast and ballistic. Hence, with release of the ball, the throwing movement is terminated, no online corrections are possible (requirement b), and all efferent and afferent information necessary for effect prediction is available. Because the ball needs some time to travel toward the target, knowledge of the action result is delayed with respect to the release (requirement d). The only point that was not met by the design of this earlier study was the prevention of action effect monitoring, because participants saw the ball flying toward the target. Nevertheless, the results of the study showed that at least movement corrections could be excluded as contributing source to an observed error-related frontal brain negativity. More specifically, in trials where participants missed the target, two negative deflections emerged after ball release. One occurred in a time window of 200–350 ms after ball release, and the second was visible from 350 ms after release until participants received KR feedback, i.e., when they could see the ball hitting or missing the target on the computer screen. The first signal was interpreted as indicative of error prediction, whereas the second was ascribed to action effect monitoring, namely, visual monitoring of the ball flight. The reaction to the result feedback (FRN) was not analyzed by Maurer et al. (2015).

Fig. 1.

A: illustration of the British pub game skittles. The green ball must be thrown so that it flies around the center post and knocks down the target object (red cylinder). B: illustration of virtual skittles used in this study. The subjects sees the virtual version in 2 dimensions from a bird’s eye perspective. C: the execution and result space depicts every angle-velocity combination with its outcome coded by color (minimal distance ball to target, DBT). The gray line represents the course of angle-velocity combinations the ball passes through at one exemplary trial. The star on the trajectory represents the angle and velocity value at the moment of ball release.

In the present study, we made use of the possibility to temporally separate movement execution and corresponding sensory online information from action effect monitoring and KR feedback with the throwing task (Fig. 2). We aimed to extend the findings from Maurer et al. (2015) by eliminating the display of the ball flight trajectory after release to eliminate action effect monitoring, and hence to test whether the second broader signal found by Maurer and colleagues (2015) was indeed caused by monitoring of the ball flight. If this was the case, the second negative deflection should at least be attenuated in the present study. In contrast, the first signal (from 200 to 350 ms after release) should still emerge in the case that it reflects predictive error processing involving action-related signals instead of visual information about ball flight. This would imply that information about the movement execution is used by a forward model to generate predictions about goal achievement and thus allows predictive detection of high-level goal errors based on low-level kinematic errors. Thus the aim of the present study is to add to the existing findings to confirm that the Ne/ERN is related to forward-model predictions of low-level errors leading to a high-level error.

Fig. 2.

The skittles task has an action and a feedback phase. During the action phase, subjects are throwing the ball with the help of the manipulandum (left). After the ball is released from the lever, the action is monitored until the thrown ball reaches the target (middle; note that in the present experiment, the ball is not visible during the action effect monitoring). At 850 ms after ball release, a static feedback about the ball flight trajectory, together with a verbal cue, is presented to provide information about the action outcome (KR feedback; right).

METHODS AND MATERIALS

Participants.

We tested 21 participants (4 males) with an average age of 22 yr (SD_age 2.3 yr). The subjects were healthy and had normal or corrected-to-normal vision. All subjects were right-handed and were recruited from the student population of the Justus-Liebig-University Giessen. They received course credit and had a chance to win up to €60. The experiment was conducted in accordance with the ethical standards laid down in the Declaration of Helsinki, and all participants provided written informed consent. Furthermore, the protocol was approved by the Ethical Review Board of the Justus-Liebig-University Giessen.

Task.

The visuomotor transformation inherent to the throwing task was inspired by a British pub game called “skittles,” where a ball is attached to a rope, which is connected to the top of a post (Fig. 1A). The goal of the task is to throw the ball in a way that a target object is hit. Accordingly, participants saw a virtual lever with which they could pick up and throw a green virtual ball (radius on screen = 2.5 mm) and a blue center post (radius on screen = 12.5 mm) around which the ball had to be thrown to hit a red target (radius on screen = 2.5 mm).

The trajectory of the thrown ball was defined by two variables captured at the moment of ball release: angle and velocity. It is important to note that the task offers redundant solutions, meaning that there exist an infinite number of angle-velocity combinations to hit the target (see Fig. 1C). Participants cannot learn the task associatively. Hence, even if they had true knowledge about their actual release angle and velocity in a throw (which is not the case due to perceptual noise), they could not derive the throwing outcome from it directly. They only receive explicit information about the throwing result when KR feedback is presented. Depending on the experimental phase, participants received continuous and/or terminal visual feedback about the movement trajectory of the ball (see Experimental design and procedure). The calculation of the ball’s elliptic pathway around the post was based on a physical model of the task (Müller and Sternad 2004). In this model the sizes and positions of the task stimuli are as follows: center post (radius = 0.25 m; position: x = 0.0 m, y = 0.0 m), target (radius = 0.05 m; position: x = 0.35 m, y = 1.0 m), and ball (radius = 0.05 m).

The participants had to throw the ball with their right hand counterclockwise around the vertical post. A miniature model of the real skittles game was used to clarify the task. To prevent a fast, rhythmic execution of subsequent trials and to trigger the movement start, the subjects were instructed to start every trial by moving the end of the virtual lever into a red circle positioned to the left of the lever. Immediately after the tip of the virtual lever reached the circle, it turned yellow. The circle turned green when the lever was held 1 s within the yellow circle. This was the signal for the participants that they were now free to start the movement at any time. Importantly, the subjects did not start the movement in reaction to the green light.

Participants were instructed to hit the target as often as possible. For motivational purposes, they could win monetary rewards for achieving high hit rates. During the 2 practice days, rewards were graded €30, €20, and €10 for the subjects with the 3 highest hit rates. In addition, they could win the same amounts of money during the EEG acquisition on days 3 and 4. In this way we could keep the participants motivated throughout the experiment.

Apparatus.

Participants sat on a stool, which was placed 100 cm in front of a 15-in., 4:3 computer monitor (model AOC 919Va2; screen resolution: 1,024 × 768 pixels). Their right arm rested on a foam-padded metal lever (the manipulandum), which was supported by a height-adjustable vertical stand. Those two components were fixed to each other at the vertical rotation axis and below the elbow joint of the participant (Fig. 1B). At the start of each experimental day, the participants could adjust the height and the position of the stool so that they felt comfortable. Movement was restricted to the horizontal plane, more specifically to a rotation around a fixed vertical axis. The horizontal movement of the lever was presented on the screen from a bird’s eye perspective. The virtual representation of the lever on the computer screen was displayed as a solid bar, which moved when the manipulandum was moved by the participant. The index finger was placed on a contact sensor at the tip of the lever, which triggered the pick-up of the virtual ball. As soon as the finger was lifted, the virtual ball was released from the virtual representation of the lever. An integrated magnetic angle sensor with a resolution of 12 bits (0.09°) measured the lever rotation with a sampling rate of 1,000 Hz.

Experimental design and procedure.

Because the goal of the study was to separate the error prediction process from the ball flight monitoring, the display of the ball flight trajectory after release had to be eliminated (see Fig. 2 for more details about task phases). Pilot experiments revealed that participants learned the task slower when they lack dynamic ball flight information in real-time. Thus, in the present experiment, real-time display of the ball trajectory was preserved for task practice and then incrementally faded out (Fig. 3). Concretely, subjects performed the task on 4 days with 400 trials each. The first 2 days were used as practice days; EEG recordings were conducted on days 3 and 4. On day 1, 100% of the ball flight trajectory was dynamically displayed immediately after ball release for the first 200 trials. In addition to this immediate dynamic feedback, a static feedback of the ball trajectory as well as KR feedback about the result was displayed on the screen (850 ms after ball release). In detail, a collision sound was played for the hit trials, the target object was knocked out of its position, and the German word for “hit” (“Treffer”) was displayed in green on the computer monitor. In case of an error, participants received the feedback “Unfortunately a miss” (“Leider vorbei”) written in red. We chose a delay of KR feedback information of 850 ms because it is the average time the ball flies to the target object (calculated using preliminary data). Static ball flight trajectory and KR feedback remained throughout the four sessions. On contrast, the display of the dynamic ball flight trajectory was decreased by 33% every 100 trials in the last 200 trials of day 1. On day 2, the subjects started with 66% dynamic ball flight display, which was again decreased every 100 trials by 33%. As a result, they executed the last 200 trials of day 2 with 0% dynamic ball flight information. In the 0% condition the ball was masked at the moment of ball release, and the subject exclusively received the static feedback and the KR feedback after 850 ms. The EEG measurements were conducted on days 3 and 4 with 0% dynamic ball flight information.

Fig. 3.

Development of hit rates (HR) over the 4 experimental days. To provide a better resolution about the trend of the HR, the 400 trials of each day are displayed in blocks of 100 trials. The EEG recordings were conducted at days 3 and 4. The second row displays the percentage of visible online feedback (ball flight) that participants received at each block.

EEG data acquisition and preprocessing.

The EEG recordings as well as an electrooculogram (EOG) were conducted using a 16-channel AC/DC amplifier with Ag-AgCl active scalp electrodes from Brain Products (Gilching, Germany). Electrodes were placed using the actiCAP electrode cap (Brain Products) according to the international standard 10–20 system (Klem et al. 1999). We used F3, Fz, F4, FCz, C3, Cz, C4, P3, Pz, and P4 and placed the ground electrode on the Fpz position. The online reference electrode was positioned on the left mastoid to avoid artifacts caused by the movement of the right arm. A second reference electrode (offline reference) was placed on the contralateral side (right mastoid) to be able to use the average signal of both electrodes in the data analysis. In addition, EOG was collected with two electrodes on the external canthi of both eyes (to measure horizontal eye movement) and two electrodes for the vertical eye movements positioned above and below the right eye.

Before the recordings started, the impedance of all electrodes was measured and kept below 20 kΩ. To check that the impedance did not increase over the time of task execution the impedances of all electrodes was measured again after recording. Data was sampled at a rate of 500 Hz. After data collection, EEG data was additionally filtered with a phase shift-free Butterworth filter at 0.2–30 Hz (bandpass). To correct for the ocular artifacts, blinks were detected by using a blink-detection algorithm (Gratton et al. 1983), and for the resulting blink intervals, an Infomax independent component analysis (ICA; Makeig et al. 1996, 1997) was conducted. Afterward, the EEG was cut into segments around the synchronization trigger representing the ball release (600 ms before and 1,200 ms after the release trigger). We applied different baseline corrections with respect to the time intervals where effects were expected (200–350 ms and 1,000–1,200 ms after ball release). Finally, data segments were visually inspected and remaining artifacts manually removed before further analyses.

Behavioral data processing: classification of errors and successful trials.

Looking at neural correlates of error prediction, behavioral data (the throws) needed to be sorted into errors and hits. Because the participants’ goal was to hit the target as often as possible, trials where the ball did not hit the target object were classified as error trials. The balls’ radii (target and thrown ball) in the physical model by Müller and Sternad (2004) were 5 cm each. Hence, a hit was achieved when the ball trajectory reached the center of the target object as close as ≤10 cm. For grand averages and according inference statistical analyses, we excluded marginal hits (minimal distance between ball and target: DBT ≥ 7 cm and ≤ 10 cm), misses (DBT ≤ 12 cm and > 10 cm) and hits of the center post from the data analysis, because predicting the action outcome in these trials is much harder than in clear hits and errors. In consequence, we included an average of 92 hits and 125 errors per subject (SD_hit = 20; SD_error = 31). The individual number of analyzed hits and errors varied for each subject in the range from 65 to 140 for hit trials and from 41 to 140 for error trials. In addition, however, we also conducted an analysis including all hit and error trials of each subject to validate the analyzation procedure.

Hypothesis and statistical analysis of error related signals.

To examine neural correlates of error prediction, the trials of the hit and error categories of each participant were averaged and compared by calculating difference curves (mean curve of hit trials was subtracted from the mean curve of error trials). According to the findings of Maurer et al. (2015), the time window to expect the Ne/ERN (effect window, EffW) was defined 200–350 ms after ball release. Because the Ne/ERN is mostly pronounced at the FCz electrode in the 10–20 system, data from this electrode was solely used for statistical analyses. With a one-sample t-test, we tested whether the mean amplitude of difference curves within the EffW_200–350 deviated negatively from 0. In addition, we also compared the mean amplitudes of hit and error curves within the time window of action effect monitoring with regard to Maurer et al. (EffW_350–850) and as a reaction to the result feedback (EffW_1000–1200). In error trials, we expected a greater negative deflection in the ERP average relative to the hit trials in the time window EffW_200–350 after ball release. Maurer et al. (2015) assumed that the effect they observed in the time window EffW_350–850 after ball release could be caused by processing the visual ball flight. If this assumption is true, this effect should be absent in the current study because the online action effect monitoring in form of the ball flight is masked. With respect to the result feedback, we expected a FRN to emerge when negative feedback (target miss) was presented. Furthermore, we expect that the brain activity should scale with the reliability of the error information; i.e., the amplitude of the FRN should be more pronounced than the Ne/ERN amplitude because the prediction of the action outcome is affected by uncertainties that are eliminated in the feedback of the actual outcome.

One can expect higher Ne/ERN amplitudes for participants with more reliable error predictions. Furthermore, more reliable predictions of the forward model should also result in a faster learning inverse model and thus a better task performance (e.g., Jordan and Rumelhart 1992). Hence, we additionally expected a negative correlation between the average neural activation in error trials within EffW_200–350 and the average hit rate (HR) in the skittles task over days 3 and 4 (EEG recordings).

For hypothesis testing, we set the nominal level to 95% prediction coverage (α = 0.05). For illustration purposes, population mean curves and confidence bands were calculated by averaging hit and error curves of all participants.

RESULTS

Behavioral results: throwing performance and kinematic data.

Participants achieved an average hit rate (HR) at days 3 and 4 (days with EEG recordings) of 74% with a range from 45% to 94% (SD_HR = 14%). Figure 3 depicts the trend of the HR over the 4 experimental days. There is a continuous increase of the HR during the practice phase until midway into day 2, when the average hit rates drop significantly from 74% to 63% [t(20) = 2.65, P = 0.02, d = 0.59]. At this point, the visible online feedback about the ball flight was reduced from 33% to 0%. However, after the drop to 63%, the participants could improve their performance back to 70% until the beginning of day 3 [t(20) = 1.85, P = 0.04, d = 0.40]. At days 3 and 4, the participants showed a slight, nonsignificant improvement of the HR from 70% to 77% [t(20) = −1.28, P = 0.21, d = 0.27].

Kinematic data of the ball trajectory as well as the throwing movement are exemplarily depicted in Fig. 4. It can be observed that neither the ball trajectories (Fig. 4, top) nor the throwing trajectories with respect to angle and velocity profiles (Fig. 4, bottom) differ substantially between hit and error trials. Furthermore, as can be seen in Fig. 4, bottom, release variables of hit and error trials come from the same population. Differences between hits and errors rather arise from temporal deviations in the moment of release (moments of release are plotted as stars on each trajectory in Fig. 4, bottom). The time window of the Ne/ERN (EffW_200–350) coincides with the reversal movement after the throw and not with the throwing movement itself. Profiles in this reversal phase are very similar between hit and error trials, as well. Thus the observed differences in the Ne/ERN cannot be attributed to differences in the kinematics.

Fig. 4.

Kinematic data (ball trajectories and corresponding movement trajectories) of all hit trials (left) and all error trials (right) of one example subject. Top, task space: single ball trajectories plotted onto the task space with center post, target, and the different release positions of the lever. Bottom, execution and result space: single throwing movement trajectories with corresponding release points (gray and red stars) and average trajectory (bold lines) plotted as a function of release variables (velocity and angle) and throwing result (distance ball to target, DBT). The background color of the execution and result space codes the distance from white, which indicates a target hit, to black, which indicates hitting the center post. The green part of the average trajectory represents the time period in the throwing trajectory where the Ne/ERN is expected to emerge.

Electrophysiological results.

Grand averages of error (red) and hit (black) trials are displayed in Fig. 5. In contrast to the hit curve, there was a negative deflection in the error curve within EffW_200–350 (peaking at 250 ms). This negative deflection in the error curve represented a significant effect in mean amplitude [t(20) = −3.43, P < 0.01, d = 0.77]. This result can only be observed when comparing clear hits (DBT < 7 cm) with clear errors (DBT > 12 cm). The integration of all hits (including marginal ones: DBT ≤ 10 cm) and errors (including close ones: DBT > 10 cm) led to an increase of effect variability. As a result, the differences between hits and errors were not significant anymore [t(20) = 0.22, P = 0.71, d = 0.05].

Fig. 5.

Grand average curves of the hit (black) and error (red) trials. Data were synchronized to the moment of release (left broken line). The gray-shaded confidence band shows significant differences between the hit and error curves when the error curve falls outside the band. The green- and blue-shaded areas highlight the regions of the hypothesized effects. Bar diagram at right shows the mean difference activation for EffW_200–350 in green and both the present effect and the effect found by Maurer et al. (2015) for EffW_350–850 in blue (error bars represent SE).

In agreement with the assumption made by Maurer et al. (2015) that the ERP found within EffW_350–850 relates to the availability of visual ball flight, we did not find a significant difference in mean amplitude [t(20) = −1.45, P = 0.09, d = 0.32] when ball flight was masked. As a reaction to the result feedback (Fig. 6), we found highly significant differences in the mean amplitude of the difference curve [error minus hit; EffW_1000–1200; t(20) = −2.07, P < 0.01, d = 1.09]. Comparing the mean amplitudes of differences curves within EffW_200–350 and EffW_1000–1200, we find a significantly higher amplitude within EffW_1000–1200 compared with EffW_200–350 [t(20) = 2.20, P = 0.02, d = 0.48]. Regarding the correlation analysis, we found a significant moderate negative correlation between the hit rates of the subjects and the neural activation level within EffW_200–350 (r = −0.46, P = 0.02; Fig. 7).

Fig. 6.

Grand average curves of the hit and error trials. The time the subjects received the result feedback (KR) is displayed as a dotted line at 850 ms after ball release. The gray band illustrates the 95% confidence band that was constructed on the basis of the hit trials. The shaded area reaching from 1,000 to 1,200 ms represents the effect window for which the mean amplitude of differences curves was calculated and used for statistical testing.

Fig. 7.

Correlation between the mean amplitude within EffW_200–350 and the average HR over EEG recording days.

DISCUSSION

The present study was designed to examine an ERP (such as the Ne/ERN) relating to predictive error processing in a throwing task. The goal was to separate this prediction process from other processes involved in movement corrections and action effect monitoring, which could possibly also contribute to the Ne/ERN. In a previous study, Maurer et al. (2015) had already ruled out movement corrections as a source of the Ne/ERN by using a fast, ballistic throwing task, which did not allow online corrections. As another crucial change, we now implemented a procedure such that any feedback about the throwing outcome (ball flight trajectory as well as information about hit or miss) was delayed by 850 ms with respect to the ball’s release (i.e., the termination of the throwing movement). A Ne/ERN signal occurring after release would then not arise from a monitoring of the action effect, but rather could be ascribed to predictive error processes pertaining to action-related efferent and afferent signals. In the following, we discuss the results with respect to this assumption and the significance of the Ne/ERN as an indicator of predictive error processes based on forward modeling.

Contribution of error prediction and action effect monitoring to the Ne/ERN.

Within the time window EffW_200–350, we found a significant negative deflection in the error trials peaking ~250 ms after the ball release that was not present in the hit trials. At this point in time, the throwing movement was terminated and information about the ball trajectory as well as the throwing result was not available yet. Hence, neither motor execution nor processing of explicit external information about the movement outcome (action effect monitoring and result feedback) could have given rise to the signal. Furthermore, during the time window where Maurer et al. (2015) found the second negative deflection (EffW_350–850), we observed clearly smaller, nonsignificant differences between the hit and error curves in our study. The effect size of the mean amplitude of the difference curve in EffW_350–850 between hits and errors in the study from Maurer et al. (2015) was d = 1.74. Assuming this effect, we receive a power of 0.99 with a sample size of 21 in the current experiment. Hence, we can conclude with high certainty a clearly smaller effect in the time interval EffW_350–850 in the present study, presumably representing other processes than in the study of Maurer et al. (2015). In conclusion, we can state that the observed signal represents a Ne/ERN based on predictive processes about the movement error.

The amplitude of our Ne/ERN signal was smaller relative to the “classical” Ne/ERN (by 2 µV; Falkenstein et al. 1991; Gehring et al. 1993). This can be explained threefold. First, it is logical that in a complex motor task predictions are also more complex, which in turn might lead to less accurate predictions. Because the Ne/ERN is quantified by averaging several trials, wrong error predictions (i.e., errors predicted as hits) as well as wrong hit predictions (i.e., hits predicted as errors) might have attenuated the effect. This explanation is supported by the observation of the difference between hits and errors in the EEG average curves vanishing when all hit and error trials (not only the clear hits and errors) were averaged, because this includes hits and errors that lie very close to each other and thus includes a larger number of wrong predictions in the analysis.

Second, another explanation for the diminished Ne/ERN amplitude is that participants might have differed with respect to the reliability of their error predictions. Furthermore, processing times could have varied interindividually. As a consequence, the peaks of the negative deflection in error trials can vary in time and thereby cause temporal smearing in the average curve.

The third explanation is based on the aspect that errors analyzed in this study were self-generated in contrast to other studies, in which errors had been induced by external perturbations (Anguera et al. 2009; Krigolson and Holroyd 2006; Torrecillos et al. 2014). These different types of errors could also account for a delayed and diminished amplitude of the Ne/ERN potential found in EffW_200–350 due to a higher level of uncertainty in the prediction of self-generated errors in contrast to predicting errors induced by external perturbations. This could lead, as a consequence, to a lower neural activation and a longer processing time.

Ne/ERN and forward models.

As explained in the Introduction, it is suggested that predictive error processing is accomplished through internal models, more concretely, through forward models using a copy of the motor commands (efference copy) that simulates the effects of our movements (Wolpert and Flanagan 2001). The simulated (or predicted) effects are then compared with the desired effects, and deviations give rise to the error prediction. However, our Ne/ERN signal appeared ~250 ms after ball release, which is similar to the onset of the error signal in the study of Maurer et al. (2015). In agreement with their study, we would therefore argue that the basis of error prediction cannot be limited to the efference copy in our case, because processing of sensory (in this study, visual and proprioceptive) online information about the movement execution (in contrast to information from monitoring the action effect) would also be possible at this point in time (Jeannerod 1988). Thus the predictive processes that manifest in the Ne/ERN signal might rather be an integration of motor commands with proprioceptive and visual online information about the throwing movement. Importantly, this sensory information gives no explicit cue about the throwing outcome, because the task we used offers redundant solutions, which does not allow associative connections between throwing execution and throwing outcome. Hence, the integration of sensory information about the movement execution can improve the error prediction by reducing uncertainties, but it cannot substitute KR feedback information. Considering the fact that in this study it was possible to integrate sensory information (coming from vision and proprioception) to generate a motor prediction, we assume that both Ne/ERN from classical findings (Falkenstein et al. 1991; Gehring et al. 1993) and our present findings (see also Maurer et al. 2015) have the same functional significance, but the prediction generation process could be different due to different inputs to the prediction models (e.g., internal forward model).

Whether all three channels of information are indeed necessary to achieve error prediction in a task like ours remains to be clarified. But evidence that humans do integrate information from the efference copy with sensory feedback in the forward model to achieve a more accurate prediction of behavioral effect has already been shown in several studies (Desmurget and Grafton 2000; Shadmehr et al. 2010; Wolpert et al. 1995). In line with this aspect, we found an error prediction-related ERP (first negative deflection within EffW_200–350) that is delayed relative to the classical Ne/ERN findings. Hence, the processing of forward model error prediction in a complex motor task might require more resources in the movement programming phase than, for example, a choice reaction time task.

Whereas information from different channels needs to be integrated to yield an error prediction, the error feedback (related to the desired movement goal) should be processed mainly through the channel that contains the feedback information. In our skittles task, KR was provided using a static visual cue 850 ms after ball release. We analyzed the reaction to the KR and found that the neural activation after negative feedback was presented is significantly more negative compared with positive feedback (Fig. 6). We interpret this negative deflection as a FRN because it is only observable after error feedback. This finding is in line with other studies finding this reaction to negative feedback (Krigolson and Holroyd 2007a; Lutz et al. 2013). In addition, we found significantly higher amplitudes for the FRN (EffW_1000–1200) compared with the Ne/ERN-like potential (EffW_200–350). With the assumption that there is a relation between the certainty of error prediction and the amplitude of the Ne/ERN, the reported differences in amplitude are not surprising. Because we observed relatively small amplitudes of the Ne/ERN-like potential compared with other studies (Falkenstein et al. 1991; Gehring et al. 1993; MacLean et al. 2015), it can be assumed that the error prediction in our study is tainted with higher uncertainty (caused by the removal of action effect monitoring). This would lead, later on, to a higher amplitude of the FRN, because now the knowledge about the failure is irrevocable of the KR.

Correlation of error prediction and task performance.

If the Ne/ERN indeed indicates forward model error prediction, we also assume that its amplitude should scale with the reliability of the prediction. One possible access to this relation is the route via task performance. From theoretical considerations it is suggested that more reliable predictions of the forward model should result in a faster learning inverse model of the task at hand (e.g., Jordan and Rumelhart 1992). Thus we expected participants with better error prediction to reach higher performances in the skittles task, which should reveal as a negative correlation (since the Ne/ERN is a signal with negative activity) between the mean EEG amplitude of the error trials within EffW_200–350 and the hit rate in the skittles task. Behavioral results show that participants increased their hitting performance significantly during the first 2 practice days and leveled off on day 4, at least. This indicates that the task was novel to them and that a learning process happened. As a result of the correlation analysis, we indeed found a significant, moderate negative correlation between mean amplitude and hit rate. This finding matches the results found in the literature (Maurer et al. 2015) and indicates that the different hit rates might be a function of the quality of the forward model. In addition, the result further illustrates that participants differed in the size of their Ne/ERN signal, which in consequence attenuated the amplitude in the grand average as already discussed above.

Conclusion.

The presence of the Ne/ERN (during EffW_200–350) and the absence of the deflection during the ball flight (EffW_350–850) as observed in the study by Maurer et al. (2015) lead us to the conclusion that the Ne/ERN signal is representative of a prediction about an outcome error and that visual action monitoring about the movement effect (e.g., the visible ball flight) is not a prerequisite for this prediction. Furthermore, we suggest that the prediction is carried out by a forward model, which seemingly integrates only sources available before termination of the throwing movement (efference copy, visual and proprioceptive online information about movement). Because the Ne/ERN was observed after termination of the throwing movement (i.e., release), it can also be excluded that corrective submovements contributed the signal.

GRANTS

This research was supported by the Deutsche Forschungsgemeinschaft-funded Collaborative Research Center on “Cardinal Mechanisms of Perception” Grant SFB-TRR 135.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

ENDNOTE

At the request of the author(s), readers are herein alerted to the fact that additional materials related to this manuscript (EEG data segments and participants' responses from the detection task) may be found athttp://doi.org/10.5281/zenodo.580200. These materials are not a part of this manuscript, and have not undergone peer review by the American Physiological Society (APS). APS and the journal editors take no responsibility for these materials, for the website address, or for any links to or from it.

AUTHOR CONTRIBUTIONS

M.J., M.H., H. Maurer, H. Müller, and L.K.M. conceived and designed research; M.J. performed experiments; M.J. and L.K.M. analyzed data; M.J., M.H., H. Müller, and L.K.M. interpreted results of experiments; M.J. prepared figures; M.J. and L.K.M. drafted manuscript; M.J., M.H., H. Maurer, H. Müller, and L.K.M. edited and revised manuscript; M.J., M.H., H. Maurer, H. Müller, and L.K.M. approved final version of manuscript.