Embodying Others in Immersive Virtual Reality: Electro-Cortical Signatures of Monitoring the Errors in the Actions of an Avatar Seen from a First-Person Perspective

Abstract

Brain monitoring of errors in one's own and other's actions is crucial for a variety of processes, ranging from the fine-tuning of motor skill learning to important social functions, such as reading out and anticipating the intentions of others. Here, we combined immersive virtual reality and EEG recording to explore whether embodying the errors of an avatar by seeing it from a first-person perspective may activate the error monitoring system in the brain of an onlooker. We asked healthy participants to observe, from a first- or third-person perspective, an avatar performing a correct or an incorrect reach-to-grasp movement toward one of two virtual mugs placed on a table. At the end of each trial, participants reported verbally how much they embodied the avatar's arm. Ratings were maximal in first-person perspective, indicating that immersive virtual reality can be a powerful tool to induce embodiment of an artificial agent, even through mere visual perception and in the absence of any cross-modal boosting. Observation of erroneous grasping from a first-person perspective enhanced error-related negativity and medial–frontal theta power in the trials where human onlookers embodied the virtual character, hinting at the tight link between early, automatic coding of error detection and sense of embodiment. Error positivity was similar in 1PP and 3PP, suggesting that conscious coding of errors is similar for self and other. Thus, embodiment plays an important role in activating specific components of the action monitoring system when others' errors are coded as if they are one's own errors.

SIGNIFICANCE STATEMENT Detecting errors in other's actions is crucial for social functions, such as reading out and anticipating the intentions of others. Using immersive virtual reality and EEG recording, we explored how the brain of an onlooker reacted to the errors of an avatar seen from a first-person perspective. We found that mere observation of erroneous actions enhances electrocortical markers of error detection in the trials where human onlookers embodied the virtual character. Thus, the cerebral system for action monitoring is maximally activated when others' errors are coded as if they are one's own errors. The results have important implications for understanding how the brain can control the external world and thus creating new brain–computer interfaces.

Introduction

Monitoring when the outcome of one's own actions deviates from the intended goal is crucial for efficient goal-directed behavior and for the fine-tuning of motor skill learning. Neuroscientific studies of error detection in humans (Taylor et al., 2007) and nonhuman primates (Michelet et al., 2009) classically focused on the detection of one's own errors and on its neural signatures. Importantly, effective social interactions are fundamentally shaped by the ability to detect errors in the actions of other individuals (de Bruijn et al., 2007). Electrophysiological studies show that performing an error triggers a negative frontocortical ERP component (error-related negativity [ERN]; or Ne) that begins around the time of an erroneous response and peaks ∼80–100 ms later (Falkenstein et al., 1991; Gehring et al., 1993; Wessel, 2012). Monitoring the conflict deriving from processing multiple stimuli and responses (Yeung et al., 2004) as well as learning complex motor skills (Holroyd and Coles, 2002) are inherently linked to the process of mapping discrepancies between predicted and actual outcomes of actions. Interestingly, changes of ERN seem to index the behavioral adjustments called into play during a variety of processes associated with error monitoring. It has been demonstrated, for example, that amplitude of ERN is modified by whether an error in a typical Flanker task is caused by internal (e.g., inability to deal with the complexity of distractors) versus external causes (e.g., technical malfunctions of a response key) (Gentsch et al., 2009; Steinhauser and Kiesel, 2011). Thus, electrocortical signatures of errors are linked to the behavioral adjustments triggered by the perception of what is responsible for performed errors.

In addition to ERN, processing of committed errors elicits a positive deflection (error positivity [Pe]) (Falkenstein et al., 2000). Whereas ERN is linked to frontal regions, including anterior cingulate and supplementary motor cortices, Pe is linked to the posterior cingulate cortex (Vocat et al., 2008). Highly relevant for the present study is that ERN (van Schie et al., 2004) as well as a Pe-like component (van Elk et al., 2012) are also elicited by the observation of errors committed by other individuals.

Inherently associated with the ERN are the cortical oscillations in the theta band recorded over medial–frontal sites that have been related to action monitoring, cognitive control, and reinforcement learning (Trujillo and Allen, 2007; Cavanagh et al., 2009). Studies indicate that medial–frontal theta may mediate information processing during conflict, error monitoring, and negative performance feedback. It is also relevant that a variety of methods [including fMRI (Carter et al., 1998; van Veen et al., 2001), dipole modeling (Holroyd et al., 1998; Miltner et al., 2003), and direct recordings in humans (Wang et al., 2005; Cohen et al., 2008)] suggest that both ERN and theta oscillations likely originate from anterior cingulate and presupplementary motor area.

The two main components of embodiment, namely, ownership (i.e., the feeling that an artificial agent is part of our body) and agency (i.e., the feeling we are responsible for its movement) are essential for self-recognition and for distinguishing self-generated from other-produced actions (Jeannerod, 2003; Kalckert and Ehrsson, 2012). Virtual-reality studies show that mere visual perception of an avatar seen from a first-person perspective (1PP) may suffice to induce a strong feeling of being embodied in a virtual avatar (Slater et al., 2010; Petkova et al., 2011; Maselli and Slater, 2013) and controlling its actions (Tieri et al., 2015a).

In the present study, we explored whether the sense of embodiment or the perspective from which an error is seen plays any specific influence on the electrocortical markers of error detection and error awareness. To achieve this aim, we recorded through EEG, error-related cortical potentials and oscillations in healthy subjects immersed in a virtual environment (Cave Automatic Virtual Environment [CAVE]) while they observed action errors made by the limb of an avatar seen in 1PP or third-person perspective (3PP) and thus perceived or not perceived as part of their body.

Materials and Methods

Participants.

Twenty healthy participants took part in the study (8 female, mean ± SD, 27.1 ± 3.7 years). All participants were right-handed, had normal or corrected-to-normal visual acuity, were naive as to the purposes of the experiment, and gave their informed consent. The experimental protocol was approved by the local Ethics Committee at the Fondazione Santa Lucia Research Hospital (Rome, Italy), and the study was conducted in accordance with the ethical standards of the 1964 Declaration of Helsinki.

Apparatus, stimuli, and procedure.

Participants sat in a four screen (3 × 3 × 2.5 m) CAVE (Cruz-Neira et al., 1993). The virtual scenario consisted of a dining room with two mugs on a table. Participants immersed in the scenario saw two avatars (one in 1PP and another in 3PP; see Fig. 1A). The mugs were placed in the avatars' peripersonal space at a distance of ∼50 cm. The avatars and the scenarios were created by means of Autodesk Maya 2011 and 3D Studio Max 2011, respectively. The kinematics of the avatars' upper limb were implemented by HALCA library (Gillies and Spanlang, 2010), and the experiment was performed in an immersive 3D virtual environment, with real-size avatars, drawn on 1:1 scale and rendered in XVR 2.1 (http://www.vrmedia.it/en.html) (Tecchia et al., 2010). Left- and right-eye virtual images were alternately displayed with a refresh rate of 60 Hz and were synchronized with Nvidia stereo glasses. Finally, an Intersense 900 ultrasonic system (Thales Visionix) with 6 degrees of freedom was used to record the head position and to adjust the participant's point of view to the whole immersive virtual scenario.

Figure 1.

Illustration of the experimental paradigm through frames extracted from the videoclips. A, Representation of the four observation conditions. B, Timeline of a typical event trial.

In two different blocks, each consisting of 100 trials, participants observed one of the avatars performing reach-to-grasp movements toward one of the two mugs. Instructions on which mug was to be grasped were provided to the participants at the beginning of each trial through a synthetized voice. In one block, the participants observed the avatar's performance in a 1PP perspective, in which the avatar's arm was projected out of their shoulder (Fig. 1A, top row), whereas in the other block participants observed the performance of the avatar seated in a 3PP (Fig. 1A, bottom row). For each perspective, the avatar's kinematics was identical for the first 700 ms. However, the kinematics of the movement could be different in the last 300 ms of the trials where correct (70%) or missing mug grasp (30%) occurred. The sequence of correct and incorrect trials was randomized. After the grasping, the avatar's arm rested for 4000 ms before returning to the starting position. Finally, an intertrial interval of 3000 ms preceded the next trial (Fig. 1B).

During each intertrial interval, participants answered the question of “how much did you feel the avatar's arm belonged to you?” by uttering a number between 1 and 5 (with 1 corresponding to “not mine” and 5 to “as if the avatar limb was my own”). This provided us with a subjective index of embodiment mainly referring to sense of ownership. In principle, a total of 4000 subjective ratings (200 trials for each of the 20 participants, 2800 ratings concerning correct trials: 1400 in 1PP and 1400 in 3PP) and 1200 ratings concerning wrong movements (600 in 1PP and 600 in 3PP) should be obtained. However, one participant was unable to provide reliable reports in 4 trials (in the 1PP correct condition), and another participant did not complete the task and provided 80 trials instead of 100 in the 3PP conditions (13 in correct and 7 in incorrect conditions; see Fig. 2). The order of 1PP and 3PP blocks was counterbalanced across participants.

Figure 2.

Subjective reports of embodiment in the different observation conditions. y-axes represent ratings of embodiment (along a 1–5 point rating scale). Because 1 indicates no embodiment, virtually no embodiment occurred in the 3PP condition. Numbers in brackets under each column indicate the number of ratings in that specific condition.

In a preliminary phase, we matched the virtual avatar's size to each participant and asked him/her to report the alignment of the avatar's right shoulder with his/her own shoulder. This allowed us to maximize the perceived spatial congruency of the real and the virtual body. Then a practice session of 30 trials (21 correct and 9 erroneous mug grasps) was performed to acquaint the participants with the virtual environment.

EEG recording and preprocessing analysis.

EEG signals were recorded and amplified using a Neuroscan SynAmps RT amplifiers system (Compumedics) and acquired from 60 tin scalp electrodes embedded in a fabric cap (Electro-Cap International), arranged according to the 10–10 system. The EEG was recorded from the following channels: Fp1, Fpz, Fp2, AF3, AF4, F7, F5, F3, F1, Fz, F2, F4, F6, F8, FC5, FC3, FC1, FCz, FC2, FC4, FC6, T7, C5, C3, C1, Cz, C2, C4, C6, T8, TP7, CP5, CP3, CP1, CPz, CP2, CP4, CP6, TP8, P7, P5, P3, P1, Pz, P2, P4, P6, P8, PO7, PO3, AF7, POz, AF8, PO4, PO8, O1, Oz, O2, FT7, and FT8. Horizontal electro-oculogram was recorded bipolarly from electrodes placed on the outer canthi of each eye. Vertical electro-oculogram was recorded from an electrode placed below the right eye. All electrodes were physically referenced to an electrode placed on the right earlobe and were algebraically rereferenced off-line to the average of both earlobe electrodes. Impedance was kept <5 kΩ for all electrodes for all duration of the experiment, amplifier hardware bandpass filter was 0.01–200 Hz, and sampling rate was 1000 Hz. To remove the eye blinks or other artifacts, EEG and horizontal electro-oculogram were processed in two separate steps. First, we applied a blind source separation method by using independent component analysis (Jung et al., 2000) to remove from EEG any components related to eye movements. Second, trials showing amplifier blocking, residual eye blink, or other types of artifacts were excluded from the analysis manually. The artifact rejection procedure (over all the 20 participants) led to the rejection of some trials. The number of trials of which ERPs and time-frequency analyses were performed is detailed in Figure 3, and Figures 5 and 6, respectively (see below).

Figure 3.

Electrophysiological results in the domain of time. A, Grand average waveforms showing: (1) ERN component extracted in two different time windows (35–135 and 135–235 ms) for 1PP and 3PP; (2) N400 extracted in the time window 400–600 ms, at FCz electrode (left side of the panel); and (3) error positivity extracted in the time window 200–700 ms at Pz electrode (right side of the panel). Solid lines indicate 1PP. Dashed lines indicate 3PP. Gray represents correct trials. Black represents incorrect trials. Numbers in brackets near each line indicate the number of trials on which the analyses were performed. Frontal and parietal peaks of the topographical voltage distribution are reported below each ERP component. B, Scalp voltage maps for ERN (left side), N400 (central), and Pe (right side) plotted by subtracting the waveforms in correct from incorrect trials.

Figure 5.

Electrophysiological results in the time-frequency domain. A, Time-frequency representation of ERD/ERS (in percentage) from baseline (from −500 to −300 ms) for the frequency range 4–30 Hz and the time interval between −0.5 and 0.5 s for the 1PP high embodiment and the 3PP low embodiment and for correct and incorrect avatar's grasps at FCz electrode. Data from the entire sample of 20 participants are included. The value 0 on the x-axis indicates the instant occurrence of the error. It is worth noting that the effect is much stronger, earlier, and longer-lasting for theta than for alpha oscillations. B, Topographical distribution of the ERD/ERS from baseline of theta (4–8 Hz, left) and alpha (8–12 Hz, right) band. Error-related activity is estimated as subtraction of 1PP incorrect − correct and 3PP incorrect − correct, in the time window between 0 and 400 ms.

Figure 6.

Time-frequency results in the 11 participants who provided a number of trials sufficient to perform a group analysis after computation of trial-by-trial embodiment ratings. A, Time-frequency representation of ERD/ERS (in percentage) from baseline (−500 to −300 ms) for the frequency range 4–30 Hz and the time interval from −0.5 to 0.5 s for the following five conditions: 1PP HE (high embodiment), correct trials; 1PP HE, incorrect trials, 1PP LE (low embodiment), incorrect trials, 3PP LE correct trials, 3PP LE, incorrect trials. Increased synchronization in theta band was found only in 1PP HE, incorrect grasp condition in the time interval from 0 to 400 ms. B, Topographical distribution of the ERD/ERS from baseline of theta and alpha band in the two 1PP incorrect conditions, separated for high and low embodiment, in the time window between 0 and 400 ms. It is worth noting that the effect is much stronger, earlier, and longer-lasting for theta than for alpha oscillations.

Analysis of action observation-related ERPs.

Electro-encephalographic signal analyses in the time domain allowed us to explore whether specific ERP components emerged in the different experimental conditions.

Two main components, already described in the ERP literature (i.e., ERN and Pe) were individuated. The ERN is a negative deflection waveform with a frontocentral midline distribution that peaks ∼80–100 ms after the occurrence of an erroneous response (Falkenstein et al., 1991; Gehring et al., 1993). Relevant to the present study is that an ERN component was elicited also when observing erroneous actions in others (van Schie et al., 2004). It is worth noting here that the ERN evoked by observation of errors in others occurred later and was smaller in amplitude with respect to the ERN evoked by actually committed errors (van Schie et al., 2004).

Pe is a P300-like component peaking at the Pz electrode likely associated with the conscious recognition of errors, either committed (Vidal et al., 2000; Pavone et al., 2009) or observed in others (de Bruijn et al., 2007).

In addition to the ERN and Pe components (which have been repeatedly associated with action errors), we individuated a large negative deflection peaking ∼400 ms over centroparietal scalp electrodes that is reminiscent of the N400 component originally found as a consequence of semantic violations (Kutas and Federmeier, 2011). It is worth noting here that mounting evidence demonstrates the N400 is not evoked only by semantic incongruence (across written, spoken, and signed language) but is also found to be elicited by observation of erroneous actions, with a more frontal scalp distribution when meaningful actions are involved (Proverbio et al., 2012, 2013; Balconi and Vitaloni, 2014).

Separate single-trial epochs of 1.5 s length were extracted in the time window starting 500 ms before the end of the avatar's movement in both 1PP and 3PP conditions. Epochs were corrected with respect to the baseline in the time window from 500 to 300 ms before the grasping/missing offset and were filtered with a low-pass digital filter of 30 Hz. An additional high-pass filter of 2 Hz was used for the ERN and N400 components to reduce the massive contribution of slow potentials that propagate from parietal to frontal areas and masked some of the frontal components relevant to our study. The mean amplitudes of the ERN component at the frontal–medial electrode site (FCz) were extracted in two different time windows, namely, 35–135 ms for 1PP and 135–235 ms for 3PP. This procedure was adopted because the ERN triggered by observed errors later than ERN triggered by committed errors (van Schie et al., 2004); and although only passive observation is used in our task, differences in embodiment may make observed 1PP errors more similar to performed errors with respect to observed 3PP errors.

The Pe mean amplitudes were measured in a time window between 200 and 700 ms at the parietal site (Pz electrode) following the avatars' grasping/missing movement. The mean amplitudes of N400 component were extracted in a time window between 400 and 600 ms at the frontal–medial electrode site (FCz). The choice of the electrode for each component was based on visual inspection of waveforms where maximal ERN, Pe, and N400 amplitudes were expected on the basis of ERPs' literature (Falkenstein et al., 2000; van Elk et al., 2008; Proverbio et al., 2012, 2013).

Separate analyses for each of the above components were performed by means of Analyzer 1.05 software (Brain-Products).

Time-frequency analysis.

Time-frequency calculations were computed using the FieldTrip routines (Donders Institute, Nijmegen) (Oostenveld et al., 2011) in MATLAB (The MathWorks). The EEG time series were obtained by segmenting the signal into epochs of 2 s length (from 1000 ms before to 1000 ms after the trigger of the avatars' grasping/missing movement), and they were bandpass filtered from 0.1 to 100 Hz and down-sampled at 500 Hz. Each epoch was transformed in the frequency domain and multiplied with the Fast Fourier Transformation power spectrum of a set of complex Morlet wavelets, defined as a Gaussian-windowed complex sine wave and taking the inverse Fast Fourier Transformation as follows:

where t is the time and f is the frequency (increased from 2 to 30 Hz in linearly spaced steps), and σ defines the width or cycles of each frequency band set according to 4/(2πf) (Cavanagh et al., 2009). A width of 4 provided an adequate trade-off between temporal and frequency resolution (Cavanagh et al., 2012). Each epoch was then cut in 50 ms bins in a −500 to 500 ms periresponse window and averaged across trials. Finally, for each estimated frequency band, results were displayed as event-related desynchronization/synchronization (ERD/ERS) with respect to a baseline from −500 to −300 ms before the avatars' grasping/missing movement. ERD and ERS represent a decrease or increase in synchrony of the recorded neuronal population (Pfurtscheller and Lopes da Silva, 1999). Positive and negative ERD/ERS values index synchronization and desynchronization with respect to a given reference interval.

To compute event-related desynchronization/synchronization, we used the following formula:

where E(t, f) represents the spectrum at a given t and f and E_ref(t, f) represents the mean power of the reference interval. For each condition, the mean value in the time window from 0 to 400 ms after the avatars' grasping/missing movement was computed at FCz on theta (4–8 Hz), alpha (8–12 Hz), and beta (12–30 Hz) bands. This choice was motivated by the results of time-domain analysis in which FCz electrode showed the largest ERN waveform peak. Moreover, FCz was the electrode that showed the largest error specific effect in the wavelet transformation.

Theta activity is thought to reflect the error-related brain activity in the frequency domain (Trujillo and Allen, 2007; Hanslmayr et al., 2008; Marco-Pallares et al., 2008; Cavanagh et al., 2009; Christie and Tata, 2009; Mazaheri et al., 2009). Studies indicate that, when an error occurs, neurons in the medial–frontal cortices specifically increase their synchronization in the theta band but hardly, if any, in other bands (Cohen, 2011). Thus, this cortical region might be considered a functional “hub” where the theta rhythm may act by coordinating local and long-range neural networks that monitor actions and detect response errors (Huerta and Lisman, 1995; Jensen and Lisman, 2000; Dragoi and Buzsáki, 2006).

Statistical analyses of behavioral, ERPs, and time-frequency power representation data were analyzed by using Statsoft Statistica 8. GLM with the Greenhouse-Geisser correction for nonsphericity was applied where appropriate (Keselman and Rogan, 1980). Post hoc comparisons were performed using the Bonferroni correction for multiple comparisons.

Results

Analysis of subjective ratings of embodiment of the avatar limb

Figure 2 shows the subjective reports concerning the participants' embodiment with the virtual avatar in the 1PP and 3PP conditions and for correct and incorrect grasping of the avatar. The distribution of the ratings was checked for normality using the Shapiro-Wilk test. The data were not normally distributed (Shapiro-Wilk test for 2 of 4 conditions: p < 0.01). Thus, nonparametric analysis, including Friedman ANOVA and Wilcoxon test for the within-group effects perspective (1PP vs 3PP) and avatar's grasp accuracy (correct vs incorrect), was used. The Bonferroni correction was adopted for the six within-group comparisons (thus setting the p level at 0.008). We found a significant effect of the level of embodiment (χ² = 51.84, p < 0.001), which was accounted for by higher sense of embodiment in 1PP than in 3PP (mean: 3.63 vs 1.21) conditions. Within-group comparisons showed significant differences between correct versus incorrect in 1PP (p < 0.001, r = 0.81), correct in 1PP versus 3PP condition (p < 0.001, r = 0.88), correct in 1PP versus incorrect in 3PP (p < 0.001, r = 0.88), incorrect in 1PP versus 3PP (p < 0.001, r = 0.88), and incorrect in 1PP versus correct in 3PP (p < 0.001, r = 0.87). No significant difference was found between correct versus incorrect trials in 3PP (p = 0.15).

Analysis of electrophysiological results

Both ERP and time-frequency ERD/ERS data were distributed normally and thus analyzed using ANOVAs. ERN and N400 amplitudes at FCz electrode and error positivity amplitude at Pz electrode were entered in three separate two-way ANOVAs with perspective (1PP vs 3PP) and avatar's grasp accuracy (correct vs incorrect) as main factors.

ERN

Inspection of Figure 3A (left panel, left part) suggests an enhanced ERN for the incorrect 1PP condition compared with the incorrect 3PP condition. Statistical analysis shows significant main effects of the perspective type (F_(1,19) = 12.47, p = 0.002, η² = 0.39; means: 1PP = −1.81 μV vs 3PP = −0.63 μV) and of avatars' grasp accuracy (F_(1,19) = 34.60, p = 0.00001, η² = 0.64; means: correct = −0.40 μV vs incorrect = −2.04 μV). The interaction between perspective and avatar's grasp accuracy (F_(1,19) = 4.69, p = 0.04, η² = 0.19) was significant. Post hoc analysis confirmed that the participants' response monitoring system detected the avatar's erroneous motor responses when the participants observed the avatar's arm in 1PP [all p values <0.0001; mean: 1PP-incorrect (−2.83 μV) vs 1PP-correct (−0.79 μV); 1PP-incorrect vs 3PP-correct (−0.02 μV); and 1PP-incorrect vs 3PP-incorrect (−1.24 μV)]. Moreover, participants' monitoring system detected erroneous avatar's grasping in the 3PP [p = 0.001; 3PP-incorrect (−1.24 μV) vs 3PP-correct (−0.02 μV)]. No other comparisons (correct trials in 1PP vs 3PP) turned out to be significant (all p values >0.5). Maps of ERN plotted as difference between incorrect minus correct trials in 1PP versus 3PP conditions are shown in Figure 3B (left part).

N400

Figure 3A (left panel, right part) shows a larger negativity waveform in 1PP and 3PP condition peaking between 400 and 600 ms after the erroneous grasping. Statistical analysis showed a significant main effect of the avatars' grasp accuracy type [F_(1,19) = 5.53, p = 0.029, η² = 0.22; means: correct = −0.40 μV vs incorrect = −0.91 μV] but not of the perspective (1PP vs 3PP) or of its interaction with grasp accuracy (all p values > 0.6). Maps of N400 plotted as difference between incorrect minus correct trials in 1PP versus 3PP conditions are shown in Figure 3B (middle part).

Error positivity (Pe)

Figure 3A (right panel) shows Pe as a huge positive wave peaking 350 ms after the end of avatars' motor response on the Pz electrode. A larger error positivity was present on the incorrect trials in both 1PP and 3PP condition with respect to the corresponding conditions on correct trials. ANOVA showed significant main effects of perspective type [F_(1,19) = 34.60, p = 0.00001, η² = 0.645; due to higher amplitude in 1PP = 3.92 μV vs 3PP = 2.42 μV] and avatar's grasp accuracy [F_(1,19) = 34.87, p = 0.00001, η² = 0.647; due to lower amplitude for correct = 2.01 μV vs incorrect = 4.32 μV]. No interaction between perspective and avatar's grasp accuracy was significant (all p values = 0.65). Maps of Pe plotted as difference between incorrect minus correct trials in 1PP versus 3PP conditions are shown in Figure 3B (right part).

It is held that high-pass filters >1 Hz may introduce artifactual polarity effects that influence ERPs (e.g., Tanner et al., 2015). However, in the present study, we used a high-pass filter of 2 Hz for the ERN and N400 components with the specific aim to reduce the massive contribution of error positivity components that propagate from parietal to frontal areas and masked some of the frontal components relevant to our study. Importantly, we followed explicit recommendations related to filtering procedures according to which it is important to check that grand average waveforms with and without filters maintain the same morphology (Acunzo et al., 2012) and do not introduce distortions that may bias the estimated parameters (Widmann et al., 2015). More specifically, to show that the adopted high-pass filter did not create any artifactual effects on ERN and N400, we computed the grand average of incorrect minus correct trials for the 1PP and 3PP conditions with and without the filter (Fig. 4A) and showed the residual effects of the filter independently from the type of trial and perspective (Fig. 4B). The figure shows that the high-pass filter removed the slow activity that characterizes the error positivity in the time window between 200 and 800 but did not generate distortions in the signal and did not introduce artifactual peaks in the waveforms.

Figure 4.

Grand average comparison of high-pass filtered and unfiltered waveforms. A, Grand average of incorrect minus correct trials for the 1PP and 3PP conditions with and without the high-pass filter. B, Residual effects of the filter independently from the type of trial and perspective.

Time-frequency analysis

Figure 5 shows the time-frequency power representation of the medial–frontal activity for correct and incorrect grasps of avatars seen from 1PP and 3PP for the theta, alpha, and beta frequency bands. Event-related desynchronization/synchronization (ERD/ERS) measures were quantified for each participant in a time interval between 0 and 400 ms after the end of the avatar's grasping/missing. ERD/ERS values recorded at the FCz electrode in the theta (4–8 Hz), alpha (8–12 Hz), and beta (12–30 Hz) bands and were entered into three (one for each band) separate repeated-measure two-ways ANOVAs with perspective type (1PP vs 3PP) and avatar's grasp accuracy (correct vs incorrect) as main factors.

Theta band

The ANOVA shows a significant main effect of perspective (F_(1,19) = 10.78, p = 0.0039, η² = 0.36) accounted for by a synchronization in the 1PP (means: 16.02%) versus a desynchronization in the 3PP (means: −6.49%). The significance of the avatars' grasp accuracy (F_(1,19) = 26.90, p < 0.001, η² = 0.59) is accounted for by synchronization in the incorrect (means: 22.39%) versus desynchronization in the correct (means: −12.86%) trials. The interaction perspective by avatar's grasp accuracy was also significant (F_(1,19) = 8.40, p = 0.009, η² = 0.31). Crucially, this effect seems to be entirely accounted for by the difference between incorrect 1PP (means = 43.46%) versus all the other conditions (incorrect 3PP [means = 1.31%] (p = 0.0018); vs correct 1PP [means = −11.41%] (p < 0.001); versus correct 3PP [means = −14.31%] (p < 0.001), which in turn did not differ from one another (all p values > 0.7).

Alpha band

The ANOVA shows a significant main effect of avatar's grasp accuracy (F_(1,19) = 41.75, p < 0.001, η² = 0.69) with positive values (synchronization) in the incorrect grasp (means: 19.33%) and negative values (desynchronization) in the correct grasp (means: −9.16%). The interaction perspective by avatar's grasp accuracy was also significant (F_(1,19) = 7.15, p = 0.014, η² = 0.27). This effect seems to be entirely accounted for by the difference between 1PP incorrect (means = 32.39%) versus all other conditions (incorrect 3PP [means = 6.27%]; p = 0.022); versus correct 1PP [means = −11.05%] (p < 0.001); versus correct 3PP [means = −7.28%] (p < 0.001), which in turn did not differ from one another (all p values > 0.24). No significant effect of perspective was found.

Beta band

No significant main effect or interactions were found for this band.

Teasing apart the role of perspective (1PP vs 3PP) from the role of sense of embodiment (high vs low)

Results from ERPs (Fig. 3) and time-frequency (Fig. 5) analyses suggest that the neuroelectric signature of detecting errors (as indexed by ERN and theta oscillations) in others is associated with seeing an avatar in 1PP perspective. However, the way in which we presented the results thus far may not distinguish whether error related medial–frontal theta and alpha bands in the incorrect 1PP condition were selectively prompted by perspective (1PP vs 3PP), sense of embodiment (high vs low), or a combination thereof. To try and tease apart the effect of participant's perspective from sense of embodiment, we used a median-split procedure to separate low and high reports of sense of embodiment in the different perspective and avatar accuracy conditions (MacCallum et al., 2002). We found that trials with scores ≤2 index a low sense of embodiment and trials rated with scores ≥3 index a high sense of embodiment. Table 1 reports the number of trials (in the 20 participants) for each of the 8 possible conditions derived from the sorting procedure. Although the median-split procedure provides a straightforward way to categorize specific trials, studies suggest that dychotomizing continuous variables may imply a gain in power (Preacher et al., 2005). However, the problem may be prominent for normally distributed data (Irwin and McClelland, 2003). It is worth noting that Likert scales are nonparametrically distributed along ordinal distributions. Thus, applying the median-split procedure in this specific case simply indicates the passage from an ordinal distribution to a simplified one. Furthermore, we applied cumulative probit models to analyze ordinal data, estimated the probability of obtaining a given value in the Likert scale independently from individual differences (Agresti, 2002), and found that the probability of getting 1 or 2 in the Likert has a confidence interval between 0.6 and 0.8. In view of this, we think that applying the median-split procedure in our case is not problematic.

Table 1.

Embodiment ratings in the entire sample of 20 participants after application of the median split procedure for sorting low- versus high-embodiment trials in 1PP and 3PP and for accuracy of avatar performance (incorrect vs correct)^a

	1PP				3PP
	Low		High		Low		High
	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect	Correct	Incorrect
	No. of trials	69	172	1327	428	1351	590	36	3
% of trials of total	4.9	28.7	95.1	71.3	97.4	99.5	2.6	0.5
Mean	1.88	1.91	3.88	3.46	1.19	1.07	3.01	3
SD	0.25	0.14	0.50	0.59	0.30	0.10	0.02	—
No. of subjects	8	11	20	20	20	20	5	1

^aIn 1PP, high embodiment was reported in an overwhelming proportion of trials. In contrast, in 3PP, low embodiment was associated with an overwhelming proportion of trials (see the first four table lines). Line 5 indicates the number of participants (of 20) who contributed to a given category of reports.

Inspection of Table 1 clearly shows that in 1PP the overwhelming majority of ratings indicate high embodiment, whereas an opposite pattern of results is found in 3PP.

The different combinations are reported in the following: 1PP high-embodiment correct (1327 of 1396 possible trials from 20 participants), 1PP high-embodiment incorrect (428 of 600 possible trials from 20 participants), 1PP low-embodiment correct (69 of 1396 possible trials, data from 8 of 20 participants), 1PP low-embodiment incorrect (172 of 600 possible trials, data from 11 of 20 participants), 3PP high-embodiment correct (36 of 1387 possible trials, data from 5 of 20 participants), 3PP high-embodiment incorrect (3 of 593 possible trials, data from 1 of 20 participants), 3PP low-embodiment correct (1351 of 1387 possible trials, from 20 participants), and 3PP low-embodiment incorrect (590 of 593 possible trials from 20 participants). Because of the small number of trials for some of the rating categories, a quantitative group analysis was possible only for 5 of 8 conditions (1PP high-embodiment correct, 1PP high-embodiment incorrect, 1PP low-embodiment incorrect, 3PP low-embodiment correct, and 3PP low-embodiment incorrect) where at least 11 of the 20 participants provided a minimum of 10 trials. Table 2 shows mean values of each ERP component in the above five conditions.

Table 2.

ERPs (ERN, N400, and Pe) of the 11 participants who provided a number of trials sufficient to perform a group analysis^a

	1PP HE (correct trials) (μV)	1PP HE (incorrect trials) (μV)	1PP LE (incorrect trials) (μV)	3PP LE (correct trials) (μV)	3PP LE (incorrect trials) (μV)
ERN	−0.93 ± 0.88	−3.14 ± 1.98	−0.94 ± 1.48	0.05 ± 0.35	−1.10 ± 1.45
N400	−0.20 ± 0.54	−0.89 ± 1.61	−1.50 ± 2.35	−0.56 ± 0.81	−1.07 ± 1.36
Pe	2.47 ± 1.55	5.38 ± 2.82	4.19 ± 3.03	1.11 ± 1.66	3.28 ± 2.84

^aData are mean ± SD. HE, High embodiment; LE, low embodiment.

Being interested in exploring the link between sense of embodiment and electrocortical indices of error processing, ERPs and time-frequency values from these 11 participants were entered in separate one-way ANOVAs where the main effect of conditions had three levels (1PP high-embodiment incorrect, 1PP low-embodiment incorrect, 3PP low-embodiment incorrect).

ERN

The significance of the main effect (F_(2,20) = 7.49, p = 0.003, η² = 0.43) is entirely accounted for by the difference between 1PP high-embodiment incorrect [means = −3.14 μV] with respect to the other two conditions (1PP low-embodiment incorrect [means = −0.94 μV]; p = 0.007); 3PP low-embodiment incorrect [means = −1.10 μV] (p = 0.012), which in turn did not differ from one another (p = 1).

N400

No significant main effect was found for this component (F_(2,20) = 0.95, p = 0.40, η² = 0.08).

Error positivity

The main effect of condition was significant (F_(2,20) = 4.06, p = 0.033, η² = 0.29). Post hoc comparisons show higher Pe amplitude in 1PP high-embodiment incorrect [means = 5.38 μV] with respect to 3PP low-embodiment incorrect [means = 3.28] (p = 0.03) but not to 1PP low-embodiment incorrect [means = 4.19] (p = 0.36). No difference between 3PP low embodiment and 1PP low embodiment was found (p = 0.69).

Time-frequency analysis

ERD/ERS values recorded at the FCz electrode in the theta (4–8 Hz), alpha (8–12 Hz), and beta (12–30 Hz) bands in the five conditions where 11 participants provided a sufficient number of trials are shown in Figure 6A.

However, being interested in the error-related activity, only the data from incorrect conditions were used for formal analysis. These data were entered into three (one for each frequency band) one-way ANOVAs with condition (three levels: 1PP high-embodiment incorrect, 1PP low-embodiment incorrect, and 3PP low-embodiment incorrect) as main factor.

Theta band

The ANOVA shows a significant main effect (F_(2,20) = 6.90 p = 0.005, η² = 0.41) that is accounted for by the difference of 1PP high-embodiment incorrect (mean = 56.6%) with respect to 1PP low-embodiment incorrect (mean = 4.64%, p = 0.026) and to 3PP low-embodiment incorrect (mean = −5.13%, p = 0.007), which in turn did not differ from one another (p = 1).

Alpha band

The ANOVA shows a significant main effect (F_(2,20) = 5.04, p = 0.017, η² = 0.33). Post hoc comparisons showed that 1PP high-embodiment incorrect (mean = 31.57%) tended to be significantly different from 1PP incorrect low (mean = 7.45%, p = 0.09) and was significantly different from 3PP low-embodiment incorrect (mean = −0.16%, p = 0.019). These last two conditions were not different from one another (p = 1). Thus, changes in the alpha band synchronization are less specifically linked to embodiment with respect to theta changes.

Beta band

No significant effect was found (F_(2,20) = 0.70 (p = 0.51), η² = 0.07).

To rule out any nonspecific effect of theta and alpha cortical oscillations found in our experimental task, we performed the same analysis on parieto-occipital electrodes (mainly from POz in a time window of 500–800 ms after error) that, unlike the frontal ones, have never been associated with the process of detecting errors. Importantly, changes of alpha synchronization in these regions have been associated with attentional monitoring (Cohen and van Gaal, 2013). While the ANOVA on alpha oscillation shows a significant main effect on the avatar's grasp accuracy (F_(1,19) = 6.18, p = 0.022, η² = 0.24) that is accounted for by the stronger desynchronization in erroneous (mean = −16.7%) versus correct grasping (mean = −7.75%), no effect of theta was found (1PP correct mean = −11.81%, 1PP incorrect mean = −3.62%, 3PP correct mean = −14.3%, 3PP incorrect mean = −15.28%). It is worth noting that, unlike the frontal effect on alpha, the parieto-occipital one is not influenced by the perspective and may thus simply reflect attentional reactivity to the erroneous grasping that is more arousing and salient because of its lower frequency (Carp et al., 2009; Mazaheri et al., 2009). More importantly, the absence of any effect on parieto-occipital theta may further hint at a specific association between frontal theta oscillations and error monitoring (Fig. 6B).

It is worth noting that we performed the analyses of the three erroneous conditions for ERP and time-frequency also by computing the median split for each participant. By using this procedure, we selected 11 of 20 participants. However, this approach further reduced the number of 1PP trials where embodiment scores can be considered low. Indeed, only 8 of 20 participants provided a number of trials sufficient to allow an analysis of the ERPs. Interestingly, the computation at individual level strengthened the conclusion that 1PP is mainly associated with high embodiment. Because having the largest possible number of participants is recommended in ERP studies, we adopted the general median-split procedure illustrated above. Importantly, however, we checked that the general pattern of electrophysiological results was comparable using the general or the individual median-split procedure. To this aim, we performed the ERP and time-frequency analyses also with the individual median-split procedure. We found that the results with the two procedures are largely overlapping. More specifically, we found the following:

ERN: The significance of the main effect (F_(2,14) = 7.02, p = 0.007, η² = 0.50) is entirely accounted for by the difference between 1PP high-embodiment incorrect (means = −3.49 μV) with respect to the other two conditions (1PP low-embodiment incorrect [means = −1.26 μV]; p = 0.031); 3PP low-embodiment incorrect [means = −0.87 μV] (p = 0.011), which in turn did not differ from one another (p = 1). N400: No significant main effect was found for this component (F_(2,14) = 0.94, p = 0.41, η² = 0.11). Error positivity: The main effect of condition was significant (F_(2,14) = 5.59, p = 0.016, η² = 0.44). Post hoc comparisons show higher Pe amplitude in 1PP high-embodiment incorrect (means = 5.66 μV) with respect to 3PP low-embodiment incorrect [means = 3.85] (p = 0.035) and with 1PP low-embodiment incorrect [means = 3.85] (p = 0.36). No difference between 3PP low embodiment and 1PP low embodiment was found (p = 0.1).

Theta band: The ANOVA shows a significant main effect (F_(2,20) = 9.35 p = 0.001, η² = 0.48) that is accounted for by the difference of 1PP high-embodiment incorrect (mean = 68.3%) with respect to 1PP low-embodiment incorrect (mean = 18.8%, p = 0.025) and to 3PP low-embodiment incorrect (mean = −3.7%, p = 0.001), which in turn did not differ from one another (p = 0.61). Alpha band: The ANOVA shows a significant main effect (F_(2,20) = 4.13, p = 0.031, η² = 0.29). Post hoc comparisons showed that 1PP high-embodiment incorrect (mean = 26.61%) was not significantly different from 1PP incorrect low (mean = 2.25%, p = 0.11) and was significantly different from 3PP low-embodiment incorrect (mean = −0.29%, p = 0.04). These last two conditions were not different from one another (p = 1). Beta band: No significant effect was found (F_(2,20) = 1.07, p = 0.36, η² = 0.09).

All in all, we found that the data concerning the indices of error monitoring linked to the embodiment (i.e., ERN and theta oscillations) were identical with the two procedures.

Functional association between ERP and time-frequency signatures of perspective taking and error observation

While both alpha and theta band oscillations showed a significant main effect of error conditions that was mainly accounted for by 1PP versus 3PP, the latter band oscillation seems to be stronger, longer-lasting, and more specific than the former one.

To explore the possible different meaning of the two time-frequency effects, we performed a multiple regression analysis to ascertain any link between ERN and alpha and theta as predictors. We found a significant negative effect of theta (β = −0.62, p = 0.030), suggesting that stronger theta synchronization predicts more negative ERN. No significant relation is found for the alpha band and ERN (β = 0.31, p = 0.24). It is worth noting that alpha and theta did not show any prediction effect on amplitude of N400 [theta (β = −0.13, p = 0.66) and alpha (β = −0.06, p = 0.84)].

Discussion

We explored the link between embodiment and error monitoring by combining immersive virtual reality and EEG recording in healthy participants who observed correct or erroneous actions performed by an avatar. We obtained three main results. First, seeing an avatar from a first-person perception is enough to embody it in the absence of any cross-modal boosting, as in rubber-hand of full-body illusion conditions (Maselli and Slater, 2013; Tieri et al., 2015b). Observation of errors in 1PP slightly reduced the sense of embodiment which, however, remained much higher with respect to when observing the avatar's actions from a third-person perspective. Second, merely observing erroneous naturalistic action induces changes of perceived embodiment, which was reflected in modulation of ERN, the ERP component that is typically linked to detection of committed errors. Third, increased synchronization of cortical oscillations in the theta band recorded over medial–frontal electrodes (centered on FCz) is specifically associated with detection of the errors of an avatar but only in the trials where human onlookers embodied the observed virtual character.

Cortical signatures of observed action error as inferred from ERPs

ERN and Pe are thought of as electrocortical signatures of the neural system that monitors one's own action and detects performance errors in humans (van Veen and Carter, 2006; Wessel, 2012) and nonhuman primates (Phillips and Everling, 2014). Our experimental manipulation was effective in eliciting a clear ERN at FCz electrode confirming its link with mid-frontal cortex activity. Importantly, ERN amplitude was much larger when participants observed, from a first-person perspective, the slips in the avatar's action. Thus, although ERN was present in all the erroneous trials, its amplitude was not enhanced during the 3PP and 1PP low-embodiment trials. Thus, we expand previous knowledge about vicarious error detection (van Schie et al., 2004) by showing that ERN not only signals the occurrence of an error seen from a specific perspective, but it may represent the marker of the feeling that the observed errors were committed by the self. Another novel result concerns Pe, the ERP component that predicts whether an error would be consciously perceived up to 400 ms before the average awareness response (Murphy et al., 2012; Wessel, 2012). Unlike ERN, Pe amplitude was comparable for 1PP and 3PP, which are typically associated with reports of high and low embodiment, respectively. That Pe amplitudes are not primarily influenced by embodiment suggests that awareness of errors (as indexed by this component) is comparable for the self and the other.

Finding a clear N400 potential elicited at the very same electrode of the ERN deserves some discussion. N400 component is traditionally linked to the occurrence of semantic violations when processing linguistic material (Kutas and Federmeier, 2011). However, more recent studies indicate that this component can also be elicited by observation of erroneous actions (Balconi and Vitaloni, 2014) and other action-related material, such as production of deceptive responses (Proverbio et al., 2013) or detection of incorrectly performed basketball actions in expert athletes (Proverbio et al., 2012). That N400 was larger for erroneous grasps is in keeping with studies suggesting its role in the process of integrating experience-related knowledge with actual information in specific contexts (Amoruso et al., 2013).

Cortical signatures of observed action error as inferred from time-frequency analysis of cortical oscillations

Brain oscillations at different frequencies provide temporal and spatial codes that may represent a mechanism for creating the dynamic, functional communication networks likely underpinning complex integrative functions (for review, see Başar et al., 2001). It is worth noting that oscillations in the theta band recorded from medial–frontal regions during error detection or other types of conflict monitoring (Cohen, 2014; Cavanagh and Shackman, 2015) seem to be inherently linked to ERN in the domain of time (Luu et al., 2004). In addition to time-locked error signals, our study shows enhanced theta (4–8 Hz) oscillations over medial–frontal electrodes (centered on FCz) during observation of the errors made by the avatar. Crucially and in keeping with what was observed for ERN, medial–frontal theta enhancement occurred when participants reported to feel embodied in the avatar, a condition mainly associated with taking a first-person perspective. However, seeing the avatar's errors in first-person perspective was necessary but not sufficient to elicit endogenous potentials that code early and purportedly unconscious error detection. Indeed, enhancement of theta power was specific for observation of erroneous grasping seen from a first-person perspective but only in the trials where the subjective ratings of embodiment with the avatar indicated high identification between participants and avatar. Thus, in our experimental conditions theta oscillations may be considered as marker of a complex code that integrates error monitoring, visual perspective, and sense of embodiment. While the results concerning enhancement of theta power is in keeping with most of the previous literature (Trujillo and Allen, 2007; Cavanagh et al., 2009; van Driel et al., 2012), the enhancement of alpha band in the very same conditions found for theta is less expected and deserves some discussion. It is important to note the interaction of different band oscillations, correlates with efficiency of performance in specific cognitive domains (e.g., theta-gamma for memory (Lisman and Jensen, 2013) or alpha-theta in attentional tasks (Song and Bédard, 2014). Modulation of alpha-theta bands has been reported also in conditions where errors were committed as a result of sustained attention lapses and/or of failures of motor control (van Driel et al., 2012). More specifically, different types of performance errors were associated with different patterns of mid-frontal theta-band and posterior alpha-band oscillatory activity. More specifically, attention lapses versus failure of motor control related-errors were associated with absence of mid-frontal theta phase synchronization and increase of parieto-occipital-frontal alpha synchronization (van Driel et al., 2012), suggesting that performance monitoring is based on different mechanisms, which are differentially recruited in different experimental conditions (Cohen and van Gaal, 2013). In particular, parieto-occipital alpha synchronization seems to be more related to attention-related errors and mid-frontal theta to motor-control related errors (van Driel et al., 2012). Although our study deals exclusively with observed action errors, we report a modulation of theta and alpha oscillations contingent upon error monitoring and a general error-related alpha desynchronization over parietal regions, which is compatible with the attentional salience of observed slips of action reported in previous research (Carp et al., 2009). However, unlike the aforementioned study, we show an increase of theta synchronization together with much a smaller effect for alpha over the same mid-frontal regions during the 1PP observation of a highly embodied avatar. It is worth noting that the effect is much stronger, earlier, and longer-lasting for theta than for alpha oscillations, which can at least partly help to reconcile with the lack of frontal alpha effect reported in previous studies (van Driel et al., 2012). At any rate, we suggest that increased synchronization (particularly in the theta band) codes genuinely motoric aspects of an action error while the parietal effects in the alpha frequency monitors the attentional dimensions accompanying errors.

All in all, both ERN and cortical oscillations in the theta band seem to represent electrocortical markers of embodying the errors of an avatar. Importantly, neural activity in the frontal node of the performance monitor system (Carter et al., 1998) may underpin error processing through a complex coding of error detection and perspective taking that ultimately induces changes in one's sense of embodiment. It is worth noting that the increased synchronization of alpha oscillations may index the attentional load required for monitoring the occurrence of infrequent errors made by an avatar.

In conclusion, using immersive virtual reality, we created in observers a strong feeling of embodiment with an avatar seen from a first-person perspective. The main point of novelty is the association of ERN and medial–frontal theta power contingent upon observation of erroneous grasping of an avatar when it was perceived as highly embodied thanks to the assumption of a first-person perspective. This effect was found only for early and possibly automatic markers of error detection but not for electrocortical markers of late conscious awareness of the errors (i.e., the Pe) in the actions of others. Thus, the suggestion is made that, in our passive observation conditions, early markers of error detection are triggered by the supposedly unconscious attribution of salience to one's own errors. In contrast, late markers of error awareness index a somewhat disembodied coding of errors in both self and others. All in all, we expand current knowledge about the electrocortical correlates of endogenous activation of a remedial action and error monitoring system by highlighting the importance of detecting others' errors as though they were made by the self. It is worth noting that the association between 1PP and high embodiment may imply that a perceptual preparation to action occurs thanks to the multisensory hand-centered reference systems based on the activity of parietal and prefrontal cortices (Makin et al., 2008; Verhagen et al., 2013; Brozzoli et al., 2014). Thus, our findings are potentially important for improving the flexibility of current brain–computer interfaces aimed at optimizing the control of artificial agents (Chavarriaga et al., 2014).