Bioelectrical brain effects of one’s own voice identification in pitch of voice auditory feedback

Abstract

Control of voice fundamental frequency (F0) relies in part on comparison of the intended F0 level and auditory feedback. This comparison impacts “sense of agency”, or SoA, commonly defined as being the agent of one's own actions and plays a key role for self-awareness and social interactions. SoA is aberrant in several psychiatric disorders. Knowledge about brain activity reflecting SoA can be used in clinical practice for these disorders. It was shown that perception of voice feedback as one’s own voice, reflecting the recognition of SoA, alters auditory sensory processing. Using a voice perturbation paradigm we contrasted vocal and bioelectrical brain responses to auditory stimuli that differed in magnitude: 100 and 400 cents. Results suggest the different magnitudes were perceived as a pitch error in self-vocalization (100 cents) or as a pitch shift generated externally (400 cents). Vocalizations and neural responses to changes in pitch of self-vocalization were defined as those made to small magnitude pitch- shifts (100 cents) and which did not show differential neural responses to upward versus downward changes in voice pitch auditory feedback. Vocal responses to large magnitude pitch shifts (400 cents) were smaller than those made to small pitch shifts, and neural responses differed according to upwards versus downward changes in pitch. Our results suggest that the presence of SoA for self-produced sounds may modify bioelectrical brain responses reflecting differences in auditory processing of the direction of a pitch shift. We suggest that this modification of bioelectrical response can be used as a biological index of SoA. Possible neuronal mechanisms of this modification of bioelectrical brain response are discussed.

1. Introduction

1.1 Experience of one’s own control of vocalization

The control of vocalization has been of interest as it pertains to speech and singing for a number of years. In recent times, this control process has received additional attention through the advent of techniques that measure how a speaker reacts to perturbations in sensory feedback of vocal output. In most cases, vocal and neural responses were measured in response to alterations in auditory feedback of vocal pitch. One aspect of this technique that has not received full attention involves the “Sense of Agency” (SoA) related to a speaker’s voice.

The SoA, is referred to as the experience of oneself as the agent of one's own actions (agency), or in other words the sense that “I am the one who is causing or controlling a movement or change(s) in the outside word that I am perceiving” (for more details see: Gallagher, 2000; J. W. Moore, 2016). SoA plays a key role in self-awareness (Gallagher, 2000), body-awareness (Haggard & Tsakiris, 2009; Kannape & Blanke, 2012), and social interaction (De Jaegher & Froese, 2009; Ruys & Aarts, 2012). Recent advances in SoA studies (for more details see: David, 2012) suggest that SoA is a complex and dynamic, multilayered, and multifactorial phenomenon that involves multiple sensory systems, various brain areas, and complex interactions between these areas. Although several theoretical accounts for the SoA were suggested (J. W. Moore, 2016; Synofzik, Vosgerau, & Newen, 2008a, 2008b; Synofzik, Vosgerau, & Voss, 2013) and scientific investigations of SoA constitute a rapidly expanding field (David, Obhi, & Moore, 2015), the neuronal mechanisms of the SoA are not well understood (Blakemore & Frith, 2003; David, Newen, & Vogeley, 2008; Haggard & Chambon, 2012; J. Moore & Haggard, 2008; Weiss, Herwig, & Schutz- Bosbach, 2011). In the present study we examine the issue of SoA related to voice control.

The neuroscientific operationalizations of SoA during voluntary actions utilize explicit and/or implicit measures (J. W. Moore, 2016; J. W. Moore, Middleton, Haggard, & Fletcher, 2012; Saito, Takahata, Murai, & Takahashi, 2015; Synofzik, et al., 2008a). Explicit measures employ explicit judgments of whether a sensory event is caused by one’s own action or results from events in the outside word. The implicit aspect of agency is a non-conceptual, usually unconscious “low-level feeling” of being an agent that is closely related to self-caused action regulation or perceptual processing (Saito, et al., 2015). Therefore for the implicit measures of SoA, the experiences of action are simply tagged as self-caused or not (J. W. Moore, et al., 2012). There is no objective, biological index for the process of implicit agency registration. Based on our everyday life observations on SoA, this process was described as the following: “Rather than being explicitly aware of the motor representations, in moment to moment situations we experience self-agency by a rather diffuse sense of a coherent, harmonious ongoing flow of anticipations and sensory feedback” (Pacherie, 2001). It was hypothesized that this implicit level of internal action-monitoring might be defective in clinical cases (Pacherie, 2001). Therefore, bioelectrical indexes of SoA associated with involuntary, reflexive actions are needed for clinical practice. The advantages of bioelectrical indexes of SoA elicited by involuntary, reflexive actions over explicit judgments about SoA following voluntary actions are obvious in light of the results showing that SoA is not “given” when people produce apparently voluntary actions (D. Wegner & Sparrow, 2004). Specifically, it was reported that explicit SoA is a fragile construct that is dependent on inferences about which agent was the most probable cause of the action and what purpose or meaning the action had (J. W. Moore, Wegner, & Haggard, 2009; D. M. Wegner & Wheatley, 1999).

Meta-analysis of brain hemodynamic changes associated with SoA suggested that the temporo-parietal junction, pre-supplementary motor area (SMA) and pre-frontal cortex might be involved in a network underlying SoA (Sperduti, Delaveau, Fossati, & Nadel, 2011). In line with this, studies of voluntary actions also suggest that the pre-SMA (known for its role in action planning and initiation) might be one of the crucial brain areas for the subjective experience of agency (Cavazzana, Penolazzi, Begliomini, & Bisiacchi, 2015; Javadi, 2015; Kuhn, Brass, & Haggard, 2013; J. W. Moore, Ruge, Wenke, Rothwell, & Haggard, 2010). Apparently, connectivity between SMA and parietal brain areas might underlie the experience of SoA (Dogge, Hofman, Boersma, Dijkerman, & Aarts, 2014; Ritterband-Rosenbaum, Nielsen, & Christensen, 2014).

A critical function for SoA is evident from clinical studies showing that lesions to the pre-SMA in humans can lead to alien-limb syndrome, with patients demonstrating involuntary actions such as grasping nearby objects - even other people - without ever intending to do so (for review see: Della Sala, Marchetti, & Spinnler, 1991; Feinberg, Schindler, Flanagan, & Haber, 1992; Sarva, Deik, & Severt, 2014). Moreover, the capacity to experience SoA might be impaired in certain pathological conditions, in which patients regularly fail to identify their own actions or thoughts, by misattributing them to external sources. This phenomenon was extensively studied in patients with schizophrenia (Asai & Tanno, 2008; Frith & Done, 1989; Kircher & Leube, 2003; Maeda, et al., 2012; Robinson, Wagner, & Northoff, 2016; Spence, et al., 1997; Waters & Badcock, 2010) and results suggest that delusions of influence are based on imprecise internal predictions about the sensory consequences of one's actions (Ford & Mathalon, 2012; Frith, 1987; Synofzik, Thier, Leube, Schlotterbeck, & Lindner, 2010). Aberrant SoA was also found in patients with obsessive-compulsive disorder (Gentsch, Schutz-Bosbach, Endrass, & Kathmann, 2012) and in individuals with autism spectrum disorders (Chiu, et al., 2008; Uddin, et al., 2008). Also, borderline personality disorder (BPD) patients have dysfunctional self-image/identity (Leichsenring, Leibing, Kruse, New, & Leweke, 2011), and diagnostic criteria for BPD specify that a person must have a significant impairment in personality functioning in relation to self. Thus an identity disturbance appears to be a core and distinctive component of BPD with patients expressing a sense of “self-fragmentation” and “falling apart” (Wilkinson-Ryan & Westen, 2000).

Internal predictions about one’s actions are theoretical constructs that have been shown to be valuable in understanding control of several types of behaviors (Blakemore, Wolpert, & Frith, 2002; Frith, Blakemore, & Wolpert, 2000). For example, when a person is speaking, if the sound of the auditory feedback of one’s voice matches what the speaker intended to say, there is a modification of bioelectrical brain activity, which has been interpreted to mean that the brain recognizes the voice as self produced (Hawco, Jones, Ferretti, & Keough, 2009; Korzyukov, et al., 2015; Scheerer, Behich, Liu, & Jones, 2013). Moreover the study of SoA for speech indicates that auditory feedback of our own voice acts as a pathway for semantic monitoring (Lind, Hall, Breidegard, Balkenius, & Johansson, 2014). These results suggest that voice-related SoA might arise from central processing incorporating internal predictions of one’s vocalizations.

1.2 Control of self-vocalizations and “sense of agency”

One of the main ways in which vocalizations are controlled is by monitoring the acoustical properties of the voice to insure its accuracy and if necessary, to correct for errors in production (Chang, Niziolek, Knight, Nagarajan, & Houde, 2013; Greenlee, et al., 2011; Houde, Nagarajan, Sekihara, & Merzenich, 2002; Rauschecker, 2011). This process involves the use of SoA to identify self-vocalization, which requires the act of vocalizing and recognizing a close similarity between what the speaker intended to produce and the sensory feedback of the voice. At the behavioral level recognition of the auditory input as one’s own vocalization is acoustically expressed as an appropriate corrective involuntary change in vocalization (e.g., voice pitch) in response to an unintentional aberration in voice auditory feedback. If the auditory feedback is not recognized as self-produced, the sound is treated by the brain as any other environmental sound, there is no SoA experience and the motor corrective actions are small in magnitude.

In concert with these studies, it has been shown that bioelectric signals representing brain activity may be sensitive to the issue of self-produced vs. externally-produced speech sounds. Previous studies showed that the N1 ERP potential is suppressed for small pitch shifts (100 cents) while the N1 is not suppressed for large pitch shifts (400 cents) in voice auditory feedback. Moreover, large pitch shift magnitudes elicit ERPs that vary in magnitude according to stimulus direction (up or down pitch shift) whereas ERPs to small magnitude pitch shifts do not vary according to stimulus direction (Liu et al., 2011). Therefore, measuring ERPs in response to voice pitch-shifted auditory feedback can be used as an indicator of SoA.

Motor control models (for review see: Haggard, 2005) suggest that SoA seems to arise from the integration of information about predicted feedback, sensory information, and efferent copies of motor control neural signals. One of the important elements of this mechanism is the modulation of sensory input during self-generated movement. For example the study of finger-muscle activity in the SMA associated with SoA suggests that this brain area provides an efferent signal to modulate somatosensory activity during self-generated movement (Haggard & Whitford, 2004).

Taking into account studies suggesting that the vocal motor system can modulate auditory cortical processing (Behroozmand, et al., 2016; Behroozmand, et al., 2015; Chang, et al., 2013; Cogan, et al., 2014; Daliri & Max, 2016; Greenlee, et al., 2013; Jenson, Harkrider, Thornton, Bowers, & Saltuklaroglu, 2015; Sitek, et al., 2013) we hypothesized that during vocalization, SoA-related motor activity should alter functional characteristics of auditory perceptual neuronal networks. Specifically we hypothesized that there should be a difference between bioelectrical brain responses with and without the presence of SoA associated with vocalization. Quantification of this difference potentially can be used as a bioelectrical index of SoA in clinical practice.

Several Event Related Potentials (ERP) studies of SoA associated with vocalization have demonstrated that if the vocal sound is recognized as self- produced, there is suppression of neural activity (N1 ERP component) related to auditory processing of the sound (Behroozmand & Larson, 2011; Curio, Neuloh, Numminen, Jousmaki, & Hari, 2000; Flinker, et al., 2010; Ford, et al., 2001; Heinks-Maldonado, Mathalon, Gray, & Ford, 2005; Houde, et al., 2002). Similar to audition, SoA-related suppression mechanisms have been observed in the visual modality. For example a reduction of the visual N1 ERP response has been observed as a marker of a self-produced action as compared to externally generated feedback (Gentsch, Kathmann, & Schutz-Bosbach, 2012; Gentsch & Schutz-Bosbach, 2011). These studies support our hypothesis that if auditory processing is altered by the act of self-produced action, the functional characteristics of auditory processing neurons should be altered and bioelectrical manifestation of this change can be used as index of SoA. Our hypothesis can be tested with the voice pitch perturbation paradigm.

1.3 Voice perturbation paradigm

In this paradigm, subjects sustain vocalization at a steady pitch level (Burnett, Freedland, Larson, & Hain, 1998; Kawahara, 1994), and voice pitch auditory feedback is changed and fed back to the speaker in real time (Figure 1A). The change in pitch of the subject’s voice is interpreted as an error in voice production, and the subject automatically corrects for the error by changing their voice fundamental frequency (F0) in the opposite direction as the pitch shift stimulus. The F0 response occurs with a latency between 100 and 200 ms, and is usually a fraction (20–30 percent) of the error introduced via the feedback signal.

Figure 1.

Schematic illustration of Voice perturbation paradigm (A) and study design (B).

The pitch-shift stimuli and the ensuing vocal responses have been shown to elicit well-documented patterns of ERPs (Behroozmand, Karvelis, Liu, & Larson, 2009; Behroozmand, Korzyukov, & Larson, 2011; Z. Chen, et al., 2013; Korzyukov, Karvelis, Behroozmand, & Larson, 2012; H. Liu, Meshman, Behroozmand, & Larson, 2011; Scheerer, et al., 2013), which closely resemble the P50-N1-P2 complex that has been routinely recorded in numerous auditory ERP studies (Burkard, Eggermont, & Don, 2007). It is very likely that neuronal generators underlying conventionally recorded auditory ERPs are also involved in the generation of the P50-N1-P2 responses elicited by pitch-shifted voice feedback (Korzyukov, Karvelis, et al., 2012). However in contrast to conventional auditory ERP studies, the P50-N1-P2 components in the voice perturbation paradigm are recorded when a subject is vocalizing, i.e. performing a complex, goal oriented motor act. Thus, the dynamic contribution of motor control-related neuronal activity elicited by pitch-shifted voice feedback is likely to affect the ERP components.

If a pitch shift stimulus of 100 cents (100 cents equals one semitone on a musical scale) is presented at voice onset, the N1 ERP is greatly suppressed (Behroozmand & Larson, 2011; Houde, et al., 2002; Scheerer, et al., 2013). The suppression of the N1 ERP has been interpreted to mean that the feedback signal is self-produced, and this suppression is an indication of SoA. On the other hand, if the feedback signal is outside the self-vocalization pitch range, e.g., greater than 250 cents (Behroozmand & Larson, 2011; Hawco, et al., 2009; Scheerer, et al., 2013), the suppression is reduced, and in this case there is a reduction in the SoA.

2. Materials and Methods

2.1. Participants and procedures

Thirteen native English speakers (6 females and 7 males, mean age: 21.38 years Std. Dev.: 1.1) participated in the study. All subjects passed a bilateral pure-tone hearing screening test at 20 dB SPL (octave frequencies between 250–8000 Hz) and reported no history of neurological disorders. All study procedures including recruitment, data acquisition, and informed consent were approved by the Northwestern University Institutional Review Board, and subjects were monetarily compensated for their participation. Written informed consent was received from all participants.

During the test, subjects were seated in a sound-attenuated room and were instructed to vocalize and sustain the vowel “ah” for approximately 4–5 seconds at their conversational pitch and loudness levels whenever they felt comfortable, i.e., without a cue. They were informed that their voice would be played back to them during their vocalizations, and they were asked to ignore changes in frequency of their voice (pitch-shifts) (Figure 1A). While they were vocalizing, participants watched several static pictures that changed every 30 sec. Subjects typically paused for 2–3 sec between vocalizations to take a breath. During each vocalization, three pitch-shift stimuli (200 ms duration) with an inter- stimulus interval of 600–800 ms were presented. In the 100 cents block of trials, the auditory feedback was shifted to 100 cents, and in the 400 cents block, pitch shifts were 400 cents in magnitude. (Behroozmand & Larson, 2011; Hawco, et al., 2009). In both experimental blocks, pitch-shift stimulus directions (Upward and Downward) were randomly varied from trial to trial to +/−100 cents in one block and +/− 400cents in another (Figure 1B). During each block, subjects vocalized 80 times and 3 pitch-shift stimuli per vocalization were presented. The rise time of the pitch shift was 10–15 ms. A total of approximately 100 trials for each stimulus condition was obtained for each subject.

2.2. Data acquisition

Subjects’ voices were recorded with an AKG boomset microphone (model C420), amplified with a Mackie mixer (model 1202-VLZ3), and pitch-shifted through an Eventide Eclipse Harmonizer. The time delay from vocal onset, the duration, direction, and magnitude of pitch shifts were controlled by MIDI software (Max/MSP v.5.0 Cycling 74). Voice and auditory feedback were sampled at 10 kHz using PowerLab A/D Converter (Model ML880, AD Instruments) and recorded onto a laboratory computer utilizing Chart software (AD Instruments). Subjects maintained their conversational F0 levels and voice loudness at about 70–75 dB. The feedback signal (i.e., the subject’s pitch-shifted voice) was delivered through Etymotic earphones (model ER1-14A) at about 80–85 dB. The 10 dB gain between voice and feedback channels (controlled by a Crown amplifier D75) was used to partially mask air-born and bone-conducted voice feedback. The time between acquisition of one time sample and playback of the pitch-shifted version varied from 20 ms to 10 ms; the average was approximately 16 ms.

The electroencephalogram (EEG) signals were recorded from 64 sites on the subject’s scalp using an Ag-AgCl electrode cap (EasyCap GmbH, Germany) in accordance with the extended international 10–20 system (Oostenveld & Praamstra, 2001) including left and right mastoids. Recordings were made using the average reference montage in which outputs of all of the amplifiers are averaged, and this averaged signal was used as the common reference for each channel. Scalp-recorded brain potentials were low-pass filtered with a 400-Hz cut-off frequency (anti-aliasing filter), digitized at 2 kHz, and recorded using a BrainVision QuickAmp amplifier (Brain Products GmbH, Germany). Electrode impedances were kept below 5 kΩ for all channels. The electro-oculogram (EOG) signals were recorded using two pairs of bipolar electrodes placed above and below the right eye to monitor vertical eye movements and at the canthus of each eye to monitor horizontal eye movements.

2.3 Data Analysis

The voice and feedback signals were first processed in Pratt (Boersma, 2001) to obtain the F0 contours. These signals along with TTL pulses corresponding to the stimulus onset and magnitude were then processed in IGOR PRO (Wavemetrics, Lake Oswego, OR). The vocal F0 responses to +100 and −100 cents and +400 and −400 cents shifts were averaged separately across a 500 ms window from 100 ms before the shift to 400 ms after the shift for each individual participant and then grand-averaged across participants. The mean magnitude of F0 responses was measured for each individual participant’s data within the window centered over the peak of the F0 responses identified at the time of the peak in the grand average data. These measurements were made separately for “Up” and “Down” stimuli for vocal F0 responses to 100 cents and 400 cents shifts. Measured data were submitted for statistical evaluation with a 2 way ANOVA with the following factors: Direction (“Up” vs. “Down”) and Magnitude (100cents vs. 400cents). Because vocal responses to upward stimuli are usually downward, with negative values for the vocal response, the absolute values of all behavioral responses were calculated and used for statistical analysis. The absolute values were measured, which allowed us to compare response magnitudes for both positive and negative values of the averaged signal.

The recorded EEG signals were filtered offline using a band-pass filter with cut-off frequencies set to 1Hz and 50Hz (48 dB/oct) and then segmented into epochs ranging from 100 ms before and 400 ms after the onset of the pitch shift. Epochs with EEG or EOG amplitudes exceeding 75 µV were removed from data analysis. At least 90 epochs were averaged for ERP calculation for all participants. Separate ERP averages were made for “Up” and “Down” stimuli for each experimental block. A grand-average of the ERP was then calculated for each electrode, stimulus direction, and magnitude across all 13 subjects. Based on visual inspection of these grand-averaged ERP waveforms, the most prominent peaks and scalp distribution of these peaks amplitudes were identified.

The mean amplitudes, peak latencies and amplitudes were measured for each individual participant’s data within the window centered over the peak identified from the grand average data. The length of this window was also determined based on the length of the grand average response. The automated peak detection procedure within this time window was applied for each individual participant’s data measurements. These measurements were submitted for statistical analyses. Data were analyzed (STATISTICA, StatSoft) with a three- way Analysis of Variance (ANOVA; General-Linear model, Repeated Measures) with the following factors: Direction (“Up” vs. “Down”), Frontality, and Laterality.

3. Results

3.1. Behavioral Data

Figure 2 displays behavioral F0 responses for all experimental conditions. The peak times of the responses were about 300 ms post stimulus, and the onset times were typically about 100 ms. Mean F0 magnitudes were measured for each experimental condition across a 260–350 ms post-stimulus time window for each subject. Two-way ANOVAs of measured mean F0 magnitudes yielded a significant (F(1,12)=39.87, p<0.0005) main effect of “Magnitude” that was due to a larger mean F0 magnitude to the 100 cents (mean=12.14 cents) as compared to the 400 cents (mean=5.00 cents) stimuli. The main effect of “Direction” was not significant: F(1, 12)=1.6234, p=0.22674. The interaction between main effects was not significant: F(1, 12)=0.072, p=0.793.

Figure 2.

Behavioral (voice) results. Grand-averaged F0 traces across all participants separately for “100 cents” (red color) and “400 cents” (blue) stimuli. Vertical dashed lines represent pitch shift stimuli onset. F0 traces peaked above zero-line represent responses to “Down” stimuli, F0 traces peaked below zero-line represent responses to “Up” stimuli. Error bars represent standard error value.

3.2. ERP components

3.2.1. ERP sensitivity to pitch-shift direction

Relatively small behavioral responses to 400 cents indicate that 400 cents change in the pitch of auditory feedback during vocalization did not elicit a substantial motor reaction. This suggests that bioelectrical brain responses to 400 cents stimuli predominantly reflected auditory sensory activity rather than motor activity (for more details see: Korzyukov, et al., 2015) and SoA. As can be seen from the grand-averaged ERP data in Figure 3, the patterns of the P50-N1-P2 ERP responses for the 400 cents ”Up” and “Down” shifts were similar to the waveforms reported in conventional auditory ERP studies (Burkard, et al., 2007). The N1 ERP component to 400 cents peaked around a latency of 120 ms. As can be seen from the scalp distribution (Figure 3, maps), the N1 response to 400 cents stimuli was maximal over frontal-central electrodes and inverted its polarity at mastoids (Figure 3). Therefore the mean N1 amplitudes (measured across a 108–138 ms latency window) were measured from 15 electrodes centered over fronto-central areas in each subject and submitted for statistical evaluation that was performed with a repeated measurements three-way ANOVA (factors: Direction (“Up” vs “Down”); Frontality (Frontal electrodes: F3, F1, Fz, F2, F4, versus Fronto-central: FC3, FC1, FCz, FC2, FC4, versus Central: C3, C1, Cz, C2, C4), and Laterality (5 rows of electrodes from left to right).

Figure 3.

Grand-averaged ERP waveforms from 5 representative electrodes and ERP scalp distribution of N1 responses elicited by the 400 cents shift of the F0. All scalp distribution maps were calculated for the mean latency that is marked for the each map. Direction of the pitch shift stimuli is labeled with “Up” and “Down”. Grey vertical columns at the ERP waveforms indicate the latency interval chosen for the statistical evaluation. Yellow label at one representative (Fz) electrode illustrates a statistically significant difference between measures of mean amplitudes.

The ANOVA results yielded a significant interaction between “Direction” and “Frontality” (F(2,24) = 5.859, p<0.009). Subsequent Bonferroni and a Newman-Keuls post hoc test revealed that this was due to significantly (p<0.002) larger mean amplitudes of the N1 response at the Fronto-central and Central electrodes for the “Down” stimuli (for the Fronto-central mean: −2.7µV; for the Central mean: −2.3µV) as compared to “Up” stimuli (for the Fronto-central mean: − 2.0µV; for the Central mean: −1.4µV). The results from the ANOVA also yielded a significant interaction between “Direction” and “Laterality” (F(4,48) = 4.676, p<0.003). Subsequent Bonferroni and Newman-Keuls post hoc testing revealed that this was due to a significantly (p<0.001) larger mean amplitude of the N1 response to the “Down” stimuli at the Left-central (mean: −2.36µV), Central (mean:−2.7µV), Right-central (mean: −2.66µV) and Right (mean:−2.37µV) electrodes as compared to “Up” stimuli at the Left-central (mean: −1.8µV), Central (mean:−1.89µV), Right-central (mean: −1.88µV) and Right (mean:−1.79µV).

Since these differences in mean amplitudes of responses to 400 cents stimuli might be explained by the differences in the peak latency (see Figure 3), the peak latency and amplitudes were measured within the latency window of 90–160ms from 15 electrodes in each subject and submitted for the same statistical evaluation that was used for the mean amplitude measures. The results from ANOVA for the N1 latency yielded a significant main effect of “Direction” (F(1, 12) = 9.146, p<0.0106). The mean peak latency of the N1 response to the “Down” stimuli was 125 ms and to the “Up” stimuli was 118 ms. The results from the ANOVA for the N1 peak amplitude also yielded a significant main effect of “Direction” (F(1, 12) = 8.09, p<0.015). The mean peak amplitude of the N1 response to the “Down” stimuli was −3.24µV and to the “Up” stimuli was −2.67µV.

3.2.2 ERP in the presence of “Sense of Agency”

The most prominent ERP response to 100 cents shift in auditory feedback was the N1 component peaking around a latency of 150 ms (Figure 4). As can be seen from the scalp distribution of the N1 (Figure 4, maps) the N1 responses to 100 cents stimuli were maximal over frontal-central electrodes and inverted in polarity at the mastoids. Therefore the mean N1 amplitudes (measured across a 140–170 ms latency window) were measured from 15 electrodes centered over fronto-central areas in each subject and submitted for statistical evaluation that was performed with a repeated measures three-way ANOVA (factors: Direction (“Up” vs “Down”); Frontality (Frontal electrodes: F3, F1, Fz, F2, F4, versus Fronto-central: FC3, FC1, FCz, FC2, FC4, versus Central: C3, C1, Cz, C2, C4), and Laterality (5 rows of electrodes from left to right). The results from the ANOVA did not yield a significant main or interaction effect(s) of Direction (main effect: F(1,12)=3.09, p=0.1 mean amplitude for “Up”: −1.0µV, for “Down”: − 1.24µV).

Figure 4.

Grand-averaged ERP waveforms from 5 representative electrodes and ERP scalp distribution of N1 response elicited by the 100 cents shift of the F0. All scalp distribution maps were calculated for the mean latency that is marked for the each map. Direction of the pitch shift stimuli is labeled with “Up” and “Down”.

3.3 Difference between the presence and absence of “Sense of Agency”

In order to statistically evaluate difference in indexes of brain activity with and without SoA, additional calculations were done. Since ERP differences between “Up” and “Down” stimuli in the present study may be indicative of SoA, the differences between the mean N1 amplitude of the ERP elicited by “Up” and “Down” stimuli were taken as a SoA-related index of brain activity for statistical evaluation. These differences were calculated by subtracting the mean N1 amplitudes that were measured for “Down” stimuli from mean the N1 amplitude measured for “Up” stimuli. Subtractions were done separately for 100 and 400 cents experimental conditions for each of the 15 electrodes that were selected in previous stages of ERP data analyses. Values obtained as a result of the subtraction were subjected to multivariate analysis of variance (MANOVA) with stimulus magnitude of 100 or 400 cents as the main factor. The MANOVA indicated that there is a statistically significant difference in brain activity between 100 and 400 cents experimental conditions. Specifically, values of SoA-related index of brain activity measured from an array of electrodes in the vicinity of the vertex were significantly larger for 400 cents (presence SoA) than for 100 cents (absence of SoA). At the level of significance of 0.05, the difference between 100 and 400 cents experimental conditions was significant at Cz (for 100 cents: 0.58µV, for 400 cents: 1.25 µV; Wilks' Lambda = 0.651, F(1,12)=6.41, p=0.02632), and C2 (for 100 cents: 0.47µV, for 400 cents: 1.05 µV; Wilks' Lambda = 0.7, F(1,12)=5.136, p=0.04272) electrodes. At C4 (100 cents: 0.31µV; 400 cents: 0.745 µV) and C1 (100 cents: 0.388 µV, for 400 cents: 0.9388 µV) electrodes, the difference was close to the level of significance at C4, Wilks' Lambda = 0.72, F(1,12)=4.655, p=0.0519 and for C1, Wilks' Lambda = 0.736, F(1,12)=4.289, p=0.0605).

4. Discussion

4.1. Behavioral responses to upward and downward pitch shifts in auditory feedback during vocalizations

Although several previous pitch-shift studies manipulated the direction of pitch feedback perturbation (S. H. Chen, Liu, Xu, & Larson, 2007; Larson, Altman, Liu, & Hain, 2008; Larson, Burnett, Bauer, Kiran, & Hain, 2001; H. Liu & Larson, 2007), the directional effect on vocal response magnitudes has been inconsistent. That is, in some studies there were larger response magnitudes with upward stimuli, and in some studies magnitudes were larger with downward stimuli. For example it was reported that only adults 51–60 years of age produced larger responses to upward stimuli as compared to downward stimuli (P. Liu, Chen, Jones, Huang, & Liu, 2011).

The main behavioral result of our study replicates findings from other studies (Behroozmand & Larson, 2011; Hawco, et al., 2009; Korzyukov, et al., 2015; Scheerer, et al., 2013) demonstrating that when the F0 magnitude of feedback perturbations is greater than 250 cents, vocal response magnitudes decreased (Scheerer, et al., 2013). Taken together the results of the present and previous studies strongly suggest that feedback perturbations larger than 250–300 cents are perceived as not self-generated but as an acoustical change in the environment. Thus our behavioral results indicate that 400 cents stimuli were perceived as non-self generated auditory stimuli with little vocal-motor involvement in the generation of ERPs to 400 cents stimuli. This means that any functional characteristics of ERPs elicited by 400 cents stimuli are not affected by SoA since there was no substantial self initiated motor activity related to vocalization error correction.

4.2. Functional characteristics of ERPs elicited by 400 cents pitch direction change

Acoustical changes in the pitch of auditory input used in the present study closely resemble pitch changes in the frequency modulated (FM) sweep stimuli used in numerous studies and that are common components of species-specific vocalizations. FM sweep-selective neurons are present in the auditory cortex of every species examined and are apparently involved in processing species- specific vocalizations (Hart, Palmer, & Hall, 2003; Heil & Irvine, 1998; Heil, Langner, & Scheich, 1992; Kowalski, Versnel, & Shamma, 1995; Mendelson & Cynader, 1985; Mendelson, Schreiner, Sutter, & Grasse, 1993; Nelken & Versnel, 2000; Suga, 1965; Taniguchi, Niwa, Wong, & Suga, 1986; Tian & Rauschecker, 1994, 2004).

In light of these findings, pitch-direction-related differences in amplitude and latency of the N1 ERP responses elicited by the 400 cents pitch shift perturbations (PSP) in the present study can be interpreted as bioelectrical manifestations of a difference in sensory brain activity between upward and downward pitch-changes in auditory stimuli. Although the 400 cents PSP in the present study occurred during vocalizations, behavioral responses to upward and downward pitch shifts in auditory feedback during these vocalizations indicate that 400 cents stimuli were not perceived as self-generated. Therefore neural processing of upward and downward pitch-changes of 400 cents stimuli does not elicit SoA effects, and should be equivalent to generalized brain processing of upward and downward pitch-changes in the auditory environment independent of self-initiated actions.

Although the exact cortical mechanisms for brain processing of upward and downward pitch-changes in the auditory environment remain elusive several studies suggest that degree of pitch-changes might be processed independently from the direction of change (for review see: Walker, Bizley, King, & Schnupp, 2011). It was shown that cortical mechanisms that detect pitch differences are neuroanatomically dissociable from those mediating pitch direction discrimination (Tramo, Cariani, Koh, Makris, & Braida, 2005; Tramo, Shah, & Braida, 2002). Similarly, an impairment in pure tone frequency direction judgments, but not frequency change detection thresholds, has been observed in patients with surgical lesions (Johnsrude, Penhune, & Zatorre, 2000). Studies that examine pitch perception in neurological patients (Stewart, von Kriegstein, Warren, & Griffiths, 2006) suggested that pitch difference detection is associated with subcortical structures and primary auditory cortex, whereas pitch direction discrimination is more often associated with lateral Heschl’s gyrus. Together, these studies indicate that the difference in pitch change direction cannot be attributed to only 400 cents pitch change but should be inherent to a broader auditory frequency range including 100 cents pitch change in the auditory environment.

4.3. Effect of “sense of agency” on ERP differences for direction of pitch changes

Previous studies showed that the vocal motor control system actively monitors voice auditory feedback for the purpose of correcting for errors in output (Burnett, et al., 1998; Hain, et al., 2000). Thus, brain systems underlying SoA should continuously monitor acoustic properties of the auditory feedback of the voice in order to validate the presence of SoA for vocalization-related auditory input. This process of monitoring apparently involves motor area(s) in order to be able initiate corrective motor action(s) as fast as possible (Cavazzana, et al., 2015; Javadi, 2015; Kuhn, et al., 2013; J. W. Moore, et al., 2010). The present data suggest that as soon as a perturbation in voice auditory feedback is recognized as not self-produced (400 cents pitch change), the SoA-related modulation (for analogy see: Gentsch & Schutz-Bosbach, 2011) of auditory brain activity is stopped, and this results in a SoA-related ERP difference between ERPs elicited by 400 cents pitch change (Figure 3) and the ERPs elicited by 100 cents (Figure 4). Basically the difference between brain responses to the auditory processing of non-self generated auditory feedback (400 cents stimuli) and the integrated auditory-vocal-motor system (100 cents stimuli) can be seen as the difference between directionality effects on the ERPs shown in Figures 3 and 4.

Since 100 cents stimuli in the present study were recognized as an error in self-produced vocalizations (evident from F0 responses, Figure 2), the involuntarily elicited corrective motor reactions are indicative of a SoA presence. When 100 cents PSPs were presented and the SoA was present (Figure 4), there were no PSP directionality effects on the ERPs. These results are in line with previous studies that used a similar paradigm and demonstrated larger N1 ERP responses to downward compared to upward stimuli for 200 and 500 cents stimulus magnitudes (H. Liu, et al., 2011). In this previous study, voice pitch feedback was shifted uniformly with a fixed direction (either upward or downward) 150 times within a block of trials. The authors hypothesized that an implicit expectation about the future stimulus direction might have caused downward stimuli to elicit larger behavioral and neural responses as compared to upward stimuli (H. Liu, et al., 2011). Subsequent studies using a similar paradigm (Korzyukov, Sattler, Behroozmand, & Larson, 2012; Scheerer & Jones, 2014) indeed provided evidence that neuronal processing of auditory feedback during vocalization can be affected by the implicit expectation about future stimulus characteristics. Taking these results into account, in the present study we excluded expectation as an experimental factor that overlaps with the directionality effect and presented our stimuli in a random sequence of stimulus directions. This modification in the experimental design allowed us to reveal additional findings that expand the results of Liu and colleagues (H. Liu, et al., 2011). Since in the present study implicit expectation about the future stimulus direction could not affect our results, we suggest that neuronal processes associated with the implicit aspect of SoA during vocalization can be probed with bioelectrical indexes reported in the present study and by Liu and colleagues (H. Liu, et al., 2011).

Apparently this SoA manifestation is mediated in the brain by a compensatory motor control-related enhancement of auditory cortex activity reported by Chang and colleagues (Chang, et al., 2013) that might modulate activity of FM-responsive neurons. Findings showing that rapid changes in activity of auditory cortex neurons can be triggered by stimulation of frontal cortical areas (Winkowski, Bandyopadhyay, Shamma, & Kanold, 2013) allow us to suggest that SoA-related rapid modulation of FM-responsive neurons also might be a factor to explain the results. Thus, we hypothesize that SoA-related modulation of FM-responsive neurons leads to rapid plasticity of FM-responsive auditory cortex neurons resulting in alternation between the ERP characteristics found in the present study for the 400 cents and the 100 cents PSPs. This explanation is supported by recent findings of rapid plasticity of FM-responsive auditory cortex neurons to optimize task performance (Yin, Fritz, & Shamma, 2014) that is in line with reports of rapid task-related plasticity of neurons in auditory cortex (Fritz, Elhilali, & Shamma, 2005).

It should be noted that not only can the motor component of SoA reduce the ERP difference for direction of FM-sweep stimuli, but apparently other cognitive functions also can reduce/eliminate a general preference of FM-responsive neurons for tone sweep direction. For example in an ERP study (Sanders & Poeppel, 2007) when subjects were instructed to pay special attention to auditory input and thus voluntarily modulate activity of auditory cortex by the attentional system, the preference of the auditory system for tone sweep direction was not found in humans.

Hypothetically, it can be assumed, that rapid plasticity of FM-responsive auditory cortex neurons resulting in the alternation between the ERP characteristics found in the present study for the 400 cents and the 100 cents PSPs might be associated with perturbation magnitude. However to the best of our knowledge there is no evidence that ERP responses reflecting differences in auditory processing of the direction of a pitch shift can be determined by the perturbation magnitude. Even if an interaction between the perturbation magnitude and ERP responses reflecting processing of the direction can be found, the SoA will always be present in 100 cents PSP trials and absent in 400 cents trials.

Future studies of N1 modulation during vocalization on patients who fail to identify their own actions or thoughts, by misattributing them to external sources are needed to investigate effects found in the present study. Such future research may validate the implementation of the proposed bioelectrical indices of implicit aspect of SoA in clinical and healthy populations.

Highlights.

Sense of Agency;

Voice control mechanisms;

Voice perturbation paradigm;

Event Related Potentials;

Auditory processing of pitch change direction.