Processing of auditory novelty in human cortex during a semantic categorization task

Abstract

Auditory semantic novelty – a new meaningful sound in the context of a predictable acoustical environment – can probe neural circuits involved in language processing. Aberrant novelty detection is a feature of many neuropsychiatric disorders. This large-scale human intracranial electrophysiology study examined the spatial distribution of gamma and alpha power and auditory evoked potentials (AEP) associated with responses to unexpected words during performance of semantic categorization tasks. Participants were neurosurgical patients undergoing monitoring for medically intractable epilepsy. Each task included repeatedly presented monosyllabic words from different talkers (“common”) and ten words presented only once (“novel”). Targets were words belonging to a specific semantic category. Novelty effects were defined as differences between neural responses to novel and common words. Novelty increased task difficulty and was associated with augmented gamma, suppressed alpha power, and AEP differences broadly distributed across the cortex. Gamma novelty effect had the highest prevalence in planum temporale, posterior superior temporal gyrus (STG) and pars triangularis of the inferior frontal gyrus; alpha in anterolateral Heschl’s gyrus (HG), anterior STG and middle anterior cingulate cortex; AEP in posteromedial HG, lower bank of the superior temporal sulcus, and planum polare. Gamma novelty effect had a higher prevalence in dorsal than ventral auditory-related areas. Novelty effects were more pronounced in the left hemisphere. Better novel target detection was associated with reduced gamma novelty effect within auditory cortex and enhanced gamma effect within prefrontal and sensorimotor cortex. Alpha and AEP novelty effects were generally more prevalent in better performing participants. Multiple areas, including auditory cortex on the superior temporal plane, featured AEP novelty effect within the time frame of P3a and N400 scalp-recorded novelty-related potentials. This work provides a detailed account of auditory novelty in a paradigm that directly examined brain regions associated with semantic processing. Future studies may aid in the development of objective measures to assess the integrity of semantic novelty processing in clinical populations.

Introduction

The human brain is adept at identifying new sounds in the environment and determining their potential ecologic relevance (Escera and Malmierca, 2014; Justo-Guillen et al., 2019). Detection of a potentially meaningful new sound is followed by attempts to place the sound into a pre-existing semantic category (Nobre and McCarthy, 1994; Mecklinger et al., 1997). Novelty detection in the auditory domain is of fundamental importance for learning across the lifespan (Nordt et al., 2016) and is present in early infancy (Katus et al., 2023). Aberrant detection of novel sounds is a feature of many neuropsychiatric disorders, including autism and schizophrenia, as well as altered states of consciousness, e.g., sleep, delirium, and coma (Lepistö et al., 2005; Higuchi et al., 2014; Morlet & Fischer, 2014; Strauss et al., 2015; Michie et al., 2016; Hudac et al., 2018; Perrottelli et al., 2021).

Auditory semantic novelty has typically been studied using either auditory-only paradigms with non-speech environmental sounds (e.g., Opitz et al., 1999) or audiovisual oddball paradigms with speech stimuli (e.g., Parmentier, 2008). Functional neuroimaging studies have revealed greater cortical activation in response to novel meaningful environmental sounds than to sounds that are not meaningful (e.g., tones) (Opitz et al., 1999). Behavioral studies have indicated that speech processing relies on specialized systems distinct from those involved in processing of music and other environmental sounds (Moskowitz et al., 2020). Likewise, results from audiovisual paradigms reflect complex interactions between responses to stimuli of both sensory modalities and associated selective attention mechanisms (Parmentier, 2008; see also Salmela et al., 2018). Thus, it is premature to extrapolate findings derived during identification of novel environmental stimuli or those obtained in audiovisual paradigms to auditory novelty detection associated with speech stimuli.

Most studies devoted to identifying the brain areas associated with novelty detection and semantic classification used electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) (e.g., Mecklinger et al., 1997, Opitz et al., 1999; Kotz et al., 2007). Auditory novelty detection within the context of semantic processing has been associated with several event-related potential (ERP) components, including P3a and N400. P3a, peaking at ~300 ms after the onset of the novel stimulus, is an endogenous automatic preattentive response, thought to have a prefrontal generator (Polich, 2007; Friedman et al., 2009). The clinical importance of examining P3a with speech stimuli is exemplified by a greater degree of P3a impairment for speech than non-speech stimuli in children with autism (Lepistö et al., 2005). N400 is an attention-dependent neural response that represents semantic classification of novel sounds, including non-speech stimuli (Opitz et al., 1999; Kotz et al., 2007). N400 elicited by speech stimuli has been suggested to be a useful biomarker for the diagnosis and management of aphasia (Meechan et al., 2021).

The current intracranial EEG (iEEG) study is an expansion of our previous work examining brain regions subserving semantic classification (Steinschneider et al., 2014; Nourski et al., 2017). These studies examined semantic processing using repeatedly presented words (“common”, COM) associated with different semantic categories. The present study focused on responses to additional, single-exemplar words (“novel”, NOV) that were intermixed with the repeatedly presented COM stimuli. Extensive electrode coverage (over 5500 recording sites across 38 participants) permitted detailed characterizations, including spatial distribution of auditory novelty responses during performance of the same task as used in the previous studies (Steinschneider et al., 2014; Nourski et al., 2017). Novelty responses to unexpected words were predicted to have a higher prevalence along the ventral than dorsal auditory cortical processing stream given the semantic nature of the task (Hickok & Poeppel, 2004; Rauschecker & Scott, 2009). For the same reason, it was predicted that there would be a left hemispheric bias (cf. Nourski et al., 2017), in contrast to a right hemispheric bias for novel non-speech environmental sounds (Mecklinger et al., 1997; Opitz et al., 1999). Additionally, we focused on examining the relationship between bottom-up and top-down cortical representation of auditory novelty and behavioral task performance. To that end, we used broadband gamma (30-150 Hz) activation and alpha (8-14 Hz) suppression as physiologic markers and examined the degree to which task performance was a function of top-down modulation. These complementary measures are sensitive indices of cortical function and represent fundamental features of predictive coding (Billig et al., 2019; Bastos et al., 2020; Nourski et al., 2021b, 2022). Gamma activation reflects feedforward signaling at the cortical level, whereas alpha suppression represents the release from inhibition due to diminished feedback from higher cortical areas (Fontolan et al., 2014; Billig et al., 2019; Bastos et al., 2020). Finally, building upon the groundbreaking work of Halgren et al. (1995, 1998), the present study examined averaged evoked potentials (AEP) to facilitate identification of the putative generators of non-invasively recorded responses.

Methods

Participants

Participants were 38 neurosurgical patients (18 female, age 18-55 years old, median age 31.5 years old) diagnosed with medically refractory epilepsy who were undergoing chronic iEEG monitoring to identify potentially resectable seizure foci. Demographic, iEEG electrode coverage, and task performance data for each participant are presented in Supplementary Table 1. Thirty-three participants were right-handed, four (R292, R334, R456, R717) were left-handed, and one (L416) was ambidextrous. The prefix of the participant code indicates the hemisphere with predominant electrode coverage: L for left (N = 19), R for right (N = 17), B for bilateral (N = 2). Thirty-two participants were left hemisphere language dominant per Wada testing, two (R292, R717) were right hemisphere dominant, and two (L640 and L702) had bilateral language dominance. In the two remaining participants (R322, R567, both right-handed), a Wada test was not clinically warranted, and thus language dominance was not formally established.

All participants were native English speakers except for L275 (a 30-year-old native Bosnian speaker) and R559 (a 27-year-old native Arabic speaker); both had English formal education and over 10 years of exposure. Out of the 38 participants, 26 had pure-tone thresholds within 25 dB HL, between 250 Hz and 4 kHz. Participant L372 had a left 30 dB HL notch at 750 Hz; R561 had a right mild notch at 2 kHz; L307, R334, R376 had a right mild notch at 4 kHz; R369 had a left mild notch at 4 kHz; L357 and L525 had left mild high frequency (≥4 kHz) loss; R728 had a bilateral mild high frequency loss; L423 had bilateral moderate (45-50 dB HL) high frequency (≥4 kHz) loss; R717 had severe (75 dB HL) loss at 2 kHz bilaterally and moderate-to-moderately severe loss at 4 kHz (left: 65 dB HL, right: 55 dB HL). Word recognition scores, evaluated as part of the audiometric assessment, were 92/92% (left/right ear) in participant L275, 96/88% in R334, 96/92% in R369, 88/100% in R399, 92/92% in L423, 92/100% in R456, 96/92% in L525, 92/96% in R559, 100/92% in L702, 100/88% in R717, and ≥96% in all other tested participants. Speech reception thresholds were within 20 dB HL in all tested participants, including those with tone audiometry thresholds outside the 25 dB HL criterion. All participants underwent a preoperative neuropsychological evaluation, and no cognitive deficits that should impact the findings presented in this study were identified.

Research protocols were approved by the University of Iowa Institutional Review Board and the National Institutes of Health. Written informed consent was obtained from all participants. Research participation did not interfere with acquisition of clinical data, and participants could rescind consent at any time without interrupting their clinical evaluation.

Stimuli and procedure

Experimental stimuli were monosyllabic words and complex tones presented in three target detection task blocks (Steinschneider et al., 2014; Nourski et al., 2017, 2021b, 2022). The common (COM) words were “cat”, “dog”, “five”, “ten”, “red” and “white”, obtained from the TIMIT speech corpus (Garofolo et al., 1993) and LibriVox (http://librivox.org/) audiobook catalog. The experimental procedure is schematically summarized in Figure 1. Every COM word was presented 20 times in each block and was spoken by a different talker each time. This resulted in a total of 20 unique exemplars, 14 spoken by different male and 6 by different female talkers. Additionally, blocks 2 and 3 each introduced 10 novel (NOV) words (5 target and 5 non-target) that were not previously presented in earlier block(s). The NOV words were “bat”, “duck”, “goat”, “pig”, “rat”, “bite”, “give”, “read”, “run” and “walk” in block 2, and “four”, “nine”, “one”, “six”, “three”, “hit”, “kiss”, “make”, “sit” and “wait” in block 3. The NOV words were spoken only by male talkers. These words, while used frequently in the English language, were unexpected in the context of the task, and therefore considered novel. The complex tones included four harmonics of fundamental frequencies of 125 Hz (28 trials) and 250 Hz (12 trials), approximating fundamental frequencies of male and female talkers, respectively. Responses to the tone stimuli have been presented elsewhere (Nourski et al., 2021b, 2022).

Figure 1.

Schematic of the experimental procedure. Exemplar random stimulus sequences presented in the three experimental blocks (tone, animal, and number target detection tasks). Block 1 was used to establish participants’ familiarity with the task and COM words. Only data from blocks 2 and 3 (words “cat”, “dog”, “five” and “ten” spoken by male talkers and all NOV words) are presented in the study.

Targets were either the complex tones (block 1) or words (both COM and NOV) belonging to the specific semantic categories of animals or numbers (blocks 2 and 3, respectively). At the beginning of each block, the participant was told by the experimenter which category was the target. The participant’s task was to push a response button on a Microsoft Sidewinder game controller (Microsoft, Redmond, WA) or a USB numeric keypad whenever they heard a target. The participants were instructed to press the button using the hand ipsilateral to the hemisphere in which the majority of electrodes were implanted. This was done to reduce contributions of preparatory, motor, and somatosensory responses associated with the button press to the recorded neural activity.

Stimuli were presented via insert earphones (ER4B, Etymotic Research, Elk Grove Village, IL) integrated into custom-fit earmolds. All stimuli had a duration of 300 ms, were normalized to the same root-mean-square amplitude and were presented in random order with an inter-stimulus interval chosen within a Gaussian distribution (mean 2 s; SD = 10 ms). Prior to each experiment, the participants were presented with a preview of a Block 1 stimulus sequence (i.e., not containing NOV stimuli) to ensure volume was at a comfortable level (typically, 55-65 dB SPL) and that the participants understood the task requirements. Experiments were carried out in a dedicated electrically shielded suite in the University of Iowa’ Institute for Clinical and Translational Science’s Clinical Research Unit or in the Stead Family Children’s Hospital. Experiments were carried out at least 3 hours after the most recent ictal event. The participants were fully awake and alert and were reclining in a hospital bed or an armchair during the experiments.

Recordings

Recordings were obtained using either subdural and depth electrodes, or depth electrodes alone, based on clinical requirements, as determined by the team of epileptologists and neurosurgeons. Details of electrode implantation, recording and iEEG data analysis have been described previously (e.g., Nourski and Howard, 2015). Electrode arrays were manufactured by Ad-Tech Medical (Racine, WI) or PMT (Chanhassen, MN). Subdural arrays, implanted in 26 participants out of 38, consisted of platinum-iridium discs (2.3 mm diameter, 5-10 mm inter-electrode center-to-center distance), embedded in a silicon membrane. Stereotactically implanted depth arrays included between 4 and 14 cylindrical contacts along the electrode shaft, with 2.2-10 mm inter-electrode distance. A subgaleal electrode, placed over the cranial vertex near midline, was used as a reference in all participants.

Data acquisition was done by a TDT RZ2 real-time processor (Tucker-Davis Technologies, Alachua, FL) in participants L275 through L357 and by a Neuralynx Atlas System (Neuralynx, Bozeman, MT) in participants L369 through R728. Recorded data were amplified, filtered (0.7–800 Hz bandpass, 5 dB/octave rolloff for TDT-recorded data; 0.1–500 Hz bandpass, 12 dB/octave rolloff for Neuralynx-recorded data), and digitized at a sampling rate of 2034.5 Hz (TDT) or 2000 Hz (Neuralynx).

Analysis

Anatomical localization of recording sites relied on post-implantation structural MRI and computed tomography (CT). Images were first aligned with pre-operative T1 MRI scans using linear co-registration implemented in FSL (FLIRT) (Jenkinson et al., 2002). Accuracy of electrode localization within the pre-operative MRI space was refined using three-dimensional non-linear thin-plate spline warping to correct for post-operative brain shift and distortion (Rohr et al., 2001). The warping was constrained within 50-100 control points manually selected throughout the brain, which were visually aligned to anatomical landmarks in the pre- and post-implantation scans.

Each recording site was assigned to one of 56 regions of interest (ROIs) based on anatomical reconstructions of electrode locations in each participant. The ROIs were organized into 8 ROI groups (Fig. 2; see also Table 1). ROI assignment was based on automated parcellation of cortical gyri, implemented in the FreeSurfer software package (Destrieux et al., 2010, 2017) and then confirmed by visual inspection of anatomical reconstruction data. For recording sites in Heschl’s gyrus, delineation of the border between core auditory cortex and adjacent non-core areas [posteromedial (HGPM) and anterolateral (HGAL) portions of Heschl’s gyrus, respectively] was performed in each participant using physiological criteria (Brugge et al., 2009; Nourski et al., 2016). Recording sites were assigned to HGPM if they exhibited phase-locked responses to 100 Hz click trains characteristic of HGPM (Brugge et al., 2009). Additionally, linear correlation coefficients (Pearson’s r) between AEPs recorded from adjacent sites were examined to identify discontinuities that could reflect a transition from HGPM to HGAL. STG was subdivided into posterior and middle non-core auditory cortex ROIs (STGP and STGM), and anterior auditory-related anterior ROI (STGA) using the transverse temporal sulcus and ascending ramus of the Sylvian fissure as boundaries. The insula was subdivided into posterior (InsP) and anterior (InsA) portions (long and short insular gyri, respectively). Middle and inferior temporal gyrus were each divided into posterior, middle, and anterior ROIs (MTGP, MTGM, MTGA, ITGP, ITGM, ITGA) by dividing the gyrus into three approximately equal-length thirds. Angular gyrus was divided into posterior (AGP) and anterior (AGA) ROIs along the angular sulcus. Anterior cingulate cortex (ACC) was identified by automatic parcellation in FreeSurfer and was considered as part of the prefrontal ROI group (i.e., separately from the rest of the cingulate gyrus). Recording sites identified as seizure foci, characterized by excessive noise, and depth electrode contacts localized outside cortical gray matter, were excluded from analyses and are not included in Figure 2b, 2c, Table 1, and Supplementary Table 1. In total, 5511 recording sites were studied across the 38 participants.

Figure 2.

ROIs and electrode coverage in all 38 participants. a: ROI parcellation scheme. b: Locations of recording sites, determined for each participant individually and color-coded according to the ROI group, are plotted in MNI coordinate space and projected onto the Freesurfer average template brain for spatial reference. Projections are shown in the lateral, top-down (superior temporal plane), ventral and mesial views (top to bottom). Sites in the amygdala, caudate nucleus, frontal operculum, hippocampus, parietal operculum, and putamen are not shown but were still analyzed. c: ROI groups, ROIs, and abbreviations used in the present study. Number of participants and number of sites that contributed to each ROI are provided in the two right-most columns (N and n, respectively).

Table 1.

Prevalence of semantic novelty effects across ROIs.

ROI group	ROI	n sites	Gamma		Alpha		AEP
			n	%	n	%	n	%
HGPM	HGPM	145	8	5.51	27	18.6	31	21.4
STP	HGAL	86	13	15.1	24	27.9	16	18.6
	PP	69	10	14.5	13	18.8	14	20.3
	PT	69	14	20.3	12	17.4	12	17.4
STG	STGM	175	20	11.4	25	14.3	13	7.43
	STGP	247	49	19.8	32	13.0	19	7.69
Ventral	MTGA	107	8	7.48	6	5.61	4	3.74
	MTGM	239	19	7.95	24	10.0	14	5.86
	MTGP	285	29	10.2	16	5.61	15	5.26
	STGA	51	3	5.88	11	21.6	4	7.84
	STSL	137	6	4.38	23	16.8	28	20.4
	STSU	62	3	4.84	8	12.9	5	8.06
Dorsal	AGA	111	13	11.7	6	5.41	3	2.70
	AGP	61	1	1.64	3	4.92	1	1.64
	SMG	254	35	13.8	21	8.27	20	7.87
Prefrontal	ACC	36	4	11.1	4	11.1	3	8.33
	FMG	10	2	20.0	0	0	0	0
	IFGop	85	12	14.1	15	17.6	9	10.6
	IFGor	29	2	6.90	3	10.3	2	6.90
	IFGtr	138	25	18.1	18	13.0	19	13.8
	MFG	326	44	13.5	21	6.44	40	12.3
	OG	341	51	15.0	28	8.21	32	9.38
	SFG	125	10	8.00	5	4.00	9	7.20
	TFG	46	4	8.70	3	6.52	6	13.0
Sensorimotor	ParaCL	12	0	0	0	0	1	8.33
	PostCG	228	11	4.82	12	5.26	19	8.33
	PreCG	289	26	9.00	25	8.65	28	9.69
Other	Amyg	119	2	1.68	7	5.88	14	11.8
	Caud	11	0	0	3	27.3	0	0
	CingMA	31	2	6.45	6	19.4	6	19.4
	CingMP	18	1	5.56	0	0	1	5.56
	CingPD	19	2	10.5	1	5.26	1	5.26
	Cun	16	0	0	5	31.3	0	0
	FG	151	15	9.93	19	12.6	5	3.31
	fOperc	11	2	18.2	1	9.09	2	18.2
	GR	63	10	15.9	3	4.76	6	9.52
	Hipp	118	6	5.08	12	10.2	11	9.32
	InsA	66	8	12.1	12	18.2	13	19.7
	InsP	86	2	2.33	12	14.0	17	19.8
	IOG	13	0	0	0	0	0	0
	IPS	10	1	10.0	0	0	1	10.0
	ITGA	123	10	8.13	12	9.76	10	8.13
	ITGM	98	9	9.18	4	4.08	3	3.06
	ITGP	75	10	13.3	8	10.7	3	4.00
	LingG	37	1	2.70	0	0	0	0
	MOG	95	4	4.21	10	10.5	8	8.42
	OP	10	0	0	1	10	0	0
	PHG	107	9	8.41	8	7.48	5	4.67
	PMC	126	6	4.76	17	13.5	13	10.3
	pOperc	11	0	0	1	9.09	4	36.4
	PreCun	32	0	0	2	6.25	2	6.25
	Put	13	1	7.69	2	15.4	3	23.1
	SOG	14	1	7.14	0	0	0	0
	SPL	38	2	5.26	0	0	3	7.89
	SubcG	7	0	0	3	42.9	1	14.3
	TP	230	29	12.6	17	7.39	19	8.26

ROIs with insufficient electrode coverage (<20 sites) are listed in italics and are not depicted in Figure 5.

Behavioral task performance was characterized in terms of hit rates (percentages of correctly identified target words), sensitivity (d′ = Z_hit-Z_{false alarm}, where Z is the inverse of the cumulative distribution function of the normal distribution), and reaction times (RT). They were calculated for all target stimuli (combined COM and NOV), and separately for COM and NOV targets. Relationships between overall hit rates and d′ values were examined using a Spearman rank correlation test.

Relationships between overall hit rates and participants’ age, audiometric evaluation results (0.5-2 kHz pure tone average thresholds, speech reception thresholds, word recognition scores) averaged between the left and the right ear, age at epilepsy onset, duration of epilepsy, time since electrode implantation, and time since most recent ictal event were examined using Spearman rank correlation tests. The relationships of overall hit rates with participants’ sex and localization of seizure focus to the left or right hemisphere were examined using two-tailed Wilcoxon rank sum tests. A complementary analysis limited to NOV target hit rates was performed as detailed above.

The difference in hit rates between COM and NOV targets was examined using a two-tailed Wilcoxon signed-rank test. RT difference between COM and NOV targets was examined using a linear mixed effects model with stimulus (COM or NOV) as a fixed effect and participant as a random intercept. The model was implemented in MATLAB R2023b (MathWorks, Natick, MA, USA).

Analysis of iEEG data was performed using custom software written in MATLAB programming environment. Recordings were downsampled to 1000 Hz for computational efficiency and de-noised using a demodulated band transform-based procedure (Kovach and Gander, 2016). Voltage deflections exceeding five standard deviations from the within-block mean for each recording site were considered artifacts, and trials containing such deflections were excluded from further analysis.

Single-trial local field potential waveforms were baseline-corrected by subtracting mean voltage in the 100 ms time window immediately preceding stimulus onset and averaged to obtain the AEP. Time-frequency analysis was implemented using the demodulated band transform-based algorithm. This was done by computing the discrete Fourier transform over the entire recording block, using bandwidths of 1, 2, 4, 10, and 20 Hz for theta (4-8 Hz), alpha (8-14 Hz), beta (14-30 Hz), low gamma (30-70 Hz) and high gamma (70-150 Hz) frequency ranges, respectively. Event-related band power (ERBP) was calculated by log-transforming power for each center frequency and normalizing it to a baseline value measured as the mean power in the prestimulus reference interval (100 to 200 ms prior to stimulus onset).

Quantitative analyses of iEEG data focused on broadband (30-150 Hz) gamma and alpha ERBP and the AEP. The choice of broadband gamma was motivated by previous work which identified concomitant power changes in both low and high gamma band in auditory and auditory-related cortex (Nourski et al., 2021b, 2022). Auditory novelty effects were defined as significant differences between responses to all NOV trials and responses to the COM words “cat”, “dog”, “five”, and “ten”, spoken by male talkers. COM words “red” and “white”, while used in the task, were not included in the analysis of iEEG data to keep the target-to-non-target trial ratio consistent between COM and NOV stimuli and thus control for the target effect (cf. Steinschneider et al., 2014; Nourski et al., 2017). Likewise, COM stimuli spoken by female talkers were excluded from iEEG data analyses to control for talker gender (as all NOV words were spoken by male talkers).

Differences in responses to COM and NOV stimuli were evaluated over 1800 ms after stimulus onset using non-parametric cluster-based permutation tests (Maris and Oostenveld, 2007; Nourski et al., 2018). The test statistic was based on grouping adjacent time points that exhibited a significant difference between COM and NOV trials. In the permutation procedure, the surrogate COM and NOV sets had the same number of trials as the real COM and NOV sets. At each recording site, t-values were calculated for each time point and compared to a threshold, corresponding to the 1st percentile tail of the T-distribution. One-tailed tests were used for gamma and alpha ERBP. Novelty effects were defined as greater gamma ERBP augmentation or greater alpha ERBP suppression associated with NOV vs. COM stimuli. Two-tailed tests were used for AEP. Clusters were defined as consecutive points for which the t-value exceeded the threshold. The cluster-level statistic was defined as the sum of the t-values within each cluster, and its significance was calculated using permutation tests (n=10,000). Novelty effects were considered significant at p < 0.05. For each of the three measures (gamma, alpha, AEP), recording sites with at least one significant cluster were considered to exhibit the corresponding novelty effect.

The prevalence of each novelty effect in a given ROI was defined as the percentage of sites within the ROI, across all participants, that exhibited the effect. Planned pairwise comparisons of novelty effect prevalence between ROIs were carried out using Fisher exact tests. Differences in the prevalence of novelty effects between left and right hemisphere were examined using Fisher exact tests at the whole brain level (all left hemisphere sites vs all right hemisphere sites) and on the ROI group level [HGPM, superior temporal plane (STP), STG, ventral, dorsal, prefrontal, and sensorimotor]. Correction for multiple comparisons was performed using the false discovery rate approach (Benjamini & Hochberg, 1995). The same method was used for comparisons of novelty effect prevalence between participants who exhibited above- vs. below-average task performance (as defined by hit rates to NOV target stimuli). This approach was taken instead of using performance as a continuous measure due to the heterogeneity of electrode coverage inherent to iEEG studies. The choice of NOV target hit rates (rather than COM targets or all targets) as the task performance criterion was motivated by the focus on the brain’s response to unexpected word stimuli. Specifically, the purpose of the analysis was to establish whether enhanced neural responses to NOV stimuli were associated with more or less accurate identification of those stimuli.

To facilitate the comparison of data collected in the present iEEG study with those previously obtained using non-invasive methods, grand average AEP waveforms for sites exhibiting significant AEP novelty effect were plotted for select ROIs. Full wave-rectified AEPs from depth electrodes, located within the STP, insula, amygdala, ACC, and middle-anterior cingulate cortex (CingMA), were calculated to provide a temporal reference for the emergence of the AEP novelty effect in these anatomical regions. Sites overlying lateral temporal cortex were divided into posterior, intermediate and anterior temporal lobe groups. To maintain consistency of AEP waveform polarity with that of scalp-recorded data, only subdural contacts were used for this analysis.

Results

Electrode coverage

Results are based on analyses of data from 5511 recording sites across 38 participants (Fig. 2). Recording sites were assigned to ROIs as shown in Figure 2a. ROIs were organized into several groups with respect to their position within the auditory cortical hierarchy, as envisioned by current models (Hickok & Poeppel, 2004; Rauschecker & Scott, 2009) and supported by response onset latency measurements (Hamilton et al., 2021; Nourski et al., 2021, 2022). The ROI groups included auditory core (HGPM), non-core STP (HGAL, PT, and PP), STG (STGM, STGP), ventral auditory-related, dorsal auditory-related, prefrontal, and sensorimotor. The distribution of recording sites across all participants is plotted in standard MNI coordinate space in Figure 2b. There was comparable sampling of both the left and right hemisphere (2816 and 2695 sites, respectively). The extensive coverage of the STP, superior temporal sulcus (STS), lateral and mesial temporal, frontal, and parietal cortices (Fig. 2c) allowed for comparisons between the two hemispheres, across ROI groups and between individual ROIs.

Task performance

Participants exhibited variable performance in the target detection task, with overall hit rates to both COM and NOV target stimuli ranging from 24.4 to 100% (median 86.7%), d′ between 1.05 and >5 (median 3.35), and median reaction times between 535 and 1308 ms (grand median 838 ms) (Supplementary Table 1). Overall hit rates and d′ values were highly correlated (ρ=0.838, p<0.0001). Six participants had d′<2, which was associated with overall hit rates of 61.1% or less and generally longer RTs. Inclusion of these poorly performing participants facilitated comparisons between physiology and behavior.

Participants’ clinical and demographic profiles were analyzed with respect to task performance. Younger age of epilepsy onset was associated with poorer task performance as measured by overall hit rates (ρ=0.362, p=0.0255), though the duration of the disorder was not (ρ=−0.228, p=0.168). There was no significant relationship between overall hit rates and participants’ sex (p = 0.342), age (ρ=0.107, p=0.523), pure tone average thresholds (ρ=−0.0334, p=0.842), speech reception thresholds (ρ=0.119, p=0.489), word recognition scores (ρ=0.224, p=0.189), time since electrode implantation (ρ=−0.101, p=0.546), time since most recent ictal event (ρ=0.0144, p=0.947), whether electrode coverage was primarily in the left or right hemisphere (p=0.837), or whether the seizure focus was localized to the left or right hemisphere (p=0.970). A complementary analysis limited to NOV target hit rates did not reveal any significant relationships with participants’ background (p>0.05 for all comparisons).

Introducing new words increased the difficulty of the task, as revealed by a separate assessment of behavioral performance for COM and NOV target stimuli (Fig. 3). NOV targets had lower hit rates compared to COM targets (median 70.0 and 88.1%, respectively, p < 0.0001) and were associated with significantly increased reaction times (RTs) (β = 144 ms, p < 0.0001). Based on NOV target hit rates, three groups of participants were identified: above-median (good) (NOV target hit rate 80-100%; 15 participants), median (NOV hit rate 70%; 8 participants), and below-median (poor) performers (NOV hit rate 0-60%; 15 participants).

Figure 3.

Target detection task performance. Hit rates and RTs were calculated separately for COM and NOV targets and pooled across the two target detection blocks (animal and number words). A: Hit rates. Symbols represent individual participants. Data from participants characterized by above-median (80-100%; N = 15), median (70%; N = 8) and below-median (0-60%; N = 15) NOV target hit rates are shown in black, dark gray, and light gray, respectively. Median NOV target hit rate (70%) is denoted by a dashed line. B: RT distributions. In violin plots, white circles denote medians, horizontal lines denote means, bars denote Q₁ and Q₃, and whiskers show the range of lower and higher adjacent values (i.e., values within 1.5 interquartile ranges below Q₁ or above Q₃, respectively). Significance of differences was established using Fisher exact test for hit rates and a linear mixed effects model for RTs.

Responses to COM and NOV stimuli in a representative participant

Auditory novelty effects were broadly distributed across studied ROIs and were present as augmented gamma power, greater alpha power suppression, and differences in AEP morphology. These effects are exemplified by participant L372 with extensive electrode coverage of the left hemisphere (Fig. 4). Here, responses were averaged separately for COM and NOV stimuli. Non-target and target word trials were combined from blocks 2 and 3. The non-target/target trial ratio was 50/50 for both COM and NOV stimuli. Significant COM/NOV difference clusters are shown underneath each panel, color-coded by measure (magenta: gamma; cyan: alpha; black: AEP). Timing of the stimuli and behavioral responses to target trials (button presses; depicted as open circles for NOV stimuli) are shown above for reference.

Figure 4.

Responses to COM and NOV stimuli in the left hemisphere in a representative participant (L372). Examples from 14 recording sites. Data are pooled across non-target and target trials in both word target detection blocks. AEP waveforms are plotted in black over time-frequency plots. Scale bar: 50 μV (negative voltage up). Stimulus schematic, distribution of RTs to COM target trials (violins) and NOV target RTs (white circles) are shown on top. Clusters representing significant gamma, alpha and AEP novelty effects are shown underneath time-frequency plots as magenta, cyan and black bars, respectively. Inset: Electrode coverage of lateral hemispheric convexity and STP in participant L372 showing locations of the exemplar sites. Sites characterized by presence of significant gamma, alpha and AEP novelty effects are shown in magenta, cyan, and black, respectively.

Gamma activity elicited by NOV stimuli was generally of greater magnitude and duration when compared against the COM condition. This effect extended beyond canonical auditory cortex into temporal and parietal auditory-related areas, as well as multiple areas in the prefrontal cortex. Gamma augmentation within ROIs outside canonical auditory cortex typically had longer onset latencies, consistent with the feedforward pattern of sensory stimulus processing. Of note, the onset of gamma augmentation within precentral gyrus was earlier than that in prefrontal cortex. The recording site shown in Figure 4 was at a distance (39 mm) from the sylvian fissure, supporting the interpretation that this was not a volume-conducted response from subjacent auditory cortex. This activity may reflect the short-latency responses in motor cortex during listening reported by Shtyrov et al. (2014).

Suppression of power in low iEEG frequency bands was another prominent novelty effect. This suppression was centered within the alpha band, though often initiated in the beta band. Typically, this suppression was of longer duration than the augmented gamma and at some sites, could be the principal response difference between COM and NOV words.

AEPs could also display differences in morphology between COM and NOV conditions. The overall distribution of recording sites that featured significant novelty effects is shown in the inset of Figure 4. All three effects could be observed in temporal, parietal and frontal cortex, though notably AEP effects were most prominent in the frontal lobe.

Topography of novelty effects

To characterize the topography of novelty effects across the studied ROIs, data were combined across all participants, and prevalences of gamma, alpha and AEP effects were calculated for each ROI (Table 1). The results of this approach are summarized in Figure 5 for ROIs with coverage of at least 20 recording sites, deemed sufficient for interpretation of prevalence.

Figure 5.

Prevalence of novelty effects across ROIs. ROIs with electrode coverage of at least 20 sites are color-coded by ROI group and rank-ordered according to prevalence of gamma, alpha and AEP novelty effect (top, middle, and bottom row, respectively). See Figure 2c for the list of ROI abbreviations.

Gamma novelty effect had the highest prevalence in the planum temporale (PT; 20.3%), STGP (19.8%) and IFGtr (18.1%). Thus, the gamma novelty effect is strongly represented at three progressive stages of the auditory cortical hierarchy. In contrast to PT, HGPM had the lowest prevalence of gamma novelty effect (5.51%) within superior temporal cortex. This was significantly lower than in PT (p=0.00154) despite anatomical proximity and functional similarities between the two areas (Hamilton et al., 2021; Nourski et al., 2022; Banks et al., 2023). Likewise, there was a difference in prevalence of gamma novelty effect between adjacent ROIs STGP (19.8%) and STGM (11.4%) (p = 0.0232). Gamma effect had a higher overall prevalence in dorsal (11.5%) than ventral (7.72%) auditory-related areas (p = 0.0297). Within the dorsal group, prominent gamma effect was observed in SMG and AGA. By contrast, AGP was largely devoid of auditory gamma novelty effect (AGA: 11.7%, AGP: 1.64%; p=0.0203).

Alpha novelty effect was particularly prominent in rostral superior temporal ROIs, with the highest prevalence in HGAL (27.9%), STGA (21.6%), and CingMA (19.4%). This effect had a higher overall prevalence in ventral (10.0%) than dorsal (7.04%) auditory-related areas, though this difference failed to reach significance (p = 0.0990). Among ventral auditory-related ROIs, the two banks of the STS, particularly the lower bank (STSL), had a high prevalence of alpha effect (STSU: 12.9%; STSL: 16.8%). Additionally, alpha novelty effect was particularly prominent in InsA (18.8%), which, along with CingMA, is considered part of the salience network (Kiehl et al, 2001; Menon and Uddin, 2010) which responds prominently to novel stimuli (Breukelaar et al., 2017).

AEP novelty effect was characterized by a group of ROIs with relatively high (>15%) prevalence. These included the STP (HGPM, PP, PT, HGAL), adjacent insular cortex (both InsA and InsP) and STSL. Of note, the combined activity of STP generators would have similar dipole orientations orthogonal to the STP, thus likely contributing to scalp-recorded novelty effects volume-conducted to the dorsolateral convexity of the brain.

As observed in Table 1 and Figure 5, there were a number of ROIs outside auditory and auditory-related areas that exhibited novelty effects in at least one of the examined measures of cortical activity (gamma, alpha, AEP). Auditory novelty effects were prominent in most ROIs within prefrontal cortex. All three effects were present (>10% prevalence) within pars opercularis and pars triangularis of the inferior frontal gyrus (IFGop and IFGtr). Other prefrontal ROIs included middle frontal gyrus (MFG; 13.5% gamma, 6.44% alpha, 12.3% AEP) and orbital gyri (OG; 15.0% gamma, 8.21% alpha, 9.38% AEP). Novelty effects within premotor cortex (PMC) were dominated by alpha suppression (13.5%) and AEP changes (10.3%), while gamma effect was uncommon (4.76%).

Both PreCG and postcentral gyrus (PostCG) were sensitive to auditory novelty (PreCG effect prevalence: 9.00% gamma, 8.65% alpha, 9.69% AEP; PostCG: 4.82% gamma, 5.26% alpha, 8.33% AEP). Along the z_MNI axis, PreCG sites that exhibited significant novelty effects were distributed with a median of 37.2 mm and interquartile range 21.0-52.2 mm. For PostCG, the topographic distribution along z_MNI axis was characterized by median 31.2 mm and interquartile range 17.3-49.6 mm. These distributions mainly overlapped with sensorimotor representation of the face, tongue, and larynx (cf. Roux et al., 2018, 2020). As participants were instructed to use their hand ipsilateral to the recording sites to press the button, it is unlikely that novelty effects in PreCG reflected neural activity associated with the button press. Instead, this response could be a manifestation of activation along the dorsal auditory cortical processing stream.

Within the medial temporal lobe, both the hippocampus and the amygdala were responsive to auditory novelty. The greatest effect within the hippocampus was alpha suppression, present in 10.2% sites, while the greatest effect in the amygdala was seen in the AEP (11.8%). Both regions displayed a low prevalence of gamma effects (5.08 and 1.68%, respectively). Occipital and ventral temporal ROIs that are typically considered as higher order visual areas, also featured auditory novelty effects to varying degrees. Here, the fusiform gyrus (FG) had the highest prevalence of both gamma (9.93%) and alpha (12.6%) effects. The relevance of the novelty effects in higher visual areas is unclear but may represent engagement of mental imagery in order to associate the NOV stimulus with the target semantic category.

Hemispheric asymmetries

The semantic nature of the task suggested the possibility of processing biases between the two hemispheres. Results of this evaluation are illustrated in Figure 6. Figure 6a depicts the locations of sites with novelty effects in all 38 participants, plotted in the MNI coordinate space and projected onto the FreeSurfer average template brain for spatial reference. Sites characterized by significant gamma, alpha and AEP novelty effects are shown in magenta, cyan, and black, respectively. Overall hemispheric asymmetries in the prevalence of novelty effects were examined by comparing percentages of sites characterized by significant gamma, alpha and AEP novelty effects in the left (L) and right (R) hemisphere (Fig. 6a, inset). There was a left hemisphere bias for all three novelty effects on the whole-brain level (gamma: p = 0.0156, alpha: p < 0.0001, AEP: p < 0.0001). This global asymmetry motivated a more detailed examination on the ROI group level (Fig. 7b). In contrast to the whole-brain analysis, no significant differences were observed for gamma novelty effect in any of the seven ROI groups (Fig. 6b, left panel). Left hemispheric bias was observed for alpha effect in STG, ventral auditory-related group, and prefrontal cortex (p = 0.00951, p = 0.000245, and p = 0.0336, respectively). The dorsal auditory-related group did not display this asymmetry. Left hemispheric bias was also observed for AEP effect in the STP and prefrontal cortex (p = 0.000410 and p < 0.0001, respectively). On the single ROI level, a significant left hemisphere bias was identified in PP for AEP effect (p = 0.0132), MFG for both alpha and AEP effect (p = 0.00849, p = 0.000202, respectively) in PMC for alpha effect (p = 0.00849). Of note, HGPM did not feature significant hemispheric differences in prevalence of any of the three effects, suggesting that hemispheric asymmetry in the processing of NOV words is an emergent property of higher auditory and auditory-related areas.

Figure 6.

Novelty effects across the hemispheres. a: Summary of data from all 38 participants plotted in the MNI coordinate space and projected onto the FreeSurfer average template brain for spatial reference. Side views of the lateral hemispheric convexity, top-down views of the superior temporal plane, ventral and mesial views are aligned along the y_MNI axis. Sites characterized by the presence of significant gamma, alpha and AEP novelty effects are shown in magenta, cyan, and black, respectively. Inset: Overall differences in the prevalence of novelty effects. Percentages of sites characterized by significant gamma, alpha and AEP novelty effects are plotted separately for sites in the left (L) vs right (R) hemisphere. Significance of differences was established using Fisher exact tests. b: Regional hemispheric asymmetries in the prevalence of novelty effects. Percentages of sites characterized by significant gamma, alpha and AEP novelty effects (left, middle, right panel, respectively) are plotted separately for left and right hemisphere locations for seven ROI groups. SensMot: sensorimotor.

Figure 7.

Relationship between behavioral task performance and prevalence of novelty effects. a: Overall differences in the prevalence of novelty effects. Percentages of sites characterized by significant gamma, alpha and AEP novelty effects are plotted separately for participants who exhibited good (G; 80-100% hit rate) and poor (P; 0-60% hit rate) Novel (NOV) target detection performance. Significance of differences was established using Fisher exact tests. b: Differences in the prevalence of novelty effects between good and poor performers across ROI groups. Percentages of sites characterized by significant gamma, alpha and AEP novelty effects (left, middle, right panel, respectively) in the seven ROI groups are plotted separately for participants who exhibited good and poor NOV target detection performance. Note that no HGPM sites in good performers exhibited gamma novelty effect. Inset in the left panel depicts a comparison of the gamma novelty effect prevalence within canonical auditory cortex (HGPM, STP, STG) between participants with good and poor NOV target detection performance. Inset in the right panel depicts the same comparison for the AEP novelty effect.

Relationship between NOV target detection and physiology of novelty effects

As noted earlier (see Fig. 3), auditory novelty was associated with lower hit rates and longer RTs for target NOV words compared to COM target trials. Hit rate for NOV targets ranged from 0 to 100% (median 70%). To examine whether this variability in performance was paralleled by differences in neural novelty responses, iEEG data were compared between participants who exhibited above-average performance (good NOV performance: 80-100%) and those who performed below average (poor NOV performance: 0-60%). The prevalence of novelty effects was calculated separately for these two groups. At the whole brain level, there were significant differences for all three measures: prevalence of gamma, alpha and AEP effects were higher in better-performing participants (p = 0.0107; p < 0.0001; p < 0.0001, respectively; Fig. 7a).

At the ROI group level, for gamma effect, the difference between good and poor performers was mainly driven by prefrontal and sensorimotor cortex, as both ROI groups had a significantly higher prevalence of gamma effect in good performers (p = 0.000119, p = 0.0109, respectively; Fig. 7b). At the individual ROI level, a significant performance bias for gamma effect was identified in PreCG (p=0.0286). Of note, the three canonical auditory cortex groups (HGPM, STP, STG) consistently featured the opposite relationship wherein the prevalence of gamma effect was higher in poor performers (p = 0.00276 when canonical auditory cortex was examined as a whole). This difference, however, did not reach significance when the three auditory ROI groups were examined individually (HGPM: p = 0.0715, STP: p = 0.114, STG: p = 0.0830).

Within HGPM, there was a dissociation between gamma and alpha novelty effects with regard to task performance. Whereas gamma novelty effect in HGPM was only observed in poor performers, alpha novelty effect was more prevalent in good performers (p = 0.000249). Positive relationships between task performance and prevalence of alpha novelty effects were also observed in STP, ventral auditory-related, prefrontal, and sensorimotor cortex (p = 0.00240, p = 0.00540, p = 0.000249, p = 0.000163, respectively). Only prefrontal and sensorimotor cortex showed a positive relationship between task performance and prevalence of both gamma and alpha novelty effects. On the individual ROI level, alpha novelty effect had a significantly higher prevalence in good performers in IFGop, PreCG, and InsP (p = 0.00636).

When canonical auditory cortex (HGPM, STP, and lateral STG) was examined as a whole, a higher prevalence of AEP effect in good performers was identified (p = 0.0174). This difference, however, was not significant when the three auditory ROI groups were examined individually (HGPM: p = 0.138, STP: p = 0.138, STG: p = 0.688). Finally, the difference in AEP novelty effect between good and poor performers was found in ventral auditory-related ROIs (p = 0.00503) and, on the individual ROI level, in Amyg, InsP, and ITGA (p = 0.0325, p = 0.0235, p = 0.0380, respectively).

Morphology and timing of AEP novelty effects

Non-invasive ERP studies have identified several scalp-recorded potentials associated with auditory novelty, including P3a and N400. To identify putative generators of these components, we first examined all sites within a given ROI and compared averaged responses to COM and NOV stimuli. For depth electrodes located within the STP (including HGPM), insula, amygdala, and anterior cingulate ROIs (ACC and CingMA), AEPs were full wave-rectified to account for variability of electrode locations relative to current dipoles. For regions located on the hemispheric convexity, only subdural contacts were used for AEP averaging to maintain consistent polarity of the AEP waveforms. To examine the potential rostro-caudal gradient in the novelty effect (cf. Friederici et al., 2003), the lateral temporal cortex was divided into three groups: posterior, intermediate, and anterior.

Inclusion of all sites in this analysis failed to identify differences between COM and NOV responses that would be consistent with the time frame of P3a and N400 (data not shown). This absence of an effect was likely due to the relatively low overall prevalence of significant AEP novelty effect in the studied ROIs. Thus, to put the present findings within the context of scalp-recorded novelty responses, only AEPs from sites with significant novelty effects were averaged. The timing of the differences between COM and NOV responses was compared with that of P3a and N400.

AEP novelty effect (i.e., difference between COM and NOV grand averaged waveforms) in the time range of P3a and N400 (dashed lines in Fig. 8) was present in each of the STP ROIs including HGPM (Fig. 8a). In each ROI, the effect continued beyond 1 s post-stimulus. AEP novelty effect within the STP, medial structures (in InsA, InsP, amygdala, and ACC/CingMA; Fig. 8b), and intermediate-to-anterior lateral temporal cortex (Fig. 8c) generally peaked within 300-400 ms after stimulus onset. Within the STP and lateral temporal cortex, more rostral ROIs generally exhibited a novelty effect that overlapped in time with P3a and N400. This effect in intermediate and anterior lateral temporal ROIs had a positive polarity between 300 and 400 ms. Additionally, positive deflections that peaked earlier were seen overlying the dorsal auditory-related ROIs. AEPs recorded from the hippocampus were too variable to warrant a meaningful interpretation (data not shown). Prefrontal, premotor, and sensorimotor cortex featured negative polarity deflections, with later onsets than those seen in the temporal and parietal cortices and peaks beyond 400 ms (Fig. 8d). Taken together, it is likely that scalp-recorded potentials P3a and N400 are predominantly generated by contributions from volume-conducted activity from multiple brain regions including auditory cortex, medial and anterior temporal structures.

Figure 8.

Morphology and timing of the AEP novelty effect. Grand average AEP waveforms from sites exhibiting significant AEP novelty effect. a: Full wave-rectified AEPs from depth electrodes located within the STP (including HGPM). b: Full wave-rectified AEPs from depth electrodes located within InsA, InsP, amygdala, and ACC/CingMA. c: AEP waveforms from subdural recording sites in posterior, intermediate, and anterior temporal ROIs, and dorsal auditory-related ROIs. d: AEP waveforms from subdural recording sites in ventro- and dorsolateral prefrontal, PMC, and sensorimotor cortex. AEP elicited by COM and NOV stimuli are depicted by thin and thick solid lines, respectively. Dashed lines provide a temporal reference for the P3a and N400 ERP components (300 and 400 ms, respectively).

Discussion

Three complementary measures - broadband gamma activity, alpha suppression, and AEPs - were used to examine auditory novelty processing within the context of a semantic categorization task. Gamma activity represents feedforward signaling while alpha suppression is a measure of release from feedback inhibition (Kandylaki et al., 2016; Bastos et al., 2020; Schmitt et al., 2021). Placed in the context of the predictive coding framework, the presence of a broadband gamma novelty effect reflects feedforward error signals caused by unexpected words. Alpha oscillations enhance processing efficiency by attenuating responses to predictable inputs (Bauer et al., 2014). Novel stimuli violate expectations, leading to a decrease in alpha power and increased gamma activity (Chao et al., 2022). This dynamic interplay highlights the complementary nature of high (gamma) and low (alpha) frequency components of iEEG and emphasizes that both measures are key to advancing our understanding of auditory processing at the cortical level (Billig et al., 2019; Nourski et al., 2021b, 2022). Additionally, examination of AEPs offers the potential to relate the results of direct recordings to more widely available non-invasive measures.

Cortical speech processing is a cascade of neural activity, including acoustic, phonemic and semantic stages subserved by the STP, STG and higher-order areas, respectively. It is important to acknowledge that the present study did not specifically determine to what extent the observed effects reflected general auditory novelty detection mechanisms or were specific for semantic processing of words. The left hemisphere bias of the novelty effects is consistent with semantic processing of NOV words, contrasting with a right hemisphere bias of novelty effects associated with environmental sounds (Opitz et al., 1999). Additionally, the possibility of repetition suppression for COM stimuli may contribute to the novelty effects reported here. While the current study included a limited set of tone stimuli, future studies using a wider array of speech and non-speech stimuli will be required to address these issues.

Gamma novelty effect

Gamma novelty effect varied across canonical auditory cortex ROIs within the STP and lateral STG. One notable difference in the distribution of the gamma novelty effect was the higher prevalence in PT compared to HGPM, even though these areas have similar short onset latencies to sound, and both exhibit narrow-band spectrotemporal receptive fields (Hamilton et al., 2018; Nourski et al., 2022). This difference in processing of NOV words is consistent with recent studies demonstrating that these two areas represent distinct nodes initiating auditory cortical processing along different but overlapping circuits (Hamilton et al., 2021; Banks et al., 2023). Thus, the high prevalence in PT may reflect its special involvement in phonological processing (Hamilton et al., 2018, 2021). The higher prevalence of the gamma novelty effect in STGP relative to STGM may be secondary to similar connectivity patterns of PT and STGP that have been reported recently (Hamilton et al., 2021; Banks et al., 2023). A notable finding was that within canonical auditory cortex, higher prevalence of gamma novelty effect was associated with poorer task performance. This seemingly counterintuitive relationship may reflect interference between NOV word encoding at the acoustic and phonemic levels and semantic categorization task demands.

Beyond canonical auditory cortex, gamma novelty effect was present in an extensive number of ROIs, including those in both ventral and dorsal auditory processing streams. Among the ventral auditory-related ROIs, the highest prevalence was found in MTGP, a key area involved in lexico-semantic processing (Hickok, 2009; Youssofzadeh et al., 2022). Within the dorsal ROI group, the SMG had the highest prevalence of gamma novelty effect. SMG plays a major role in phonemic processing, sensory-motor integration, and verbal working memory (Turkeltaub and Coslett, 2010; Zevin et al., 2010; Deschamps et al., 2014). Additionally, AGA, but not AGP, also displayed a high prevalence of gamma novelty effect. AGA is implicated in semantic processing, especially during more difficult tasks (Hartwigsen et al., 2016; Kuhnke et al., 2023). In the present study, NOV targets were associated with greater task difficulty manifested by lower hit rates and longer RTs than to COM targets.

Multiple prefrontal ROIs exhibited a gamma novelty effect, particularly prominent in IFGtr and adjacent IFGop. It is of note that in the current study, no significant hemispheric differences were observed for the gamma novelty effect within these two areas. This should not imply that the IFG is performing similar functions in both hemispheres. Specifically, left IFG is implicated in speech processing (IFGtr: semantic processing; IFGop: phonological and syntactic processing; Hartwigsen et al., 2016; Graessner et al., 2021; van der Burght et al., 2022). By contrast, right IFG has been shown to be involved in semantic classification of novel non-speech environmental sounds (Opitz et al., 1999). Thus, activation of the left IFG may be driven by semantic processing of NOV words, whereas the right IFG may be primarily activated by the novelty of these stimuli. Finally, the high prevalence of gamma novelty effect in ventral frontal cortex, including OG and adjacent GR, can be related to the broad role of orbitofrontal cortex in decision making and prediction (Kringelbach, 2005). Taken together, prefrontal cortex is likely subserving multiple functions associated with detection and processing of auditory novelty during semantic categorization tasks. These functions span from speech processing to identification and semantic classification of unexpected stimuli, ultimately contributing to successful task performance. A significantly higher prevalence of gamma novelty effect in better performers in prefrontal cortex is consistent with this interpretation.

PreCG is traditionally considered a motor area, yet a growing body of literature supports its involvement in speech perception. Multiple functions of PreCG have been proposed in this regard. These include (1) the process of using articulatory cues to promote speech perception (Pulvermüller et al., 2006; D’Ausilio et al., 2009; Schomers et al., 2015), (2) a separate mapping of acoustic speech attributes superimposed on articulatory representations (Cheung et al., 2016), and (3) rapidly relating the semantic content of words to their somatotopic correlate within the homunculus (Shtyrov et al., 2014). PreCG sites demonstrating the gamma novelty effect were located mostly in the face area as defined by z_MNI coordinates (Roux et al., 2020). Thus, current results are consistent with the first two interpretations, yet do not refute the third proposal. Additionally, PreCG has been interpreted to be a component of the predictive coding network for speech (Cope et al., 2023), therefore suggesting a more general role in auditory novelty detection.

Finally, prominent gamma novelty effect was present in the TP, a key site of pathology in frontotemporal aphasia of the semantic type (Amici et al., 2006; Gorno-Tempini et al., 2011). InsA also had a high prevalence of the gamma novelty effect. InsA is a node in the salience network that is involved in multiple aspects of cognitive and behavioral control (Kiehl et al, 2001; Menon et al., 2010; Niendam et al., 2012; Breukelaar et al., 2017). This area is activated by novel environmental sounds, during articulation of novel syllables, and contributes to recognition memory, wherein the salience of novel stimuli is remembered (Kiehl et al, 2001; Downar et al., 2002; Bermudez-Rattoni, 2014). Thus, activation of InsA likely reflects a general novelty mechanism for behaviorally salient stimuli.

Alpha suppression

While both alpha suppression and gamma augmentation are measures of excitatory neuronal activity, they have distinct regional distributions. Within core auditory cortex and surrounding areas in the STP there was a strong association between prevalence of the alpha effect and above-average task performance. This relationship was the opposite of that seen for gamma effect. The reason for this discrepancy is unclear but may reflect fundamental differences in the modulation of gamma feedforward and alpha feedback signals within the context of a mismatch between novelty detection and detection of a semantic target. Further, it is consistent with a more general phenomenon that the physiological responses associated with speech processing are strongly modulated by the specifics of the experimental task (Hickok and Poeppel, 2007).

Distinct regional distributions of gamma and alpha novelty effects extended beyond canonical auditory cortex and were observed along the dorsal and ventral auditory processing streams. Caudal and dorsal temporo-parietal ROIs (PT, STGP, AGA, SMG) exhibited a high prevalence of the gamma novelty effect. By contrast, more rostral and ventral ROIs (HGAL, PP, STGA, STSU, STSL) represented novelty primarily in the form of alpha suppression. The highest prevalence of the alpha novelty effect was within HGAL, STGA and PP, suggesting that modulation of alpha power occurs in regions involved in both the acoustic encoding (HGAL) and higher-order processing of the NOV stimuli (STGA, PP). STGA and PP have been implicated in semantic encoding at the sentence level (Friederici et al., 2000; 2003; Humphries et al., 2005). Results of the current study support a role for these two regions in semantic processing at the single word level (cf. Dewitt and Rauschecker, 2012, 2016). Overall, the strong left hemispheric bias for alpha novelty effect within the STG, ventral (but not dorsal) auditory cortical processing stream, and prefrontal ROIs is consistent with their contribution to semantic processing of NOV words.

These findings highlight the translational importance of alpha suppression as a measure of excitatory cortical activity. Unlike high gamma, alpha-band activity is readily accessible with non-invasive neuroimaging methods (EEG, MEG). Caution, however, must be exercised, as these two measures, while complementary, do not provide the same information. Specifically, they have distinct laminar profiles, different spatial distributions throughout the cortex, and ultimately reflect distinct feedforward and feedback circuits associated with predictive coding (Bauer et al., 2014; Fontolan et al., 2014; Bastos et al., 2020; Chao et al., 2022). It is also plausible that feedforward encoding involves neuronal activity at a detailed level that cannot always be readily detectable with high gamma activity recorded from low impedance electrodes.

Averaged evoked potentials

AEPs index synaptic activity generated by neuronal populations that are both time- and phase-locked (Eggermont & Ponton, 2002). Thus, AEPs provide complementary information to gamma ERBP, a surrogate mesoscopic measure of action potentials (Steinschneider et al., 2008). The highest prevalence of AEP novelty effect was found in the auditory cortex in the STP (including HGPM) as well as both subdivisions of the insula, STSL, and CingMA. Typically, these regions exhibited an AEP novelty effect within the time frames of the P3a and N400. However, each region has its own geometry and dipole orientation; deeper structures may generate closed electrical fields or produce responses that cancel out from superposition with other signals when measured at the scalp (e.g., Halgren, 2008; Steinschneider et al., 2010). Methods based on inverse solutions for localization of generators may not adequately account for the contributions of all the sources of AEPs reflecting auditory novelty. Functional neuroimaging may assist in this process, but results would remain an approximation. Unambiguous methods such as tracing a potential from the cortical surface to the generator are rarely feasible in human neurosurgical patients. Despite these limitations, certain conclusions are possible if accounting for source cytoarchitectonics, location, and orientation.

Novelty effects emanating from auditory cortex on the STP likely contribute to the scalp-recorded P3a and N400. Prevalence of the AEP novelty effect was high throughout the STP with potentials that spanned the time frames of both P3a and N400. Volume-conducted activity from the STP is maximal over the dorsolateral convexity near Fz and Cz (Vaughan & Ritter, 1970), and thus likely contribute to the morphology of the scalp-recorded novelty responses. Previous non-invasive and intracranial results support this conclusion (Alho et al., 1998; Halgren et al., 1998; Friederici et al., 2003; Strobel et al., 2008).

AEP novelty responses that spanned 300-400 ms also occurred in InsA, InsP, amygdala and CingMA. InsA and CingMA have strong functional connections and are key elements of the salience network (Taylor et al., 2009; Menon and Uddin, 2010). Novel and potentially salient sounds activate both regions (Kiehl et al., 2001; Downar et al., 2002; Friederici et al., 2003). InsP and the amygdala have also been implicated in generating auditory novelty potentials (Halgren et al., 1995; Friederici et al., 2003), but the degree to which these regions contribute to scalp-recorded novelty ERPs remains unclear.

In contrast to depth recordings, recordings from subdural electrodes overlying the cortical convexity permit analysis of AEP morphology with respect to both timing and polarity (Howard et al., 2000). Intermediate and anterior regions of the temporal lobe as well as ROIs along the dorsal pathway featured positive waves peaking as early as 200 ms (parietal ROIs) and between 300-450 ms over middle and anterior portions of the temporal lobe (see Fig. 9c). These regions may thus contribute to P3a (cf. Halgren et al., 1998; Opitz et al., 1999; Kiehl et al., 2001; Strobel et al., 2008).

We were not able to identify potential generators of P3a and N400 within prefrontal cortex. Prominent negative deflections occurred after 400-500 ms over prefrontal, premotor and sensorimotor cortex, suggesting limited contribution to the P3a and N400. This contrasts with previous work that identified the right prefrontal cortex as a generator of AEP novelty responses, including P3a (Opitz et al., 1999). However, novel environmental sounds were used as opposed to novel words, and thus differences in topography, latency and hemispheric bias may be secondary to the stimuli used.

Concluding remarks

Younger age of epilepsy onset was associated with poorer task performance (cf. Trimmel et al., 2021; Crow et al., 2023), indicating the importance of examining participants’ clinical and demographic backgrounds in iEEG studies. Gamma novelty effect had a consistently higher prevalence in poor performers than better performers within canonical auditory cortex. This may be a consequence of long-standing epilepsy in poor performers. The elevated gamma activity in poor performers argues against a simple attentional mechanism in auditory cortex to explain their low hit rates, as attention should enhance gamma within non-core auditory cortex on the STP and STG (Mesgarani & Chang, 2012; Nourski et al., 2021a). The novelty effects reported here likely reflect a combination of neural activity directly associated with semantic processing and more general mechanisms of novelty detection and repetition suppression. Interactions between novelty detection and semantic processing will need to be disambiguated in future studies by including additional categories of meaningful and non-meaningful novel stimuli. Inclusion of paradigms that examine the effects of novel sound repetition may offer new insights into the interplay between novelty and habituation. Understanding these processes may be especially important when considering the many neuropsychiatric disorders characterized by aberrant semantic processing, novelty detection, and habituation (e.g., Hudac et al., 2018; Justo-Guillén et al., 2019; Ippolito et al., 2022).

Highlights.

• First large scale iEEG study of auditory novelty using words in a semantic task

• Analysis of gamma activity, alpha suppression, and intracranial evoked potentials

• Novelty effects are widely distributed within and beyond auditory cortex

• Novelty effects elicited by unexpected words display a left hemispheric bias

• Greater effect prevalence is generally associated with better task performance