Neural Dynamics Associated with Semantic and Episodic Memory for Faces: Evidence from Multiple Frequency Bands

Abstract

Prior semantic knowledge facilitates episodic recognition memory for faces. To examine the neural manifestation of the interplay between semantic and episodic memory, we investigated neuroelectric dynamics during the creation (study) and the retrieval (test) of episodic memories for famous and nonfamous faces. Episodic memory effects were evident in several EEG frequency bands: theta (4–8 Hz), alpha (9–13 Hz), and gamma (40–100 Hz). Activity in these bands was differentially modulated by preexisting semantic knowledge and by episodic memory, implicating their different functional roles in memory. More specifically, theta activity and alpha suppression were larger for old compared to new faces at test regardless of fame, but were both larger for famous faces during study. This pattern of selective semantic effects suggests that the theta and alpha responses, which are primarily associated with episodic memory, reflect utilization of semantic information only when it is beneficial for task performance. In contrast, gamma activity decreased between the first (study) and second (test) presentation of a face, but overall was larger for famous than nonfamous faces. Hence, the gamma rhythm seems to be primarily related to activation of preexisting neural representations that may contribute to the formation of new episodic traces. Although the latter process is affected by the episodic status of a stimulus, gamma activity might not be a direct index of episodic memory. Taken together, these data provide new insights into the complex interaction between semantic and episodic memory for faces and the neural dynamics associated with mnemonic processes.

INTRODUCTION

Face recognition is a commonplace act that all humans perform daily. It is an essential part of effective social interactions, which rely on the ability to learn new faces, to associate them with contextual information while storing them in memory and, when appropriate, to correctly retrieve this knowledge from memory. As individuals develop personal relationships, new episodic representations are established in long-term memory, which are integrated with, and enriched by semantic knowledge. This then raises the question of what role prior semantic knowledge might play in the creation and/or retrieval of new episodic traces for faces, and what type of brain dynamics may mediate this cognitive interplay.

Neuropsychological and brain imaging studies have contributed considerably to our understanding of the brain areas involved in episodic and semantic memory generally and for face processing in particular (e.g., Gobbini & Haxby, 2006; Haxby, Hoffman, & Gobbini, 2000; Grill-Spector et al., 1999). These studies have suggested that recognition of familiar or known faces (i.e., those associated with semantic information) involves different neural mechanisms than those involved in the recognition of newly learned, unfamiliar, relatively unknown faces (Leube, Erb, Grodd, Bartels, & Kircher, 2003; Leveroni et al., 2000). Nonetheless, remarkably little is known about the neural dynamics leading to and accompanying successful episodic memory encoding and retrieval, and whether, and if so how, the neural dynamics associated with these processes are influenced by semantic knowledge. Due to its exquisite temporal resolution, the electroencephalogram (EEG) can provide a good measure of neural dynamics.

Over the last decade, neural oscillations in multiple frequency bands have been implicated in coordinating the precise timing of neuronal activity (Fries, Nikolić, & Singer, 2007; Jacobs, Kahana, Ekstrom, & Fried, 2007; Fries, 2005), and facilitating cognitive processes such as perception, attention, long-term coding, and retrieval of information (Jensen, Kaiser, & Lachaux, 2007; Buzsáki, 2005; Buzsáki & Draguhn, 2004; Engel, Fries, & Singer, 2001; Singer & Gray, 1995). With regard to human memory, investigations using EEG, MEG, and intracranial recordings have revealed modulations of oscillatory activity in the theta (4–8 Hz), alpha (9–13 Hz), and high-gamma (>40 Hz) frequency bands during successful episodic encoding and retrieval. Many of these studies used a classic memory recognition paradigm in which participants first learn a series of items and are subsequently tested for recognition of these (old) items, interspersed among new ones.

The primary EEG rhythm associated with memory is theta—the dominant rhythm in hippocampus (Buzsáki, 2002). A positive correlation between hippocampal theta amplitude and successful memory has been reported in human intracranial recordings (Mormann et al., 2005; Fell et al., 2003; Sederberg, Kahana, Howard, Donner, & Madsen, 2003). Likewise, theta activity at frontal, temporal, and posterior scalp is typically modulated by successful encoding and retrieval; its amplitude is larger for remembered items compared to new or forgotten items (Osipova et al., 2006; Düzel, Neufang, & Heinze, 2005; Burgess & Gruzelier, 1997, 2000; Doppelmayr, Klimesch, Schwaiger, Stadler, & Röhm, 2000; Weiss & Rappelsberger, 2000; Doppelmayr, Klimesch, Schwaiger, Auinger, & Winkler, 1998; Klimesch, Doppelmayr, Schimke, & Ripper, 1997). The similar modulation of scalp-recorded theta and hippocampal theta by memory processes has led some researchers to suggest that the theta rhythm might mediate a dynamic link between hippocampus and neocorti-cal areas that store and process long-term memory traces, as first proposed by Miller (1991).

The alpha rhythm is also modulated by memory among other cognitive activities. Alpha is the dominant rhythm during rest, and desynchronizes during mental processing. This desynchronization is manifest as a decrease in scalp-measured alpha amplitude, primarily over occipital regions. A prevalent view holds that alpha amplitude reflects cortical deactivation or inhibition and, accordingly, that alpha desynchronization reflects enhanced cortical activation (Klimesch, Sauseng, & Hanslmayr, 2007; Pfurtscheller, 2003; Ray & Cole, 1985; but cf. Palva & Palva, 2007). The amount of alpha desynchronization has been correlated with intelligence as well as with individual memory ability (Doppelmayr, Klimesch, Hödlmoser, Sauseng, & Gruber, 2005; Doppelmayr et al., 2000; Vogt, Klimesch, & Doppelmayr, 1998; Klimesch, Doppelmayr, Pachinger, & Ripper, 1997). During memory tasks, alpha desynchronization is influenced by semantic processing (Klimesch, Doppelmayr, & Hanslmayr, 2006; Klimesch, Doppelmayr, Pachinger, & Russegger, 1997; Klimesch, Schimke, & Schwaiger, 1994) and episodic memory alike (Burgess & Gruzelier, 2000; Weiss & Rappelsberger, 2000; Klimesch, Doppelmayr, Pachinger, & Ripper, 1997), although, to date, there has been no direct comparison of these effects.

Gamma band activity also seems to bear some relation to memory processes. Gamma band activity in primary and associative sensory cortices, for example, has been linked to the activation and maintenance of neural representations corresponding to an incoming stimulus (Gruber, Maess, Trujillo-Barreto, & Müller, 2008; Jensen et al., 2007; Fries, Schröder, Roelfsema, Singer, & Engel, 2002; Engel et al., 2001; Singer & Gray, 1995). In working memory tasks, for example, gamma activity in visual areas is sustained throughout the retention of a stimulus’ perceptual properties (Jokisch & Jensen, 2007; Mainy et al., 2007; Howard et al., 2003). These studies and others have demonstrated that gamma activity not only reflects the processing of physical stimulus attributes but is also modulated by cognitive factors such as attention, target-matching, and working memory load (Jeschke, Lenz, Budinger, Herrmann, & Ohl, 2007; Lenz et al., 2007; Tallon-Baudry, Bertrand, Hénaff, Isnard, & Fischer, 2005). Furthermore, gamma activity in extrastriate regions during memory encoding predicts the likelihood of subsequent retrieval (Mainy et al., 2007; Osipova et al., 2006; Sederberg et al., 2003, 2006). It is less clear whether gamma is also increased during successful retrieval, as the reports are contradictory (Osipova et al., 2006, but cf. Düzel et al., 2005 and see Discussion). In addition, like alpha, gamma activity is also influenced by semantic variables (Gruber, Maess, et al., 2008; Lachaux et al., 2005, 2007), as reflected in greater gamma for meaningful compared to meaningless stimuli.¹

The emerging picture from the literature then is that all three of these EEG frequency bands are modulated by episodic memory. However, there are still important questions to be addressed. For example, the extent to which the various reported memory effects on these rhythms reflect the actual creation and/or activation of new episodic memory traces versus other cognitive processes that contribute to or are affected by episodic memory remains unclear. Further, to a large extent, studies of neural dynamics have overlooked the critical relationship between episodic memory and pre-experimentally existing semantic knowledge. Certainly, theories of memory have hypothesized that the creation of new episodic memories benefits from preexisting knowledge in semantic memory (Tulving, 1983, 1995). Such theories have been influenced by Craik and Lockhart’s (1972) Levels-of-Processing framework (see also Craik, 2002) by which episodic recognition depends on the extent of elaboration at study, that is, the richness of the preexistent associative network activated during encoding into episodic memory. Thus, it is likely that the extent of prior familiarity and/or the amount of semantic information associated with a stimulus affects the quality of episodic memory, and its neural correlates during encoding and retrieval.

According to a prevalent view, semantic memory consists of multiple subsystems encoding different types of information, for instance, verbal and perceptual (e.g., Coltheart et al., 1998; Gainotti, Silveri, Daniele, & Giustolisi, 1995; Allport, 1985). Likewise, it has been suggested that these different types of information might be stored in various brain areas (e.g., Kellenbach, Brett, & Patterson, 2001). In the case of faces, it is difficult to distinguish the effects of perceptual-semantics and verbal-semantics as the two are inherently confounded: Faces for which verbal-semantic information is available will also have extensive perceptual-semantic representations. Throughout this report, the term “semantic” knowledge will refer to all information—verbal and perceptual—stored in long-term memory in association with a particular face.

Surprisingly, the impact of prior semantic knowledge on episodic memory has not been directly addressed in previous investigations of the neural correlates of memory. In fact, a great majority of the studies cited above used familiar stimuli (e.g., words, pictures), which undoubtedly activated semantic memory traces, thus confounding episodic and semantic factors (for similar criticism, see Yovel & Paller, 2004). This criticism is especially relevant for our understanding of the roles of alpha and gamma activity in episodic memory, as both have also been explicitly associated with perceptual and semantic processing.

To shed light on these issues, in the present study we directly tested the influence of pre-experimental familiarity with a face on the neural correlates associated with episodic memory formation and retrieval. In a classic memory recognition task, we systematically manipulated the level of semantic and episodic information available to participants. To this end, we employed both famous faces, for which participants had rich preexisting memory representations, as well as nonfamous, unfamiliar faces, for which the participants had no prior memory representations. Our rationale was that for nonfamous faces, participants’ memory performance would rely wholly on the perceptual and episodic representations formed during study, whereas for famous faces episodic memory would likely be facilitated by pre-experimental semantic knowledge associated with those faces. Accordingly, and in line with the Levels-of-Processing theory, we expected episodic memory performance to be better for famous faces than for unfamiliar faces.

The primary goal of this study was to see whether and, if so, how neural activity in the theta, alpha, and gamma frequency bands, reported to be associated with episodic memory, is modulated by prior semantic knowledge during episodic memory encoding and retrieval. Our specific aims thus were twofold: First, by determining which of the frequency bands reflects processes penetrable by pre-existing semantic knowledge, we hoped to discover the neural basis for the relationship between episodic memory encoding and depth of processing. Second, by comparing old and new faces during test, we could replicate episodic memory effects and extend the findings to include the role of semantic factors on retrieval. It is important to note that although we had a specific hypothesis regarding the famous faces advantage over to nonfamous faces during encoding (at study), we had no specific hypotheses regarding how pre-experimental familiarity with the faces might affect the neural correlates of episodic memory retrieval. At the same time, we did expect to find overall similar neural responses during study and test, as both processes are thought to involve similar neural mechanisms (Mesulam, 1998; Fuster, 1995).

METHODS

Participants

The participants were 20 undergraduates (14 women) from the Hebrew University ranging in age from 22 to 29 years (median age = 25.5). All participants reported normal or corrected-to-normal vision and had no history of psychiatric or neurological disorders. Among them, one was left-handed. They signed an informed consent approved by the institutional review board of the Hebrew University, and were paid for participation or received class credit.

Stimuli

The stimuli consisted of 100 photographs of male and female unfamiliar faces and an equal number of male and female famous faces (face familiarity was determined separately for each participant; see below). All the pictures were equated for mean luminance and contrast, and consisted of an equal number of male and female faces. The stimuli were presented at fixation and, seen from a distance of approximately 60 cm, occupied 9.5° × 13° in the visual field (10 cm × 14 cm).

Task and Procedure

The experiment consisted of 10 consecutive blocks, each containing a study session followed by a test session. In the study session, in each block participants viewed 20 different face photographs (which were not repeated across blocks) and were instructed to judge the beauty of each face via a button press (1 = pretty; 2 = average; 3 = ugly). They were also instructed to remember the faces for subsequent testing. In the test sessions, the studied faces appeared interspersed among 20 new ones. Participants were instructed to decide whether each face was “old” or “new.” They were also instructed to rate their confidence (1 = absolutely sure; 2 = pretty sure; 3 = not so sure; 4 = not sure at all). Both at study and at test, stimuli were presented for 1500 msec, after which participants were cued to give their response. There was an ISI of 500 msec between the response and the presentation of the next stimulus. Famous and unfamiliar faces were presented in alternating blocks, with the starting block counterbalanced across participants. To ensure that participants were, in fact, familiar with the famous faces presented in the experiment, they performed a familiarity validation test at the end of the experiment. In this part, they were presented with all the famous faces and instructed to write the name and profession of the person pictured. Only faces that the participant could name or state other specific personal information about were included in the analysis (on average, subjects were familiar with 94% of the famous faces). The experiment was run in an acoustically treated and electrically isolated booth.

EEG Recording

The EEG analog signals were recorded continuously by 64 Ag–AgCl pin-type active electrodes mounted on an elastic cap (ECI) according to the extended 10–20 system. Eye movements, as well as blinks, were monitored using bipolar horizontal and vertical EOG derivations via two pairs of electrodes, one pair attached to the external canthi, and the other to the infraorbital and supraorbital regions of the right eye. Both EEG an EOG were sampled at 1024 Hz using a BioSemi Active II system (www.biosemi.com). A low-pass filter of 256 Hz was applied at recording to avoid aliasing. An additional electrode was placed on the tip of the nose to be used as a reference.

Data Preprocessing

Data were analyzed using Brain Vision Analyzer (Brain Products GmBH; www.brainproducts.com) as well as in-house Matlab routines. Raw EEG data were high-pass filtered at 1.0 Hz (24 dB) and referenced to the tip of the nose. Eye movements and gross muscular activity were corrected using independent component analysis. Remaining artifacts exceeding ±100 μV in amplitude were rejected. Only correctly categorized faces were analyzed. That is, at study these included only faces that were subsequently remembered and at test these included only faces that were correctly categorized as old or new. Participants’ performance was exceptionally good, with few misses or false alarms (less than 3% on average); therefore, there were an insufficient number of trials to allow analysis of mistakes. The raw EEG signal was then segmented according to these conditions into epochs ranging from 400 msec before to 1000 msec after stimulus onset.

Local Synchronization

We performed a wavelet analysis on the single trials at each recording site in order to obtain the amplitude and phase of the EEG activity at all frequencies ranging from 1 to 100 Hz. To account for the different time–frequency tradeoffs for low and high frequencies, we used slightly different parameters for these two frequency bands. In all cases, we used a complex Gaussian Morlet wavelet, w(t, f) = Aexp(−t²/2σ_t²)exp(2iπft), where A was the normalization factor A = (σ_tπ)^−1/2. For frequencies between 1 and 20 Hz, the width of the wavelet was determined according to the Morlet parameter 2πσ_t f = 5 and the procedure was applied in steps of 1 Hz, whereas for frequencies between 20 and 100 Hz, a Morlet parameter of 2πσ_t f = 8 was applied in 2-Hz steps. The motivation for choosing a lower Morlet parameter for the low-frequency range was to improve the temporal resolution of these frequencies which tend to be “smeared” over time. The wavelet analysis yields a complex result ψ(t, f) = x(t, f) + iy(t, f) from which the amplitude Amp(t, f = x(t, f)² + y(t, f)² and phase $\phi (t,f)={tan}^{-1}\frac{y(t,f)}{x(t,f)}$ at each time–frequency point are extracted. Data were down-sampled in time by a factor of 4 to reduce the set size after the wavelet analysis was performed. We averaged the amplitudes at each time–frequency point across trials in each condition, and normalized them by subtracting the mean activity at each frequency level during an epoch between −300 and −100 msec from stimulus onset and dividing by the standard deviation during that time.

To reduce the possible contamination of the data by muscular artifacts related to microsaccades, which might be mistaken for cortical gamma activity (Yuval-Greenberg, Tommer, Keren, Nelken, & Deuoell, 2008), we based the time–frequency analysis of gamma on the current source density signal (CSD; second spatial derivative of the voltage). The CSD is a reference-free signal, thus restricting the possible contamination of scalp electrodes by eye movements to those electrodes near the eyes, as opposed to the commonly used nose reference which introduces this artifact at all scalp sites (for a similar approach, see Burgess & Gruzelier, 2000; Pulvermüller, Birbaumer, Lutzenberger, & Mohr, 1997; Lutzenberger, Pulvermüller, Elbert, & Birbaumer, 1995). CSD was calculated on the raw EEG segments using a spherical Laplacian (with four-spline interpolations; see Perrin, Pernier, Bertrand, & Echallier, 1989). The raw CSD segments were then subjected to wavelet analysis, as described above. The gamma activity obtained using this method was similar in its properties to that recorded intracranial being sustained over time and limited to frequencies over ~40 Hz, thus lending credibility to our findings. In addition, we recorded EMG from the sternocleidomastoid (SCM) and trapezius muscles of a subset of participants to rule out the possibility that the gamma effects are contaminated by muscular activity (as postulated by Whitham et al., 2008). The correlation and cross-correlation between the spectral content of the event-related EMG and EEG signals were not significant (p < .05), and the EMG was not modulated by the present experimental manipulations. We thus feel confident that the “gamma activity” reported here reflects, indeed, cortical activity in the gamma band.

It is customary to distinguish an increase of oscillatory activity that is phase-locked to the stimulus from that which is non-phase-locked activity (often termed “induced”). To differentiate these two types of neural dynamics, researchers often calculate the intertrial coherence (ITC), which is a measure of phase consistency across trials. A concurrent increase in amplitude and ITC reflects a phase-locked response, whereas an increase in amplitude that is not accompanied by a parallel increase in ITC reflects induced activity (Shah et al., 2004; Makeig et al., 2002). We calculated the ITC based on the phase information at each time–frequency point using a measure of spherical variance across trials $\text{ITC}(t,f)=\frac{1}{N}{\sum }_{j=1}^N{e}^{i{\phi }^j}$. These values were Z-Fisher transformed to maintain a normal distribution and baseline-corrected relative to an epoch between −300 and −100 msec before stimulus onset.

Statistical analysis was performed separately for each of the frequency bands (theta = 4–8 Hz, alpha = 9–13 Hz, gamma = 40–100 Hz). For each frequency band, we defined windows of interest (WOI), which restricted the analysis to the time windows during which a conspicuous change in amplitude relative to baseline was observed, and to recording sites exhibiting the strongest activity during this interval. These were determined based on inspection of the average response to all stimuli, collapsed across conditions. In addition, we verified that the mean activity in each WOI significantly differed from baseline using a paired t test.

Differences between conditions in each frequency band were established by two statistical analyses of the mean amplitudes in each WOI. In the study phase, we investigated the semantic effect of preexisting knowledge during the formation of new episodic traces. A paired t test was used to compare the activity elicited by famous and nonfamous faces. In the test phase, we investigated both the semantic factor (famous vs. nonfamous faces) and the episodic factor (old vs. new faces) using a two-way ANOVA with repeated measures. In cases where bilateral activity was found, the factor hemisphere (left, right) was added into the ANOVA, making it a three-way ANOVA. In addition, in order to determine whether the activity in each WOI is phase-locked or not phase-locked (induced), we performed a t test on the mean ITC in each WOI, collapsed across conditions. If the ITC differed significantly from zero, the activity in that WOI was described as phase-locked. Otherwise, it was considered induced.

Because we cannot assume that the time–frequency amplitudes and ITC values are normally distributed, the significance levels in all statistical tests were determined using a permutation test. For each subject, we randomly reassigned values to conditions and reran the relevant ANOVA or t test. This procedure was repeated 1000 times, yielding a data-driven distribution of t values or F values. The significance levels reported are relative to this distribution (Jokisch & Jensen, 2007; Nichols & Holmes, 2002; Burgess & Gruzelier, 2000).

RESULTS

Behavioral Results

Overall, participants were very good at remembering all of the studied faces, but did perform better with famous than nonfamous faces (Figure 1A). To reduce the likelihood of including trials on which participants guessed, we discarded trials on which participants’ confidence was low (rated 3 or 4; approximately 9% of the trials). For the remaining trials, participants correctly recognized 95.55% of the old famous faces and 86.45% the old nonfamous faces. Of the new faces, participants erroneously categorized 1.75% famous faces and 10.5% nonfamous faces as old. For each participant, we calculated the d-prime statistic (d′), which is a measure of sensitivity for discriminating between any two categories (in this case, “old” and “new” items); d′ is calculated by obtaining the Z-scores for the hit rate and false alarm rate individually for each participant, and subtracting the two. These statistics were subjected to a paired t test comparing memory for famous and nonfamous faces, which yielded a significant semantic effect [t(19) = 11.0, p < .001].

Figure 1.

(A) d′ scores on the old/new recognition test for famous and nonfamous faces. Overall, participants performed well on this task, and had better recognition memory for famous than nonfamous faces. (B) The proportion of high confidence responses (Confidence Level 1) was higher for both old and new famous faces, relative to nonfamous faces.

Moreover, even with low confidence trials removed, participants were more confident in their responses for famous compared with nonfamous faces (Figure 1B). Among the correctly recognized old faces that were given high confidence rates (1 or 2), the relative percentage of responses with confidence rating 1 was 95% for famous faces and 80% for nonfamous faces. Similarly, among the correctly categorized new faces, the relative percentage of responses with confidence rating 1 was 85% for famous faces and 52% for nonfamous faces. The difference between the percent of confidence 1 responses for famous versus nonfamous faces was statistically significant [t(19) = 7.98, p < .0001 for old; t(19) = 5.85, p < .0001 for new; Table 1].

Table 1.

Behavioral Results on the Old/New Recognition Test for Famous and Nonfamous Faces

	Old				New
Old/New Response	Correct Answer (Hit)		Wrong Answer (Miss)		Correct Answer (Correct Rejection)		Wrong Answer (False Alarm)
Confidence Level Response	1	2	1	2	1	2	1	2
Famous faces	91.85%	3.7%	0.85%	0.85%	82%	13.25%	1.2%	0.55%
Nonfamous faces	70.6%	15.85%	1.35%	3.55%	36.4%	29.95%	4.3%	6.25%

Recognition memory was better and the participants responded with higher confidence levels for famous than nonfamous faces.

In order to assist episodic memory formation, during study participants were asked to evaluate the beauty of each face. On average, more of the famous faces were judged as pretty than nonfamous faces [t(19) = 12.4, p < .0001], and more of the nonfamous faces were considered ugly than famous faces [t(19) = 3.4, p < .05].

Electrophysiological Results²

Early Phase-locked Response

Examination of the time–frequency plots obtained from averaging together all the conditions revealed an early phase-locked response during the first 400 msec after stimulus onset, spanning the entire spectrum from 1 to 40 Hz [see selection of WOIs in Figure 2; ITC t(19) = 7.9, p < .001]. The amplitude of this response decreased at higher frequencies and it was strongest in the theta range. This phase-locked response was maximal at centro-occipital site Oz, and probably reflects the spectral breakdown of the P1–N1 complex of the sensory ERP (Gruber, Klimesch, Sauseng, & Doppelmayr, 2005). Statistical analysis focused on these two electrodes and the factor hemisphere (left, right) was added into a three-way ANOVA with repeated measures. The phase-locked response was not modulated by the experimental conditions, nor were there any reliable effects or interactions [study: hemisphere, F(1, 19) = 2.53, p = .12; semantic, F(1, 19) < 1.0; test: hemisphere, F(1, 19) = 2.08, p = .17; semantic, F(1, 19) = 1.97, p = .17; episodic, F(1, 19) = 1.21, p = .28; Figure 3]. Due to the lack of experimental modulation, this early phase-locked response will not be further discussed.

Figure 2.

Demonstration of the time–frequency WOIs chosen for statistic analysis in (A) the low-frequency range (1–20 Hz) and (B) the high-frequency range (20–100 Hz; based on CSD). WOIs were chosen based on the mean time–frequency plots of the amplitude (top) averaged across all experimental conditions. Hot colors indicate amplitudes higher than baseline, whereas cold colors indicate amplitudes lower than baseline (i.e., desynchronization). Each WOI was categorized as phase-locked or induced based on the mean ITC in the WOI (bottom). The phase-locked WOI is delineated by a solid rectangle, and the induced WOIs are delineated by a dashed rectangle. (C) Scalp distributions of the amplitudes in each WOI are depicted to illustrate where the maximal activity was found.

Figure 3.

(A) Time course of the phase-locked response in the low-frequency range during the study and test sessions at electrode Oz. (B) Mean amplitude of the phase-locked response at electrode O2 between 0 and 400 msec. The phase-locked response was not significantly modulated by the present experimental manipulations.

Late Induced Response

Induced theta activity (4–8 Hz) was observed between 400 and 800 msec [see selection of WOIs in Figure 2; ITC t(19) < 1.0), primarily at frontal sites. Electrode Fz was chosen for statistical analysis. At study, frontal theta amplitudes were larger for famous than for nonfamous faces [t(19) = 1.75, p < .04]. At test, old faces elicited higher theta than new faces [F(1, 19) = 5.946, p < .02]. There was no main effect of the semantic factor [F(1, 19) = 1.65, p = .24], and no reliable interactions (F < 1.0; Figure 4A).

Figure 4.

(A) Time course of induced theta amplitude during the study and test sessions at electrodes Fz (left) and mean theta amplitudes in the chosen WOI in all conditions (right). (B) Time course of induced alpha amplitude during the study and test sessions at electrodes Oz (left) and mean alpha amplitudes in the chosen WOI in all conditions (right). Both theta and alpha amplitudes were significantly modulated by the semantic factor during study and by the episodic factor during test.

Alpha amplitude (8–12 Hz) decreased below baseline starting around 200 msec and lasted throughout the analyzed epoch, reflecting desynchronization. This desynchronization was of an induced nature [ITC t(19) < 1.0] and was maximal at Oz (see selection of WOIs in Figure 2). A significant modulation of alpha desynchronization by semantic knowledge was observed during study [t(16) = 2.14, p < .02], with a larger amplitude decrease for famous than for nonfamous faces. During test, there was no semantic effect (F < 1) but the desynchronization was larger for old than for new faces [F(1, 19) = 6.99, p < .01; Figure 4B]. Some authors report differences between lower (8–10 Hz) and upper (10–12 Hz) alpha activity (e.g., Klimesch et al., 2006). We also performed the analysis separately for these two frequency bands, and found both these ranges were similarly modulated by our manipulations with no distinction between them.

Induced gamma activity (40–100 Hz) increased relative to baseline at occipital sites starting from 100 msec and was sustained throughout the epoch, although the amplitude decreased over time [see selection of WOIs in Figure 2; ITC t(19) < 1.0]. During study there was a significant semantic effect [t(19) = −2.31, p < .05], with famous faces exhibiting higher amplitudes than nonfamous faces. At test, a significant episodic effect [F(1, 19) = 4.66, p < .05] revealed higher amplitudes for new items than for old items. A semantic effect showed a similar but nonsignificant trend as during study [F(1, 19) = 3.24, p = .065]. The semantic and episodic interaction also was not significant (F < 1.0; Figure 5).

Figure 5.

(A) Time–frequency plots illustrating the amplitude of the CSD signal in the high-frequency range, between 20 and 100 Hz. (B) Mean induced gamma amplitude in the chosen WOI. During study, famous faces elicited higher gamma than nonfamous faces, and a similar trend was found during test (gray asterisks). In addition, during test, gamma amplitude decreased for old faces relative to new faces.

DISCUSSION

The present study investigated neural activity in three frequency ranges during the encoding and the retrieval of episodic memories of human faces with a focus on the extent to which memory-related activities are modulated by pre-experimental semantic knowledge. As expected, recognition memory was better and participants’ associated confidence levels were higher for famous than for nonfamous faces. Clearly, prior semantic knowledge about faces can assist episodic recognition under these conditions. Critically, induced activity in the theta, alpha, and gamma bands were differentially modulated by both episodic and semantic factors, enabling us to explore the neural underpinnings of the various interactions between semantic and episodic factors in face recognition memory.

Pre-experimental semantic knowledge affected neural activity in these three frequency bands primarily during the formation/encoding of new episodic memories (i.e., during study). The advantage for famous faces during study was manifest in increased frontal theta and occipital gamma amplitudes as well as a larger decrease in occipital alpha for famous (compared to nonfamous) faces.

Moreover, during episodic retrieval (i.e., during test), all three frequency bands were affected by the old/new status of a stimulus, albeit differently in the different frequency band. Frontal theta and occipital alpha responses³ were larger in response to old than to new faces. In contrast, the occipital gamma response decreased for old faces compared to new faces. Furthermore, although gamma activity maintained a trend of a semantic effect during the test stage, which was in a similar direction as that found at study, the semantic effect was totally eliminated at test in the theta and alpha bands. Overall, this pattern supports the association of theta, alpha, and gamma activity to episodic retrieval; however, it seems that the two former frequency bands contribute differently to episodic memory than the latter.

Because theta and alpha activity were similarly modulated, we discuss their effects jointly but follow with some ideas about their distinctions. The episodic memory effect on theta activity reported here was manifest in larger amplitudes for old compared to new faces. This pattern is congruent with previous studies of episodic memory from human intracranial and scalp recordings (see Introduction). Indeed, theta involvement in long-term potentiation as well as in memory encoding, maintenance, and retrieval are well established (Greenstein, Pavlides, & Winson, 1988; Larson, Wong, & Lynch, 1986). Similarly, episodic memory has been linked to increased alpha desynchronization (i.e., decrease relative to baseline) in a similar manner to that reported here (Burgess & Gruzelier, 2000; Weiss & Rappelsberger, 2000; Klimesch, Doppelmayr, Pachinger, & Ripper, 1997). The agreement between the present results and the literature lends credence to our experimental setup while also establishing theta and alpha as reliable correlates of episodic memory retrieval, and allows us to hypothesize about the modulation of this activity by semantic factors.

Pre-experimental semantic knowledge increased theta and alpha responses during study (encoding) but not during the test (retrieval) stage. These findings were initially surprising, as many researchers have suggested that encoding and retrieval rely on similar neural mechanisms (e.g., Mesulam, 1998; Fuster, 1995). However, this pattern makes sense from a perspective of task-driven utilization of prior information. At study, pre-experimental information is beneficial for encoding a face into memory. In line with the Levels-of-Processing framework, integrating a newly formed episodic trace into an extensive network of associations activated by a familiar stimulus renders the trace stronger, thereby facilitating its subsequent retrieval (Craik & Tulving, 1975; Craik & Lockhart, 1972). In contrast, at test, deciding whether or not a certain stimulus has been seen during study might, in fact, be hindered by semantic familiarity. If both old and new familiar items activate extensive associative networks that are irrelevant to their episodic status, it would be more difficult to distinguish among members of the two categories. Therefore, we suggest that the memory processes reflected by theta and alpha activity are not impervious to preexisting semantic information, but utilize it selectively, namely, only when it is beneficial to the task at hand.

This line of reasoning leads to a number of testable predictions for future investigation. For example, intermingling famous and nonfamous face in the same memory retrieval block might render the semantic information relevant for retrieval per se, thereby leading to its use. Similarly, to the extent that the recognition task is made more difficult and/or the face identity becomes more task relevant, preexisting information becomes more useful, and thus, more likely to affect the memory retrieval increases manifest in theta and alpha activity.

The modulation of alpha desynchronization by semantic processing has been observed in several studies (e.g., Freunberger, Klimesch, Griesmayr, Sauseng, & Gruber, 2008; Klimesch, Doppelmayr, Pachinger, & Ripper, 1997; Klimesch et al., 1994). Röhm, Klimesch, Haider, and Doppelmayr (2001), for example, found that alpha desynchronization increases according to the semantic demands imposed by different tasks, which supports the view that alpha desynchronization is influenced by semantics selectively. By contrast, modulation of theta activity by semantic processing has not been reported to our knowledge, although its involvement in episodic memory and working memory is well established (Kahana, Seelig, & Madsen, 2001). Klimesch et al. (2006) attributed the differential sensitivity of theta and alpha to semantics as due to their involvement in different types of memory processing. Specifically, they suggested that alpha desynchronization is related to long-term memory, whereas theta is associated with short-term/working memory. The present results demonstrate for the first time that even if, as suggested, theta is related primarily to short-term memory, this process is not impervious to influences from long-term semantic memory. Rather, semantics probably assists the memory encoding process, which is reflected by modulation in theta activity.

The induced theta and alpha responses had a similar time course, exhibiting a relatively late onset (~200 msec after stimulus presentation) and a prolonged duration (compared, for example, with the early phase-locked theta response). This time course is more consistent with a secondary/recurrent rather than initial processing stage. Theories of memory retrieval suggest that hippocampal activation is followed by (back-) propagation from hippocampus to other cortical areas that originally supplied it with input and/or are involved in storing long-term memory representations (e.g., Moscovitch, Nadel, Winocur, Gilboa, & Rosenbaum, 2006; Nadel & Moscovitch, 1997; Treves & Rolls, 1994). It is possible that our observations of semantic and episodic memory on induced frontal theta and occipital alpha reflect the interaction of hippocampal regions with frontal and occipital regions, respectively.

Several lines of evidence support this hypothesis, albeit indirectly. For one, theta rhythm has been explicitly implicated as providing a dynamic link between hippocampus and frontal, prefrontal, and temporal neocortex involved in storing long-term memory representations (Düzel et al., 2005; Kirk & Mackay, 2003; Newman & Grace, 1999; Miller, 1991). This association is supported primarily by the similarity in the modulations of cortical and hippocampal theta in memory tasks. Moreover, in a recent study, Siapas, Lubenov, and Wilson (2005) demonstrated that prefrontal theta in the rat brain is phase-locked to hippocampal theta. This important finding provides additional support for the notion that the theta rhythm mediates communication between these two regions. Another line of evidence comes from findings demonstrating back-propagation from the monkey hippocampus to extrastriate regions. Face-specific responses in inferior temporal cortex (IT) are elicited at about 200–350 msec, whereas perirhinal activation is observed as early as 90 msec (Henson et al., 2003; Brown & Xiang, 1998; Treves & Rolls, 1994; Perrett, Rolls, & Caan, 1982). Although no direct connection has been made between this back-propagation and the alpha rhythm, there are several reasons why the memory effects on alpha might be related to hippocampal inputs. First, the onset of alpha suppression fits with the presumed timing of feedback responses from hippocampus to extrastriate regions. Second, multimodal cortical areas in superior temporal sulcus—the presumed source of the cortical alpha rhythm—receive inputs directly from hippocampus (Steriade, Gloor, Llinás, Lopes de Silva, & Mesulam, 1990). In addition, Von Stein, Chiang, and König (2000) implied the susceptibility of occipital alpha to top–down influences by demonstrating that alpha activity in primary cat visual cortex is driven by input from higher-order areas. In short, although to date there is no direct evidence attributing semantic memory effects on frontal theta and occipital alpha to episodic influences from hippocampus, we believe that this is a plausible and testable hypothesis.

The third frequency band of interest in the current study was the gamma band. We found that gamma activity was influenced by semantic knowledge during study, displaying higher amplitudes for famous than for nonfamous faces. Our results support the association of gamma activity in the occipital areas with activation of preexisting neural representation corresponding to an incoming visual stimulus (Jensen et al., 2007). In concert, this pattern is in line with demonstrations of larger gamma activity for coherent and meaningful stimuli compared to meaningless stimuli (Gruber, Maess, et al., 2008; Zion-Golumbic & Bentin, 2007; Lachaux et al., 2005; Singer & Gray, 1995). The greater gamma activity in response to famous faces probably reflects activation of a more elaborate neural representation for these faces compared to unfamiliar ones (see also Anaki, Zion-Golumbic, & Bentin, 2007). This elaborate semantic information may have facilitated episodic encoding by reinforcing the already existent associations. Supporting this view, previous studies showed that the degree of gamma activity in extrastriate regions during episodic encoding has been found to predict subsequent episodic memory performance (Mainy et al., 2007; Osipova et al., 2006).

The differences between famous and nonfamous faces may further suggest that gamma activity reflects activation of the most specific and individualized neural representation. For famous faces, representations corresponding to individual exemplars (e.g., “Jerry Seinfeld,” “Margaret Thatcher”) are accessed, whereas for unfamiliar faces, only categorical or subordinate-categorical representation can, in principle, be activated (e.g., “man,” “woman”; regarding the default level of face categorization, see Tanaka, 2001). A similar interpretation of modulations in gamma amplitude as reflecting different levels of categorization was given to explain higher gamma activity for faces compared to nonface stimuli (Zion-Golumbic et al., 2008).

Although the semantic effect in the current study was significant during study, it showed only a trend in that direction at test. As discussed above for the theta and alpha activity, just how much and what information is “automatically” activated in association with a particular stimulus may be driven by the demand characteristics of the task. Perhaps episodic recognition relies on semantic information less than encoding does, as reflected in the reduced semantic effect at test. This is, of course, a testable hypothesis. The notion of selective and task-dependent activation of the neural representations is supported by the significantly smaller gamma amplitudes for old faces at test compared to both new faces and to the same faces presented during study. This pattern is similar to previous findings reporting larger gamma amplitudes for correctly classified “new” words versus correctly recognized “old” words (Düzel et al., 2005). It is also consistent with gamma repetition suppression, that is, smaller gamma amplitudes elicited by repeated items compared to those presented for the first time (Trautner et al., 2006; Gruber & Müller, 2002, 2005) as well as with the smaller gamma amplitudes for primed versus unprimed words (semantic factors; Düzel, Habib, Guderian, & Heinze, 2004). A similar pattern of repetition suppression has also been observed in monkey inferior temporal cortex (Li, Miller, & Desimone, 1993) and in human BOLD signal in fMRI studies (Fiebach, Gruber, & Supp, 2005; Henson & Rugg, 2003; on the relationship between gamma activity and the BOLD signal, see Lachaux et al., 2007; Nir et al., 2007).

It has been suggested that gamma activity suppression when accessing a neural representation/semantic neighborhood for a second (vs. initial) time may reflect activation of a sparser network due to a “sharpening” of the representation (Axmacher, Elger, & Fell, 2008; Pexman, Hargreaves, Edwards, Henry, & Goodyear, 2007). Gamma repetition suppression is also a consequence of episodic factors to the extent that it reflects the fact that an item has recently been seen and, therefore, less information needs to be reactivated. However, we note an MEG study reporting patterns inconsistent with gamma repetition suppression (Osipova et al., 2006); specifically, these authors found a small but significant increase in gamma amplitude for correctly remembered old landscapes versus correctly classified new ones, although not for old versus new buildings. These discrepant patterns of gamma clearly need to be investigated.

One alternative explanation for the semantic effects reported here is that famous faces attract more attention than nonfamous faces. Indeed, famous and distinct faces tend to be processed more efficiently, a trait which may be due to allocation of attention (Ryu & Chaudhuri, 2007; Jackson & Raymond, 2006). In addition, the bias found here for judging famous faces as more attractive than non-famous faces might be related to their attracting more attention (although this may also be due to a social bias in deeming celebrities as beautiful, see also Halberstadt, 2006; Peskin & Newell, 2004). This is a viable interpretation especially because theta, alpha, and gamma synchronization are also known to be influenced by attention (Jensen et al., 2007; Sauseng, Hoppe, Klimesch, Gerloff, & Hummel, 2007). However, on this hypothesis, we would expect attentional effects to be similar in both the study and test sessions, which was not the case in any of our measures. We thus believe that the present data do not support the hypothesis that differences between theta, alpha, and gamma synchronization for famous and non-famous faces in this particular task stem from differential allocation of attention, but rather from differential utilization of semantic information for mnemonic tasks at hand.

Finally, we note that famous and nonfamous faces differ from each other not only in the richness of the information associated with each but also in perceptual familiarity, which is also greater for famous faces. The design of the present study does not enable us to determine to what extent the observed differences between famous and nonfamous faces during study reflect either or both of these differences. This issue can be addressed in studies that compare famous faces to perceptually similar faces that are not characterized by an elaborate verbal semantic network.

Conclusion

We find that the formation of episodic memory for faces is facilitated by preexisting semantic information as reflected in performance on the episodic recognition task. This facilitation is manifest in all three frequency bands investigated here—theta and gamma synchronization and alpha desynchronization—all enhanced for famous face compared to nonfamous faces during study. By contrast, pre-existing semantic knowledge played a marginal role at test as reflected by nonreliable semantic effects in these three frequency bands. This pattern suggests a fundamental difference in the reliance of encoding and retrieval processes on semantic information and is indicative of the distinction between these processes.

Moreover, we find that during test, activity in the theta and alpha bands is differentially modulated by episodic memory compared to gamma band activity. The pattern of these modulations, taken together with the extensive literature on the various factors affecting each frequency band, suggests they have different relationships to episodic retrieval and different roles in this process. Specifically, we hypothesize that gamma activity reflects the extent of the preexisting neural associations activated in response to a visual stimulus, whereas frontal theta synchronization and occipital alpha desynchronizations reflect episodic memory processes per se, which utilize semantic information when it is beneficial for task performance. This claim is in line with a recent study demonstrating that occipital gamma activity is modulated by stimulus familiarity, whereas theta activity is modulated by explicit episodic recollection (Gruber, Tsivilis, Giabbiconi, & Müller, 2008).

We believe that, in concert, our broad-range analysis of several EEG frequency bands provides new insights into the complex interplay between neural mechanisms contributing to semantic and episodic memory processes for faces. Specifically, we find that although there is clearly a strong interdependence between the neural mechanisms supporting episodic and semantic memory, the various influences of these two distinct memory processes can be differentiated by investigating their effects on neural synchronization in multiple frequency bands. The present study invites aggressive investigation of the functional roles of neural synchronization on the different aspects of recognition memory in general, and for faces in particular.

UNCITED REFERENCES

Guderian & Düzel, 2005

Rizzuto et al., 2003

Tulving & Markowitsch, 1997

Weiss, Müller, & Rappelsberger, 2000