Differentiating amodal familiarity from modality-specific memory processes: An ERP study

Abstract

Distinct event-related potential effects have been related to familiarity and recollection processes underlying recognition memory. Familiarity has been conceptualized as similar either to perceptual priming mechanisms supporting implicit memory or to amodal global-matching processes that should show little sensitivity to perceptual variables. The present experiment manipulated the study modality of words (auditory, visual) that were visually tested for recognition memory. The mid-frontal (300–500 ms) old/new effect often attributed to familiarity was not affected by study modality, so it appears related to an amodal familiarity process. An earlier (176–260 ms) fronto-polar old/new effect was perceptually specific in that it was observed only following visual study. The parietal old/new effect (400–800 ms), often attributed to recollection, was similar following both visual and auditory study. Temporal-spatial PCA clarified the separability of these effects.

Dual-process theories assume that recognition memory is supported by separate processes of recollection and familiarity (reviewed in Yonelinas, 2002). Recollection refers to a recall-like process that retrieves detailed memories about individual items or episodes. Familiarity has been conceptualized in various ways, but generally refers to a memory process that supports recognition without the detailed information that is characteristic of recollection. Some dual-process theories have suggested that common processes may underlie both familiarity in recognition memory and priming on implicit memory tests (Jacoby, 1991; Mandler, 1980). According to this view, for example, if you studied the word TRUCK, the same processes that make TRUCK familiar on a recognition memory test would also make you more likely to use that word as a completion for TRU__ on a word-stem completion task. On the other hand, evidence suggests that implicit memory and familiarity can be dissociated (Wagner & Gabrieli, 1998). For example, amnesic patients showing normal implicit stem-completion priming were impaired on recognition memory tests designed to foster reliance on familiarity (Stark & Squire, 2000). Conversely, a patient with damage to the right occipital cortex showed normal familiarity-based recognition memory despite impaired perceptual priming (Wagner, Stebbins, Masciari, Fleischman, & Gabrieli, 1998). Additionally, numerous experimental variables have been shown to differentially affect familiarity and implicit memory (especially on perceptual priming tasks in which test cues are physically, rather than semantically/conceptually, related to studied stimuli, as reviewed by Wagner & Gabrieli, 1998; Yonelinas, 2002).

Event-related brain potentials (ERPs) have been used to study distinctions between explicit/implicit memory as well as the distinction between recollection and familiarity. There has been some consensus that an ERP difference observed between correctly recognized old and new items from about 400 to 800 ms (often called the “parietal old/new effect”) is related to recollection because it is primarily observed under conditions in which subjects are able to recollect detailed information about studied items (Allan, Wilding, & Rugg, 1998; Curran, 2000; Friedman & Johnson, 2000; Mecklinger, 2000). Other ERP effects are considered to be memory related because they respond differentially to old and new items, but they are thought to reflect the activity of processes distinct from recollection. Rugg et al. (1998) observed an ERP difference between old items and new items (300–500 ms, maximal over parietal channels) that was attributed to implicit memory because it was equally apparent whether or not subjects explicitly recognized the test words (ERPs to hits=misses >correct rejections; also observed by Düzel, Vargha-Khadem, Heinze, & Mishkin, 2001). Paller and colleagues have suggested that processes underlying perceptual word priming are reflected within ERP differences (300–500 ms, maximal over occipital channels) that, like behavioral priming effects, are sensitive to changes in the perceptual format of repeated items (Paller & Gross, 1998; Paller, Kutas, & McIsaac, 1998).

Other research has suggested that frontally distributed ERP old/new differences (also appearing between 300 and 500 ms) may be related to familiarity (Curran, 1999, 2000; Curran & Cleary, 2003; Curran, Tanaka, & Weiskopf, 2002; Guillem, Bicu, & Debruille, 2001; Mecklinger, 2000; Nessler, Mecklinger, & Penney, 2001; Rugg et al., 1998). We have called this the “FN400 old/new effect” because of its similarity to the N400 component related to semantic processing (Kutas & Iragui, 1998; Kutas & Van Petten, 1994), but it is more frontally distributed than the centro-parietal N400 typically observed in language studies. The FN400 responds similarly to studied items and similar lures (plurality reversed words, Curran, 2000; mirror-reversed pictures, Curran & Cleary, 2003; geometrically similar shapes, Curran et al., 2002; and semantically similar words, Nessler et al., 2001). Thus, assuming that familiarity is a process that responds to the overall similarity between studied and tested items, like the familiarity-based global matching models of recognition memory (Gillund & Shiffrin, 1984; Hintzman, 1988; Hintzman & Curran, 1994; Shiffrin & Steyvers, 1997), such sensitivity to study-test similarity (e.g., original plurality of words or orientation of pictures) would be consistent with the activity of a familiarity-based process.

The observation that different ERP effects have been associated with implicit priming and familiarity is consistent with the notion that separate processes underlie these phenomena (Wagner & Gabrieli, 1998; Yonelinas, 2002). On the other hand, there are reasons to investigate the possibility that the FN400 old/new effect maybe yet another manifestation of perceptual priming. Curran, Schacter, Johnson, & Spinks (2001) used a variant of the Deese/Roediger-McDermott (DRM) false memory paradigm (Deese, 1959; Roediger & McDermott, 1995) to investigate the sensitivity of the FN400 to semantically similar lures. Surprisingly, no FN400 old/new effect was observed even when comparing studied items to semantically unrelated lures. Curran et al. (2001) speculated that the FN400 old/new effect may have been absent because the presentation modality of the words changed between study (auditory) and test (visual). Such sensitivity to study-test modality raises the possibility that the FN400 old/new effect is related to perceptual priming more so than familiarity because perceptual priming is sensitive to manipulations of the physical similarity between study and test conditions (e.g., Curran & Schacter, 1996; Curran, Schacter, & Galluccio, 1999). However, a subsequent ERP experiment using categorically related nouns as lures did observe significant FN400 old/new effects despite also switching from auditory study to visual test (Nessler et al., 2001).¹ Apart from these “false memory” experiments, two other ERP experiments have manipulated the study modality prior to memory tests (Joyce, Paller, Schwartz, & Kutas, 1999; Wilding, Doyle, & Rugg, 1995), and modality effects within the 300–500-ms temporal window of the FN400 were mixed. Wilding et al. found no modality effects whereas Joyce et al. observed differences (across the entire scalp) between visual and auditory study conditions between 300 and 400 ms. Reasons for these discrepant results are unclear, but they underscore the importance of directly investigating the manner in which modality influences the FN400 old/new effect.

In summary, previous research suggests that the familiarity process(es) supporting recognition memory are distinct from the perceptual priming processes that support implicit memory. The FN400 old/new effect has been hypothesized to be related to an amodal familiarity signal arising from a global-matching process, so the present experiment tests the alternative possibility that the FN400 is related to perceptual priming by manipulating the study modality (auditory or visual) of visually tested words. Each subject completed four study-test blocks, half containing visually studied words and half containing auditorally studied words. If the FN400 old/new effect is not affected by modality, the results would be consistent with the idea that the FN400 is related to an amodal familiarity process that is unrelated to perceptual priming. If the FN400 old/new effect is greater following visual than auditory study, the results would suggest that the FN400 reflects the activity of a process that could contribute to perceptual priming.

Method

Participants

Twenty-seven, right-handed students from the University of Colorado participated in one session (approximately 2 hr) in partial fulfillment of a course requirement. Data from 3 participants were discarded because of excessive eye blinking. Ten of the 24 participants retained for analyses were male.

Stimuli

Stimuli were 320 common English words. Visual words were centrally presented in white against a black background. Visual display used a Mitsubishi 15-in. LCD Color Display monitor. Auditory words were prerecorded digitally in a single male voice, and presented over Harmon/Kardon Multimedia Speakers placed behind the monitor. The words were divided into four sets of 80 that were roughly equated for word frequency (M=14.64, SD=18.54, range=0:99; Kucera & Francis, 1967) and number of letters (M=5.42, SD=1.06, range=4:7). These sets were counterbalanced through the modality and old/new conditions across participants. Additional words with similar characteristics were used as practice and buffer items.

Design

Study modality (auditory, visual) and memory status (old, new) of the words were manipulated within participants. Study modality was alternated between four study/test blocks such that odd/even study lists were auditory/visual (actual order was counterbalanced across participants). Study lists contained 40 words. Test lists contained 40 old words randomly intermixed with 40 new words.

Procedure

Participants were instructed before completing two short practice blocks (one block per study modality, 20 studied items/block, 8 test items/block). A Geodesic Sensor Net was applied between the practice and experimental blocks.

Each study block contained 40 words surrounded by 2-word primacy and recency buffers. Both auditory and visual words were presented at a rate of one every 1,600 ms. A 600-ms fixation sign (+) preceded the presentation of each test word. Visual words were presented for 1,000 ms. Auditory sound files lasted 1,000 ms including a variable silent period following the word's end (due to differences in pronunciation duration). In other words, both visual and auditory words were presented at a rate of 1 per 1,600 ms. Subjects were instructed to try to remember each word, but were given no formal encoding task. A rest period of at least 2 min intervened between each study and test list.

Each test block contained 80 test words that were divided into four subblocks, so that participants could periodically rest their eyes. Each subblock began with a new word that was not included in analyses to minimize postbreak movement artifacts. All test-trial events were synchronized to the refresh cycle of the monitor. Trials began with a variable duration plus sign (500 to 1,000 ms), followed by test word presentation for 2,000 ms, followed by a question mark. Participants were instructed to withhold their responses until the question mark appeared, and to minimize blinking and other movements. Participants pressed “old” or “new” keys with the index finger of each hand (assignment of old/new to left/right keys was counterbalanced across participants).

EEG/ERP Methods

Scalp voltages were collected with a 128-channel Geodesic Sensor Nett™ (Tucker, 1993) connected to an AC-coupled, 128-channel, high-input impedance amplifier (200 MΩ, Net Amps™, Electrical Geodesics Inc., Eugene, OR). Amplified analog voltages (0.1–100 Hz bandpass, −3 dB) were digitized at 250 Hz. Individual sensors were adjusted until impedances were less than 50 kΩ.

ERPs were baseline corrected with respect to a 100-ms prestimulus recording interval and digitally low-pass filtered at 40 Hz. Trials were discarded from analyses if they contained incorrect responses, eye movements (EOG over 70 μV) or more than 20% of channels were bad (average amplitude over 100 μV or transit amplitude over 50 μV). At least 27 acceptable trials were retained for each participant in each condition. The mean number of trials in each condition was: auditory/new=57; auditory/old=53; visual/new=54; visual/old=51. Individual bad channels were replaced on a trial-by-trial basis with a spherical spline algorithm (Srinivasan, Nunez, Silberstein, Tucker, & Cadusch, 1996). EEG was measured with respect to a vertex reference (Cz), but an average-reference transformation was used to minimize the effects of reference-site activity and accurately estimate the scalp topography of the measured electrical fields (Bertrand, Perin, & Pernier, 1985; Curran, Tucker, Kutas, & Posner, 1993; Dien, 1998b; Lehman & Skrandies, 1985; Picton, Lins, & Scherg, 1995; Tucker, Liotti, Potts, Russell, & Posner, 1994). Average-reference ERPs are computed for each channel as the voltage difference between that channel and the average of all channels. The average reference was corrected for the polar average reference effect (Junghöfer, Elbert, Tucker, & Braun, 1999).

Results

Behavioral Results

A modality×old/new ANOVA indicated that accuracy was higher following auditory than visual study, F(1,22)=7.51, MSE=0.004, p<.05. The modality difference was significant for correct rejections (auditory=81%, visual=77%), t(23)=2.27, SE=.02, p<.05; but not for hits (auditory=76%, visual=72%), t(23)=1.61, SE=.02, p>.10. Computing d′between old and new items also showed that discrimination was higher following auditory (M=1.77) than visual study (1.48), t(23)=2.75, SE=.10, p<.01. These d′ values are somewhat low, and might be attributable to not having a formal encoding task during study.

ERP Results

ERPs recorded near selected locations from the international 10-20 system are shown in Figure 1 (Jasper, 1958). In addition to the FN400 and parietal old/new effects that were the a priori focus of this study, interesting old/new effects were also observed during the time of the P2 component. The FN400 was analyzed over two anterior, superior channels groups centered near the standard F3 and F4 locations (following Curran, 2000; Curran & Cleary, 2003; Curran et al., 2001, 2002). These are labeled left and right anterior/superior regions (LAS and RAS, see Figure 2). The P2 effects also peaked near these regions, so they also were used in the initial P2 analyses. The parietal old/new effect was analyzed over two posterior, superior channels groups that included the standard P3/P4 locations in addition to more anterior channels (following Curran, 2000; Curran & Cleary, 2003; Curran et al., 2001, 2002). These are labeled left and right posterior/superior regions (LPS and RPS, see Figure 2). For completeness, we also analyzed late (1,000–1,500 ms) frontal ERP effects (old>new) that are often observed in ERP studies of recognition memory (Allan et al., 1998; Curran & Friedman, in press; Curran et al., 2001; Johnson, Kounios, & Nolde, 1996; Ranganath & Paller, 2000; Wilding, 1999; Wilding & Rugg, 1997a, 1997b).²

Figure 1.

Grand-average ERPs from channels representative of the international 10–20 system (Jasper, 1958). Channels are labeled according to Geodesic Electrode Net numbers (see Figure 2) along with their nearest 10–20 equivalent location.

Figure 2.

Grand-average ERPs from regions of interest used in ANOVAs. Regions are depicted with black channel groups. LAS is left/anterior/superior. RAS is right/anterior/superior. LPS is left/posterior/superior. RPS is right/posterior/superior.

P2 (176–260 ms)

The following analyses examined possible perceptual priming effects that were apparent from inspection of early ERPs. Visual inspection of the P2 component indicated that it was more positive for old than new words following visual study, but not following auditory study (see LAS and RAS regions of Figure 2). When averaged across all conditions, the P2 peaked at 218 ms. Mean amplitudes from the LAS and RAS were calculated within a 176–260-ms window defined by the mean latency (218 ms) ±2 standard deviations (rounded to the nearest time sample). An old/new×modality×hemisphere ANOVA resulted in two significant effects. Mean amplitude was more positive for old than new items, F(1, 23)=11.81, MSE=0.72, p<.01, and the old/new difference interacted with modality, F(1,23)=10.36, MSE=0.77, p<.01. Separate ANOVAs for each modality revealed a significant old/new effect following visual study, F(1,23)=24.11, MSE=0.68, p<.001, but not following auditory study, F<1, MSE=0.81. Thus, as illustrated in Figure 3a, the P2 showed a modality specific old/new effect. The new topography of the P2 old/new effect is shown in Figure 4 (first column).

Figure 3.

Mean amplitudes for each spatiotemporal region of interest. Error bars represent the standard error of the old/new difference. a: Mean amplitudes across the LAS and RAS regions from 176 to 260 ms. b: Mean amplitudes across the LAS and RAS regions from 300 to 500 ms. c: Mean amplitudes within the LPS and RPS regions from 400 to 800 ms.

Figure 4.

Topography of the old/new differences estimated by spherical-spline interpolations (Srinivasan et al., 1996). The front of the head is depicted at the top of each oval.

FN400 Results (300–500 ms)

Mean LAS and RAS amplitudes between 300 and 500 ms were the dependent measures. An old/new×modality×hemisphere ANOVA indicated that amplitudes were more negative for new than old items, F(1,23)=26.05, MSE=0.65, p<.001. The old/new effect did not interact with modality, F(1,23)=1.16, MSE=1.04, but the old/new effect appeared slightly larger following visual than auditory study (see Figure 3b). Separate ANOVAs confirmed that the old/new difference was significant for each modality alone: auditory, F(1,23)=6.46, MSE=0.70, p<.05; visual, F(1,23)=13.79, MSE=0.98, p<.01. The topography of the FN400 old/new effect is shown in Figure 4 (second column).

Parietal Results (400–800 ms)

Mean LPS and RPS amplitudes across each channel group between 400 and 800 ms were the dependent measures. An old/new×modality×hemisphere ANOVA indicated that amplitudes were more positive for old than new items, F(1,23)=16.56, MSE=2.54, p<.001. The parietal old/new effect typically is larger over the left than right hemisphere, but the old/new×hemisphere interaction was not significant, F(1, 23)=2.77, MSE=0.28, p=.11. There was a significant old/new×modality×hemisphere interaction, F(1,23)=4.18, MSE=0.21, p=.05, but the interaction was not significant when the amplitudes were rescaled to remove overall amplitude differences between conditions (vector length method; McCarthy & Wood, 1985). The topography of the parietal old/new effect is shown in Figure 4 (third column).

Late Frontal Results (1,000–1,500 ms)

Mean amplitudes for left (F7, Fp1) and right (F8, Fp2) frontal channels between 1,000 and 1,500 ms were the dependent measures. An old/new×modality×hemisphere×channel ANOVA indicated that amplitudes were more positive for old than new items, F(1, 23)=5.56, MSE=6.22, p<.05. These old/new differences did not significantly interact with any of the other variables, so they are not considered further.

Principal Components Analysis (PCA)

A temporal-spatial PCA was performed to better understand the spatiotemporal relationship among the ERP effects reported above (Spencer, Dien, & Donchin, 1999, 2001). A temporal PCA was followed by a spatial PCA.³ The temporal PCA was calculated from −96 to 900 ms with 12,384 observations per time point (24 participants×4 conditions×129 channels=12,384). Covariance was used as the measure of association. A Promax rotation (Dien, 1998a; Hendrickson & White, 1964) was used, which involves first applying a Varimax rotation and then relaxing it to allow for correlated factors. A screen test indicated that 14 factors should be retained (95% of variance accounted for by Varimax solution, 79% of variance accounted for by Promax solution). We chose to focus exclusively on factors with time courses similar to the primary effects described above. Figure 5 shows the time course of three temporal factors (TF) that followed the timing of the old/new effects described previously: P2 (TF3, peak latency = 220 ms), FN400 (TF4, peak latency = 408 ms), and parietal (TF1, peak latency = 596 ms). These factors accounted for 22% (TF3), 33% (TF4), and 43% (TF1) of the variance. The only other factors accounting for greater than 15% of the variance were TF2 (40%) and TF5 (29%) (percentages sum to greater than 100% because an oblique rotation allows the factors to be correlated). TF2 corresponded to a commonly observed temporal factor peaking near the end of the epoch (e.g., Donchin & Heffley, 1979; Spencer et al., 2001). TF5 peaked at 328 ms. ANOVAs suggested that neither of these factors were reliably affected by the present experimental manipulations (p>.20 for all effects), so they are not considered further.

Figure 5.

The time course of the three primary temporal factors (TF) estimated by PCA.

Separate spatial PCAs were performed on each of the 14 temporal factors. The data matrix consisted of 129 channels by 96 observations (24 participants × 4 conditions = 96). Five spatial factors were retained for each temporal factor. Based on visual inspection of the timing, topography, and condition effects, four spatiotemporal factors were analyzed as latent components possibly underlying the main experimental effects described previously. The topography of the old/new differences for each of these spatiotemporal factors is shown in Figure 6. The spatial PCA procedure enforces identical scalp topographies for a given factor across all conditions, although they are free to differ in amplitude.

Figure 6.

The topography of the old/new differences for the four primary spatiotemporal factors. VIS is visual. AUD is auditory.

The second spatial factor of the third temporal factor (TF3/SF2, peaking at 220 ms) was consistent with the P2 (first column of Figure 6). ANOVA on the factor scores revealed a significant old/new difference, F(1,23) = 4.13, MSE = 1.55, p = .05, that marginally interacted with modality, F(1,23) = 3.84, MSE = 1.02, p = .06. Separate ANOVAs on each study modality indicated that the old/new effect was significant following visual study, F(1,23) = 7.00, MSE = 1.10, p = .01, but not following auditory study, F<1, MSE = 0.84.

TF4 (408 ms) was decomposed into two spatial factors possibly related to the FN400. TF4/SF2 had a fronto-polar distribution (second column of Figure 6). Like the FN400, TF4/SF1 showed a significant old/new difference, F(1,23) = 13.77, MSE = 1.56, p<.01, that did not interact with modality, F<1, MSE = 1.74. Old/new differences were significant for both visual, F(1,23) = 7.64, MSE = 1.89, p<.01, and auditory study, F(1,23) = 5.40, MSE = 1.41, p<.05. The second FN400–like factor, TF4/SF1, was distributed around the vertex (third column of Figure 6). Again, it showed a significant old/new difference, F(1,23) = 14.26, MSE = 1.07, p<.001, that did not interact with modality, F<1, MSE = 0.83. The old/new difference was significant for the both modalities: visual: F(1, 23) = 7.66, MSE = 1.20, p = .01; auditory: F(1,23) = 8.87, MSE = 0.71, p<.01.

TF1/SF3 (596 ms) was distributed similarly to the parietal old/new effects (fourth column of Figure 6). Like the parietal old/new effect, the factor scores showed a significant old/new difference, F(1,23) = 35.90, MSE = 1.52, p<.001, that did not interact with modality, F<1, MSE = 0.46.

Discussion

The present experiment investigated the effect of study modality on ERP old/new effects recorded during visual recognition memory tests. An early old/new effect coincident with the P2 component (176–260 ms) showed old/new effects when words were studied visually, but not auditorally. Later (FN400: 300–500 ms; parietal: 400–800 ms) old/new differences did not interact with modality.

Recognition discrimination (d′) was higher following auditory than visual study. Previous investigations of modality effects on recognition memory have yielded mixed results. Some studies have failed to find significant modality effects (Craik, Moscovitch, & McDowd, 1994; Hayman & Rickards, 1995; Hintzman, Block, & Inskeep, 1972; Joyce et al., 1999; Smith & Hunt, 1998), others have found an advantage when study and test modality match (Gallo, McDermott, Percer, & Roediger, 2001; Gregg & Gardiner, 1994; Jacoby & Dallas, 1981; Kirsner, 1974; Maylor & Mo, 1999; Mulligan & Hirshman, 1995; Wilding et al., 1995),⁴ and one other study, like the present study, found that performance on visual recognition tests can be better following auditory study (Hintzman & Caulton, 1997).

One might think that a visual study advantage arising from perceptual priming mechanisms would have been more likely if speeded responses were given rather than the delayed responses required in the present experiment. However, the results of two previous studies argue against this possibility. Speed-accuracy trade-off (SAT) experiments have been used to study the time course over which different types of information/processes influence performance by requiring subjects to make speeded responses at unpredictable times. SAT studies have generally supported the idea that familiarity can influence performance prior to recollection (Hintzman & Curran, 1994; McElree, Dolan, & Jacoby, 1999). Two SAT studies have measured visual recognition memory following visual or auditory study (Hintzman & Caulton, 1997; Mulligan & Hirshman, 1995). Mulligan and Hirshman found that slow responses at asymptotic discrimination levels were more accurate following visual than auditory study, whereas Hintzman and Caulton found asymptotic accuracy to be higher for auditory than visual study. Most importantly for the present purposes, neither study suggested that study modality affected fast responses. Thus, these studies suggest that slow responding did not necessarily mask any modality matching effects in the present study. As a whole, modality effects on recognition memory performance tend to be small and unreliable. This is one reason to be doubtful that familiarity is entirely attributable to perceptual priming. In turn, the amodal nature of the FN400 old/new effect is consistent with its hypothesized relationship to familiarity.

ERP old/new effects coinciding with the P2 (176–260 ms) were specific to words studied and tested in the visual modality, so they may be related to visually specific memory process. Because behavioral modality effects are a hallmark of perceptual implicit memory (e.g., Curran & Schacter, 1996; Curran et al., 1999), it is natural to consider these visually specific ERP old/new effects to be related to perceptual priming mechanisms. Because visual study did not lead to better recognition performance than auditory study in the present study, any such priming mechanisms may not have contributed to performance. These visually specific old/new differences appeared earlier than the 300–500-ms ERP effects that have previously been attributable to visual word priming (Paller & Gross, 1998; Paller et al., 1998) or implicit memory (Rugg et al., 1998). Spatiotemporal PCA indicated that P2 old/new difference was maximal over fronto-polar regions, unlike the posterior distribution of the visual word priming (Paller & Gross, 1998; Paller et al., 1998) and implicit memory effects (Rugg et al., 1998).

Early (i.e., around 200 ms) modality-specific repetition effects have been observed when repetitions are more closely spaced (zero to six items between repetitions: Rugg, Doyle, & Melan, 1993; Rugg, Doyle, & Wells, 1995; Rugg & Nieto-Vegas, 1999), but have been less commonly observed in study-test recognition memory designs in which many items and several minutes intervene between study and test presentations of each item. Such interrepetition interval differences may explain why P2 old/new differences were observed in the present experiment, but not in a previous ERP study of recognition memory that manipulated study/test modality (Wilding et al., 1995). The present study used 44-word study lists, 2-min study/test breaks, and 80-word recognition test lists. Wilding et al. had a 240-word study list, a 5-min break, and a 480-word recognition test. Similar early, frontal old/new differences were observed for visual pseudo-words but not for words (Curran, 1999). Interestingly, these early pseudoword old/new effects were observed during recognition memory tests, but not during lexical decision tests. A previous experiment with list lengths similar to the present (30 study, 60 test) also found no P2 old/new effects on a lexical decision test (Joyce et al., 1999). Thus, contrary to the perspective that these modality-specific priming effects are related to perceptual priming, they may be exclusively observed under explicit memory testing. Thus, one must entertain the possibility that the present early old/new effect is related to intentional retrieval of modality-specific information rather than an implicit perceptual priming mechanism. Unfortunately, the present experiment provides no direct evidence regarding the implicit/explicit nature of this component, although relevant analyses were attempted (see footnote ²). Very early (100–300 ms) old/new differences with a similar frontopolar distribution also were observed in a recognition memory experiment testing memory for visual objects presented against varying environmental backgrounds (Tsivilis, Otten, & Rugg, 2001). One conjecture is that the early perceptual-specific effects are primarily observed when the visual modality is made salient by varying modality at study (the present experiment), using semantically impoverished stimuli (pseudowords in Curran, 1999), or using visually rich stimuli (Tsivilis et al., 2001).

The primary question driving the present research concerned the possible modality specificity of the FN400 old/new effect. The FN400 old/new effect was observed following both visual and auditory study. As reviewed in the introduction, familiarity has variously been considered to be related to (Jacoby, 1991; Mandler, 1980) or distinct from (Stark & Squire, 2000; Wagner & Gabrieli, 1998; Wagner et al., 1998; Yonelinas, 2002) perceptual implicit memory. Our results are more consistent with the latter view in that the familiarity-like process indexed by the FN400 old/new effect is not modality specific. There was a nonsignificant trend toward a greater FN400 old/new effect for visual than auditory conditions. This slight perceptual enhancement would be expected of a familiarity process that includes perceptual attributes as a minor aspect of the overall global matching process, but it is likely that other attributes (e.g., semantics) play a greater role.

The temporal-spatial PCA suggested that the FN400 may be multiply determined. A temporal factor peaking at 408 ms (TF4) was spatially decomposed into separate fronto-polar (TF4/SF2) and vertex (TF4/SF1) factors with similarities to the FN400. It is possible that the overall distribution of the FN400, often described as mid-frontal (Tsivilis et al., 2001) or medial frontal (Friedman & Johnson, 2000), represents the superposition of these two distinct sources. The “FN400” typically is distributed more frontally than the classic centro-parietal N400 that has primarily been described in psycholinguistic studies of semantic processing (e.g., Kutas & Iragui, 1998; Kutas & Van Petten, 1994), but the true relationship between these effects remains uncertain. When PCA was applied to the results of the sentence congruity paradigm most often used to elicit the classic N400, a factor distributed similarly to the present 408 ms vertex factor was identified (Dien, Frishkoff, Cerbone, & Tucker, 2003). This suggests the possibility that the familiarity-related FN400 may represent the superposition of the classic N400 with a temporally coincident fronto-polar effect that results in an overall mid-frontal distribution. Given the well-documented sensitivity of the classic N400 to semantic rather than perceptual processing, the insensitivity of the 408 ms vertex factor to modality changes is consistent with this particular factor being associated with the classic N400.

The topographic similarity between frontopolar factors peaking at 220 ms (TF3/SF2) and 408 ms (TF4/SF2) invites speculation that these factors reflect the time-varying activity of a common mechanism (Figure 6). The contrast between the modality specificity of the earlier factor and the amodality of the later factor constrains any such interpretation. On the one hand, the different effects of modality on the two factors may suggest the existence of separate underlying processes generating coincidentally similar topographies. On the other hand, a common process may be initially sensitive to modality specific aspects of memory, but later become sensitive to high-level information that takes more time to accrue. Further research is necessary to examine these possibilities.

The two 408-ms spatial factors may be related to previous reports of distinct 300–500-ms, memory-related ERP effects over frontal and parietal regions (Düzel et al., 2001; Rugg et al., 1998). Both effects discriminate ERPs associated with hits from those associated with correct rejections, but only the parietal effect differed between misses and correct rejections. Thus, the 300–500-ms parietal effect may be related to implicit memory because it discriminates old from new items in the absence of behavioral evidence of explicit recognition. PCA including trials with misses failed to confirm this possibility, but this null effect may be related to the low number of misses obtained in the present experiment (see footnote ²).

Conventional analyses revealed little difference between the FN400 and parietal old/new effects. Both old/new effects were observed after either visual or auditory study. Although the FN400 old/new difference tended to be larger following visual study, this does not constitute a convincing dissociation. Furthermore, the topography of these effects were somewhat similar (Figure 4). Several other experiments, as reviewed in the introduction, have functionally dissociated the FN400 and parietal old/new effects by manipulating variables that differentially affect them (Curran, 1999, 2000; Curran & Cleary, 2003; Curran et al., 2002; Nessler et al., 2001; Rugg et al., 1998; Tsivilis et al., 2001). In the present experiment, we were able to objectively differentiate the FN400 and parietal old/new effects with PCA. This was primarily accomplished through the temporal PCA that revealed distinct factors peaking near 400 and 600 ms. Thus, PCA appears to be a promising method for dissociating these memory-related ERP effects despite the spatiotemporal and functional overlap.

In summary, three distinct ERP old/new effects were identified through analysis of mean amplitudes and temporal-spatial PCA. First, an early old/new effect coincident with the P2 (176–260 ms) was specific to the visual modality. Its modality specificity invites the possibility that it is related to implicit perceptual priming, but the present experiment provides no direct information on the implicit/explicit nature of this effect. Future research is needed to better understand the nature of the underlying processes. Second, the mid-frontal FN400 old/new effect (300–500 ms) was observed following both visual and auditory study modalities. Combined with previous evidence relating the FN400 to familiarity, this suggests that the FN400 is more likely related to an amodal familiarity process than a process supporting perceptual priming. Third, a parietal old/new effect (400–800 ms) was also observed following both visual and auditory study. The parietal old/new effect is often thought to underlie the recollection of specific information. Even though the parietal and FN400 old/new effects could not be functionally dissociated with the modality manipulation used in the present experiment, we found that PCA clearly associated these old/new effects with distinct temporal-spatial sources of variability.