Prefrontal Engagement during Source Memory Retrieval Depends on the Prior Encoding Task

Abstract

The prefrontal cortex is strongly engaged by some, but not all, episodic memory tests. Prior work has shown that source recognition tests—those that require memory for conjunctions of studied attributes—yield deficient performance in patients with prefrontal damage and greater prefrontal activity in healthy subjects, as compared to simple recognition tests. Here, we tested the hypothesis that there is no intrinsic relationship between the prefrontal cortex and source memory, but that the prefrontal cortex is engaged by the demand to retrieve weakly encoded relationships. Subjects attempted to remember object/color conjunctions after an encoding task that focused on object identity alone, and an integrative encoding task that encouraged attention to object/color relationships. After the integrative encoding task, the late prefrontal brain electrical activity that typically occurs in source memory tests was eliminated. Earlier brain electrical activity related to successful recognition of the objects was unaffected by the nature of prior encoding.

INTRODUCTION

The likelihood of successful memory performance, as well as the content of what is remembered, depends on processes engaged during both initial learning and the subsequent retrieval attempt. Measures of human brain activity during memory tests have confirmed two traditional ideas about encoding/retrieval relationships. Brain electrical activity (event-related potentials [ERPs]) shows a stronger differentiation between studied and unstudied material if the study-phase task had focused on the meanings of words as compared to their letters (Paller & Kutas, 1992). Hemodynamic images similarly show greater activity in numerous brain areas after semantic encoding as compared to judgments about superficial detail (Mandzia, Black, McAndrews, Grady, & Graham, 2004). These findings support the levels-of-processing perspective on encoding, that some study tasks are simply better than others for creating strong memory traces that will be accessible later (Craik & Lockhart, 1972). Other experiments show that the specific brain regions active during retrieval depend on the content of the material retrieved: the auditory cortex when cued with visual words that had been paired with sounds in the study phase, picture-processing regions of visual cortex when cued with verbal labels corresponding to pictures from the study phase, and motor association cortex when cued with photos of objects that were associated with actions during the study phase (Senkfor, Van Petten, & Kutas, 2002; Vaidya, Zhao, Desmond, & Gabrieli, 2002; Nyberg, Habib, Mcintosh, & Tulving, 2000; Wheeler, Peterson, & Buckner, 2000). These latter findings support the venerable idea that successful retrieval involves at least a partial recapitulation of the pattern of brain activity present during initial encoding (Hebb, 1949).

The current experiment was designed to examine a facet of the encoding/retrieval relationship that has received much less attention, namely, whether executive processes engaged during retrieval might also be dependent on the nature of prior encoding. Executive processes are those that link the fundamental processes of memory encoding and retrieval to current task demands and overt behavior. During a memory test, a partial list of executive functions will include setting a response criterion (How strong must a memory be to warrant a response of “old”?), evaluating the strength of retrieved information in light of the criterion, instigating a secondary search of memory if the initial products of retrieval are close to the criterion or simply inadequate for the task at hand, making a decision, and, finally, mapping the decision onto an overt response. Executive processes in memory have been associated with the prefrontal cortex in both human and nonhuman primates, and executive failure is the most prominent account of the selective memory deficits shown in patients with prefrontal damage (Moscovitch & Winocur, 2002; Shimamura, 2002).

We used a type of memory test known to place a heavy reliance on the prefrontal cortex—one that requires judgments about remembered conjunctions of stimulus attributes (also called source memory tests; Johnson, Hashtroudi, & Lindsay, 1993). In a groundbreaking study, patients with frontal damage were as accurate as controls in recognizing made-up trivia “facts” but showed a deficit when asked whether the facts had been learned in the laboratory or elsewhere (Janowsky, Shimamura, & Squire, 1989). Frontal patients thus showed a specific impairment in remembering associations between core (item) and contextual (source) information from the learning episode. Studies in healthy participants have used a variety of paradigms for which the distinction between “core” and “contextual” is less cut-and-dried¹ but that clearly demand memory for conjunctions of studied attributes. Tested conjunctions have included words and the voice that spoke them, actions and the actor who performed them, words and their presentation modality (auditory or visual), pictures and their spatial locations, and others. Hemodynamic imaging methods show greater prefrontal activity when subjects attempt to remember conjunctions than when they make simple old/new recognition judgments based on the “items” (such as words alone, independent of voice or modality; Dobbins, Foley, Schacter, & Wagner, 2002; Raye, Johnson, & Mitchell, 2000; Rugg, Fletcher, Chua, & Dolan, 1999). ERPs show a much larger prefrontal difference between studied and unstudied items during conjunction/source memory tests than during simple recognition tests (Johansson, Stenberg, Lindgren, & Rosen, 2002; Senkfor, 2002; Ranganath & Paller, 2000; Van Petten, Senkfor, & Newberg, 2000; Senkfor & Van Petten, 1998; Johnson, Kounios, & Nolde, 1997; Trott, Friedman, & Ritter, 1997; Wilding, Doyle, & Rugg, 1995). The present study examines whether prefrontal engagement in source memory tests is mandatory or occurs only when the tested conjunctions have not been well learned.

In two sessions, subjects attempted to remember the conjunction of two stimulus attributes: the identity and the depicted color of an object in a line drawing. Test drawings were judged as either new, old and in the same color as the study phase, or old but in a different color than in the study phase. In one session, prior to the memory test, subjects performed an item-oriented encoding task of judging the size of the depicted object in real life (larger or smaller than the computer monitor, size task). In the other session, they performed an integrative encoding task of judging the relationship between the object and the color of the drawing (color congruity task). In both encoding phases, half of the objects were presented in semantically congruent colors (e.g., red stop sign); the other half were presented in semantically incongruent colors (e.g., blue eggplant), as shown in Figure 1. Because both encoding tasks required identification of the depicted objects and some thought about their real-life properties, we predicted little difference between sessions in the ability to recognize studied objects (items) later. Matched levels of object recognition across encoding tasks is a critical feature of the current design because we wanted to isolate source memory processes rather than compare globally strong and weak memory. We predicted that the color congruity task would lead to tighter integration between object identities and studied colors during encoding, and thus produce higher accuracy in remembering object/color relationships. In addition to behavioral data, ERPs were recorded during both the study and test phases to evaluate the impact of integrative encoding on prefrontal and more posterior cortical activity.

Figure 1.

Illustration of the experimental paradigm. In one session, participants judged whether the depicted objects were larger or smaller than the computer monitor (42 × 38 cm). In the other session, they judged whether the depicted color was a good or poor match for the object. In both sessions, the subsequent memory test included new pictures in both congruent and incongruent colors, old pictures in the studied color, and old pictures in a different color than during the study phase. For the old-different trials, the colors had been studied, but paired with different objects.

Prior work from our laboratory and others has identified two ERP components that differentiate studied and unstudied items in recognition tests. Items that have been studied and are recognized as such during the retrieval phase elicit more positive ERPs than do new items correctly rejected, new items falsely judged as old, and old items that are unrecognized. This early old/new effect thus reflects successful retrieval, and is observed beginning at 200 to 400 msec after stimulus onset (Rubin, Van Petten, Glisky, & Newberg, 1999; Van Petten & Senkfor, 1996). The early old/new effect appears across widespread regions of the scalp, but recordings from patients with organic amnesia show that it is dependent on the integrity of the medial temporal lobe and diencephalon (Olichney, et al., 2000). In source or conjunction memory tests, the early old/new effect is much the same as in simple recognition tests, but is accompanied by an additional, later old/new effect with a focal distribution over the prefrontal cortex (Johansson et al., 2002; Senkfor, 2002; Ranganath & Paller, 2000; Van Petten et al., 2000; Senkfor & Van Petten, 1998; Johnson, Kounios, & Nolde, 1997; Trott et al., 1997; Wilding & Rugg, 1996; Wilding et al., 1995). Because the prefrontal old/new effect begins only some 700 to 800 msec after stimulus onset, it reflects a mnemonic process that is engaged after initial item recognition has occurred or is at least well underway. In our studies, the prefrontal old/new effect has been insensitive to whether or not the source information accompanying each item is successfully recovered. Instead, late activity (>800 msec) over posterior scalp differentiates trials with accurate source judgments as compared to trials in which the items are recognized as old, but the source judgment is incorrect (Van Petten et al., 2000; Senkfor & Van Petten, 1998). These results indicate that successful retrieval of both attributes and conjunctions is indexed by ERPs recorded over posterior brain regions, but that prefrontal executive processes are taxed by source memory tests. The similarity of the prefrontal old/new effect across paradigms with different materials suggests that these executive processes are largely independent of memory content, but may instead be specific to the task of remembering and judging relationships between stimulus attributes. We have argued that one such executive process consists of directing a secondary memory search for the link between two attributes after the individual attributes have been recognized as old (Van Petten et al., 2000; Senkfor & Van Petten, 1998).

Our framework suggests that a secondary memory search will not be necessary if two attributes are tightly bound during initial encoding, and can more easily be retrieved as an integrated unit. We thus hypothesized that the integrative orienting task used here would reduce or eliminate prefrontal involvement during retrieval, as compared to the item-oriented study task that more closely resembles those of prior studies.

METHODS

Subjects

Twenty-four young adults (8 men, 16 women, age range 20–28 years) with normal color vision (assessed by the Ishihara plates) and no history of neurological or psychiatric disorder were paid for their participation. The project was approved by the University of Arizona Biomedical Institutional Review Board, and all subjects offered informed consent.

Materials

Stimuli were 448 line drawings of natural and artifactual objects, each in a congruent and an incongruent color. The same 17 colors, in the same proportions, were used to create the congruent and incongruent versions of the drawings. Stimuli were presented on a CRT monitor, and averaged 4.4 by 2.6 degrees of visual angle. The experimenters judged half of the depicted objects as larger, in real life, than the computer monitor used to present them, and half as smaller than the monitor. Across the 24 subjects, each object occurred equally often in each of the 12 test conditions: (1) congruent color, new, after the size study task; (2) congruent color, new, after the color study task; (3) incongruent color, new, after the size study task; (4) incongruent color, new, after the color study task; (5) congruent color, studied as congruent in the size task; (6) congruent color, studied as congruent in the color task; (7) incongruent color, studied as incongruent in the size task; (8) incongruent color, studied as incongruent in the color task; (9) congruent color, studied as incongruent in the size task; (10) congruent color, studied as incongruent in the color task; (11) incongruent color, studied as congruent in the size task; and (12) incongruent color, studied as congruent in the color task. No object occurred in more than one test condition for any individual subject.

Procedure

Each subject participated in two sessions conducted about 5 days apart; half of the subjects performed the size judgment study task in the first session and the color congruity judgment in the second session; half performed the tasks in the reverse order. During each study block, 16 drawings were presented for 500 msec, with a stimulus onset asynchrony of 4000 msec. Subjects signaled “large” or “small” (or “good” vs. “bad color”) by key presses with the right and left index fingers (assignment of hands counterbalanced across subjects). Three minutes after completion of a study block, 32 drawings were presented for test. One quarter of the test objects were unstudied and in a congruent color; one quarter were unstudied and in an incongruent color; both of these conditions called for a response of “new.” One quarter of the test objects were presented in the same color as during the study phase (evenly divided between congruent and incongruent) and called for a response of “old same.” One quarter of the test objects were presented in a different color as during the study phase (evenly divided between congruent and incongruent) and called for a response of “old different.” Old-same and old-different responses were signaled by key presses with the index and middle fingers of one hand, new responses were signaled by a key press with the index finger of the other hand (hand and finger assignments counterbalanced across subjects). Each session comprised seven study/test cycles and began with a practice block of 24 drawings not used in the main experiment.

Electrophysiological Recording

The electroencephalogram was recorded from 29 scalp sites, including 27 standard locations: 7 spanning the midline of the scalp from prefrontal to occipital (Fpz, Fz, Fcz, Cz, Cpz, Pz, Oz), 6 lateral pairs closer to the midline (Fp1, Fp2, F3, F4, Fc3, Fc4, C3, C4, P3, P4, O1, O2), and 4 lateral pairs farther from the midline (F7, F8, Ft7, Ft8, Tp7, Tp8, T5, T6). Two additional electrodes (far lateral prefrontal, Fp5, Fp6) were placed 10% of the head circumference lateral to Fpz. Electrodes below the right eye and at the external canthi of the two eyes were used to detect blinks and eye movements. Amplifier band pass was 0.01 to 100 Hz; sampling rate was 250 Hz, and gain was 50,000. Trials contaminated by blink, eye movement, or amplifier saturation artifacts were rejected prior to averaging the trials into ERPs for each condition. Scalp ERPs were referenced to an average of the right and left mastoids.

Measurement and Analyses

Item recognition accuracy was computed as the number of recognized old objects (old responses, regardless of the accuracy of the “same” or “different” aspect of the judgment) plus the number of rejected new objects, divided by the total (chance = 50%). Source accuracy was computed as correct same and correct different judgments, divided by the total number of trials with correct old responses (chance = 50%). ERP analyses were based on trials with correct judgments only. A parietotemporal region of interest (ROI) included scalp sites Pz, P3, P4, T5, and T6; a prefrontal region of interest included scalp sites Fpz, Fp1, Fp2, Fp5, and Fp6. ERP measurements were mean amplitudes in latency windows based on prior studies (for the test phase data) or visual inspection of the data (for the encoding phase data), relative to a 200-msec prestimulus baseline. Most statistical analyses were analyses of variance (ANOVAs); repeated measures were encoding task, ROI (prefrontal vs. parietotemporal), and electrode site within each region of interest (five levels). One analysis included a between-subjects factor of encoding-task order; because there was no significant effect of task order, other analyses collapse across order. When initial analyses included significant interactions between encoding task and ROI, follow-up analyses on each ROI were conducted. Pearson correlations were used to examine relationships between item and source accuracy.

RESULTS

Behavioral Performance

During the encoding phases, the size (item oriented) and color congruity (integrative) tasks produced similar agreement rates between the participants' decisions and the size or color categories we had assigned to the drawings (86.4% and 83.6%, respectively, ns). Size judgments were made somewhat more quickly than color judgments: 1162 vs. 1254 msec, F(1,23) = 5.98, p < .05.

Recognition rates for the objects alone (item recognition independent of color) were equally high after the two encoding tasks. Hit rates for old objects were 97.7% (SE = 0.6) and 96.8% (SE = 0.7) after the item-oriented and integrative study tasks, respectively (F < 1). Correct rejection rates for new objects were 96.7% (SE = 0.7) and 97.4% (SE = 0.5) after item and integrative encoding, respectively, also a nonsignificant difference (F < 1).

Table 1 shows source memory accuracies and reaction times (RTs) for old items judged as old. Source accuracy in judging the conjunction between object and color was substantially improved by integrative encoding, F(1,23) = 71.0, p < .0001. Overall RTs were equivalent during the two memory tests, F(1,23) = 1.4.

Table 1.

Source Memory Accuracy and Reaction Time for Correct Trials (Mean and SE)

Encoding Task	Study and Test Colors	Accuracy (%)	RT (msec)
Size	Old same	80.7 (2.5)	1319 (46)
	Old different	73.3 (1.8)	1410 (51)
	All	77.0 (1.8)	1362 (47)
Color congruity	Old same	94.1 (1.1)	1280 (51)
	Old different	87.9 (1.9)	1411 (47)
	All	91.1 (1.2)	1342 (47)

After item-oriented study, accuracy in judging pictures as old or new was uncorrelated with accuracy in judging the object/color conjunction as old or new, r = .17, ns. The lack of association between these two memory measures within individual subjects is consistent with the view that conjunction (source) memory tests typically demand additional processes that are not necessary for simple item recognition. In contrast, the two accuracy measures were correlated after integrative study (r = .41, p < .05). The correlation results are consistent with our account of the source memory advantage from integrative study: Performing the color congruity task helped to bind the object and color attributes into a single memory trace, making the conjunction memory test more like a simple item recognition test.

Source accuracy was higher for drawings remaining in the same color than for those that switched color from study to test (87.0% vs. 80.6%), F(1,23) = 12.5, p < .002, but this effect did not depend on encoding task (Encoding Task × Same/Switch, F < 1). Correct source judgments were also faster for same- than switched-color items, F(1,23) = 26.7, p < .0001, and the RT effect was larger after integrative than item-oriented study (131 vs. 91 msec, respectively), F(1,23) = 5.20, p < .05.

Event-related Potential Data

Encoding Phases

Because the encoding manipulation used here was novel, we had no strong a priori predictions about the nature of any brain activity difference between the two study phases. The question of central interest was whether any observed difference would appear over prefrontal regions, or over more posterior scalp sites. Figure 2A shows that color congruity judgments elicited more positive ERPs than size judgments, but that this difference was most prominent over parietal scalp (Figure 2B). Study-phase ERPs were segmented into three latency ranges for analysis: 200–500, 500–800, and 800–1200 msec poststimulus onset. Across all scalp sites, the effect of encoding task was significant during only the 500- to 800-msec range, F(1,23) = 5.55, p < .05. The difference between encoding tasks was significant for the parietotemporal region in the 500- to 800-msec latency range, F(1,23) = 6.34, p < .02, but not for the prefrontal region in any latency window (Fs < 1).² The encoding phase results indicate that the differential processing required to make object size versus object color congruity judgments had a detectable impact on scalp-recorded brain activity, but suggests that this differential processing was confined to posterior cortical regions supporting perceptual and basic mnemonic functions, rather than the prefrontal cortex.

Figure 2.

(A) Grand-average ERPs from 24 subjects performing the size judgment (item oriented) and color congruity (integrative) tasks during the study phases. From left to right, the prefrontal scalp sites are Fp5, Fp1, Fpz, Fp2, and Fp5, and the parietotemporal sites are T5, P3, Pz, P4, and T6. (B) Topographic map showing the spatial distribution of the significant difference between the two study tasks from 500 to 800 msec after stimulus onset; earlier (200–500 msec) and later (800–1200 msec) latency windows showed no statistically significant differences. Negative voltage is plotted upward in this and all subsequent figures.

Retrieval Phases

Test-phase waveforms were segmented into two latency ranges for analysis: 200–600 and 800–1200 msec poststimulus onset. The standard early old/new effect was robust: in the 200- to 600-msec latency window; hits elicited more positive ERPs than correct rejections, F(1,23) = 112.2, p < .0001 (see Figure 3). The early old/new effect was evident at all scalp sites, although somewhat larger over parietotemporal scalp. The type of prior encoding was not significant as a main effect, nor did it modulate the amplitude or scalp distribution of the early old/new effect (Encoding Task × Hit/CR, F < 1; Encoding Task × Hit/CR × Prefrontal/Parietotemporal, F < 1). Overall, the early retrieval effect was strikingly similar across sessions with different study tasks.

Figure 3.

Grand-average ERPs from 24 subjects during the memory test, contrasting new pictures correctly judged as new to old pictures accompanied by correct source judgments (old same and old different collapsed). Midline scalp sites from most anterior (prefrontal) to most posterior (occipital) are shown.

The late prefrontal differentiation between old and new items was evident only after item-oriented study (Figure 4; Encoding task × Hit/CR × Prefrontal/Parietotemporal, F(1,23) = 4.41, p < .05).³ After size judgment, the late old/new effect was significant for the prefrontal region, F(1,23) = 18.2, p < .0005, but not for the parietotemporal region, F < 1, much like previous studies using an item-oriented encoding task (Van Petten et al., 2000; Senkfor & Van Petten, 1998). After color congruity judgment, the late old/new contrast was not significant for either region: prefrontal, F(1,23) = 1.78, p = .20; parietotemporal, F < 1. In other words, attending to the relationship between the object and color attributes during the study phase eliminated the additional brain activity that is typically elicited when subjects attempt to remember conjunctions of attributes. The elimination of the late prefrontal effect by integrative study did not depend on the order in which subjects performed the two study tasks (Encoding Task × Hit/CR × Prefrontal/Parietotemporal × Order, F < 1).

Figure 4.

Grand-average ERPs from 24 subjects during the source memory test, from scalp sites Fp6 (right prefrontal) and T5 (left temporal). On the left side, thin vertical lines mark the 800- to 1200-msec latency window; on the right they mark the 200- to 600-msec window.

The reduction of the late old/new effect by integrative study was due to altered responses to studied pictures. New pictures, which could be rejected on the basis of object identity alone, elicited nearly identical activity in the two memory tests (prefrontal and parietotemporal regions, both latency ranges, all Fs < 1.3; see Figure 5). Differences in neural activity elicited by unstudied items have been considered indicative of “retrieval orientations”—flexible strategies for processing a retrieval cue depending on the nature of what is sought in memory and its similarity to the cue (Hornberger, Morcom, & Rugg, 2004). The lack of differentiation between ERPs for correct rejections indicates that the encoding manipulation here has no relationship to the retrieval orientation manipulations of other studies. Instead, the nature of the prior encoding task influenced brain activity for studied pictures only because only these trials demanded a judgment about the remembered conjunction of object and color. Figure 6A contrasts responses to recognized pictures after the two different encoding tasks, and Figure 6B shows the prefrontal locus of the study-phase manipulation on brain activity during retrieval.

Figure 5.

Grand-average ERPs from 24 subjects during the memory test, contrasting new pictures correctly judged as new after the two study tasks. Midline scalp sites from most anterior (prefrontal) to most posterior (occipital) are shown.

Figure 6.

(A) Grand-average ERPs from 24 subjects performing the source memory test. “Correct old” are trials in which studied objects were recognized as old, and the source judgment about the object/color relationship was also correct (old same and old different trials collapsed). From left to right, the prefrontal scalp sites are Fp5, Fp1, Fpz, Fp2, and Fp6, and the parietotemporal sites are T5, P3, Pz, P4, and T6. (B) Topographic map showing the spatial distribution of the significant difference between the hit trials in the two memory tests, dependent on which study task had preceded the test. The map depicts the 800- to 1200-msec latency window; earlier latency windows yielded no significant differences.

Same versus switched colors during retrieval

Figure 7 shows that old-same and old-different trials elicited different prefrontal activity, but indistinguishable ERPs over the posterior cortex (both latency windows, Fs < 1 for parietotemporal sites). The same–different effect was contingent on the nature of the prior encoding task and was most evident over the left hemisphere. Analyses of individual prefrontal sites showed significant interactions between encoding task and same–different for both of the left prefrontal sites, F(1,23) = 4.41, p < .05 at Fp5; F(1,23) = 5.24, p < .05 at Fp1, a trend for the midline prefrontal site, F(1,23) = 3.09, p < .09, and no significant interaction for the right hemisphere sites. Right prefrontal sites instead showed a main effect of encoding task, independent of whether the object–color conjunction was the same or different as during the study phase, F(1,23) = 5.33, p < .05 for Fp2; F(1,23) = 4.06, p < .05 for Fp6. For the left prefrontal sites, the nature of the interaction was that integrative encoding reduced prefrontal activity to a greater degree for old-same trials—those that remained in the same color from study to test. In contrast, old-different trials showed little benefit from the integrative task at these left prefrontal sites.

Figure 7.

Grand-average ERPs from 24 subjects performing the source memory test, from the prefrontal and parietotemporal scalp sites.

Late negativity for old items

In addition to the early, spatially widespread positivity and late prefrontal positivity elicited by studied items, we also observed a third aspect of the ERPs that differentiated old (studied) and new items. Beginning ∼1000 msec poststimulus onset, the ERPs elicited by old and new items cross over at some scalp sites, so that old items elicit relatively more negative potentials. The late negativity for old items was broadly distributed over frontal to occipital scalp, but not evident at prefrontal sites, and small at sites over the temporal lobe. Because the mean RT for new items was ∼1000 msec, but participants needed an additional 300 msec to respond to old items, we hypothesized that this late crossover indexed response preparation rather than a mnemonic process. We thus sorted the data according to whether the right or left hand was assigned to old responses (opposite hand always assigned to new responses). Each subject used two fingers of the right hand to signal old in one session (finger selection signaled “old same” vs. “old different”), and the two fingers of the left hand in the other session. Figure 8 shows that the late negativity for old items showed a pronounced asymmetry contingent on hand of response and was evident primarily at scalp sites contralateral to the hand used for old. Mean amplitudes between 1000 and 1400 msec poststimulus onset were analyzed via ANOVAs with factors of Hand assigned for old (right vs. left), Old/New, and Laterality (left vs. right scalp), together with a between-subjects factor reflecting whether the right hand was used for old after the item-oriented study task or after the integrative study task. In these analyses, an old/new effect that reverses asymmetry depending on response hand will result in an interaction between Hand, Old/New, and Laterality. Results for each pair of lateral scalp sites are shown in Table 2. Along the anterior–posterior axis, scalp sites from frontal to occipital showed reversing asymmetries depending on response hand. Two of the electrode pairs over the temporal lobe showed a weaker reversal. It is important to note that the prefrontal sites were not influenced by the response mapping manipulation, nor did the preceding study task interact with response hand for any pair of sites (Hand × Old/New × Laterality × Study Task, all null results).

Figure 8.

Grand-average ERPs from 24 subjects performing the source memory test. Each subject used the right hand to signal old (different fingers for same vs. different) and their left hand to signal new in one session, and the reverse in the other session.

Table 2.

Statistical Results for Lateral Asymmetry of the Late Negative Old/New Effect

Electrode Pair	Response Hand × Old/New × Laterality F(1,22)		Study Task × Response Hand × Old/New × Laterality F(1,22)
Fp1, Fp2	1.22		1.81
F3, F4	9.54	***	0.07
Fc3, Fc4	71.05	****	0.70
C3, C4	92.21	****	0.31
P3, P4	10.31	***	0.49
O1, O2	6.50	*	0.01
Fp5, Fp6	0.04		0.93
F7, F8	0.40		2.19
Ft7, Ft8	5.93	*	0.35
Tp7, Tp8	0.00		1.84
T5, T6	4.25	*	0.20

Listed first are the seven lateral electrode pairs closer to the midline, followed by the five pairs farther from the midline (primarily over the temporal lobe, with the exception of Fp5 and Fp6 over lateral prefrontal cortex, and F7 and F8 over lateral frontal/prefrontal cortex).

^*p < .05.

^***p < .005.

^****p < .0001.

DISCUSSION

Three ERP Differences between Old and New Items

Three aspects of the ERPs recorded at test distinguished correct old and correct new trials: (1) an early-onset (∼200 msec), spatially widespread positivity for old items, (2) a late-onset (∼700 msec) prefrontal positivity, and (3) a very late (∼1000 msec) negativity for old items. All three effects are similar to prior studies from our laboratory using qualitatively different materials (words and voices, Senkfor & Van Petten, 1998; drawings and spatial locations, Van Petten et al., 2000). However, our previous articles focused on the first two effects and included no analyses or discussion of the late negativity. We thus comment on the commonalities and differences among studies from different laboratories before proceeding to the core goal of the current study—the impact of encoding on source memory retrieval.

Cycowicz, Friedman, and colleagues have reported three studies in which participants were tested on their memory for line drawings and their colors. All used nonintegrative study tasks. In the first two of these reports, the late prefrontal old/new effect typical of source memory tests was not evident, although other studies from this laboratory using different stimulus attributes included the typical late prefrontal effect (Cycowicz & Friedman, 2003; Cycowicz, Friedman, & Snodgrass, 2001 vs. Trott et al., 1997). A more recent report clarifies this puzzling absence as due to the choice of reference electrode (Friedman, Cycowicz, & Bersick, 2005). The electroencephalogram is always recorded from differential amplifiers that amplify the difference between an active scalp site and a reference site selected to pick up less brain activity, but the same electrical noise from the environment. In the present study, as in our past work, the reference site has been an average of electrodes over the two mastoids; mastoid or earlobe references are commonly used because the thick mastoid process provides substantial (although not absolute) electrical insulation from the brain (Katznelson, 1981). An alternative reference site is the tip of the nose, used by Cycowicz, Friedman, and colleagues. In their 2005 article, Friedman et al. present both mastoid-referenced and nosetip-referenced data, and note that a prefrontal positivity for old items is apparent in the former but not the latter. This result strongly suggests that similar brain activity is present at both the prefrontal (forehead) and nosetip locations so that differential recordings between the two show little activity, in contrast to differential recordings between prefrontal and mastoid locations.

The drawing-plus-color memory studies of Friedman et al. (2005), Cycowicz and Friedman (2003), and Cycowicz et al. (2001) have consistently reported that old items elicit a large late negative potential over widespread (but not prefrontal) regions of the scalp. In a recent review, Johansson and Mecklinger (2003) suggest that such late negative potentials may frequently reflect reconstruction of “the prior study episode when task-relevant attribute conjunctions are not readily recovered or need continued evaluation” (p. 64), much the same functional description that we have offered for prefrontal activity. In the current study, we also observe a late negativity for old items, but analyses based on hand of response strongly suggest that this late effect indexed response preparation rather than a mnemonic process per se. The general morphology of the late negativity strongly resembles the Readiness Potential component of the ERP. More critically, the late negative-going potential for old items was largest over motor, premotor, and somatomotor cortical regions (frontocentral, central, and parietal scalp), and was much larger contralateral to the hand used to signal old responses. Old and new items would be expected to elicit differential motor preparation in the latest portion of the epoch (after 1000 msec) because the mean RT for new items hovered around 1000 msec, but mean RT for old items was more than 300 msec later. We can thus assume that for the majority of trials, old and new items had been distinguished by 1000 msec, and the responding hand had been selected (for new trials, the response had actually been executed for some half of those trials). However, for old trials, preparation of that hand had to be maintained while the mnemonic processes distinguishing old-same from old-different trials proceeded, leading finally to selection of the appropriate finger on the responding hand. Overall, the data suggest that the late negativity for old trials is a temporally extended Readiness Potential. The late negative effect was functionally dissociated from the late prefrontal positivity in two ways: Only the prefrontal effect was influenced by the preceding study task (securely linking it to mnemonic processes), and only the late negative effect was influenced by response hand. Because of numerous procedural differences among studies, we cannot conclude with any certainty how much the late negative effect here overlaps with that observed in other laboratories. However, the current results do suggest that response preparation should be considered a strong candidate for the functional process indexed by late negative potentials.

Influence of Encoding Task during Source Memory Retrieval

A well-established effect in memory research is transfer-appropriate processing (TAP; Morris, Bransford, & Franks, 1977)—that performance improves to the extent that encoding and retrieval processes overlap. In the current experiment, the source accuracy benefit conferred by the integrative study task over the item-oriented task provides a clear example of TAP. Directing attention toward the relationship between the depicted color and the identity of an object during encoding improved memory for object/color relationships during the test phase. Memory for object identity alone (item recognition accuracy) was unaffected by the nature of the study task, because judging the real-life size of the depicted object and judging object/color congruity both required access to stored knowledge about the objects themselves. Although the typical color of an object may sometimes be spontaneously accessed when thinking about other object properties (Joseph & Proffitt, 1996), the color–congruity task mandated that this property be considered on every trial, and explicitly conjoined with object identity.

A small number of prior experiments have confirmed a prediction about brain activity that can be drawn from TAP, namely, that successful retrieval will be accompanied by reengagement of the same brain areas active during encoding (Vaidya et al., 2002; Nyberg et al., 2000; Wheeler, Peterson, & Buckner, 2000). In these studies, qualitatively different patterns of brain activity were produced by including physically distinct classes of stimuli (e.g., words vs. pictures) during the study phase. These experiments have thus evaluated an outcome of TAP, in designs that necessitated comparisons across physically distinct classes of stimuli. The current experiment is unique in comparing brain activity elicited by the same type of material at both study and test, and manipulating only the encoding operations applied to those stimuli. The results are novel in showing another consequence of transfer-appropriate processing, namely, a reduced need to exert prefrontal control processes during the attempt to retrieve information.

The first neural differentiation between studied and unstudied pictures began ∼200 msec after stimulus presentation, was widespread across the scalp, and did not vary according to the nature of the study task. This initial effect appeared much the same as that observed in numerous ERP studies using simple old/new recognition tasks that do not require memory for conjunctions of attributes (Rugg & Allan, 2000; Van Petten & Senkfor, 1996). Following the item-oriented study task, studied and unstudied pictures also elicited differential electrical activity over the prefrontal cortex in a later time epoch (beginning ∼700 msec), because only the recognized pictures required search or evaluation of the weakly encoded relationship between object identity and depicted color. This late effect appeared much the same as in previous ERP studies of source memory (Johansson et al., 2002; Ranganath & Paller, 2000; Van Petten et al., 2000; Senkfor & Van Petten, 1998; Trott et al., 1997; Wilding & Rugg, 1996; Wilding et al., 1995). In the item-oriented session, there was no significant correlation between an individual's accuracy in recognizing a picture (independent of color) and accuracy in recognizing object/color relationships, consistent with the conclusion that these reflect dissociable neural processes. When object/color relationships were more strongly encoded, additional executive processes devoted to recovering object/color relationships—after an object had been recognized—were no longer needed. Individual accuracies for object recognition and object/color recognition were significantly correlated in the integrative session, consistent with the conclusion that they were now largely dependent on the same neural processes.

In the current results, lower source accuracy was associated with greater prefrontal engagement during memory retrieval, and higher source accuracy with lesser engagement. It might be tempting to summarize this pattern of results as “greater prefrontal activity when retrieval is difficult.” However, this summary is not specific enough. It is important to remember that task difficulty is an outcome (inferred by lower accuracy and/or longer RTs) rather than an experimental manipulation, and that there are multiple processing bottlenecks that can lead to reduced memory accuracy. Among the many manipulations that can reduce recognition accuracy, many do not appear to be associated with greater prefrontal activity. For instance, meaningful material is more easily recognized than novel and less meaningful material, but a contrast between recognition of words and novel geometric patterns did not yield any difference in the topography of the ERP old/new effect (Van Petten & Senkfor, 1996). In a positron emission tomography study, drawings of impossible three-dimensional objects were less likely to be recognized than drawings of possible objects, but were associated with less (rather than more) blood flow in the dorsolateral prefrontal cortex (Schacter et al., 1995). Moreover, standard levels of processing manipulations lead to lower accuracy for items studied in “shallow” tasks relative to those studied in “deep” tasks, but test-phase differences in electrical and hemodynamic measures have been reported in posterior rather than prefrontal regions (Gonsalves & Paller, 2000; Rugg, Walla, et al., 1998). These examples are drawn from simple old/new recognition tasks in which prefrontal damage has a relatively minor impact on accuracy (Swick & Knight, 1999; Wheeler, Stuss, & Tulving, 1995), likely because performance is limited by the efficiency of basic encoding and retrieval operations served by medial temporal and posterior neocortical regions. In contrast, when a difficult version of a task does reliably engage prefrontal cortex—as in the n-back working memory task—parametric reductions of task difficulty have led to parallel reductions in prefrontal activity (Braver et al., 1997).

Although a large number of studies using both ERP and hemodynamic imaging methods have confirmed that conjunction (or source) memory tests preferentially engage the prefrontal cortex (Dobbins et al., 2002; Johansson et al., 2002; Senkfor, 2002; Ranganath & Paller, 2000; Raye et al., 2000; Van Petten et al., 2000; Rugg, Fletcher, et al., 1999; Senkfor & Van Petten, 1998; Johnson, Kounios, & Nolde, 1997; Trott et al., 1997; Wilding et al., 1995), the current results indicate that this need not be so. Instead, the results show that the role of the prefrontal cortex in these tasks is to aid in the recovery of weakly encoded relationships. In the present case, a simple instructional manipulation was sufficient to strengthen the encoding of the relationship between attributes, which may or may not be possible in other cases with qualitatively different attributes or different degrees of similarity among attributes. The impact of encoding task and instruction observed here is consistent with two recent behavioral studies. Thaiss and Petrides (2003), after observing an exception to the typical finding that patients with prefrontal damage perform poorly in source memory tests, argued that the high accuracy of their patients was due to explicit instruction during the encoding phase to try to learn relationships. Glisky, Polster, and Routhieaux (1995) have consistently reported that healthy older adults who perform poorly in neuropsychological tests traditionally thought to tap prefrontal function also have special difficulty with source memory tests. However, this group has also reported that integrative encoding tasks eliminate the source memory deficit in this subset of older adults and indeed boosts their performance to the level of young adults (Glisky, Rubin, & Davidson, 2001). These behavioral results are equally consistent with two accounts: (1) that patients with frontal damage and older adults with reduced prefrontal function fail to spontaneously select an appropriate encoding strategy, and this deficit can be overcome by explicit instruction, or (2) that the prefrontal cortex serves an important role in the retrieval of attribute conjunctions, and that this burden can be alleviated by the right sort of encoding—encoding that serves to bind attributes more closely. These two accounts are not mutually exclusive, and the current results cannot rule out the possibility that the prefrontal cortex is essential for initial task set during encoding—deciding how to focus attention on upcoming trials (Hopfinger, Buonocore, & Mangun, 2000). Although we observed no sign of differential prefrontal activity during performance of the two encoding tasks (instead, the difference between encoding tasks had a parietotemporal maximum), the single moment of deciding to adopt a particular encoding strategy could not be visualized in the design used here. In contrast, the current results do provide strong support for the latter account because the nature of the prior encoding task influenced prefrontal activity during the retrieval phase.

The present results, and the behavioral experiments above (Thaiss & Petrides, 2003; Glisky, Rubin, & Davidson, 2001) argue that there may be no intrinsic relationship between the retrieval of attribute conjunctions and the prefrontal cortex. Some investigators have postulated a special role for this brain region in strategic retrieval, as compared to the spontaneous recovery of memories in the presence of an appropriate cue (Moscovitch & Winocur, 2002; Petrides, 1996). The distinction between strategic and spontaneous retrieval is often defined as the difference between free recall and recognition tests, and one motivation for the strategic retrieval hypothesis is the impairment of frontal patients in free recall in the face of normal or near-normal recognition performance (Wheeler, Stuss, & Tulving, 1995). Source memory tests—even when administered in a recognition format as here—may typically stimulate strategic retrieval because of the relative ease of remembering some aspect of a studied stimulus combined with the difficulty of remembering conjunctions of attributes. Spontaneous recovery of some information from memory may be a particularly potent trigger to initiate a prolonged attempt to retrieve additional, task-relevant information. This model is consistent with the temporal sequence of ERP memory effects in source memory tests (Johansson et al., 2002; Van Petten et al., 2000; Senkfor & Van Petten, 1998; Trott et al., 1997; Wilding et al., 1995): an early differentiation between completely new stimuli and those that closely resemble studied stimuli (such as old objects in a different color), followed only later by a prefrontal old/new effect. However, if disparate attributes of a stimulus or event are more tightly bound during initial encoding, recognition of even conjunctions of attributes may occur spontaneously without the need for strategic retrieval.

There is a caveat to the superiority of integrative encoding, namely, that it is most beneficial when retrieval cues form a good match with what was encoded, as in the “old-same” trials here. Higher accuracy for old-same than old-different trials is typical of source recognition tests (Dodson & Shimamura, 2000; Van Petten et al., 2000; Senkfor & Van Petten, 1998; Wilding et al., 1995), and was also observed here. In both sessions, pictures that remained in the same color from study to test yielded higher source accuracy and faster responses, although the RT benefit was larger after integrative than item-oriented study. The accuracy difference may have two causes: on the one hand, a benefit from reinstating the details of the study episode on old-same trials, and on the other hand, a cost when familiar but inappropriate retrieval cues are presented on old-different trials (Dodson & Shimamura, 2000). Until now, there have been few hints about the neural correlates of either the cost or the benefit. Successful source retrieval is associated with greater activity in posterior (but not prefrontal) brain regions (Dobbins et al., 2002; Van Petten et al., 2000; Senkfor & Van Petten, 1998), but prior studies have shown no difference between old-same and old-different conditions when source retrieval is successful (Van Petten et al., 2000; Senkfor & Van Petten, 1998; Wilding et al., 1995). These results might suggest that brain activity during successful source retrieval is constant, regardless of the ease or difficulty in reaching this endpoint. A new observation here was reduced prefrontal engagement for old-same trials—those reinstating the study episode exactly—but only when the original pair of attributes had been tightly bound by the integrative encoding task. Integrative encoding also influenced prefrontal activity for old-different trials, but to a lesser extent. It is possible that the prefrontal same–different effect reflects the specific benefit of reinstatement, whereas retrieval accuracy suffers an additional cost from mismatching retrieval cues whose neural basis has not been identified. Confirmation of this proposal and neural characterization of both halves of the cost–benefit equation awaits further research.