Functional Dissociation among Components of Remembering: Control, Perceived Oldness, and Content

Abstract

Remembering is the ability to bring back to mind episodes from one's past and is presumably accomplished by multiple, interdependent processes. In the present functional magnetic resonance imaging study, neural correlates of three hypothesized components of remembering were explored, including those associated with control, perceived oldness, and retrieved content. Levels of each component were separately manipulated by varying study procedures and sorting trials by subject response. Results suggest that specific regions in the left prefrontal cortex, including anterior-ventral Brodmann's Area (BA) 45/47 and more dorsal BA 44, increase activity when high levels of control are required but do not necessarily modulate on the basis of perceived oldness. Parietal and frontal regions, particularly the left parietal cortex near BA 40/39, associate with the perception that information is old and generalize across levels of control and retrieved content. Activity in the parietal cortex correlated with perceived oldness even when judgments were in error. The inferior temporal cortex near BA 19/37 associated differentially with retrieval of visual object content. Within the ventral visual processing stream, content-based modulation was specific to late object-responsive regions, suggesting an efficient retrieval process that spares areas that process more primitive retinotopically mapped visual features. Taken collectively, the results identify neural correlates of distinct components of remembering and provide evidence for a functional dissociation. Frontal regions may contribute to control processes that interact with different posterior regions that contribute a signal that information is old and support the contents of retrieval.

Introduction

Remembering draws on controlled, goal-directed processes to construct a representation that is experienced as an episode from the past (Tulving, 1983). As such, remembering is one of many task forms that require the use of flexible, context-relevant representations. During remembering, unlike other forms of controlled-processing tasks, the sought representation is a perception of the past that includes content details of the earlier experience and is accompanied by a distinct perception that information is old. In the present study, component processes of remembering were experimentally dissociated through the combined use of behavioral manipulations that influence remembering and event-related functional magnetic resonance imaging (fMRI) procedures that allow sorting of subject responses based on recollective experience. The procedures were guided by previous theories of remembering and controlled processing (Shiffrin and Schneider, 1977; Gillund and Shiffrin, 1984; Jacoby, 1991; Johnson, 1992; Moscovitch, 1992; Burgess and Shallice, 1996; Rugg and Wilding, 2000) and recent neuroimaging studies that suggested the involvement of specific cortical regions.

A common feature among theories of remembering is that control processes guide retrieval (Bartlett, 1932; Tulving, 1983; Jacoby, 1991;Burgess and Shallice, 1996; Koriat, 2000; Moscovitch, 2000). Control processes are intentional processes initiated to solve a task goal. In remembering, the goal is to retrieve information from the past. Control processes require sequential steps of processing, are capacity-limited, and operate when the task cannot be accomplished through automated stimulus–response mappings (Shiffrin and Schneider, 1977). Here, the amount of controlled processing was manipulated by varying the extent of study, with more extensive study allowing controlled processing at the time of retrieval to decrease because of the stronger association between the studied cue and the information to be retrieved. Candidate structures within the prefrontal cortex (Buckner et al., 1998a; Nolde et al., 1998; Rugg and Wilding, 2000; Buckner and Wheeler, 2001) were explored by tracking the extent to which they modulated with the demand for controlled processing.

Successful recovery of information from the past is often accompanied by a perception that information is old and not reconstructed de novo (James, 1890; Tulving, 1983). Neural correlates of perceived oldness were studied by comparing responses for OLD and NEW items. Memory errors were also examined to determine whether regions predicting successful retrieval also correlated with whether new items were mistakenly endorsed as studied and whether studied items were missed. Particular emphasis was placed on a parietal region previously associated with successful recovery of episodic information (Henson et al., 1999; Konishi et al., 2000; McDermott et al., 2000; Donaldson et al., 2001).

When retrieval is successful, a representation of the original content is constructed that may include both sensory-specific and abstract details of the original episode. Content-specific retrieval processes were explored by varying the modality of materials. We focused on inferior temporal regions that have been activated in previous studies of visual remembering (Owen et al., 1996; Wheeler et al., 2000) and visual imagery (D'Esposito et al., 1997; Thompson et al., 2001). The central question was whether these visual-processing regions would be reactivated during remembering and how such reactivation would modulate across conditions. An across-study analysis was performed to further explore the anatomic selectivity with which the visual cortex was reactivated during retrieval of visual object content.

A prominent feature of the present study is that the manipulations mentioned above targeted distinct components of remembering and were manipulated separately within the same study. In this manner, component processes associated with level of control, the perceived oldness of information, and retrieval content could be dissociated.

Materials and Methods

Overview. Subjects studied paired associates before fMRI imaging. During imaging, subjects performed a source retrieval task. By manipulating conditions at study and sorting trials on the basis of subject response, component processes involved in remembering were modulated. Specifically, subjects studied paired associates under either extensive (many repetitions) or minimal (one repetition) study conditions. Within each study condition, the study content was manipulated: half of the words were paired with auditory associates, and the other half were paired with visual associates (Fig.1A, left panel). Then, during the imaged source retrieval task, subjects viewed word cues and decided whether each was new, previously studied with an auditory associate (sound), or previously studied with a visual associate (picture) (Fig. 1A, right panel). In this manner, three factors were crossed at the time of retrieval: (1) the amount of controlled processing required (being greater for minimally studied associates and less for extensively studied associates), (2) whether the cue was previously studied or new, and (3) the content of retrieved information (auditory or visual).

Fig. 1.

Sample study and test trials and the subsequent item categorization used in the analyses. A, Word cues paired with pictures or sounds were studied 20× or 1×. In the study example, APPLE, TRAIN, BANJO, and DOG were studied 20×. At the test, subjects decided if the cues were (1) old and studied with a picture (no matter how often studied), (2) old and studied with a sound, or (3) new. In the test example, DOG and TRAIN are 20× test cues. Cross-hair (+) trials served as baseline. B, Items were categorized in different ways to examine components of remembering. Test items were either OLD or NEW. OLD items were divided into pictures and sounds to identify regions that modulated on the basis of memory content. OLD items were also separated by 20× or 1× study to examine effects of level of control (see Materials and Methods).

As a beginning point for analyses, specific regions within the prefrontal, parietal, and temporal cortices that have been previously implicated in remembering were selected a priori. Hemodynamic response magnitudes within these regions were analyzed using a mixed effects model (including subjects as a random factor) to determine whether the regions modulated activity based on each of these three factors. Additional whole-brain analyses and projection of data in relation to a flattened cortical representation were then conducted to further explore the data. A final analysis, which pooled data from the present study with data from Wheeler et al. (2000), was conducted to explore the anatomic specificity of activation within visual cortex.

Subjects. Thirty-three right-handed (Raczkowski et al., 1974) subjects from the Washington University community participated. Subjects were 18 to 31 years old (mean, 21 years; 19 female), had normal or corrected-to-normal vision, were native English speakers, and reported no history of significant neurological problems. Subjects provided informed consent in accordance with the Washington University Human Studies Committee guidelines and received payment for their participation. Two subjects were excluded for noncompliance, another for excessive movement during the scans, and a final subject because of technical data recording problems. Of the successfully imaged subjects, response time data for two were not recorded. Therefore, imaging data for 29 subjects (17 female) and behavioral data for 27 subjects (17 female) are reported.

Prescan study phase. Approximately 24 hr before scanned testing, subjects studied 80 word cues paired with associates. Half of the word cues were studied with pictures, and half were studied with sounds. Of these, half were studied 20 times, and half were studied once. A total pool of 120 word cues was divided into six lists of 20 cues each, and the lists were then assigned to each condition, with one list for each of the four studied conditions (n = 80) and the remaining two lists comprising the new lures used at the test (n = 40; see Scanned retrieval task). Lists were equated for (1) word length (range, 3–10 letters; mean, 5.7), (2) word frequency, based on norms reported by Francis and Kucera (1982)(range, 1–350 instances per million; mean, 26.7), (3) width of the pictures (range, 3–11° visual angle; mean, 6.5°), and (4) duration of the sounds (range, 1000–2500 msec; mean, 1756 msec). A unique picture and a unique sound item were prepared for each cue, and lists of cue–item pairs were counterbalanced across subjects such that each cue appeared equally often in each study condition.

Scanned retrieval task. The basic imaged task was a source retrieval task in which subjects viewed a total of 120 cue words that were new (n = 40), previously studied with an auditory associate (n = 40), or previously studied with a visual associate (n = 40). The test was given over four separate runs. Subjects indicated, by right-hand key press, into which of the three categories each item fell [similar to the procedure ofWheeler et al. (2000) but extended to include new items]. The source retrieval test used a rapid presentation event-related design (Dale and Buckner, 1997; Miezin et al., 2000) in which trials were spaced 5 sec apart. Trials consisted of (1) cues studied 20 times with a picture (PIC20×; n = 20), (2) cues studied one time with a picture (PIC1×; n = 20), (3) cues studied 20 times with a sound (SND20×; n = 20), 4) cues studied one time with a sound (SND1×; n = 20), (5) cues not studied (NEW; n = 40), or (6) a fixation crosshair (n = 40) for which no response was required. For trial types 1–5, subjects saw a word cue for 750 msec followed by 4250 msec of fixation. For trial type 6, a fixation crosshair (+) was presented throughout the trial (5 sec). Word cues were presented centrally in black 24 point Geneva bold font on a white background. Trial types were ordered pseudorandomly within a test run so that each trial type was equally likely to be preceded and followed by each other trial type (Buckner et al., 1998b).

Behavioral data were scored for both recognition and source accuracy. Recognized items were those that were correctly identified as having been studied. Source accuracy additionally required correct identification of whether an old cue was studied with a picture or a sound (see Functional MRI data analysis). Using source accuracy criteria, correctly identified old items that were attributed to an incorrect source were scored as incorrect. New lures were also scored for accuracy.

Functional MRI procedures. Functional imaging was conducted on a 1.5 tesla Vision MRI system (Siemens, Erlangen, Germany). Visual stimuli were generated on an Apple Power Macintosh G4 computer (Apple Computer, Cupertino, CA) using PsyScope software (Cohen et al., 1993) and projected (Epson PowerLite 500c liquid crystal display) onto a screen positioned at the head of the magnet bore. Subjects viewed the screen using a mirror attached to the head coil. Headphones were used to dampen scanner noise. Responses were recorded on a fiber-optic light-sensitive key press interfaced with a PsyScope button box (Carnegie Mellon University, Pittsburgh, PA). Foam pillows and a thermoplastic face mask were used to minimize head movement. Structural images were acquired first using a sagittal magnetization preparation-rapid acquisition gradient echo T1-weighted sequence [repetition time (TR), 9.7 msec; echo time (TE), 4 msec; flip angle, 10°, inversion time, 20 msec; and delay time, 500 msec]. Subsequently, four runs of 90 whole-brain (16 8-mm-thick slices oriented along the transverse plane) functional images were collected using an asymmetric spin-echo echo-planar sequence sensitive to blood oxygen level-dependent (BOLD) contrast (T2*; TR, 2.5 sec; TE, 37 msec; and 3.75 × 3.75 mm in-plane resolution) (Kwong et al., 1992; Ogawa et al., 1992). The first four images in each run were discarded to allow magnetization to stabilize.

Functional MRI data analysis. Functional data were first corrected for odd–even slice intensity differences and then motion-corrected using a rigid-body rotation and translation correction (Snyder, 1996). Between-slice timing differences caused by slice acquisition order were adjusted using sinc interpolation. Linear slope was removed on a voxel-by-voxel basis to correct for drift (Bandettini et al., 1993). Data were normalized to a mean magnitude value of 1000 and transformed into the stereotaxic atlas space of the Talairach and Tournoux (1988) atlas (using 2 mm isotropic voxels) to permit across-subject analyses (Maccotta et al., 2001). Data were smoothed using a one-voxel-wide isotropic Hanning filter.

Signal magnitudes were obtained in relation to task trials by selectively averaging the BOLD fMRI data (Dale and Buckner, 1997;Buckner et al., 1998b). The conditions for analysis consisted of combinations of various trial types to investigate three main component processes associated with (1) the level of controlled processing, (2) whether there was successful recovery of information, and (3) retrieval content (Fig. 1B). These components were targeted by comparing (1) the OLD items studied minimally (1×) with those studied extensively (20×) (to contrast high vs low control), (2) the NEW with OLD items (to isolate successful recovery of information), and (3) the OLD items studied with PIC with those studied with SND (to explore retrieval content). For several additional analyses, items were further classified as CORRECT and INCORRECT on the basis of subject response and source accuracy or subdivided on the basis of interactions among the three main factors. Subdivided conditions were labeled by their specific subdivisions (e.g., PIC20×-CORRECT, SND1×-INCORRECT, and NEW-INCORRECT). Furthermore, various analyses used combinations of these conditions and were named accordingly. For instance, PIC20×-CORRECT and PIC1×-CORRECT grouped together as one condition are referred to as PIC-CORRECT. When conditions were combined, response magnitudes for the conditions were averaged and weighted as necessary (i.e., when only a subset of the 29 subjects contributed responses). Estimates of signal change were not obtained for PIC20×- and SND20×-INCORRECT, because too few subjects contributed trials.

Hypothesis-driven regional analyses. Hypothesis-driven analyses were performed on a small number of regions. This approach affords considerable power by reducing the number of multiple comparisons (to four in this instance) and averaging the many voxels within each region (thereby increasing the signal-to-noise ratio). Magnitude estimates were calculated for each subject and for each condition in four a priori-defined regions selected because of their hypothesized association with the component processes of interest: (1) a left inferior frontal region at or near Brodmann's Area (BA) 45/47 with peak location at (x, y, and z) −45, 35, and −4; (2) a dorsal left frontal region near BA 44 (−47, 17, and 24) (Gold and Buckner, 2002; Logan et al., 2002); (3) a left lateral parietal region near BA 40/39 (−39, −55, and 36) (Konishi et al., 2000); and (4) a left ventral temporal-occipital region near BA 19/37 (−45, −69, and −6) (Wheeler et al., 2000). Regions are referred to by their approximate Brodmann's classification.

BA 45/47 and BA 44 have been associated with increased demands for controlled processing in both episodic retrieval tasks and other task forms that require verbal elaboration (Buckner et al., 1998a; Nolde et al., 1998; Rugg et al., 1999; Ranganath et al., 2000; Buckner and Wheeler, 2001; Gold and Buckner, 2002). BA 40/39 has been suggested to increase activity during the successful recovery of previously experienced information, relative to the successful identification of new information (Habib and Lepage, 1999; Henson et al., 1999; Konishi et al., 2000; McDermott et al., 2000; Donaldson et al., 2001). Regions in and around BA 19/37 have been associated with the perception (Malach et al., 1995; Owen et al., 1996; Grill-Spector et al., 2000; Wheeler et al., 2000; Kourtzi and Kanwisher, 2001) and retrieval of visual objects from memory (Owen et al., 1996; D'Esposito et al., 1997; Köhler et al., 1998; Wheeler et al., 2000).

Many additional regions could have been selected for a priori analysis but were not to focus power on regions most consistently associated with the proposed processes. In particular, prefrontal regions involved in control processes have extended bilaterally and into anterior regions (near BA 10) that may associate with retrieval mode and ongoing monitoring processes. Although important to control processes, the present study, which uses a rapid event-related design, was not designed to fully explore their contributions to control processes (for discussion, see Buckner, 2003).

For a priori-defined regions, magnitude estimates were computed by first subtracting signal change for the fixation condition from each of the other conditions to obtain an estimate of signal change relative to fixation baseline, with overlap of the hemodynamic response between trials removed (Dale and Buckner, 1997; Buckner et al., 1998b). The mean signal change across all voxels in each region for each condition was then computed. Response magnitude estimates were calculated by subtracting the mean signal corresponding to times 0 and 17.5 sec (representing the baseline) in the epoch from the average signal at times 5 and 7.5 sec (representing the peak), relative to the start of the trial. This estimation procedure, which assumes a simple model, is stable even when the mean magnitudes approach zero. The magnitude estimates for each subject were entered into a mixed-effects model, and specific comparisons were made using ANOVA and t tests.

Whole-brain exploratory analyses. Exploratory analyses were performed independent of any a priori expectations. Such analyses allow the full data set to be explored but are less powerful than regional analyses, because they necessarily require correction for large numbers of multiple comparisons and do not benefit (beyond smoothing) from signal averaging across voxels within a region. Statistical activation maps were constructed for each condition on a voxel-by-voxel basis using a t statistic (Dale and Buckner, 1997). Contrasts of interest were regressed against a set of eight time-lagged (offset by 1 sec) γ functions that approximate the range of hemodynamic responses typically encountered (Boynton et al., 1996; Dale and Buckner, 1997). Regions of activation were identified using a significance threshold ofp < 0.0001 and 19 or more contiguous significant voxels (152 mm³). This threshold has yielded few false-positive findings in control data sets (Konishi et al., 2000).

An across-study analysis was performed to investigate the anatomic specificity of reactivation related to retrieval of visual object content. For this analysis, the PIC20× and SND20× data from the current experiment (29 subjects) were averaged with similar a data set from a previous experiment with 18 subjects (Wheeler et al., 2000). In the previous experiment, subjects studied a set of word cues paired with pictures or sounds 20 times over 2 d. During a scanned retrieval test, they were given those words and asked to judge whether the words were paired with pictures or sounds at study and to retrieve the pictures and sounds in as much detail as possible before making a response. Only studied words were presented at the test. Thus, the across-study analysis included trials, from 47 subjects, in which often-studied word cues were correctly identified as having been studied with a picture or sound. In both experiments, subjects were instructed to vividly remember the pictures and sounds before making a response.

Projection to cortical flat maps. To display functional data associated with retrieval of visual objects in relation to human retinotopically defined areas, statistical activation maps were projected onto a surface representation. The structural image used to generate the flattened cortical surface consisted of the average of 20 high-resolution (0.5 mm isotropic voxels) T1-weighted images on the same individual (Holmes et al., 1998). This template brain was segmented and flattened using methods described by Van Essen et al. (1998, 2001a) and is similar to methods described by Dale et al. (1999). Briefly, the algorithm used in this procedure extracts the surface, computed as the midpoint between the gray–white matter and gray matter–CSF boundaries (which approximates layer 4; Dale et al., 1999). This surface representation was smoothed to suppress high curvature artifacts attributable to the sampling grid to generate a fiducial surface. The fiducial surface was flattened using an algorithm that minimizes local distortions. The flattening procedure required placing cuts on the surface boundary to prevent major distortions. Next, the template brain was registered onto an atlas-representative target image, which had been made to conform to the Talairach and Tournoux (1988) atlas using the spatial normalization procedure of Lancaster et al. (1995). Finally, the functional data (in atlas space) were plotted in relation to the atlas-transformed target image surface representation. The approximate locations of retinotopically mapped visual areas were generated on the basis of data from a series of fMRI studies (Tootell et al., 1995, 1997; Hadjikhani et al., 1998) and plotted in a similar manner. These procedures were implemented using Caret and SureFit software [for software, atlases, and additional information, see Drury et al. (1996) and Van Essen et al. (1998, 2001a,b)].

Results

Behavioral results

Study conditions were manipulated to influence the level of control required during subsequent retrieval. We expected that retrieval of items studied 1× would require a greater degree of control than would retrieval of items studied extensively (20×). Consistent with this expectation (Posner and Snyder, 1975; Schneider and Shiffrin, 1977; Jacoby et al., 1999; Koriat, 2000), subjects took longer to identify 1× than 20× items (t₍₂₆₎ = 4.09; p < 0.001; Fig. 2B). Although not a process-pure index of level of control, here response time serves as one index of control during retrieval of studied information and suggests that the 20×/1× manipulation was successful. Additionally, 1× study was associated with less accurate source identification than 20× study (t₍₂₆₎= 11.14; p < 0.0001; Fig. 2A). Participants also responded more slowly to SND than to PIC items (t₍₂₆₎ = 8.10; p < 0.0001), which was paralleled by a small but reliable difference in accuracy (t₍₂₆₎ = 2.1;p < 0.05). Categorization of items by source accuracy (Fig. 2C) showed that subjects were also slower to respond to NEW-INCORRECT than to NEW-CORRECT (t₍₂₀₎ = 3.33; p < 0.01; Fig. 2C). Mean source accuracy performances for correct trials were 97.6% for PIC20×, 95.2% for SND20×, 68.9% for PIC1×, and 63.5% for SND1×. Mean correct recognition scores were 98.3% for PIC20×, 96.7% for SND20×, 73.0% for PIC1×, 69.4% for SND1×, and 90.5% for NEW items.

Fig. 2.

A, B, Response accuracy (A) and response time (B) data are shown for NEW, OLD-20×, OLD-1×, OLD-PIC, and OLD-SND conditions. Error bars indicate SEM. C, Response time data across conditions when trials were sorted on the basis of response accuracy. Asterisks and bars spanning different conditions represent different levels of significance (not all differences are illustrated): *p < 0.05; **p < 0.01; ***p < .001. NA, Not available.

Left prefrontal activity tracks controlled processing during retrieval

Regional analyses began with two left prefrontal regions, BA 45/47 and BA 44 (Fig. 3A,D), which are hypothesized to associate with controlled processing demands (Buckner et al., 1998a; Nolde et al., 1998; Rugg et al., 1999;Ranganath et al., 2000; Gold and Buckner, 2002). Consistent with a role in controlled processing, responses in both BA 45/47 and BA 44 tracked the level of control, increasing activity more after 1× study than 20× study (t₍₂₈₎ = 2.73;p < 0.05; and t₍₂₈₎ = 4.88; p < .0001, respectively; Fig. 3B,E). Paralleling the behavioral data, which showed SND retrieval to take longer than PIC retrieval, the two prefrontal regions showed greater response to SND than PIC trials (Fig. 3B,E). However, this effect was only significant in BA 44 (t₍₂₈₎ = 2.09; p < 0.05).

Fig. 3.

Frontal regions modulated activity according to the level of controlled processing. A, Horizontal section showing the BA 45/47 region overlaid onto the average anatomical image (z = −4). B, E, Signal magnitude estimates in percentage signal change for BA 45/47 across NEW, 20×, 1×, PIC, and SND conditions. Percentage signal change is relative to baseline fixation. C, F, Magnitude estimates categorized by response accuracy. D, Horizontal section showing the left dorsal frontal BA 44 region (z = 20). NA, Not available.

The pattern of responses across conditions suggested that these regions were associated with controlled processing independent of whether information was old. For 20× OLD items, which were associated with response times and accuracy rates similar to those of NEW items, the response magnitudes in both BA 45/47 and BA 44 were nearly identical to those of the NEW items (Fig. 3B,E).

Exploration of trial types based on subjects' responses also reinforced the hypothesis that these prefrontal regions associate with controlled processing demands. When trials were sorted by response accuracy, responses in the two prefrontal regions again tracked the behavioral data. NEW trials associated with INCORRECT responses, which had slower response times (Fig. 3C), showed greater prefrontal responses than NEW trials associated with CORRECT responses (Fig. 3B,E), with a significant difference noted in BA 44 (t₍₂₄₎ = 3.65; p < 0.01).

Exploratory analyses showed that the different levels of controlled processing were associated with a variety of brain regions, including those investigated in the hypothesis-driven analyses above (see Fig.6A). The more controlled (1×) condition led to increased activity in left inferior frontal cortex near BA 45/47, bilateral dorsolateral prefrontal cortex near BA 44 (seez = 32), medial frontal cortex near BA 8, bilateral frontal operculum and anterior insular cortex, bilateral caudate nucleus, and regions of bilateral medial occipital cortex near lingual gyrus and cuneus (BA 17/18). Complete coordinate lists can be obtained from the authors on request.

Fig. 6.

Statistical activation maps showing the results of exploratory whole-brain analyses that involved comparisons between 1× and 20×, OLD-20× and NEW, and PIC and SND trials (only positive activations shown). Activations are overlaid onto the average subject anatomy and shown in horizontal slices. The approximate distance of each horizontal slice from the anterior commissure–posterior commissure plane is indicated by the z coordinate (Talairach and Tournoux, 1988). Activations are scaled by significance value, with lower limits in darker colors (p< 0.001) and upper limits in brighter colors (p < 0.00000001). A, Regions associated with increased controlled processing are shown. Approximate locations are listed in Results, but note the presence of the left dorsal frontal region of interest in slice z = 32.B, Regions associated with successful recovery of old information (20×-OLD) are shown relative to NEW items. Note the presence of a large region of activation on the left atz = 44 and 32, located in parietal cortex near BA 40/39. C, Comparison of PIC and SND conditions revealing regions associated with retrieval of pictures from memory. The left BA 19/37 region can be seen at z = −8, at the occipital–temporal junction near the anterior occipital sulcus.

Parietal cortex modulates activity when information is perceived as old

Studies of episodic retrieval have suggested that a left lateral parietal region near BA 40/39 increases activity when items are correctly remembered on tests of episodic retrieval (Habib and Lepage, 1999; Konishi et al., 2000; McDermott et al., 2000; Donaldson et al., 2001). In the current experiment, BOLD responses in BA 40/39 were significantly increased in each of the four OLD trial conditions relative to the NEW trials (Fig.4B; allp < .0001). This pattern can be starkly contrasted to that found for the prefrontal regions, which did not increase responses in the OLD conditions associated with minimal controlled processing (e.g., the 20× condition; Fig. 3B,E). For BA 40/39, OLD items in the 20× condition, which took minimal time and were most consistently identified as OLD by subjects, showed the greatest response (t₍₂₈₎ = 3.63;p < 0.01 comparing 20× with 1×; Fig.4B). This region also showed strong (and similar) responses to OLD items in both the PIC and SND conditions. Therefore, left BA 40/39 increased activity for each OLD condition relative to the NEW condition, independent of the amount of controlled processing and the modality from which information was being retrieved, suggesting that it contributes to processes associated with the recovery or identification of previously experienced information.

Fig. 4.

Left parietal cortex increased activity for OLD responses. Format is similar to that of Figure 3. A, Left lateral parietal BA 40/39 region (z = 40).B, Magnitude estimates of signal change in BA 40/39 for each condition. A significant OLD–NEW difference is noted by the bracket spanning the OLD conditions and the bar spanning NEW and OLD conditions. C, Magnitude estimates when categorized by response accuracy. NA, Not available.

Sorting trials on the basis of response accuracy suggested that responses in this region correlated with the perception that information is old. Within the NEW condition, BA 40/39 responses significantly increased when subjects erroneously reported that information was old relative to when they were correct (t₍₂₄₎ = 3.70; p < 0.01). Within the OLD condition, the opposite pattern was found: responses were significantly reduced during errors when subjects incorrectly indicated that old information was new relative to when they were correct (t₍₂₈₎ = 3.85;p < 0.001). This analysis could not be performed for the 20× study items, because few errors occurred. It is also interesting to note that the level of response to the correctly remembered items in the 1× condition reached nearly the same level as correctly remembered items in the 20× condition (Fig. 4C), suggesting that the response reaches a similar level for recovered information independent of the amount of controlled processing required on a trial-by-trial basis to access that information. This collective pattern suggests that BA 40/39 activity correlates strongly with whether a subject perceives the cue as being old, in this instance, independent of how much effort was expended to achieve that perception.

Exploratory analyses were conducted to investigate neural correlates of successful recovery of information from memory. To do this, and to minimize confounds associated with differing levels of controlled processing, the higher-accuracy OLD (20×) items were contrasted with the NEW items (see Fig. 6B). These two conditions were associated with similar response times and accuracy rates but differed in terms of whether the items were OLD or NEW. Activations were detected in the left lateral parietal cortex overlapping BA 40/39, confirming the regional analyses above. Other regions that were more strongly associated with 20× items included the right and medial parietal (at or near BA 40/39), bilateral precuneus (BA 7), and bilateral parahippocampal gyri (perhaps including hippocampus proper; see z = −16), bilateral inferior and middle frontal gyri extending from BA 47 dorsally to BA 6/8, medial frontal cortex near BA 6, bilateral anterior frontal cortex near BA 10, bilateral cerebellum, and bilateral caudate nucleus. Complete coordinate lists can be obtained from the authors on request.

Inferior temporal regions are associated with retrieval of visual content

Neural correlates of content-specific processes associated with remembering were next explored by examining the inferior temporal cortex near BA 19/37 (Fig.5A), which was reported byWheeler et al. (2000) to modulate depending on content modality. Retrieval of PIC items was associated with greater activity than retrieval of SND items (t₍₂₈₎ = 3.95;p < 0.001). PIC items were also significantly more active than NEW items [t₍₂₈₎ = 3.96;p < 0.001), whereas SND and NEW items showed highly similar magnitudes (t₍₂₈₎ = 0.69). Thus, BA 19/37 showed a selective increase in response to OLD items that were associated with visual pictures during study.

Fig. 5.

Left inferior temporal cortex near BA 19/37 increased activity during retrieval of picture content. Format is similar to that of Figure 3. A, Left BA 19/37 region near the anterior occipital sulcus (z = −8).B, Magnitude estimates of signal change in BA 40/39 for each condition. C, Magnitude estimates when categorized by response accuracy.

Examination of trials based on response accuracy confirmed these observations. Neither of the miss conditions was different from the NEW-CORRECT condition, suggesting that modulation of BA 19/37 only occurred when subjects perceived the remembered visual content. By contrast, activity was greater for PIC-CORRECT than SND-CORRECT (t₍₂₈₎ = 4.33; p < 0.001), and SND-CORRECT was no different than NEW-CORRECT (t₍₂₈₎ = 0.51). The results show that BA 19/37 was most active when retrieval involved visual object content.

Exploratory analyses indicated that the correct identification of pictures at the test was associated with increased activity in bilateral posterior anterior occipital sulcus and fusiform gyrus near BA 19/37 (at or near the left-lateralized region used in the hypothesis-driven analysis) extending to the posterior parahippocampal gyrus near BA 20/36, bilateral cuneus extending from BA 19 into the parietal cortex near BA 7, and medial (BA 9/10) and right (BA 10/6) frontal cortex (Fig. 6C). Correct identification of sounds resulted in increased activity in the left inferior frontal cortex near BA 45 (data not shown). Sounds studied extensively were associated with increased activity in left superior temporal gyrus near BA 22 (similar to the results of Wheeler et al., 2000), but this result did not extend to sounds studied minimally. To further characterize this finding, we explored regional activation in the auditory association cortex using a region taken fromWheeler et al. (2000) with peak coordinates at x = −57; y = −39; and z = 14. Within this approximate BA 22 region, activity for correctly identified 20× SND (0.09%) was quantitatively larger than for 20× PIC (0.05%), but this effect failed to reach significance (t₍₂₈₎ = 1.71; p = 0.10). In addition to the study by Wheeler et al. (2000), two previous experiments have also shown increases in secondary auditory cortex during retrieval of auditory information (Zatorre et al., 1996; Nyberg et al., 2000), suggesting that the failure to reach significance in the present study is likely a type II error. Nonetheless, because of the marginal effect in the auditory cortex during retrieval of auditory content, we do not consider this effect further in the present article.

Content-specific visual retrieval selectively modulates late visual areas

An open question relates to whether early retinotopic areas are involved in visual remembering. Several studies report activation near V1 during visual mental imagery, suggesting the earliest cortical visual areas are accessed during remembering (Kosslyn et al., 1993,1995, 1999; Le Bihan et al., 1993; Owen et al., 1996; Chen et al., 1998; Klein et al., 2000). Other studies have failed to detect activation at or near the primary visual cortex. These findings suggest that retrieval processes may selectively affect visual areas involved in late stages of object perception (homologues to the primate inferior temporal cortex) (Roland and Gulyás, 1995; Mellet et al., 1996,2000; D'Esposito et al., 1997; Goebel et al., 1998; Ishai et al., 2000; Knauff et al., 2000). However, because of limited data, it has been difficult to quantify and directly compare the magnitude of activation across separate visual areas. To explore this issue, we combined the PIC20X-CORRECT and SND20X-CORRECT data from the present study (n = 29) with the analogous picture and sound conditions from an earlier study (n = 18; Wheeler et al., 2000), for a total n = 47. Quantitative regional analyses were then performed to determine the anatomic specificity of content-specific retrieval effects in visual cortex.

Four visual regions were obtained from an independent fMRI study byMaccotta and Buckner (2002), in which subjects viewed intact and scrambled objects as a targeted localizer of early and late visual areas (Fig. 7). Two of the regions, one near left calcarine sulcus (CS+) at or near BA 17/18 (peak coordinates, −21, −91, and −8; Fig. 7A) and one near the left middle occipital gyrus (MOG) at or near BA 18/19 (−29, −87, and 2; Fig.7B), were obtained by comparing both intact and scrambled objects with fixation. The other two regions, one near the left anterior occipital sulcus (AOS) at or near BA 19/37 (−36, −73, and −12; Fig. 7C) and the other near the left anterior fusiform gyrus (AFG) at or near BA 20 (−25, −37, and −20; Fig.7D), were obtained by comparing intact objects with scrambled objects. Peak activation and region definition were the same as described earlier. The regions can be seen in Figure 7 against a backdrop of the average anatomy of the 47 subjects.

Fig. 7.

Across-study analysis of 47 subjects demonstrating that late, but not early, regions of the visual cortex were sensitive to content-dependent modulations during retrieval. A, B, Percentage signal change for PIC (filled circles) and SND (open squares) conditions shown over eight time points spanning 17.5 sec, beginning with the onset of the trial. Early visual regions near the CS+ (A; ∼BA 17/18) and the MOG (B; ∼BA 18/19) increased activity equally for PIC and SND. C, D, Late visual regions near the AOS (C; ∼BA 19/37) and AFG (D; ∼BA 20) also increased activity for both PIC and SND trials but modulated on the basis of content (PIC > SND).

Time courses for PIC and SND retrieval were extracted for each region and each subject. Magnitude estimates for the PIC and SND conditions were entered into an ANOVA, which revealed a main effect of region (F_(3,138) = 12.67; p< 0.0001). As shown in Figure 7, in later visual regions (near AOS and AFG), PIC responses were reliably greater than SND responses (AOS,t₍₄₆₎ = 4.67; p < 0.0001; AFG, t₍₄₆₎ = 4.90;p < 0.0001). By direct contrast, early visual regions (CS+ and MOG) showed no difference between the PIC and SND conditions (CS+, t₍₄₆₎ = 0.38; MOG,t₍₄₆₎ = 0.20). Thus, in an analysis that affords considerable power (containing an extremely largen and using a priori-defined regions), early regions, which most likely overlap with retinotopically mapped visual areas (∼BA 17–19), revealed nearly identical quantitative values during picture and sound retrieval. Later visual regions, by contrast, showed strong and significant modulation based on retrieval content.

One limitation of the above analysis concerns the possibility that by selecting a priori regions, activation within early visual areas is missed. Moreover, the above analysis does not allow direct visualization of memory-associated activation in relation to retinotopically defined areas. To overcome these limitations, the full data set (n = 47) was used to construct activation maps of the PIC − SND retrieval effect (Fig.8A). Retrieval of pictures was associated with bilateral increases in lateral inferior temporal cortex near BA 19/37, with the left lateralized activation located near the left BA 19/37 region used in the hypothesis-driven analyses. Picture retrieval was also associated with increases in the bilateral anterior fusiform/parahippocampal gyrus (near BA 20), bilateral cuneus extending from the dorsal occipital cortex (near BA 19) to the parietal cortex (near BA 39/7), and bilateral frontal cortex (near BA 6).

Fig. 8.

Statistical activation maps from the across-study exploratory analysis showing regions more active for PIC than SND (SND > PIC activations are not shown). A, Activation maps, overlaid on top of the average anatomy of 47 subjects, showing regions of activation (red, orange) associated with retrieval of PIC items from long-term memory. The level of statistical significance is shown in the color scale bar. B, Activations in A (orange, red) shown projected onto a representative flattened cortical surface of the left hemisphere, in relation to the estimated boundaries of the retinotopically mapped visual cortex [color-coded stripes, green (MT+) on the right]. The frontal lobe is to the left; the occipital lobe is to the right.

The PIC − SND activation map was then projected onto a flattened cortical representation that also mapped estimated retinotopic boundaries. Figure 8B illustrates that the regions associated with picture retrieval (yellow, orange, red) were outside early retinotopically mapped areas of the visual cortex. A significant activation can be seen overlapping ventral portions of area MT+ (middle temporal and medial superior temporal, dark green) (Zeki et al., 1991; Tootell et al., 1995, 1997; Wandell et al., 1999; Kourtzi and Kanwisher, 2000; Huk and Heeger, 2002; Kourtzi et al., 2002). This region is located near the lateral occipital complex (LOC), a region of the cortex that has been associated with visual object processing (Malach et al., 1995; Grill-Spector et al., 2000; Kourtzi and Kanwisher, 2001). Two other regions were located anterior to area V8 (light green) and extended anteriorly into the inferior temporal cortex. Area V8 has been associated with color processing (Hadjikhani et al., 1998). These latter two regions are shown in a horizontal cross section in Figure 8A at z = −16 (left) and were located near anterior fusiform/parahippocampal gyrus. The MT+–LOC region is shown (left) in Figure 8A atz = −8. Another cluster of activations was located just dorsal and anterior to V3a, extending from occipital cortex near BA 19 to parietal cortex near BA 39/7.

One caveat regarding the present analysis is that the estimated sizes of the retinotopically mapped areas have an uncertainty (approximately ±1cm) that is greater than the width of the depicted extent (Drury et al., 1996). Therefore, the precise extent of overlap with MT+ and V8 cannot be determined. Despite this uncertainty, it is evident that the visual regions activated during remembered visual content were outside areas V1–V3, ventral posterior, and V4 and were most prominent in late visual areas associated with object and color processing.

Discussion

Results suggest dissociation in the functional anatomy supporting three classes of process associated with remembering. First, specific frontal regions near BA 44 and BA 45/47 associate with control processes. These regions are not primarily modulated by mnemonic status (e.g., whether information is old or new) but rather provide a set of processing resources for strategic access, monitoring, or both during remembering. Second, regions in the parietal cortex associate with the perception that information is old, independent of content. Examining trials associated with errors suggested that activity within parietal regions could even predict whether old cues would be missed and new cues would be inappropriately endorsed as old. Finally, regions in the inferior temporal cortex, near those associated with late stages of visual object processing, reactivate during retrieval of visual memories. Early retinotopically defined visual areas do not show such effects, suggesting that modulation of late visual areas during remembering is anatomically specific.

Controlled processing demands are reflected in frontal cortex activity

When remembering is difficult and information is not readily available, the demand for controlled search processes increases, and the act of remembering can be extended in time (Koriat, 2000). In this experiment, items studied once took more time to retrieve at the test and were less accurately identified as having been studied. Items studied once likely required a greater degree of controlled processing during retrieval than those studied 20 times (Posner and Snyder, 1975;Schneider and Shiffrin, 1977; Shiffrin and Schneider, 1977).

Paralleling the pattern of behavioral data, two left frontal regions, BA 45/47 and BA 44, tracked the level of controlled processing, increasing activity during retrieval of 1× items relative to 20× items. The more dorsal BA 44 showed additional modulations that also tracked the behavioral data, becoming more active during retrieval of words that were paired with sounds at study, relative to those paired with pictures (see Figs. 2B, 3D). Importantly, the level of activity in both frontal regions was the same for new items as it was for 20× items, indicating that the increase in activity in these regions was related to the increased demand for controlled processing and was not associated with previous experience with the items (i.e., oldness). Comparison of 1× and 20× conditions in the exploratory analysis also revealed a right dorsal frontal region near BA 44 (see Fig. 6A; z = 32) that was more active during the more controlled 1× condition. The right dorsolateral prefrontal cortex has previously been implicated in control and, more specifically, retrieval monitoring (Fletcher et al., 1998; Wagner et al., 2001; Rugg et al., 2003). Although not addressing the role of the regions in monitoring specifically, our findings are consistent with the hypothesis that right BA 44 modulates with the level of control.

The results are consistent with neuropsychological studies that indicate that damage to the frontal lobes is associated with impairment of a variety of executive and organizational functions. Patients can show decreased ability to plan and organize events (Shallice and Burgess, 1991), impaired strategic planning on memory tests with few cues or structure, such as free recall (Gershberg and Shimamura, 1995), impaired memory for source (Janowsky et al., 1989), or temporal order (Milner et al., 1985) (for review, see Schacter, 1987 and Shimamura et al., 1991). Some frontal patients also confabulate, or provide false accounts of their past, which suggests a diminished ability to properly monitor or access memories (Moscovitch, 1989; Burgess and Shallice, 1996). The results also support findings from neuroimaging studies that show that left frontal regions are associated with controlled processes during retrieval (Schacter et al., 1996; Buckner et al., 1998a; Nolde et al., 1998; Ranganath et al., 2000; Jansma et al., 2001; Dobbins et al., 2002; Buckner, 2003).

Perception of oldness associates with left lateral parietal cortex activity

Remembering is accompanied by a perception that the information is from the past. Multiple parietal and frontal regions of the brain were associated with the perception of oldness, including the lateral parietal cortex near BA 40/39. BA 40/39 increased activity during retrieval of old (studied) items relative to identification of new items. BA 40/39 was also more active during retrieval of the 20× items compared with the 1× items (Fig. 4B), a pattern opposite to that seen in the frontal regions (Fig. 3B,E). In previous studies of episodic retrieval, activity in BA 40/39 increased for trials in which old items were correctly identified, relative to trials in which new items were correctly rejected (sometimes referred to as retrieval success; Wilding and Rugg, 1996; Habib and Lepage, 1999; Henson et al., 1999, 2000; Konishi et al., 2000; McDermott et al., 2000; Rugg and Allan, 2000; Donaldson et al., 2001). This difference was present in the current experiment (Fig. 4). Furthermore, although activity was greater for correct OLD items than for NEW items, there were no differences between correctly identified 1× and 20× items (see Fig. 4C), or between correct PIC and SND items. The importance of this finding is that, in the present study, BA 40/39 modulated activity according to perceived oldness independently of level of control and type of content retrieved.

Another important finding was that BA 40/39 increased activity when subjects decided that items were old, regardless of whether the items were actually old or new (Fig. 4C). For example, when subjects mistakenly made an old response to new items, activity significantly increased more than when they made a new response. The implication is that one could predict, from the level of activity in this region, whether items were perceived as either old or new. Overall, the results suggest that left lateral parietal cortex is associated, in some manner, with a perception that the information has been previously experienced. Open questions persist, such as whether this correlate relates more to processes associated with familiarity or recollection.

Inferior temporal cortex activates during retrieval of visual object content

Remembering allows us to bring back to mind vivid details of experiences from our past. Evidence suggests that the regions of the brain that support these processes vary by content, such that, for example, the visual cortex supports the reinstatement of visual memories (Finke, 1985; Farah et al, 1988; Kosslyn et al., 1993, 1999;Roland and Gulyás, 1995; Ishai et al., 2000; Mellet et al., 2000;O'Craven and Kanwisher, 2000; Wheeler et al., 2000), the auditory cortex supports auditory memories (Zatorre and Halpern, 1993; Zatorre et al., 1996; Nyberg et al., 2000; Wheeler et al., 2000), and the motor cortex supports memory for actions (Nyberg et al., 2001). In particular, regions of the anterior occipital lobe near BA 19/37 and the fusiform gyrus, which have been implicated in visual object processing, have also been associated with retrieval of visual object information from memory (Malach et al., 1995; Owen et al., 1996;D'Esposito et al., 1997; Grill-Spector et al., 2000; Wheeler et al., 2000; Kourtzi and Kanwisher, 2001; Thompson et al., 2001). The current data also indicate that BA 19/37 is associated with retrieval of visual object content.

A significant unresolved issue is the extent to which early visual areas are involved in the reconstruction of visual memories. Some evidence suggests that reconstruction of visual content during retrieval relies on multiple levels of the visual cortical hierarchy, from early areas that process simple features of visual stimuli to late areas that process more complex object-level properties (Kosslyn et al., 1993, 1995, 1999; Le Bihan et al., 1993; Owen et al., 1996; Chen et al., 1998; Klein et al., 2000). However, other evidence suggests that retrieval of visual content can occur without the involvement of early visual areas (Roland and Gulyás, 1995; Mellet et al., 1996;D'Esposito et al., 1997; Goebel et al., 1998; Ishai et al., 2000;Knauff et al., 2000; Mellet et al., 2000; Wheeler et al., 2000). The results of our across-study analysis, which combined data from this study with similar data from a previous study (Wheeler et al., 2000), showed that, although late regions of the visual cortex were sensitive to content-dependent modulations, early regions located near BA 17/18 and BA 18/19 were not (Fig. 7). This supports the hypothesis that retrieval of visual content (relatively complex objects in this experiment) involves a top-down cascade of neural activity, which includes visual regions that process relatively high-level visual attributes associated with object perception. Early visual areas, which have more primitive retinotopically mapped receptive field properties, were not activated, suggesting an efficient retrieval process (Hebb, 1968; Roland and Gulyás, 1994; Buckner and Wheeler, 2001).

Conclusion

This experiment demonstrates that multiple component processes of remembering can be functionally dissociated. By independently manipulating different components of remembering, we identified frontal regions that modulated as a function of level of control but not mnemonic status, parietal regions that modulated according to perceived oldness but not content, and inferior temporal regions that showed content-related activity.