Neural correlates of strategic memory retrieval: Differentiating between spatial‐associative and temporal‐associative strategies

Abstract

Remembering complex, multidimensional information typically requires strategic memory retrieval, during which information is structured, for instance by spatial‐ or temporal associations. Although brain regions involved in strategic memory retrieval in general have been identified, differences in retrieval operations related to distinct retrieval strategies are not well‐understood. Thus, our aim was to identify brain regions whose activity is differentially involved in spatial‐associative and temporal‐associative retrieval. First, we showed that our behavioral paradigm probing memory for a set of object‐location associations promoted the use of a spatial‐associative structure following an encoding condition that provided multiple associations to neighboring objects (spatial‐associative condition) and the use of a temporal‐associative structure following another study condition that provided predominantly temporal associations between sequentially presented items (temporal‐associative condition). Next, we used an adapted version of this paradigm for functional MRI, where we contrasted brain activity related to the recall of object‐location associations that were either encoded in the spatial‐ or the temporal‐associative condition. In addition to brain regions generally involved in recall, we found that activity in higher‐order visual regions, including the fusiform gyrus, the lingual gyrus, and the cuneus, was relatively enhanced when subjects used a spatial‐associative structure for retrieval. In contrast, activity in the globus pallidus and the thalamus was relatively enhanced when subjects used a temporal‐associative structure for retrieval. In conclusion, we provide evidence for differential involvement of these brain regions related to different types of strategic memory retrieval and the neural structures described play a role in either spatial‐associative or temporal‐associative memory retrieval. Hum Brain Mapp 2008. © 2007 Wiley‐Liss, Inc.

INTRODUCTION

Episodic memories of previous experience are based on spatially and temporally defined sequences of events. The focus of this study is on episodic memory retrieval of these sequences of events, rather than single item memory. In general, neural operations in distinct brain regions including the medial temporal lobe, the prefrontal and temporal‐parietal cortices support episodic memory retrieval enabling people to travel backwards in subjective time to remember specific sequences of events from their personal past (Baddeley, 2001; Tulving, 2002; Wheeler, 2000). When the episodic memories to be retrieved represent complex, associative information, strategic processing (e.g., focusing on specific aspects of the memory representations or types of stored information) is beneficial in addition to the necessary operations supporting the retrieval of simple item information. Although still not well‐understood, strategic processing, or working with memory, is essential for the organization (e.g., clustering or chunking along various dimensions), selection, integration, and monitoring of the output from memory stores (Moscovitch, 1994).

Within the concept of strategic processing, there are several different problem‐solving strategies possible (Kirchhoff and Buckner, 2006) as for instance the use of the spatial and/or temporal context of the encoded information (Henson et al., 1999). Several brain regions involved in the retrieval of spatial and/or temporal context have been identified, as for instance prefrontal and parietal networks involved in performing temporal memory tasks (Cabeza et al., 1997; Dobbins et al., 2003; Simons et al., 2005). Further, dorsal midlateral right prefrontal cortex was shown to be involved in context monitoring (Henson et al., 1999). Upon retrieval of the temporal order in which words were presented, dorsal prefrontal, cuneus/precuneus and right posterior parietal regions were also shown activated in a PET study (Cabeza et al., 1997). However, when sequences of events, like object‐location associations, are to be retrieved, the spatial or temporal structure of the sequence itself is not necessarily retrieved, but may implicitly facilitate retrieval of object‐location associations. A prefrontal‐parietal network was found to be involved in implicit forms of strategic memory, as for instance in recoding information within working memory (Bor and Owen, 2007; Petersson et al., 2006). It is currently unclear whether this network is involved in different forms of implicit strategic memory, or whether the implicit use of spatial context involves different brain areas than the implicit use of temporal context in strategic episodic memory.

In this study, we focus on episodic memory for sequences of object‐location associations, whereby the location of each object has to be remembered and the spatial or temporal structure of the sequence may only be implicitly used as a retrieval strategy. When strategic memory retrieval is predominantly driven by a strategy using spatial‐associative representations, brain regions known to be involved in higher order visual processing and mental imagery like the cuneus and precuneus seem to be involved (Fletcher et al., 1995, 1996; Kosslyn et al., 1995; Wheeler et al., 2000). In contrast, the thalamus as well as the basal ganglia, in particular their output structure, the globus pallidus, are involved in forms of memory in which temporal information is crucial, for example in implicit learning of motor as well as nonmotor sequences (Forkstam and Petersson, 2005; Forkstam et al., 2006; Lehericy et al., 2005; Packard and Knowlton, 2002; Smith and McDowall, 2006; Vakil et al., 2000). In line with this, patients with basal ganglia degeneration like Parkinson's disease show deficits in implicit sequence learning (Smith and McDowall, 2006) and strategic spatial location memory (Pillon et al., 1998). Furthermore, the basal ganglia are known to be involved in behavioral sequencing and strategy selection (Cools, 1980) and the basal ganglia as well as the cerebellum are thought to be involved in internal time keeping function (Harrington and Haaland, 1999; Ivry and Spencer, 2004; Lalonde and Hannequin, 1999; Meck, 2005). Thus, it appears that the thalamus and the basal ganglia can support episodic memory retrieval of object‐location associations when this is facilitated by a strategy organizing the information along a temporal‐associative structure.

The primary objective of the present study was to identify brain regions whose activity is differentially involved in implicit use of spatial‐associatively and temporal‐associatively driven strategies. To this end, we designed two experiments (Fig. 1) in which sets of object‐location associations were encoded under two different conditions: (1) Spatial‐associative condition: Encoding of object‐location associations, providing multiple spatial associations between the item location to be learned and its neighboring objects; (2) Temporal‐associative condition: Encoding of object‐location associations by providing temporal order associations and reducing the availability of spatial associations. Importantly, the retrieval conditions in the present study differed only in the way the information was encoded. Therefore, we expected to find only those brain areas activated that are specifically involved in strategies focusing on spatial‐associative or temporal‐associative representations during object‐location retrieval. In contrast, we did not expect to find any differential activation in those brain regions that are involved in episodic retrieval of temporal structures themselves (as outlined above; Konishi et al., 2002, 2006).

Figure 1.

Experimental design, showing the timeline for a single phase. Three‐by‐three grids with simple line drawings were shown. (a) For a spatial‐associative encoding block, the subjects saw all the nine pictures at once (presented simultaneously) and had to make a living–nonliving decision on each of the pictures in a fixed pseudo random order as indicated by the moving red frame. (b) Half of the cycles started with a temporal‐associative encoding block in which participants looked at the nine pictures one‐by‐one, while the others were hidden and a living–nonliving decision had to be made on each picture in a random but predefined order. (c) In both cases, the encoding phase was followed by a 1‐back working memory task, which served as distraction. In this task, subjects had to indicate whether or not the picture in the blue frame was the same as the previous picture in the blue frame, irrespective of the location of the picture. (d) The 1‐back working memory task was always followed by a recall phase in which each of the nine pictures shown in the preceding encoding phase were shown again and subjects had to indicate their original location in the grid. In the behavioral experiment (Experiment 1), subjects were allowed to order their recall responses freely. They were presented with the studied objects each printed on a small paper card, which had to be put on the grid on the positions studied in the preceding encoding phase. (e) In the fMRI experiment (Experiment 2), the objects studied in the preceding encoding block were shown below the grid, all nine of them sequentially but in random order and subjects had to indicate their locations as A1, B2, etc., by appropriate button presses. The recall phase was always followed by a rest phase, a simple fixation cross (not shown here) after which the next cycle started with the same sequence of tasks, randomly starting with either a spatial‐associative or a temporal‐associative encoding condition.

First we investigated our experimental paradigm in a behavioral experiment to show that these encoding conditions elicited the use of either a spatial‐associative or temporal‐associative strategy during recall with unconstrained response order. Next, we measured brain activity during the recall test, using functional magnetic resonance imaging (fMRI), contrasting activity related to retrieval of associations memorized either in the spatial or the temporal encoding condition. In contrast to the behavioral experiment, the response order during the recall test was predefined in the fMRI experiment. Thus, differences in brain activity related to different retrieval strategies were not confounded by any overt response differences. We expected both recall conditions to engage overlapping retrieval networks, including temporal, parietal, and prefrontal brain regions. These networks are generally considered to be involved in different aspects of object‐location retrieval, including the hippocampus (Smith and Milner, 1989), the parahippocampal gyrus (Düzel et al., 2003; Epstein et al., 2003; Hayes et al., 2004; Sommer et al., 2005), parietal cortex (Konishi et al., 2000), and parts of the prefrontal cortex (Cabeza et al., 2003; Dobbins and Han, 2006; Dobbins et al., 2002). Because the retrieval aspects mediated by these networks might be recruited to the same or similar extent in both retrieval conditions in this study, we did not expect these brain regions to be differentially activated. In contrast, we hypothesized that the two recall conditions would differentially activate brain regions involved in the use of the specific associative representations used for strategic memory retrieval. The parietal lobe is important for spatial memory, but not necessarily for mental imagination of a coherent picture. Therefore, we predicted an increased activity in the cuneus and/or precuneus during the recall of spatial‐associatively encoded object‐location associations and in the basal ganglia and/or the thalamus during the recall of temporal‐associatively encoded object‐location associations.

MATERIALS AND METHODS

Experiment 1—The Use of Different Retrieval Strategies During Recall

Participants

Six young healthy volunteers participated in the first experiment (3 female; mean age = 27 years, SD = 3.6, range 21–31). The mean number of years of formal education was 18 (SD = 1.0). All subjects were right‐handed as indexed by an Edinburgh handedness index (≤90; Oldfield, 1971). Vision was normal or corrected‐to‐normal in every participant. All subjects gave written informed consent according to the Helsinki Declaration and the local medical ethics committee.

Stimulus material and experimental procedure

We selected 117 black‐on‐white line drawings of common living and nonliving objects (Snodgrass and Vanderwart, 1980). For each subject, we randomly chose nine drawings (five living and four nonliving) for the distraction task, 54 drawings (27 living and 27 nonliving) for the spatial‐associative encoding condition and 54 drawings (27 living and 27 nonliving) for the temporal‐associative encoding condition. In line with the subsequent fMRI experiment, this behavioral experiment was structured in 12 cycles each including four phases: encoding, distraction, visual fixation, and recall test (Fig. 1). Each cycle started with either a spatial‐associative or a temporal‐associative encoding condition, in which object‐location associations were memorized, and ended with an object‐location cued‐recall memory test. During encoding, subjects were required to memorize nine objects and their particular location in a 3 × 3 grid displayed on a computer screen. The subjects were instructed to make a living‐nonliving decision on each object and to respond verbally to ensure active participation and good recall performance. In the spatial‐associative encoding condition, a red frame moved through the grid in a fixed pseudo random order highlighting each item for 3 s, one item at a time, on which the living‐nonliving decision was made (Fig. 1a). The complete grid‐display with all nine objects was visible during the entire encoding phase providing a rich spatial‐associative encoding context, in which each item location could easily be associated with neighboring objects and the entire grid. The temporal‐associative encoding condition was identical to the spatial study condition except that each object was only transiently visible for 3 s, highlighted by the red frame and while all other items were hidden by non‐informative masks (Fig. 1b). Thus, this condition did not provide the entire grid with all objects as an associatively rich spatial structure and its structure was therefore dominated by the sequence of the presented objects.

To overwrite potentially maintained working memory of the previous encoding phase, we introduced a one‐back object memory distraction task (Fig. 1c; Baddeley, 1995). Subjects were shown a 3 × 3 grid with nine novel objects. In this distraction condition, the sequential, random movement of a blue frame over each grid‐box was accompanied by a random rearrangement of objects within the grid every 3 s. For each highlighted (blue frame) object, subjects had to indicate whether this object was identical with the one shown previously in the blue frame independently of the location within the grid over nine successive trials. To parallelize this experiment as much as possible with the subsequent fMRI experiment, we included a visual fixation phase that was equally timed to the other phases (such that every phase lasted 27 s). During this condition, a white, central fixation cross on a black background was displayed. No response was required in this condition. Subjects were instructed to attentively fixate the cross.

During the recall task, which was identical for the spatial‐associative and the temporal‐associative encoding cycles, subjects were presented with a 3 × 3 grid in carton paperboard, without drawings, as well as the studied objects each printed on a small paper card and provided at once in random spatial positions (Fig. 1d). Subjects were instructed to put the cards on the 3 × 3 grid on the positions studied during the encoding phase in any order.

Before the actual experiment, subjects practiced the task with two cycles (one spatial‐associative and one temporal‐associative study condition) with additional line drawings, which were not otherwise used during the experiment. Participants were comfortably seated at a desk with a computer monitor for stimulus presentation and the 3 × 3 grid in front of them. We used a video camera to record the responses made by the subjects for further analysis. First, the recall performance was analyzed per subject, by dividing the number of correct answers by the total number of answers, to check that all subjects performed above chance level. Next, to investigate the retrieval strategies used during recall, we analyzed the correct answers only. Specifically, we analyzed the relationship between the spatial structure of the grid and the recall order chosen by the subjects, to determine whether subjects used the spatial structure of the grid during retrieval in either of the two conditions. The number of successive correct answers in contiguous positions in the grid (Fig. 1: for instance B1 followed by B2 or B1 followed by A1 is a contiguous answer, but B1 followed by C3 is a noncontiguous answer) was counted per subject and cycle. For example, in the case of one cycle containing only two successive correct answers being B1 and A1, the actual number of contiguous correct answers would be 1. To correct this number of contiguous correct answers for differences in performance, it was expressed as a percentage of the number of contiguous correct answers that would be expected by chance. The chance level of contiguous correct answers was calculated as the number of contiguous correct answers divided by the total number of correct answers available in the grid and then set at 100%. In the example outlined above of a cycle containing only two successive correct answers the first one being B1, the chance that the next correct answer is in a contiguous position is 0.375; following B1 there are three available contiguous answers: A1, B2, and C1, divided by eight available correct answers (all nine positions except B1). The chance level of contiguous correct answers would then be set at 100%; so, in this example, 0.375 contiguous correct answers are expected by chance. Since the actual number of contiguous correct answers in this example cycle is 1, the percentage of contiguous correct answers is 1/0.375 × 100% = 267% of chance level (= 0.375 = 100%) for this particular example cycle. The percentage of contiguous answers was calculated as a measure of the use of the spatial‐associative structure of the grid during retrieval independent of the actual performance level in either of the two conditions. To investigate whether subjects used the encoding order during retrieval in either of the two conditions, we also analyzed the relationship between the given encoding order and the recall order chosen by the subjects. Our measure of interest in this analysis was the Pearson's correlation between the encoding order and the retrieval order.

Experiment 2—Brain Activity During Recall

Participants

Twenty young healthy volunteers participated in the second experiment (not included in Experiment 1; 10 female; mean age = 25 years, SD = 4, range 19–33). The mean number of years of formal education was 19 (SD = 3). All remaining subject characteristics were identical to the ones described for Experiment 1.

Stimulus material and experimental procedure

Experiment 2 was identical to experiment 1 except for: (1) To obtain sufficient power; the second experiment consisted of 20 instead of 12 cycles, each including four phases: encoding, distraction, visual fixation, and recall test (Fig. 1). Every phase lasted 29.7 s (nine items, 3.3 s each). We selected 189 black‐on‐white line drawings [nine drawings (five living and four nonliving) for the distraction task, 90 drawings (45 living and 45 nonliving) for the spatial‐associative encoding condition and 90 drawings (45 living and 45 nonliving) for the temporal‐associative encoding condition]; (2) Responses during encoding, distraction, and recall were made by appropriate button presses; (3) During the recall task, subjects were presented with the 3 × 3 grid without drawings. The participants could read the coordinates of each grid box, A1, A2, …, C3 in the corresponding box. The encoded objects were shown one at a time below the grid in random order (Fig. 1e; 3.3 s per item). Subjects were instructed to indicate the position in the grid in which the object was presented during the study phase by an appropriate combination of left and right hand key presses. The assignment of the items to the different conditions and grid positions as well as the condition with which the experiment started was randomized across subjects.

Before going into the scanner, subjects practiced the task in four cycles (two spatial‐associative and two temporal‐associative study conditions) with additional line drawings, which were not otherwise used during the experiment. We used the Presentation software (www.nbs.neuro‐bs.com) to present the stimuli and recorded the responses made by the subjects. Stimuli were back‐projected via an LCD‐projector onto a translucent screen that subjects viewed through a mirror mounted on the head coil. Subjects responded with two optical key‐devices, one in each hand. The subject's head was immobilized with a vacuum cushion to reduce head motion during fMRI data acquisition. The behavioral responses subjects made while in the scanner were analyzed for accuracy and reaction time. The use of different retrieval strategies during recall was analyzed in Experiment 1 and thus not further analyzed in Experiment 2. This approach was chosen, because it allowed us to predefine the response order during recall in the scanner, so that differences in brain activity related to the different retrieval strategies would not be confounded by any overt differences in responses.

MRI Data Acquisition

Whole head T2*‐weighted EPI‐BOLD fMRI data were acquired with a Siemens Sonata 1.5T MR scanner using an interleaved slice acquisition sequence (EPI; volume TR = 2.93 s, TE = 40 ms, 90° flip‐angle, 37 axial slices, slice‐ matrix size = 64 × 64, slice thickness = 3.5 mm, no slice gap, FOV = 224 mm, isotropic voxel‐size = 3.5 × 3.5 × 3.5 mm³). High‐resolution structural MR images were acquired with a T1‐weighted MP‐RAGE sequence (volume TR = 2.25 s, TE = 3.93 ms, 15° flip‐angle, 176 sagittal slices, slice‐matrix size = 256 × 256, slice thickness = 1 mm, no slice gap, voxel‐size = 1 × 1 × 1 mm³).

MR Image Preprocessing and Statistical Analysis

Image preprocessing and statistical analysis was executed with the SPM2 software (http://www.fil.ion.ucl.ac.uk). The functional EPI‐BOLD images were realigned and the subject‐mean functional MR images were coregistered with the corresponding structural MR images using mutual information optimization. These were subsequently spatially normalized (i.e., the normalization transformations were generated from the structural MR images and applied to the functional MR images) and transformed into a common approximate Talairach space (Talairach and Tournoux, 1988) defined by the SPM2 MNI T1 template, and finally spatially filtered by convolving the functional images with an isotropic 3D spatial Gaussian filter kernel (8 mm FWHM; Hayasaka and Nichols, 2003; Petersson et al., 1999). The fMRI data was proportionally scaled to account for global effects and analyzed statistically using the general linear model and statistical parametric mapping (Friston et al., 1995). The linear model included convolved explanatory variables (box‐car regressors) modeling the experimental conditions in a blocked fMRI design. The explanatory variables were temporally convolved with the canonical hemodynamic response function provided by SPM2. In addition, the linear model included as effects of no‐interest: session/subject‐effects, realignment parameters, and a temporal high‐pass filter to account for various low‐frequency effects. In the statistical analysis, relevant contrasts corresponding to null‐hypotheses were used to generate contrast images for each subject, which were subsequently subjected to a second‐level random effects analysis. The suprathreshold cluster‐size was used as test statistic. In the random effects analyses, the suprathreshold clusters were defined by the threshold t ₁₉ = 2.54 (except for the baseline comparisons, in which the threshold was set at t ₁₉ = 3.11 to generate clusters that were not too large to render their interpretation difficult). Only clusters significant at P < 0.05 corrected for multiple nonindependent comparisons based on the family‐wise error rate (Worsley et al., 1996) are reported. Subsequently, the significant clusters were resolved into local maxima and only local maxima significant at P < 0.05 corrected for multiple nonindependent comparisons based on the false discovery rate (Genovese et al., 2002) are reported and no masking procedure was applied. The terms of activation and deactivation are used as synonyms for a relative increase and decrease in BOLD signal, respectively.

RESULTS

Experiment 1—The Use of Different Retrieval Strategies During Recall

The cued object‐location recall performance was significantly above chance level (= 11% = 1/9 items × 100%) for both the spatial‐associatively and temporal‐associatively encoded pictures (spatial: mean correct = 74%, SD = 27, t ₅ = 5.7, P = 0.002; temporal: mean correct = 71%, SD = 26, t ₅ = 5.6, P = 0.002) and at a similar level for both conditions (t ₅ = 0.87, n.s.). To investigate whether subjects used the spatial‐associative structure of the grid during retrieval in either of the two conditions, independent of performance, we analyzed the positions in the grid of the correct answers only. The number of spatial contiguous correct answers, in the recall order chosen by the subjects, was compared to the number of spatial contiguous correct answers to be expected by chance (chance level was set at 100% for every sequence of correct answers). Only correct answers were included in this analysis. On average, for spatial‐associatively encoded items, in the sequence provided by the subjects during free recall, the percentage of correct answers that were in spatially contiguous positions in the grid, was significantly above the defined chance level (mean: 132% SD = 26%; one‐sample t‐test, test value = 100, P = 0.031). Thus, of all correct answers there are significantly more answers in contiguous positions than would be expected by chance. On the other hand, for temporal‐associatively encoded items, the percentage of spatial contiguous correct answers was not significantly different from chance level (mean: 78.4% SD = 46%, one‐sample t‐test, test value = 100, P = 0.3). To investigate whether subjects used the encoding order during retrieval in either of the conditions, we analyzed the order of the correct answers. There was a significant negative correlation between the given encoding order and the recall order chosen by the subjects for the temporal‐associatively encoded object‐location associations (r = –0.78; P < 0.05), but not for the spatial‐associatively encoded associations (r = –0.30, n.s.). This negative correlation represents a memory retrieval strategy rather than a simple recency effect, since it was absent in retrieval of the spatial‐associatively encoded associations, where a recency effect would be expected to play a similar role. Thus, these findings suggest that during the recall test without a constrained retrieval order subjects tend to utilize two different retrieval structures, depending on the way the information was encoded. For spatial‐associatively encoded items, retrieval is predominately driven by a strategy using a spatial‐associative structure, whereas subjects used the inverted encoding order as a structure for object‐location associations learned in the temporal‐associative encoding condition.

Experiment 2—Brain Activity During Recall

Behavioral data

The cued recall performance was robust and significantly above the chance level of 11% in both conditions (spatial: mean correct = 73%, SD = 16, t ₁₉ = 17.2, P < 0.001; temporal: mean correct = 67%, SD = 16, t ₁₉ = 16.1, P < 0.001) and slightly better for spatial‐ than for temporal‐associatively studied object‐location associations (t ₁₉ = 4.5, P < 0.001). To further investigate this small but significant difference in behavioral performance, we did a correlation between behavioral performance (accuracy score) and fMRI data everywhere in the brain (not restricted to the identified peak voxels or ROI data). We found that the performance scores did not explain any variance in the fMRI data, not even at high α‐levels (P > 0.5). Therefore, the performance scores were not analyzed further. The reaction times for correct responses during recall did not differ significantly between spatial‐associatively compared with temporal‐associatively encoded information (spatial: mean RT = 1.55 s, SD = 0.17; temporal: mean RT = 1.57 s, SD = 0.14; t ₁₉ = 1.45, n.s.). However, these data may be difficult to interpret, since two button presses were required per answer, whereby a right hand button press indicated A, B, or C, and a left hand button press indicated 1, 2, or 3. The reaction times mentioned are the times until the first button press (A, B, or C). Since the complete answer has not been given at that time point, putative differences in reaction time between the conditions may have been diluted. The use of two different retrieval strategies for spatial‐associatively and temporal‐associatively encoded information was already shown in Experiment 1. Relying on this result, we had the unique opportunity to focus on the quality of our fMRI data in Experiment 2. Therefore, the response order during the recall test in the scanner was predefined so that differences in brain activity related to different retrieval strategies and would not be confounded by any overt differences in responses. Thus, the use of retrieval strategies was not further analyzed in the fMRI experiment.

Imaging data

Compared to the visual‐fixation baseline condition, recall of both spatial‐associatively and temporal‐associatively studied object‐location associations activated similar brain regions, including visual and associative processing regions in the occipital [Brodmann's area (BA) 17/18/19], parietal (BA 7, 23/31, and 39/40), and temporal lobes (BA 37), as well as medial temporal (BA 36) and prefrontal (BA 6/8/9/44/45/46) regions (Fig. 2; all clusters P < 0.05, corrected). Additional significant bilateral clusters were observed in the anterior cingulate (BA 6/32), the frontal operculum/anterior insula (BA 13/15/47), and the anterior middle frontal region (BA 10/11) in the temporal‐associative retrieval condition only. With respect to subcortical structures, we observed significant activations in the thalamus and the basal ganglia in both the spatial‐associative condition [right thalamus, (x, y, z) = (14, −14, 18), P = 0.001; (16, −6, 12), P = 0.001; left thalamus, (−16, −10, 16), P = 0.001; (−12, −16, 8), P = 0.001; right caudate/putamen: (20, 6, 8), P = 0.008; left caudate/putamen: (−20, 6, 8), P = 0.008; left putamen/globus pallidus: (−18, 0, 10), P = 0.003; (−22, −6, 6), P = 0.003] and the temporal‐associative condition [right thalamus, (14, −16, 16), P < 0.001; left thalamus, (−12, −14, 10), P < 0.001; right caudate/putamen: (18, 6, 6), P < 0.001; left caudate/putamen: (−18, 4, 6), P < 0.001; left putamen/globus pallidus: (−14, −2, 12), P = 0.003; all local maxima P‐values reported were corrected for multiple nonindependent comparisons].

Figure 2.

Brain regions activated in the comparison of the recall of spatial‐associatively encoded object‐location associations (left hand columns) or the recall of temporal‐associatively encoded object‐location associations (right hand columns) versus the visual fixation condition. Activations are shown on an individual brain rendered in 3D. Only significant clusters are shown.

A direct comparison of brain activity associated with recall of spatial‐associatively and temporal‐associatively encoded object‐location associations revealed that the globus pallidus and the thalamus were significantly more active during recall of temporal‐associatively compared to spatial‐associatively studied object‐location conjunctions [bilateral cluster P = 0.029, corrected; including local maxima in left thalamus: (−10, −14, 0), Z = 3.53; right thalamus: (8, −8, 4), Z = 3.25; left thalamus/globus pallidus: (−14, −6, 0), Z = 3.33; right thalamus/globus pallidus: (12, −2, 4), Z = 3.42; right thalamus: (28, −22, 4), Z = 3.24; left globus pallidus: (−18, −4, 0), Z = 3.48; left globus pallidus: (−18, −8, 4), Z = 3.00; Fig. 3 and Table I]. Conversely, the cuneus extending into the lingual and fusiform gyri (BA 17/18/19) were significantly more active during recall of spatial‐associatively compared to temporal‐associatively encoded object‐location conjunctions [bilateral cluster P < 0.001, corrected; including local maxima in right cuneus: (6, −84, 26), Z = 3.72; left cuneus: (−4, −90, 10), Z = 3.99; right lingual gyrus: (10, −72, 0), Z = 4.05; left lingual gyrus: (−8, −72, 12), Z = 3.52; right posterior fusiform gyrus: (26, −62, −2), Z = 3.02; Fig. 3 and Table I].

Figure 3.

Brain regions activated in the direct comparison of the recall of spatial‐associatively encoded object‐location associations versus the recall of temporal‐associatively encoded object‐location associations (a) and the converse comparison (b). Activations are shown superimposed onto selected slices of an individual high‐resolution T1‐weighted volume (left and middle figures) and as intensity projections (right‐hand figures). Only significant clusters are shown.

Table I.

Brain regions differentially activated during recall of temporal‐associatively and spatial‐associatively encoded object‐location associations

Brain region	BA	Z‐score	Local maxima
Spatial versus temporal, Cuneus cluster, P < 0.001
Right cuneus	18	3.72	(6, −84, 26)
Left cuneus	17	3.99	(−4, −90, 10)
Left cuneus	17/18	4.33	(−6, −86, 16)
Left cuneus	17/18	4.46	(−8, −80, 20)
Right lingual gyrus	18	4.05	(10, −72, 0)
Right lingual gyrus	18	3.84	(10, −74, 4)
Right lingual gyrus	18	3.76	(14, −70, −6)
Right lingual gyrus	18/19	3.39	(4, −72, 8)
Left lingual gyrus	17/18	3.52	(−8, −72, 12)
Right posterior fusiform gyrus	19	3.02	(26, −62, −2)
Right posterior fusiform gyrus	19	3.01	(24, −66, −2)
Right posterior fusiform gyrus	19	2.98	(20, −62, 2)
Temporal versus spatial, Basal ganglia cluster, P = 0.029
Left thalamus		3.53	(−10, −14, 0)
Left thalamus		3.52	(−10, −20, −2)
Left thalamus		3.49	(−8, −10, 0)
Left thalamus		3.36	(−6, −14, 4)
Right thalamus		3.25	(8, −8, 4)
Right thalamus		3.14	(10, −8, 8)
Right thalamus		3.02	(6, −10, 8)
Left thalamus/globus pallidus		3.33	(−14, −6, 0)
Right thalamus/globus pallidus		3.42	(12, −2, 4)
Left globus pallidus/putamen		3.48	(−18, −4, 0)
Left globus pallidus		3.00	(−18, −8, 4)
Right thalamus/posterior putamen		3.19	(28, −22, 4)
Right posterior lateral putamen		3.24	(34, −20, 4)

BA = Brodmann's area.

DISCUSSION

In addition to operations supporting retrieval of item information, recall of complex sequences of information, as commonly required for episodic retrieval, typically entails strategic processing, and this has been conceptualized in terms of working with memory (Gabrieli et al., 1996; Moscovitch, 1994). Our fMRI results show differential recruitment of distinct brain regions during retrieval, depending only on the way in which the information was encoded. Together with the results from the behavioral experiment, these data suggest that two different types of representations are used in strategic retrieval processing. In other words, the use of a temporal‐associative structure versus a spatial‐associative structure differentially recruits distinct brain regions. More specifically, activity in the globus pallidus and the thalamus is associated with memory retrieval when subjects implicitly use the temporal‐associative structure of the encoding sequence. On the other hand, increased activity in posterior midline structures like the lingual/fusiform gyrus and the cuneus is associated with memory retrieval when subjects implicitly use a spatial‐associative structure. These results provide initial evidence for a functional differentiation between two important processes in strategic memory retrieval and also suggest that the role of the basal ganglia in processing sequences extends to the declarative memory system.

If subjects are allowed to order their recall responses freely (as in Experiment 1), they tend to order their responses along the type of structure that was predominantly provided during the preceding encoding condition. Thus, the participants ordered their responses along the spatial‐associative structure provided during the spatial‐associative encoding condition, and along the temporal‐associative structure provided during the temporal‐associative encoding condition. This result suggests that our experimental design manipulates the utilization of structured memory representation for complex information provided by the nine object‐location associations during strategic memory retrieval. Although the two recall conditions were identical in Experiment 2, fMRI revealed a differential involvement of distinct neural substrates: one for the retrieval of spatial‐associatively encoded and another for temporal‐associatively encoded object‐location associations. While the overlapping brain regions are well‐known to be involved in associative memory retrieval (Buckner et al., 1998a, b; Konishi et al., 2000) but not in temporal order retrieval itself or in recency judgment (Cabeza et al., 1997; Dobbins et al., 2003), the present results support the idea that the implicit use of different strategies during retrieval involve distinct neural substrates. Our results cannot be explained by differences in overt behavior, because the order of recall responses was randomly predefined in the fMRI experiment and thus, in contrast to an unconstrained retrieval order, in our design, every subject retrieved the same object‐location association on exactly the same (relative) point in time. The two conditions of main interest differed only in the way in which the information was encoded. Therefore, the identical design of the two recall conditions prevents our findings from being confounded by external factors, including differential visual or motor demands. A possible limitation of this approach is that an objective measure confirming the differential use of the two different strategies in the scanner is lacking. In this respect, we rely on the behavioral experiment showing that our two different encoding conditions lead to the use of two different retrieval strategies. This is a strong argument for assuming that, upon presentation of the same types of encoding conditions, different retrieval strategies would be used in the scanner as well.

Thus, the result that activity in posterior midline structures like the lingual/fusiform gyrus and the cuneus is increased during retrieval of spatial‐associatively encoded object‐location associations may be explained by the use of a spatial‐associatively driven strategy. It is conceivable that this spatial‐associatively driven strategy depended on imagining the spatial‐associative structure, in other words, imagining the grid containing the nine objects as encountered during the entire encoding phase as a coherent image (Fletcher et al., 1995, 1996; Kosslyn et al., 1995; Wheeler et al., 2000). The higher order visual processing areas, which we found to be recruited in relation to the use of spatial‐associative representations, are located in the parietal and inferior temporal lobes. The parietal lobe contributes importantly to aspects of episodic memory retrieval (Shannon and Buckner, 2004; Wagner et al., 2005). Reactivation of higher order visual processing areas that were activated during perception of items was shown to mediate their retrieval (Wheeler et al., 2000). The mechanism mediating this reactivation during retrieval remains as yet unclear. Anatomical data in monkeys showed the presence of neuronal projections between the medial temporal lobe and the parietal lobe enabling interactions between the two (Insausti et al., 1987; Kobayashi and Amaral, 2003; Suzuki and Amaral, 1994). Possibly, parts of the parietal lobe act as an interface between the prefrontal cortex (executive functions) and the medial temporal lobe (declarative memory functions; Kobayashi and Amaral, 2003; Shannon and Buckner, 2004; Valenstein et al., 1987), facilitating mental imagery and thus mediating retrieval using a spatial‐associative strategy. In support of this hypothesis, parts of the parietal lobe were found activated in tandem with the prefrontal cortex during retrieval of imagined pictures (Lundstrom et al., 2003, 2005).

The result that activity in the globus pallidus and the thalamus increased during retrieval of temporal‐associatively encoded object‐location associations may be explained by the use of a temporal‐associatively driven strategy. It is likely that the temporal‐associatively driven strategy depended on sequencing of the objects, or ordering them chronologically as encountered during the encoding phase. The globus pallidus is part of the output system of the basal ganglia and projects via the thalamus to the prefrontal cortex (Alexander et al., 1986; Lawrence et al., 1998) and thus is part of the cortico‐striatal circuit (Middleton and Strick, 2000, 2001), which is considered to be crucial for interval timing and coincidence detection (Hinton and Meck, 2004; Matell and Meck, 2004; Meck and Benson, 2002). Both the globus pallidus and the thalamus, in which we observed activation specifically related to the retrieval of temporal‐associatively encoded information, have also been observed to be activated in multiple forms of sequencing, including memory‐guided movement sequencing (Menon et al., 2000), motor sequencing (Chan et al., 2006), and semantic event sequencing (Tinaz et al., 2006). Our results thus provide initial evidence that these subcortical structures are also involved in organizing declarative memories into a temporal‐associative structure. Moreover, our findings suggest yet another role of the basal ganglia in the declarative memory system (Gabrieli, 1998; Poldrack et al., 2001; Voermans et al., 2004).

It has been proposed that, by inhibiting cortical areas not needed for the task in question, the globus pallidus subserves a focusing role and thus might provide a selection mechanism in its interaction with the prefrontal cortex (Mink and Thach, 1991). This focusing function of the globus pallidus was recently considered to be crucial to accomplish a picture sequencing task (Tinaz et al., 2006). Subthalamic inputs can excite pallidal neurons via the indirect pathway, whereas striatal inputs inhibit pallidal neurons via the direct pathway (Parent and Hazrati, 1993). Depending on the relative activity of the direct and the indirect pathway, this can lead to activity in the globus pallidus that is important for selection of prefrontal areas required in a particular cognitive task (Levy and Dubois, 2005). A task‐related increase in globus pallidus activity, as observed here during recall of temporal‐associatively encoded object‐location associations, might thus reflect a focusing on or selection of specific prefrontal areas. Therefore, the deficits in strategic memory retrieval seen in patients with basal ganglia dysfunction are likely to be the result of abnormal input to the prefrontal cortex caused by malfunctioning basal ganglia circuitry rather than of intrinsic prefrontal dysfunction per se (Berger et al., 2004; Owen et al., 1998).

Nevertheless, strategic memory is likely to recruit parts of the prefrontal cortex. It is clear that strategic processing, which is here conceptualized in terms of working with or reasoning about encoded information retrieved from memory stores, engages prefrontal functions (Bor et al., 2003, 2004; Fletcher and Henson, 2001; Henson et al., 1999; Simons and Spiers, 2003). It was previously shown that the use of the spatiotemporal context during retrieval was associated with activation of a dorsal midlateral region of the right prefrontal cortex (Henson et al., 1999). In the present study, both recall conditions activated several prefrontal regions when compared with the visual‐fixation baseline condition. These included both dorsolateral and ventrolateral prefrontal regions (Fig. 2). However, we did not observe any significant difference in prefrontal activity between the two recall conditions. The lack of differential activation of the prefrontal cortex in the present study might be due to an interaction of both sets of brain regions (or a third set) with the prefrontal cortex to a similar degree. If this is the case, the absence of a differential prefrontal response can be explained by the similar extent to which both strategies involve prefrontal activity.

It has been shown that the basal ganglia and the medial temporal lobe memory system can interact under certain conditions in declarative memory (Poldrack et al., 2001; Voermans et al., 2004) and it is well established that parts of the thalamus form an integral part of the declarative memory system (Aggleton and Brown, 1999). It therefore appears that the globus pallidus and the thalamus are well suited to regulate one form of strategic memory retrieval. In interaction with the prefrontal cortex, the globus pallidus and thalamus may select and organize information retrieved from the medial temporal lobe memory system, thus extending the role of the basal ganglia in processing sequences to the declarative memory system.

In conclusion, our study provides new evidence for differential involvement of distinct neural correlates in different types of strategic memory retrieval and shows that the role of the basal ganglia in processing sequences extends to the declarative memory system. The neural pathway used for retrieval may be flexible, depending on the way in which complex information becomes structured during retrieval contingent on encoding conditions. It is thus possible that, parts of the parietal and/or temporal lobe (spatial‐associative representations), or the globus pallidus and the thalamus (temporal‐associative representations), function as distinct interfaces between the medial temporal lobe and the prefrontal cortex. These two routes into our personal past can be flexibly used when traveling backward in subjective time to remember specific events within their spatial‐ and temporal‐associative structures.