Neural Correlates of Recency Judgment

Abstract

The prefrontal cortex plays a critical role in recollecting the temporal context of past events. The present study used event-related functional magnetic resonance imaging (fMRI) and explored the neural correlates of temporal-order retrieval during a recency judgment paradigm. In this paradigm, after study of a list of words presented sequentially, subjects were presented with two of the studied words simultaneously and were asked which of the two words was studied more recently. Two types of such retrieval trials with varied (high and low) levels of demand for temporal-order retrieval were intermixed and compared using event-related fMRI. The intraparadigm comparison of high versus low demand trials revealed brain regions with activation that was modulated on the basis of demand for temporal-order retrieval. Multiple lateral prefrontal regions including the middle and inferior lateral prefrontal cortex were prominently activated. Activation was also observed in the anterior prefrontal cortex and the medial temporal cortex, regions well documented to be related to memory retrieval in general. The modulation of brain activity in these regions suggests a detailed pathway that is engaged during recency judgment.

The prefrontal cortex has been implicated in several types of mnemonic functions (Stuss and Benson, 1986; Fuster, 1997). Among them is recollection of the temporal context of past events, an ability that has most often been tested using recency judgment paradigms in which two events are to be judged as to which has occurred more recently (Yntema and Trask, 1963). Since the initial report in Milner (1971), several neuropsychological studies of humans and monkeys have provided evidence that damage to the lateral prefrontal cortex impairs temporal-order retrieval and that the effect of damage is greater in retrieving the temporal order of past events than in retrieving the past events themselves (Shimamura et al., 1990;Milner et al., 1991; Petrides, 1991; Butters et al., 1994). Previous neuroimaging studies investigating recency judgment used this temporal-order versus item retrieval contrast and revealed prefrontal activation associated with temporal-order retrieval relative to item retrieval (Eyler Zorrilla et al., 1996; Cabeza et al., 1997, 2000).

The contrast of the dichotomous temporal-order versus item retrieval is useful in detecting functional characteristics that are differential among particular brain regions, as is most typically used in the demonstration of double dissociation between regions. However, this approach leaves unspecified the brain activity related to temporal-order retrieval itself at a whole-brain level because, for instance, it is possible that brain activity common to temporal-order and item retrieval is subtracted out even when the activity is related to temporal-order retrieval. An alternative approach complements the previous approach and allows us to uncover the whole neural correlates of temporal-order retrieval, independently of a reference task that may belong to a separate psychological category. The present event-related functional magnetic resonance imaging (fMRI) study used intraparadigm comparison and detected brain activity that is modulated depending on the demand for temporal-order retrieval itself. The basic logic and task design of the present study are similar to those of previous studies on retrieval success during old/new item recognition paradigms (Tulving, 1983; Rugg et al., 1996; Schacter et al., 1997; Buckner et al., 1998b; Henson et al., 1999a, 2000; Konishi et al., 2000; Donaldson et al., 2001). Retrieval success is revealed most efficiently by contrasting trials with a high level of retrieval success (“hit” trials) with trials with a low level of retrieval success (“correct rejection” trials). The recency judgment task devised in the present study similarly contained two types of trials with varied (high and low) levels of demand for temporal-order retrieval. The level of retrieval demand was manipulated on the basis of procedures used in previous neuropsychological studies on frontal cortical contribution to temporal-order retrieval: demand for temporal-order retrieval is greater when the temporal distance of a word pair is shorter (Milner et al., 1991) and when a word pair is not located at end positions in the list (Petrides, 1991). Trials with varied levels of retrieval demand manipulated by these features were embedded within fMRI runs, and the differential activity during high versus low demand trials was explored.

MATERIALS AND METHODS

Subjects and fMRI procedures. Informed consent was obtained from 16 healthy right-handed subjects (10 males, 6 females; age, 21–33 years). They were scanned using experimental procedures approved by the institutional review board of the University of Tokyo School of Medicine. Experiments were conducted using a 1.5 T fMRI system. Scout images were first collected to align the field of view centered on the subject's brain. T2-weighted spin-echo images were then obtained for anatomical reference [repetition time (TR) = 5.5 sec; echo time (TE) = 30 msec; 90 slices; slice thickness = 2 mm; in-plane resolution = 2 × 2 mm²). For functional imaging, gradient echo echo-planar sequences were used (TR = 3 sec; TE = 50 msec; flip angle = 90°). Each functional run consisted of 15 whole-brain acquisitions (21 6 mm slices; in-plane resolution, 4 mm), and the first four functional images in each run were excluded from analysis to take into account the equilibrium of longitudinal magnetization.

Behavioral procedures. Visual stimuli were presented to subjects by projecting the stimuli onto a screen. Subjects viewed the screen through prism glasses. A magnet-compatible button press based on a fiber-optic switch was used to record subjects' performance.

The recency judgment task consisted of two main phases, study and test (Fig. 1). During the study phase, the subjects were sequentially presented with 10 words and instructed to remember the order of the presented words for a later test phase. Each word was presented for 3 sec with an interstimulus interval of 1 sec. The words were concrete nouns taken from object stimuli in Snodgrass and Vanderwart (1980) and were presented in strings of Japanese characters.

Fig. 1.

The recency judgment task used in this study consisted of two main phases: study and test. In the test phase subjects judged which of the two words was studied more recently. The retrieval demand was made high and low in the HIGH and LOW trials, respectively. W, Word.

During the test phase, for one recency judgment trial, two words in the studied list were simultaneously presented, one each to the right and left for 3 sec with a stimulus onset asynchrony of 4 sec (that is, 3 sec word plus 1 sec fixation). The subjects were instructed to choose which word had been studied more recently. The right or left word was chosen by the subjects by pressing a right or left button, respectively, using the right thumb. Fixation gap trials (4 sec) were also included to examine the main effects relative to baseline. Only the test phase was scanned. The word pairs from the studied word lists were systematically organized to modify the level of load for temporal-order retrieval. There were two types of trials: trials with high demand for temporal-order retrieval (HIGH trials) and trials with low demand for temporal-order retrieval (LOW trials). It is known that demand for temporal-order retrieval is greater when the temporal distance of the word pair is shorter (Milner et al., 1991), consistent with the psychological literature of recency judgment (Yntema and Trask, 1963). It is also known that the demand is minimal when the pair contains words at the end in the list (Petrides, 1991), presumably because of the distinctiveness of the end words. These features were incorporated into the modification for the recency judgment trials (Fig. 1). In LOW trials, the word pair was separated by eight words and included an end word in the list (W1 or W10), and the pair was, more specifically, W1–W9 or W2–W10. In HIGH trials, the word pair was separated by three words and did not include an end word, and the pair was, more specifically, W3–W6, W4–W7, or W5–W8. Therefore, the contrast “HIGH minus LOW” is expected to reveal efficiently activation that is modulated depending on the load of temporal-order retrieval. To investigate whether there were processes specific to LOW trials (e.g., possible processes related to primacy effect), the opposite contrast of “LOW minus HIGH” was also calculated. The four recency judgment trials (two HIGH and two LOW trials) and two fixation gap trials were pseudorandomly counterbalanced in each run. The pseudorandom sequences were different from those used in event-related fMRI based on selective averaging, which were not necessary for event-related fMRI based on a general linear model (GLM). The location of a correct choice (right or left side) for the word pairs was counterbalanced across subjects by presenting the words in the study lists in a forward or backward order. The trial type (HIGH or LOW) for the word pairs was also counterbalanced across subjects by modifying the word orders in the study lists. Twelve runs were administered for each subject, each with different word lists.

To prevent the subjects from rehearsing the words between the study and test phases, a modified Wisconsin card sorting test (WCST) (Konishi et al., 1999) was performed for ∼30 sec as a distracter task. The modified WCST contained verbal demand such as the maintenance of current dimensions and the reception of verbal instruction of next dimensions and is considered to be sufficient as a distracter (Petersen and Petersen, 1959). The subjects were familiarized with the above procedures in the study–distracter–test sequence before scanning sessions.

Data analysis. Data were analyzed using SPM99 (http://www.fil.ion. ucl.ac.uk/spm/). Functional images were first realigned, slice timing corrected, normalized to the default template with interpolation to a 2 × 2 × 2 mm space, and spatially smoothed (FWHM = 8 mm). Default nonlinear parameters, brain masks, and Montreal Neurological Institute template images were used for the normalization. Event timing was then coded into a GLM (Worsley and Friston 1995; Josephs et al., 1997). Three types of transient events (correct HIGH trials, correct LOW trials, and error trials) were coded using the canonical hemodynamic response function in SPM99, time locked to the onset of these trials. In a separate analysis, to account for different reaction times in HIGH and LOW trials, the average reaction time for each trial type for each subject was convolved with the canonical function, similar to the approach ofChristoff et al. (2001). Images of parameter estimates for signal response magnitudes in the trial types were analyzed for group analysis using a random effect model. To avoid possible interference effects between word stimuli, only a limited number of images was collected for each subject (132 images), and reliable activation was revealed only after group analysis was conducted. Peak coordinate locations in activation maps were generated using a threshold of 19 or more contiguous significant voxels above p < 0.001 (t > 4.1) (each voxel 2 × 2 × 2 mm³), calculated using an empirical analysis of control data sets (Konishi et al., 2000). Note that all the detected peaks in Table 1 survived the threshold determined by the false discovery rate (Genovese et al., 2002), which yielded values oft > 3.9 for p < 0.05 (corrected) in the present data set.

Table 1.

Brain regions showing signal increase in the contrast HIGH minus LOW

	Coordinates			tvalue	Z value	BA/Area
	X	Y	Z
Lateral frontal cortex	−52	16	34	6.6	4.5	9
	−46	30	32	5.8	4.1	9
	54	14	36	5.7	4.1	9
	50	24	34	5.6	4.1	9
	−46	10	24	5.1	3.8	45/44
	−36	54	6	4.5	3.5	10/46
	−24	18	48	4.3	3.4	8/6
Medial frontal cortex	−8	18	50	8.3	5.0	8/6
	8	22	48	7.5	4.8	8/6
	−6	20	38	5.8	4.1	32
	6	28	34	5.1	3.8	32
Temporal cortex	−22	−32	−8	6.8	4.5	28/35
	30	−24	−8	5.5	4.0	28/35
Parietal cortex	−36	−46	38	6.2	4.3	40
	−46	−34	40	5.8	4.1	40
	32	−44	42	5.6	4.1	40
	12	−72	26	5.3	3.9	31/18
	−20	−68	36	5.1	3.8	7
	14	−58	60	4.4	3.5	7
Occipital cortex	−42	−70	−14	5.4	4.0	19/37
	−4	−82	24	4.9	3.7	18
	−18	−84	−18	4.6	3.6	18/19
Somatosensory/motor cortex	−30	−12	48	5.0	3.8	6
	36	−22	48	4.6	3.6	4/3
Others	−2	−12	10	6.6	4.4	Thalamus
	10	−24	−10	5.7	4.1	Brain stem
	−2	−46	−14	5.5	4.0	Cerebellum
	16	0	12	5.3	3.9	Caudate/thalamus
	−16	−4	12	4.9	3.7	Caudate/thalamus
	−2	−60	−16	4.7	3.6	Cerebellum

RESULTS

Behavioral results

Performance in the recency judgment task during fMRI scans was high: 87.0 ± 1.7% (mean ± SEM) in HIGH trials and 97.1 ± 1.4% (mean ± SEM) in LOW trials. The difference was statistically significant (paired t test;t₍₁₅₎ = 5.5; p < 0.001). Mean reaction time in correct trials was 1998 ± 56 msec (mean ± SEM) in HIGH trials and 1609 ± 48 msec (mean ± SEM) in LOW trials. The difference was also significant (pairedt test; t₍₁₅₎ = 15.7;p < .001), which is consistent with the presumed increase in temporal-order retrieval demands.

fMRI results

Compared with low-level fixation trials, a number of regions were commonly activated in both HIGH and LOW trials (Fig.2). Prominent activation was observed in occipital regions, the left primary motor cortex, and multiple frontal and parietal regions. A smaller number of regions showed differential activation for HIGH trials as compared with LOW trials. As shown in Figure 3, differential signal increase (HIGH minus LOW) was detected in multiple regions, including bilateral middle lateral prefrontal areas [near Brodman's area (BA) 9], a left inferior lateral prefrontal area (near BA 45/44), a left anterior prefrontal area (near BA 10/46), and bilateral medial temporal areas (near BA 28/35). A complete list of the differential activation is presented in Table 1. In contrast, no significant signal decrease (LOW minus HIGH) was observed. This suggests that there were no significant processes specific to LOW trials.

Fig. 2.

Statistical activation maps for signal increase in the contrasts HIGH minus FIXATION (top row) and LOW minus FIXATION (bottom row). The color scale in the maps reflects statistical significance as shown by the color bar to thetop right (above t > 4.07;p < 0.001; uncorrected). Activation maps are displayed as transverse sections and overlaid on top of the anatomic image averaged across subjects. The transverse section level is indicated by the Z coordinates of Talairach space at thebottom (Talairach and Tournoux, 1988).

Fig. 3.

Statistical activation maps for signal increase in the contrasts HIGH minus LOW (top row) and LOW minus HIGH (bottom row). Format is similar to Figure 2.

Of particular interest are the multiple activations observed in the lateral prefrontal cortex. Although relatively close to one another, close inspection of the multiple activations reveals an elaborate functional organization within the lateral prefrontal cortex. The spatial relationship among the activations is demonstrated in Figure 4. The transverse plane (Fig. 4,left) indicates that there are two bilateral sets of activations near BA 9, namely, focus 1 (−52, 16, 34) and (54, 14, 36) and focus 2 (−46, 30, 32) and (50, 24, 34). Each of these two bilateral pairs, foci 1 and 2, can be regarded as anatomically symmetrical because the distance between the bilateral pairs is <8 mm for each of the foci 1 and 2 (neglecting the laterality of xcoordinates), where the FWHM for spatial smoothing was 8 mm. The parasagittal plane (Fig. 4, right) indicates that there is another adjacent activation in the left hemisphere (focus 3) near BA 45/44 (also shown at Z = 24 in Fig. 3).

Fig. 4.

Spatial relationship of middle and inferior lateral prefrontal activations detected in the contrast HIGH minus LOW. The numbers 1, 2, and3 in the slices indicate focus 1 (−52, 16, 34) and (54, 14, 36) near BA 9, focus 2 (−46, 30, 32) and (50, 24, 34) near BA 9, and focus 3 (−46, 10, 24) near BA 45/44 (see alsoZ = 24 in Fig. 3), respectively. Thearrowhead on the transverse section shown in theleft panel indicates the plane for the parasagittal section shown in the right panel.

The time courses of the MRI signals were examined for HIGH and LOW trials relative to fixation in four regions of interest selected from the peak coordinates listed in Table 1 (Fig.5). Two regions were selected from the middle and inferior lateral prefrontal areas shown in Figure 4, and the other two regions were from the anterior prefrontal cortex and medial temporal areas, regions previously well documented in relation to memory retrieval. Differential activation was observed in these four regions, as revealed by larger peak amplitudes in the signal change for HIGH trials. A regional analysis was also performed for control visual and motor regions. In this time course analysis, peak locations were determined on the basis of the average of HIGH and LOW trials minus fixation. The visual area was selected as the largest activation in or near the primary visual cortex, and the motor area was selected as the largest left-dominant activation relative to the right hemisphere. The MRI signals for HIGH and LOW trials showed minimal differences in the visual and motor regions, indicating that the differential time courses are anatomically selective, consistent with the activation maps shown in Figure 3.

Fig. 5.

The event-related percentage MRI signal for HIGH and LOW trials displayed for four differential and two control regions. The four differential regions of interest were selected from Table 1, and the two control regions were selected on the basis of the average of HIGH and LOW trials minus fixation. The peak location (shown inyellow) is displayed to the left in each panel, and the time course for each trial type is displayed to theright.

Final analyses relate to the longer reaction time in HIGH trials relative to LOW trials. This longer reaction time may explain the differential activation shown in Figure 3. To account for this possibility, average reaction time for each trial type in each subject was convolved with the default canonical function in coding each trial type into the GLM, similar to the approach of Christoff et al. (2001), and then group analysis was conducted. The resultant activation can be interpreted as reflecting average activity per unit time. As shown in Figure 6, activation detected in Figure 3, including the multiple lateral prefrontal activations, still appears to be significant. Indeed, all of the peaks listed in Table 1 survived the level of p < 0.05 (uncorrected; testing of hypothesis in Fig. 3) in Figure 6, suggesting that these activations cannot be explained only by longer reaction time in HIGH trials relative to LOW trials. Further analysis was conducted using three regressors based on canonical hemodynamic response functions. The first regressor coded the main effect of HIGH and LOW trials, the second one coded reaction time for HIGH and LOW trials using parametric modulation, and the third one coded the trial type of HIGH or LOW using parametric modulation (+1 for HIGH and −1 for LOW trials). All of the peaks listed in Table 1 were still significant atp < 0.05 (uncorrected), confirming that the activations do not simply reflect reaction time difference.

Fig. 6.

Reaction time corrected activation maps for signal increase (top row) and decrease (bottom row) in the contrast HIGH minus LOW. Format is similar to Figure 2.

DISCUSSION

The present event-related fMRI study used a recency judgment paradigm that contained trials with high and low levels of demand for temporal-order retrieval. The intraparadigm comparison (HIGH minus LOW trials) using event-related fMRI revealed brain regions with activity that was modulated depending on retrieval demand. These regions included bilateral middle lateral prefrontal areas (near BA 9), a left inferior lateral prefrontal area (near BA 45/44), a left anterior prefrontal area (near BA 10/46), and bilateral medial temporal areas (near BA 28/35). These data suggest a detailed network of regions engaged during temporal-order retrieval.

Of central interest in this study is the prominent activation in the middle part of the middle frontal gyrus near BA 9 (Fig. 4, foci 1 and 2) detected in the contrast of HIGH minus LOW trials. The activation is located within the critical foci responsible for temporal-order retrieval previously specified in human neuropsychological studies (Milner et al., 1991) and also cytoarchitechtonically corresponds to the foci determined by neuropsychological studies on monkeys (Petrides, 1991). The activation result is also consistent with that from previous neuroimaging studies contrasting temporal-order retrieval with item retrieval that demonstrated dorsal prefrontal activation at (−37, 9, 31) and (38, 20, 31) (Eyler Zorrilla et al., 1996) and (40, 18, 32) (Cabeza et al., 1997). The converging evidence suggests the critical role of this region in temporal-order retrieval. Although it is not clear at this point whether foci 1 and 2 in Figure 4 are separate functional entities, the foci are ∼2 cm anterior to the posterior frontal regions (near BA 6/44) activated during memory encoding (not significantly activated in this study) (Kelley et al., 1998; Wagner et al., 1998b; McDermott et al., 1999; Konishi et al., 2001). Additionally, the foci are ∼2 cm dorsal and anterior to focus 3 (−46, 10, 24), which appears to correspond to the region near BA 45/44 activated during memory retrieval among competitive items (Thompson-Schill et al., 1997) and during inhibitory operations instantiated in classical neuropsychological tasks such as the Stroop task and Wisconsin card sorting task (Taylor et al., 1997; Konishi et al., 2002) (see also Konishi et al., 2001, their Table 2).

The middle prefrontal activation was modulated on the basis of the levels of demand for temporal-order retrieval. The activation in the left hemisphere, in particular, is consistent with the severe impairment of temporal-order retrieval using verbal materials (Milner, 1971, 1991). Another line of studies have shown left middle prefrontal activation during source memory (relative to item retrieval) (Nolde et al., 1998; Henson et al., 1999b) that requires another type of context retrieval the impairment of which has been implicated in frontal lobe dysfunction (Schacter et al., 1984; Janowsky et al., 1989). The left middle prefrontal activation related to both temporal-order and source retrieval suggests a common process associated with context retrieval. Moreover, the anterior prefrontal region (near BA 10/46) showed differential activation in the present study. Although several interpretations are available at this point, including retrieval success (Rugg et al., 1996; Buckner et al., 1998a;Henson et al., 1999a, 2000; Konishi et al., 2000), retrieval mode (Duzel et al., 1999), and working memory (MacLeod et al., 1998; Braver et al., 2001), the activation might also be considered in terms of context memory, because other neuroimaging studies revealed anterior prefrontal activation during context retrieval (Rugg et al., 1999;Cabeza et al., 2000).

The middle prefrontal region in the right hemisphere was also differentially activated in the present study. Right prefrontal activation is observed during retrieval tasks in general (Tulving et al., 1994) and can be interpreted as reflecting strategic search or post-retrieval processes (Wagner et al., 1998a) supported by material-independent manipulation of representations implemented by working memory (Wagner, 1999), as has been illustrated by neuroimaging studies of working memory (Petrides et al., 1993; McCarthy et al., 1994; Cohen et al., 1997; Courtney et al., 1997). Consistent with the right prefrontal activation, neuropsychological studies have demonstrated that the right, but to a lesser degree than the left, frontal damage impaired temporal-order retrieval performance when verbal materials were to be remembered (Milner 1971, 1991).

Another aspect of the activation pattern presented in this study is the left inferior prefrontal region (near BA 45/44), which was differentially activated during temporal-order retrieval. This region is known to be activated during executive operations for response inhibition instantiated in common neuropsychological tasks such as the Stroop task and the Wisconsin card sorting task (Rushworth et al., 1997; Taylor et al., 1997; Konishi et al., 2002; Nakahara et al., 2002). In the context of memory retrieval, the region is activated during memory retrieval among competitive items (Thompson-Schill et al., 1997), with damage to this region leading to impaired performance of such tasks (Thompson-Schill et al., 1998). It is suggested, on the basis of the above knowledge, that the left inferior prefrontal activation identified in this study reflects increased executive load of the selection of more recent events.

Outside the prefrontal cortex, the present study revealed differential activation in multiple posterior regions including, most notably, medial temporal cortex. Previous neuroimaging studies of temporal-order retrieval demonstrated enhanced medial temporal activation related to item retrieval relative to temporal-order retrieval (Cabeza et al., 1997, 2000). The present study used the intraparadigm comparison of HIGH minus LOW trials, and therefore the observed modulation of brain activity is based on temporal-order retrieval itself. Although the medial temporal activity is likely to be less heightened during temporal-order retrieval than during item retrieval, the medial temporal activity modulated in a load-dependent manner suggests the involvement of the medial temporal cortex in temporal-order retrieval. It is also possible that the temporal-order retrieval demands emphasized by the use of study lists as short as the present ones (10 words) increased the sensitivity to the medial temporal involvement in temporal-order retrieval. Other than the subset of regions of interest discussed above, prominent differential activation was also observed in the medial frontal, lateral and medial parietal, and subcortical regions. Although full interpretation of the functional roles of these regions remains a question for the future, the present intraparadigm comparison using event-related fMRI revealed a detailed network of brain regions involved in temporal-order retrieval that is suggestive of various aspects of psychological components that support performance of the recency judgment paradigms.