Medial temporal lobe activity at recognition increases with the duration of mnemonic delay during an object working memory task

Abstract

Object working memory (WM) engages a disseminated neural network, although the extent to which the length of time that data is held in WM influences regional activity within this network is unclear. We used functional magnetic resonance imaging to study a delayed matching to sample task in 14 healthy subjects, manipulating the duration of mnemonic delay. Across all lengths of delay, successful recognition was associated with the bilateral engagement of the inferior and middle frontal gyri and insula, the medial and inferior temporal, dorsal anterior cingulate and the posterior parietal cortices. As the length of time that data was held in WM increased, activation at recognition increased in the medial temporal, medial occipito‐temporal, anterior cingulate and posterior parietal cortices. These results confirm the components of an object WM network required for successful recognition, and suggest that parts of this network, including the medial temporal cortex, are sensitive to the duration of mnemonic delay. Hum Brain Mapp 2007. © 2006 Wiley‐Liss, Inc.

INTRODUCTION

Visuo‐spatial working memory (WM) is a core component of general intellectual ability [Gathercole et al., 2004; Verstijnen et al., 1998]. It is conceptually and physiologically divided into object and spatial WM components [Mishkin et al., 1983; Pickering, 2001; Smith et al., 1995], and may act as an intermediary between perception and long term memory, providing a limited capacity mechanism to maintain visual information in an on‐line and manipulatable form [Baddeley, 2003].

Object WM can be studied using a delayed matching to sample (DMTS) task [Gaffan, 1974], classically divided into encoding, mnemonic maintenance and recognition phases. Task load can be manipulated by increasing the duration of the maintenance delay, with a corresponding increase in response latency and reduction in response accuracy [Colom et al., 2004; Robbins et al., 1998].

Lesion [Milner, 1964; Milner et al., 1985] and animal models [Eichenbaum, 2000; Meunier et al., 1993, 1996, 1997] suggest that both intact frontal and temporal lobe function are critical for accurate DMTS performance, and that the length of the mnemonic delay, among other factors, determines the neural substrate maintaining the mnemonic signal [Andrew, 1999; Kesslak et al., 1998; Rosenzweig, 1996; Rosenzweig et al., 1993]. Though the issue of how frontal and temporal lobe activity contribute to maintain and access the mnemonic signal after a WM delay remains highly contentious. In an attempt to explore the influences on regional function Petrides [2000] in an object WM task independently manipulated task difficulty through the duration of mnemonic delay and image complexity. The author reported that lesions in the antero‐inferior temporal but not the frontal cortex caused deficits in the recognition phase that were determined by the duration of the mnemonic delay, while frontal cortex lesions were sensitive to increases in image number. Owen et al. [1995] using a similar DMTS task to this study reported impairments in recognition after a mnemonic delay in patients with temporal but not frontal lobe lesions. These results seem to suggest that while both play key roles in WM, the time sensitive substrate for the mnemonic signal may be in temporal cortex, while frontal function may relate more to an executive role [Fransen, 2005].

Functional neuroimaging studies of object WM in humans have confirmed the engagement of an extensive network incorporating nodes in the occipital, parietal [Ishai et al., 2000], lateral prefrontal [Haxby et al., 1995], and inferior [Ranganath et al., 2004] and medial temporal cortex [Ishai et al., 1999; Monk et al., 2002]. Furthermore, through manipulation of object WM load via image number or complexity, studies have identified load sensitive regions in lateral prefrontal [Klingberg et al., 1997], inferior temporal (IT) [Jha and McCarthy, 2000; Mecklinger and Pfeifer, 1996] and parietal association cortex [Jha and McCarthy, 2000].

However, the influence of the duration of the preceding mnemonic delay on the neural activity associated with successful recognition remains an intriguing question. Relatively few functional imaging studies have explored the effect of manipulating this aspect of task load. The results of these few studies have been very mixed, but suggest decreased activation in occipital cortex at greater delays, and increased activation in ventro‐lateral prefrontal (VLPFC) and IT [Goldberg et al., 1996], increased inferior and medial temporal [Elliott and Dolan, 1999], increased medial [Gabrieli et al., 1997] and increased medial but decreased IT cortex [Haxby et al., 1995].

The two earlier studies used PET imaging and consequently lacked the temporal resolution to discriminate between activity associated with mnemonic maintenance and recognition, or to deal with the confounding effect of performance deterioration associated with longer mnemonic delays. The study by Haxby et al. did however directly explore the effects of varying the length of mnemonic delay. The most recent study [Elliott and Dolan, 1999] utilised fMRI but in a block experimental design and so could not specifically examine activity at successful recognition and could only covary for task performance.

We used functional magnetic resonance imaging (fMRI) to examine the neural correlates of the DMTS task from the Cambridge Neuropsychological Test Automated Battery (CANTAB) (Cambridge Cognition, Cambridge, UK). This task has been used extensively in research and clinical settings and utilises abstract visual targets to minimise mnemonic strategies based on verbal encoding. We studied healthy volunteers, focusing in particular on the recognition phase of the experiment in order to explore how neural activity associated with correct recognition varied as a function of the duration of the preceding mnemonic delay. By employing an event‐related design we were able to improve on the methodological limitations of earlier studies. We were able to differentiate between activation associated with successful and unsuccessful trials and to exclude the potentially confounding effects of poorer task performance at increasing maintenance delay. Finally we were able to focus on activity associated with the recognition phase in particular. The aim of the study were to identify areas at recognition sensitive to the length of time that data had been already been held in WM.

We tested the following hypotheses:

• iAccurate object recognition would engage a distributed network comprising the VLPFC and anterior cingulate cortex (ACC), the medial (MT) and IT cortex and parietal association and occipital cortex.
• iiWith increasing length of mnemonic delay, activation at successful recognition would increase in the MT and VLPF cortex.

MATERIALS AND METHODS

Subjects

The study was approved by the Institute of Psychiatry Ethical Committee. Subjects were recruited by advertisement from the local community. 14 (9 male, 5 female) healthy right‐handed subjects participated. Their mean age was 26.1 years (SD 4.6, range 19–35). General intellectual function was estimated using the National Adult Reading Test [Nelson and O'Connell, 1978], mean score 114 (SD 8, range 99–128). After a complete description of the study aims and design, all subjects gave written informed consent before participating.

Experimental Task

Functional MRI data were acquired while subjects performed a modified version of the delayed matching to sample (DMTS) test from the CANTAB.

Stimuli, each subtending an angle of 5°, were presented using Visual Basic (Microsoft, Redmond) visually on a black screen, viewed through a mirror. Subjects were instructed to initially maintain visual fixation on a central cross. Each trial consisted of four phases (Fig. 1). During the initial ‘encoding’ phase, subjects were presented with a complex abstract pattern (the sample) for 5,000 ms in the centre of the screen. The sample consisted of a rectangular target divided into quadrants, each differed in shape and colour. Subjects were instructed to remember the sample as they would be asked to identify it later. The next ‘maintenance’ phase involved a delay during which the subject was instructed to hold the sample in memory while maintaining fixation on the central cross. In the third ‘recognition’ phase, subjects were shown four patterns in a North, South, East and West distribution around the central location for 6,000 ms and asked to identify the sample by pressing a joystick in the corresponding direction with their right hand. One pattern was identical to the sample, one a novel distractor (D‐Error), one the same colour but with a different shape distribution (S‐Error), and the other the same shape but with different colour distribution to the sample (C‐Error). To discourage the use of mnemonic strategies based on encoding a single quadrant, all four choice patterns shared one random quadrant in common with the sample. The duration of the maintenance delay varied across trials. Simultaneous trials involved no delay, and the sample and choice patterns were shown together at recognition. In the other trials there was a delay of 4,000 or 12,000 ms between encoding and recognition, and only the choice patterns were displayed. The final phase of each trial involved a delay during which the subjects were instructed to maintain visual fixation on a central cross. This delay was designed to equalise the inter‐trial (encoding) interval to 27,000 ms, while randomly varying the inter‐stimulus (recognition) interval, with the length of the delay dependent upon the duration of the preceding maintenance phase. Subjects were trained on 7 practice trials. In the experimental task there were 42 trials, 14 for each maintenance delay (simultaneous, 4,000 and 12,000 ms), presented in a random order in two runs of approximately 10 min.

Figure 1.

Representation of experimental paradigm.

Behavioural Data

All behavioural data, response accuracy and response latency, were recorded on a personal computer using Visual Basic (Microsoft) and analysed in SPSS Version 11.0 (SPSS, Chicago).

Image Acquisition

Gradient echo echoplanar imaging (EPI) data were acquired on a GE Signa 1.5 T system (General Electric, Milwaukee) at the Maudsley Hospital. A quadrature birdcage head coil was used for RF transmission and reception. 324 T2*‐weighted images depicting BOLD contrast [Ogawa et al., 1990] were acquired over 10 min (for each run) at each of 22 near‐axial non‐contiguous 7‐mm thick planes parallel to the intercommissural (AC‐PC) line: TE 40 ms, TR 2 s, in‐plane resolution 7 mm, interslice gap 0.7 mm. This EPI dataset provided almost complete brain coverage. The first four images were discarded to allow the magnetization to reach equilibrium amplitude. A jittered acquisition sequence optimised sampling of the BOLD response. Individual brain activation maps were co‐registered to a ‘whole head’ gradient echo image of superior spatial resolution acquired on each subject. This structural scan had the following acquisition parameters: TE 40 ms, TR 3 s, 43 slices, in‐plane resolution 3 mm, interslice gap 0.3 mm.

Individual Analysis

Data were analyzed with XBAM v3.4 (Institute of Psychiatry, London, UK) on a Sun Workstation (Sun Microsystems, Santa Clara.). Images were first realigned [Bullmore et al., 1999a] to minimise motion‐related artefacts and smoothed using a Gaussian filter (5 mm). Responses to the experimental paradigms were detected by time‐series analysis using gamma variate functions (peak responses weighted between 4 and 8 s) to model the blood oxygen level‐dependent response.

The three experimental conditions of interest were correct recognition responses after simultaneous, 4,000 or 12,000 ms maintenance delays, contrasted with baseline (visual fixation on the central cross).

To exclude colinearity for covariation associated with encoding and maintenance, which could also vary with the duration of the mnemonic delay, the design model explicitly incorporated these phases as separate conditions, as part of the linear model.

The analysis was implemented as follows. First, each experimental condition was convolved separately with the 4 and 8 s gamma functions to yield two models of the expected haemodynamic response to that condition. The weighted sum of these two convolutions that gave the best fit to the time series at each voxel was then computed. This weighted sum effectively allows voxel‐wise variability in time to peak haemodynamic response. To constrain the possible range of fits to physiologically plausible blood oxygen level‐dependent responses and improve signal detection performance, the constrained fitting procedure suggested by Friman et al. [2003] was adopted. This constrains the regression weights in the model fitting procedure for the 4 and 8 s components to non‐negative solutions. Following this fitting operation, a goodness of fit statistic, a measure of the mean power of neural response, was computed at each voxel. This was the ratio of the sum of squares of deviations from the mean intensity value due to the model (fitted time series) divided by the sum of squares due to the residuals (original time series minus model time series). This statistic is called the SSQ ratio. The percentage BOLD signal change at each voxel was also calculated. This was ((fitmax − fitmin)/mean signal intensity) × 100, where fitmax and fitmin were the maximum and minimum values of the fitted response for the time series in question.

To sample the distribution of SSQ ratio under the null hypothesis that observed values of the SSQ ratio were not determined by experimental design (with minimal assumptions), the time series at each voxel was permuted using a wavelet‐based resampling method [Breakspear et al., 2004; Bullmore et al., 2003]. This process was repeated 20 times at each voxel and the data combined over all voxels, resulting in 20 permuted parametric maps of SSQ ratio at each plane for each subject. The same permutation strategy was applied at each voxel to preserve spatial correlational structure in the data during randomisation. Combining the randomised data over all voxels yields the distribution of SSQ ratio under the null hypothesis. A test that any given voxel is activated at any required type I error can then be carried out by obtaining the appropriate critical value of SSQ ratio from the null distribution. For example, SSQ ratio values in the observed data lying above the 99th percentile of the null distribution have a probability under the null hypothesis of ≤0.01. We have shown that this permutation method gives very good type I error control with minimal distributional assumptions [Breakspear et al., 2003; Bullmore et al., 2001].

Group Mapping

To extend inference for each condition of interest (recognition after simultaneous, 4,000 and 12,000 ms mnemonic delays) to the group level, the observed and randomised SSQ ratio maps for each subject were transformed into standard space by a two stage process involving first a rigid body transformation of the fMRI data into the gradient echo image of the same subject, followed by an affine transformation onto a Talairach template [Brammer et al., 1997]. By applying the two spatial transformations computed above for each subject to the statistic maps obtained by analysing the observed and wavelet‐randomised data, a generic brain activation map was produced for each experimental condition, firstly across the two runs for each subject, then across subjects. The median observed SSQ ratio over all subjects at each voxel (median values were used to minimise outlier effects) can then be tested at each intracerebral voxel in standard space [Schmahmann et al., 1999; Talairach et al., 1988] against a critical value of the permutation distribution for median SSQ ratio ascertained from the spatially transformed wavelet‐permuted data [Brammer et al., 1997]. To increase sensitivity and reduce the multiple comparison problem encountered in fMRI, hypothesis testing was carried out at the cluster level using method developed by Bullmore et al. [1999b], shown to give excellent cluster‐wise type I error control in both structural and functional fMRI analysis. When applied to fMRI data, this method estimates the probability of occurrence of clusters under the null hypothesis using the distribution of median SSQ ratios computed from spatially transformed data obtained from wavelet permutation of the time series at each voxel. Image‐wise expectation of the number of false positive clusters under the null hypothesis is set for each analysis at less than one.

Conjunction Analysis

To identify regions of activation at successful recognition common to the three lengths of maintenance delay (simultaneous, 4,000 and 12,000 ms), we performed a conjunction analysis, based on the method described by Nichols et al. [2005], but applying non‐parametric assumptions. To set an image wide expectation of less than one false positive cluster; we set a voxel‐wise P value of 0.001 and a cluster‐wise P value of 0.01. The method identifies voxels where the minimum SSQ at successful recognition following each of the three delay conditions exceeds the critical threshold for significance, and calculates the median statistic across the three. Cluster‐level maps were then obtained as described above.

Trend Analysis (Analysis of Variance)

To examine the data for a relationship between neural activity at successful recognition and the duration of the preceding mnemonic delay, we performed a trend analysis across the three delay conditions constraining the model to identify areas at recognition where simultaneous >4,000 ms >12,000 ms and simultaneous <4,000 ms <12,000 ms.

Analysis of variance was carried out on the effect size maps representing % change in BOLD response in standard space, by first computing the difference in median SSQ ratio between groups at each voxel. Subsequent inference of the probability of this difference under the null hypothesis was made by reference to the null distribution obtained by repeated random permutation of group membership and recomputation of the difference in median SSQ ratios between the two groups obtained from the resampling process. Cluster‐level maps were then obtained as described above. We set a voxel‐wise P value of 0.05 and a cluster‐wise P value of 0.0025. This method ensured a total number of false positive clusters of less than one. Corrections for multiple comparisons were not required, as thresholds were set on an image‐wide, not a voxel‐wise basis.

Between Condition Differences (Analysis of Variance)

To more precisely identify the relationship between the duration of the preceding mnemonic delay and neural activity, post‐hoc comparisons were made between the respective delay conditions (simultaneous, 4,000 and 12,000 ms).

Finally, to clarify those parts of the network particularly associated with successful as opposed to unsuccessful recognition, we made a post‐hoc comparison between correct and incorrect trials within each maintenance delay. This analysis was restricted to the 12,000 ms delay condition as this was the only delay with sufficient incorrect responses (mean = 20%, 95%CI = 15–28%) for the comparison. Even within this condition, four subjects made no incorrect responses, and so their data were excluded and this contrast involved data from 10 subjects.

Analysis of variance was carried out on the effect size maps in standard space by first computing the difference in median SSQ ratio between groups at each voxel. Subsequent inference of the probability of this difference under the null hypothesis was made by reference to the null distribution obtained by repeated random permutation of group membership and recomputation of the difference in median SSQ ratios between the two groups obtained from the resampling process. Cluster‐level maps were then obtained as described above. We set a an image‐wide voxel‐wise P value of 0.05 and a cluster‐wise P value of 0.0025, to ensure a total number of false positives clusters of less than one.

RESULTS

Behavioural Data

Response accuracy and latency varied significantly with duration of the preceding maintenance delay (Table I, Fig. 2). As the maintenance delay increased, response latency increased and response accuracy decreased.

Table I.

Behavioural response data

	Interval			Test statistic[F (df) P]
	Simultaneous	4,000 ms	12,000 ms
Correct response rate [mean (SD)]	0.92 (0.27)	0.86 (0.35)	0.81 (0.39)	12.38 (2) <0.001
Response Latency Secs. [mean (SD)]	2.32 (1.02)	2.50 (1.12)	2.66 (1.16)	7.33 (2) 0.001

Figure 2.

Response accuracy and latency for each delay condition.

Post‐hoc tests revealed significant differences in both correct response rate and response latency between simultaneous and 4,000 ms (t = 3.19 (223 df), P = 0.02 and t = −3.94 (223 df) P = 0.027) and simultaneous and 12,000 ms delays (t = 4.95 (223 df), P < 0.001 and t = −3.94 (233 df), P < 0.001 respectively). There was also a trend to significant differences in correct response rate and response latency between the 4,000 and 12,000 ms delays (t = 1.90 (223 df), P = 0.059 and t = −1.60 (223 df), P = 0.11 respectively).

There was a significant interaction between duration of mnemonic delay and error type (F = 20.53 (6 df), P < 0.001). The number of C‐Error, D‐Error and no response errors remained similar, although the number of S‐Errors rose with duration of mnemonic delay (Fig. 3).

Figure 3.

Mean number of errors by subtype for each delay condition.

Areas Activated in Association With Successful Recognition Common to All Three Delay Conditions

Activation at correct recognition common to the three lengths of preceding maintenance delay was detected bilaterally in the inferior and middle frontal gyri and insula, the dorsal bank of the anterior cingulate gyrus, the precuneus, the inferior and middle occipital gyri, the fusiform gyri and the cerebellar cortex. Additional right sided activation was evident in a cluster that included the caudate and the brain stem, extending into the posterior hippocampal gyrus (Table II and Fig. 4).

Table II.

Areas of common activation associated with correct recognition across the three conditions

Size	Side	Cerebral region	Tal(x)	Tal(y)	Tal(z)	Brodmann's area
60	L	Cerebellum	−29	−78	−18	*
55	R	Cerebellum	25	−74	−18	*
64	L	Fusiform Gyrus	−22	−78	−13	19
44	R	Fusiform gyrus	29	−63	−13	19
43	R	Middle occipital gyrus	29	−81	−7	18
42	L	Middle occipital gyrus	−22	−70	−7	18
13	R	Brain stem/hippocampal gyrus	7	−26	−7	*/35
12	L	Inferior frontal gyrus	−29	26	−7	47
11	R	Inferior frontal gyrus	40	19	−7	47
24	R	Insula	32	22	−2	13
12	L	Insula	−29	22	−2	13
22	R	Lingual gyrus	14	−70	4	18
14	R	Caudate	11	7	4	*
8	L	Thalamus medial dorsal nucleus	−7	−19	4	*
9	R	Thalamus medial dorsal nucleus	11	−19	9	*
5	R	Putamen	18	0	9	*
6	R	Cuneus	14	−70	9	31
32	R	Inferior frontal gyrus	43	4	26	44
30	L	Inferior frontal gyrus	−47	4	26	44
8	R	Inferior parietal lobule	40	−33	31	40
37	L	Middle frontal gyrus	−43	0	37	6
52	L	Precuneus	−25	−52	37	31
44	R	Precuneus	29	−74	37	19
19	R	Anterior cingulate gyrus	4	15	37	32
64	R	Middle frontal gyrus	47	0	42	6
27	L	Anterior cingulate gyrus	0	7	42	32
33	R	Precentral gyrus	29	−11	48	4
50	L	Precentral Gyrus	−32	−19	53	4

L, left; R, right.

All clusters reported at voxel P = 0.001 and cluster P = 0.01, giving less than one false positive cluster. Only the cluster with the largest number of voxels within each region is reported, and is limited to clusters with more than five voxels. Talairach coordinates refer to the voxel with the maximum sum of squares ratio, a measure of neural response, in each cluster. Only in phase results are reported.

Asterisk indicates that no Brodmann's area number correspond to that region.

Figure 4.

Conjunction analysis of activation at correct recognition, common to the three lengths of preceding maintenance delay. Clusters are shown bilaterally in the inferior and middle frontal gyri, the anterior cingulate gyrus, the precuneus, the inferior and middle occipital gyri and the fusiform gyri. Results are superimposed onto a 3D rendered cortical surface of a template brain. (A) Coronal, (B) sagittal and (C) axial views. Voxel P = 0.001 and cluster P = 0.01, giving less than one false positive cluster.

Changes in Activation Associated With Increasing Delay

As the duration of the preceding mnemonic delay increased (simultaneous, 4,000 and 12,000 ms) activation at recognition increased bilaterally in the hippocampal gyri, the medial portion of the lingual gyri and the dorsal bank of the anterior cingulate gyri, as well as a region spanning the left cuneus and precuneus. Conversely, increasing mnemonic delay was associated with reduced engagement at recognition of more ventral temporo‐occipital cortex bilaterally, and the middle occipital gyri (Table III and Fig. 5).

Table III.

Areas showing an increase and decrease in neural response associated with correct recognition across the three delay conditions

Size	Side	Cerebral region	Tal(x)	Tal(y)	Tal(z)	Brodmann's area
Positive
21	R	Cerebellum‐vermis	11	−44	−29	*
32	R	Brain stem/hippocampal gyrus	4	−19	−24	*/35
25	L	Cerebellum‐vermis	−7	−48	−18	*
8	L	Hippocampal gyrus	−25	−30	−18	36
31	L	Lingual gyrus	−11	−74	−7	18
7	R	Lingual gyrus	18	−81	4	18
24	L	Cuneus	−11	−85	9	17
10	R	Posterior cingulate gyrus	18	−67	15	31
6	R	Middle occipital gyrus	14	−78	20	18
12	R	Anterior cingulate gyrus	11	37	26	32
9	L	Anterior cingulate gyrus	−11	19	37	32
12	L	Postcentral gyrus	−25	−30	42	1
5	L	Superior frontal gyrus	−4	22	48	8
8	L	Precuneus	−18	−44	53	7
7	L	Precentral gyrus	−14	−30	53	4
Negative
10	R	Cerebellum	36	−59	−29	*
14	L	Cerebellum	−32	−70	−18	*
12	L	Fusiform gyrus	−43	−67	−13	19
6	R	Fusiform gyrus	43	−56	−13	37
17	R	Inferior occipital gyrus	36	−74	−13	19
24	L	Lingual gyrus	−32	−56	−7	19
40	R	Middle occipital gyrus	47	−59	−7	19
12	L	Middle occipital gyrus	−32	−74	4	19
31	R	Thalamus dorsal medial nucleus	7	−22	9	*
29	R	Insula	25	−4	15	*
18	R	Inferior parietal lobule	32	−41	31	40
17	R	Precuneus	25	−63	42	19

L, left; R, right.

All clusters reported at voxel P = 0.05 and cluster P = 0.0025, giving less than one false positive cluster. Only the cluster with the largest number of voxels within each region is reported, and is limited to clusters with more than five voxels. Talairach coordinates refer to the voxel with the maximum sum of squares ratio, a measure of neural response, in each cluster. Only in phase results are reported.

Asterisk indicates that no Brodmann's area number correspond to that region.

Figure 5.

Neural Response across delay conditions in inferior frontal and inferior and medial temporal cortex.

Differences in Activation Between Lengths of Maintenance Delay

4,000 or 12,000 ms delay > simultaneous

Relative to simultaneous recognition, successful recognition after both a 4,000 and a 12,000 ms maintenance delay was associated with greater activation in the left parahippocampal, hippocampal and fusiform gyri, although the activation in these areas was more extensive following the 12,000 ms delay. The 12,000 ms delay (but not the 4,000 ms delay) was also associated with greater activation relative to simultaneous recognition in the anterior cingulate gyrus bilaterally, and in the right superior frontal gyrus (Table IV).

Table IV.

Differences in neural response at correct recognition between delay conditions

Conditions	Size	Side	Cerebral Region	x	y	z	Brodmann's area
Sim>4
	14	R	Cerebellum	32	−59	−29	*
	12	L	Cerebellum	−40	−56	−24	*
	37	R	Fusiform gyrus	29	−78	−18	19
	31	L	Fusiform gyrus	−32	−81	−13	19
	42	R	Middle occipital gyrus	25	−81	−7	18
	9	L	Parahippocampal gyrus	−7	−37	−2	30
	20	L	Inferior/middle occipital gyrus	−40	−74	4	19/37
	7	R	Parahippocampal gyrus	11	−41	4	30
	25	R	Thalamus medial dorsal nucleus	7	−22	9	*
	16	R	Posterior cingulate gyrus	14	−41	9	29
	5	L	Cuneus	−18	−85	20	19
	6	R	Cuneus	25	−78	31	7
	11	R	Precuneus	29	−70	31	19
Sim<4
	29	––	Cerebellum‐vermis	0	−59	−18	*
	13	L	Cerebellum	−4	−59	−13	*
	7	R	Cerebellum	11	−52	−13	*
	13	L	Fusiform gyrus	−11	−59	−7	19
	5	R	Fusiform/hippocampal gyrus	14	−48	−7	19/37
Sim>12
	31	L	Cerebellum	−25	−85	−18	*
	14	R	Cerebellum	29	−78	−18	*
	17	L	Fusiform gyrus	−43	−67	−13	19
	31	R	Fusiform gyrus	43	−67	−7	19
	10	L	Inferior/middle occipital gyrus	−40	−70	−2	19
	7	R	Inferior/middle occipital gyrus	36	−74	4	19
	15	R	Putamen	22	7	4	*
	30	R	Thalamus dorsal medial nucleus	7	−22	9	*
	12	R	Posterior cingulate gyrus	11	−41	15	30
	3	––	Cuneus	0	56	29	23
Sim<12
	26	R	Cerebellum‐vermis	4	−63	−24	*
	31	L	Cerebellum	−11	−74	−18	71
	10	L	Hippocampal gyrus	−14	−37	−13	36
	27	L	Lingual/fusiform gyrus	−11	−81	−7	18/19
	14	L	Cuneus	−4	−74	9	31
	11	L	Anterior cingulate gyrus	−7	33	15	24/32
	19	R	Anterior cingulate gyrus	14	37	15	32
	5	R	Superior/middle frontal gyrus	18	44	20	10/9
	13	L	Posterior cingulate gyrus	−18	−33	42	31
	12	L	Precuneus	−25	−37	48	7
4 > 12
			Nil
4 < 12
	7	R	Cerebellum	18	−67	−24	*
	14	L	Cerebellum	−18	−63	−18	*
	12	R	Brain stem/hippocampal gyrus	4	−19	−7	34
	9	R	Anterior cingulate gyrus	11	26	−7	32
	20	––	Anterior cingulate gyrus	0	15	−2	25
	9	L	Inferior occipital gyrus	−22	−85	−2	18
	5	R	Lingual gyrus	11	−74	−2	18
	14	L	Lingual gyrus	−4	−70	4	18
	8	R	Middle temporal gyrus	51	−48	4	21
	8	R	Superior temporal gyrus	54	−48	9	22
	31	L	Cuneus	−11	−70	9	30
	20	R	Cuneus	18	−67	15	18
	20	R	Precuneus	18	−74	26	31
	7	R	Superior frontal gyrus	4	48	31	9
	10	R	Precuneus	4	−63	37	7

L, left R; right; sim, simultaneous 4, 4,000 ms; 12, 12,000 ms.

All clusters reported at voxel P = 0.05 and cluster P = 0.0025, yielding less than one false positive cluster. Only the cluster with the largest number of voxels in each region is reported, and is limited to clusters with more than five voxels. Talairach coordinates refer to the voxel with the largest sum of squares ratio, a measure of power of neural response, in each cluster.

Asterisk indicates that no Brodmann's area correspond to those coordinates.

12,000 delay > 4,000 ms delay

Successful recognition after a 12,000 ms delay was associated with activation relative to recognition after a 4000 ms delay in a region with a focus close to the right red nucleus which extended into the right hippocampal gyrus, the anterior cingulate gyrus bilaterally and the right superior frontal gyrus.

Correct versus incorrect recognition after 12,000 ms delay

Relative to incorrect recognition, correct recognition after a 12,000 ms delay was associated with greater activation bilaterally in the anterior insula, the ventro‐lateral region of the inferior frontal gyrus, the rostral portion of the anterior cingulate gyrus and the precuneus. There was also greater engagement of the right IT gyrus (Table V).

Table V.

Differences in neural response between correct and incorrect recognition (after 12,000 ms delay)

Size	Side	Cerebral region	Tal(x)	Tal(y)	Tal(z)	Brodmann'sarea
Corr > Incorr
25	L	Cerebellum	−14	−44	−40	*
12	R	Cerebellum	36	−52	−24	*
12	R	Inferior temporal gyrus	40	−59	−13	37
19	R	Lingual gyrus	14	−70	−7	18
11	L	Insula	−25	22	−2	13
37	L	Anterior cingulate gyrus	−4	30	−2	24
32	R	Anterior cingulate gyrus	4	26	4	24
14	L	Inferior frontal gyrus	−32	30	4	45
13	R	Insula	29	19	9	13
13	R	Cuneus	18	−85	9	18
32	L	Cuneus	−7	−74	15	18
10	R	Inferior frontal gyrus	36	22	15	45
11	R	Middle frontal gyrus	29	33	20	46/10
19	R	Superior occipital gyrus	25	−78	31	19
9	L	Precuneus	−18	−63	48	7
12	R	Precuneus	0	−63	48	7

L, left; R, right.

All clusters reported at voxel P = 0.05 and cluster P value 0.001 yielding less than one false positive cluster. Only the cluster with the largest number of voxels in each region is reported. Talairach coordinates refer to the voxel with the largest sum of squares ratio, a measure of power of neural response, in each cluster.

Asterisk indicates that no Brodmann's area correspond to those coordinates.

DISCUSSION

Object Recognition WM Network

We used a DMTS task and fMRI in 14 healthy volunteers to identify the neural substrate associated with successful object WM. By manipulating the length of mnemonic delay, we altered task difficulty with consequent effects on behavioural response, and determined how activity within the neural network varied as the length of time information was held in WM increased.

Frontal Lobe Function

We identified robust activation in the VLPFC and ACC across all durations of mnemonic delay. Moreover, the greater engagement of these two regions during correct relative to incorrect trials particularly implicated them in successful (as opposed to unsuccessful) recognition.

Studies using a variety of WM tasks have emphasised the role of lateral prefrontal cortex in WM, although the nature of this role remains contentious. ‘Material specific’ models [Ungerleider et al., 1998; Wilson et al., 1993] have emphasised a role as the mnemonic substrate itself, and propose that prefrontal regions are parcellated according to the nature of the data being stored, with spatial material in the dorsal, and object material in the ventral pathways. By contrast functional models propose that prefrontal regional activity can be divided according to functional specialisation, VLPFC managing active retrieval and DLPFC familiarity monitoring [Passingham et al., 2000; Petrides, 1998].

VLPFC activity at encoding is sensitive to memoranda load [Klingberg et al., 1997], and predicts future response accuracy [Pessoa et al., 2002]. It is also sensitive to interference during mnemonic rehearsal [Sakai and Passingham, 2004; Sakai et al., 2002a] and to the manipulation of data in WM [Carpenter et al., 1999]. However, VLPFC is not essential to accurate object WM [Rushworth et al., 1997] and Petrides [2000] has suggested that prefrontal cortex lesions are specifically associated with executive processing deficits in WM rather than impaired mnemonic accuracy per se. Additionally VLPFC is engaged by a wide variety of non‐mnemonic cognitive tasks, including paradigms that recruit processes germane to a multiple option delayed response task like our DMTS experiment, including congruent versus incongruent, Go‐NoGo and response inhibition [Ridderinkhof et al., 2004] tasks. We manipulated task load via the duration of the mnemonic delay between encoding and retrieval. In contrast, other aspects of the paradigm that could have influenced task difficulty, specifically image complexity and number, were fixed. We failed to detect any evidence, either in the trend analysis or the post‐hoc between condition analyses, of a relationship between the magnitude of activation in VLPFC and the duration of the mnemonic delay (Fig. 5). This suggests a regional role that is independent of delay or that is saturated at a short delay [Rypma and D'Esposito, 1999].

Given that we detected activation in the VLPFC in association with successful recognition across all lengths of maintenance delay, and that this activation seemed independent of the duration of that delay, the data are consistent with models implicating this region in the strategic organisation of WM [Buckner, 2003; Bunge et al., 2002; Curtis and D'Esposito, 2003; Duncan and Owen, 2000; Petrides, 2000; Rypma and D'Esposito, 1999, 2003; Rypma et al., 2002] and response selection [Rowe et al., 2000]. These findings implicate VLPFC in a top‐down processing role [Pasternak and Greenlee, 2005], providing executive control of object WM, possibly managing object information held in other cortical regions and resisting interference [Miyashita and Hayashi, 2000; Ranganath et al., 2004; Sakai and Passingham, 2004] rather than as the mnemonic substrate itself.

In contrast the ACC, and specifically its dorsal bank, showed progressively increasing activity with increasing mnemonic delay (Fig. 6). While this could reflect a primary role for the ACC as the mnemonic substrate itself, there is little evidence to support this conclusion. Bunge [Bunge et al., 2004] detected similar responses in the ACC in an associative retrieval task, detecting greater ACC response under conditions of uncertainty, where reliance was less on the controlled recollection of data, putatively mediated by VLPFC, and more on familiarity, hypothetically marshalled via the ACC [Maril et al., 2001]. Our findings are thus consistent with the involvement of the ACC in concentration and mnemonic conflict management [Bunge et al., 2004] and response anticipation and selection [Matsumoto and Tanaka, 2004; Turken and Swick, 1999], under circumstances of greater mnemonic delay where data is less accurately recalled from WM.

Figure 6.

Increasing activation in the anterior cingulate (upper panel) and in the medial temporal lobe (lower panel) with increasing maintenance delay across conditions (12,000 ms > 4,000 ms > simultaneous). Results are superimposed onto a standard template brain. (A) Coronal, (B) sagittal and (C) Axial views. Voxel P = 0.05 and cluster P = 0.0025, giving less than one false positive cluster.

Temporal Lobe Function

We confirmed that both IT and MT activity were associated with successful recognition [Milner et al., 1998; Monk et al., 2002] irrespective of the length of the delay. Furthermore and consistent with our hypothesis, we detected increased activation in MT (but reduced activation in IT) with increasing mnemonic delay (Figs. 5 and 6), as well as when long mnemonic delays were directly contrasted with short ones. This suggests a critical, and time dependent role for temporal lobe structures in object WM.

During the encoding phase of the task, subjects could not anticipate the duration of the subsequent mnemonic delay, and so the initial demands at encoding and maintenance should have been equal across delay conditions. Our results therefore suggest that as the length of the maintenance delay increased, the neural resources in the temporal lobe contributing to the maintenance and subsequent recall of the memory trace altered.

Animal lesion work has shown that IT lobe integrity is central to accurate object recognition after a delay [Easton and Gaffan, 2000; Easton et al., 2001; Merigan and Saunders, 2004] and may represent the neural substrate maintaining object information in WM [Yakovlev et al., 1998]. Functional imaging studies have confirmed that IT activity exhibits specificity for object identity [Chao et al., 1999; Courtney et al., 1997; Ishai et al., 1999, 2000] and colour [Beauchamp et al., 1999], and that object WM and its interaction with longer term memory require sustained IT activity [Haxby et al., 2000; Ranganath et al., 2004]. IT activity may thus contribute directly to the neural substrate maintaining an object representation in WM, and this region may represent the zenith in a hierarchical model of short term visual information processing, possibly acting as the point of integration between perception and WM [Murray and Bussey, 1999].

While IT activity was common to successful recognition at all lengths of maintenance delay and to successful compared to unsuccessful trials the magnitude of IT activity decayed with the duration of the mnemonic delay. Neither deteriorating task performance nor increased response latency could have contributed to this result because of the experimental and analytical designs. Studies that have examined long term visual memory and visual imagery emphasise that while IT activity may have decayed it remains a key component of successful object recognition. The neural response to familiarity in IT cortex is complex. Electrophysiological studies have identified populations of neurones in this region that respond to stimulus familiarity through firing ‘suppression’ [Brown et al., 1987; Brown and Xiang, 1998; Fahy et al., 1993; Miller et al., 1991] as well as firing ‘enhancement’ [Li et al., 1993]. It is possible that the balance of these responses may alter as a function of task demand [Brown and Aggleton, 2001; Murray and Bussey, 1999], determined in this case by the duration of the mnemonic delay. Indeed other imaging studies [Jiang et al., 2000; Vandenberghe et al., 1995] have detected reductions in cortical activity in IT cortex specifically associated with image familiarity.

Studies of the effects of experimental MT lesions [Liu et al., 2000; Liu and Richmond, 2000; Spitzer and Richmond, 1991; Wiener et al., 2001] have suggested that intact MT lobe function plays a central role in delay dependent learning. Fernandez [Fernandez et al., 1999a, b] has proposed a model of sequentially correlated neural activity between subunits within the MT lobe supporting the transition from WM to longer term memory. Recent work [Brown and Aggleton, 2001; Eichenbaum, 2000; Gabrieli et al., 1997; Gilbert and Kesner, 2003] is consistent with such a hierarchical model and indicates that parahippocampal, perirhinal and adjacent visual association cortex are directly involved in ‘recency’ or familiarity recall via cortical reactivation [Sakai et al., 2002b]. In contrast, hippocampal and subicular activity may be related to the integration of more permanent mnemonic signals for associative learning, possibly by synaptic modification, with a significant delay dependent role [Hammond et al., 2004; Kesner et al., 2002].

In the present study, while recognition after all lengths of delay was associated with activation in ventro‐lateral regions of temporo‐occipital cortex (particularly the fusiform gyrus), this decayed with time. As the length of the maintenance delay increased, activity within more rostro‐medial components, particularly the posterior hippocampal gyri, increased, also consistent with the HIPER model emphasising the importance of posterior MT lobe to data retrieval [Lepage et al., 1998; Prince et al., 2005].

While our ‘trial unique’ variant of DMTS placed relatively greater demand on MT activity [Eacott et al., 1994], this effect will have been common across all lengths of mnemonic delay. Thus, the increase in MT activity observed with time may reflect a progressive shift of relative activity between time dependent phases of WM and more speculatively its integration with longer term memory. This reflects the neural substrate for a fractionation within object WM [Baddeley, 2003; Logie, 1995] between an ultra‐short‐term visual cache (IT cortex) and a more dynamic rehearsal and retrieval system, the episodic buffer [Baddeley, 2000]. This buffer has been proposed to represent an additional component of WM, processing WM data into longer term memory possibly mediated via the MT cortex [Brown and Aggleton, 2001; Fernandez et al., 1999b; Lisman and Grace, 2005] but for which learning by synaptic modification has not yet occurred [Fransen, 2005]. This proposal would support a hierarchical model of object WM based in multiple cortical regions encoding increasingly complex aspects of object and contextual information. Low level visual features in occipital with more complex and abstract object representations in IT cortex, while MT cortex provides top‐down modulation via rapidly formed long term memory traces to reactivate object representations in IT [Ranganath, 2005; Ranganath et al., 2004]. Unfortunately we did not explore mnemonic delays greater than 12,000 ms, and so cannot speculate any further on the relationship between MT activity and longer term memory.

Activity in Other regions

Across the three delay conditions, activity in other cortical areas at recognition decayed with increasing mnemonic delay, particularly the lateral cerebellar, occipital and parietal association cortices. These same regions were also more engaged at simultaneous recognition than after either the 4,000 or the 12,000 ms conditions. The lateral cerebellar hemispheres are implicated in visual search and attention strategies [Barrett et al., 2001; Beauchamp et al., 1999, 2001; Yantis et al., 2002] and spatial discrimination. The occipital regions are part of the dorsal ‘where’ visual processing stream, but have also been shown Sereno and Maunsell, 1998] to have shape selectivity. Their engagement in this context may thus reflect greater demands on target selection and eye movement planning during the simultaneous than the delay conditions. These findings are consistent with previous suggestions [Barrett et al., 2001; Elliott and Dolan, 1999; LaBar et al., 1999] that simultaneous recognition engages a caudal visual attention, and possibly ultra‐short term memory circuit, based primarily on perceptual memory processing.

The involvement of parietal association cortex in sensory WM is well recognised [Constantinidis and Steinmetz, 1996; Pasternak and Greenlee, 2005]. It has been suggested that this region via it's extensive connections with primary sensory, prefrontal and medial temporal cortices plays an integrative role between WM and current sensory input [Ashby et al., 2005], possibly providing the spatial information required for directing attention to the salient stimulus in a complex scene [Constantinidis and Steinmetz, 2001]. However, other theories relate this region's activity to the subject's perception of success at retrieval, whether that recognition is accompanied by remembering or familiarity, or finally a correlate of the subject's effortful attention to internal representations [Wagner et al., 2005].

Study Design

We used a jittered image acquisition sequence that allowed us to optimally model individual time series data [Dale, 1999], and improve the signal to noise ratio, plus an event‐related design [Postle et al., 2000], with an analysis model informed by the behavioural response to each trial. The latter allowed us to confirm that encoding and maintenance were performed satisfactorily, while mnemonic delay varied across trials in a pseudo‐random fashion and it is unlikely that subjects were able to predict the forthcoming delay at the start of a given trial.

Since we used a task with four choice options, there was a 1 in 4 probability of a correct response by chance alone. However, this will have applied equally to all three lengths of mnemonic delay. We limited our experimental condition of interest to successful trials and so excluded the possibility of the confounding effect of error trials in the analysis. While response latency varied statistically significantly with the duration of mnemonic delay, the order of magnitude of this variation over the different lengths of mnemonic delay was hundreds of ms. This is unlikely to represent a distinct confound in its own right as the haemodynamic model fitting process separately convolved each event of interest with 4 and 8 s gamma functions and then computed the weighted sum of these two convolutions to give the best fit to the time series data at each voxel. This weighted sum effectively allows voxel‐wise variability in time to peak haemodynamic response of up to 4,000 ms. In essence, the model fitting procedure is sufficiently flexible to accommodate much greater variations in response latency than were detected in the behavioural data, with no significant degradation of the model fit.

We employed a constrained model (simultaneous >4,000 ms >12,000 ms and simultaneous <4,000 ms <12,000 ms) for the analysis of variance across the three delay conditions. This was selected because it is the most parsimonious model and it is consistent with the relationship between the length of delay and both the accuracy and latency of the behavioural responses. Error types in this sample were consistent with previous data [Robbins et al., 1994] that the majority of errors were colour correct, shape incorrect (S‐Error). This suggests that through the use of encoding strategies, certain stimulus aspects are more accurately maintained than others over the mnemonic delay. It seems likely that this is mediated by an encoding strategy based on verbal labels.

The relationship between stimulus duration, neural response and % BOLD signal change is complex [Birn and Bandettini, 2005]. Nonetheless, in this experiment, stimulus (recognition phase) duration were equal, and it was the duration of the preceding mnemonic delay that was experimentally varied thus variations in duty cycle length should not be a source of confound. While the experimental requirements of the three conditions differed principally in the duration of time that data were held in WM, it would be an oversimplification to conclude that the between‐condition differences in activation were purely related to delay dependent activity, particularly when considering the contrasts between the simultaneous and the delay conditions.

One disadvantage of our design was that by varying the mnemonic delay, we also influenced the temporal relationship between cognitive events that may share neural resources, specifically encoding, mnemonic maintenance and recognition, thereby influencing the degree of shared variance between events. We guarded against this by explicitly modelling encoding and maintenance in the experimental model as separate events from correct recognition, including a sufficient number of trials to partition out their effect, and varying the inter‐trial (recognition) interval in a random fashion.

CONCLUSION

The results are consistent with a top‐down model of anterior cingulate and MT activity manipulating cortical objects representations in IT cortex. The VLPFC may be involved in managing information in WM, monitoring context and retrieval effort [Miyashita and Hayashi, 2000]. MT structures, specifically the entorhinal cortex and the hippocampus, may contribute to a different process of dynamic rehearsal and retrieval, the episodic buffer, within WM, and thus possibly the maintenance and management in WM of objects as they interact with LTM [Ranganath et al., 2004].