Towards understanding of the cortical network underlying associative memory

Abstract

Declarative knowledge and experiences are represented in the association cortex and are recalled by reactivation of the neural representation. Electrophysiological experiments have revealed that associations between semantically linked visual objects are formed in neural representations in the temporal and limbic cortices. Memory traces are created by the reorganization of neural circuits. These regions are reactivated during retrieval and contribute to the contents of a memory. Two different types of retrieval signals are suggested as follows: automatic and active. One flows backward from the medial temporal lobe during the automatic retrieval process, whereas the other is conveyed as a top-down signal from the prefrontal cortex to the temporal cortex during the active retrieval process. By sending the top-down signal, the prefrontal cortex manipulates and organizes to-be-remembered information, devises strategies for retrieval and monitors the outcome. To further understand the neural mechanism of memory, the following two complementary views are needed: how the multiple cortical areas in the brain-wide network interact to orchestrate cognitive functions and how the properties of single neurons and their synaptic connections with neighbouring neurons combine to form local circuits and to exhibit the function of each cortical area. We will discuss some new methodological innovations that tackle these challenges.

1. Introduction

Neuropsychological, single-unit recording and neuroimaging studies have advanced our understanding of the localization of cognitive functions in the brain. However, many topics in current cognitive neuroscience go beyond simple functional localizations and ask questions about information processing in the neural network at different scales, from local circuits within the cortical column to brain-wide connections among cortical areas (Felleman & Van Essen 1991). These questions can largely be classified into two complementary viewpoints: one considers how the brain areas in a brain-wide network interact with each other to orchestrate cognitive functions (‘global network’ viewpoint) and the other considers how the properties of single neurons and their synaptic connections with neighbouring neurons combine to form local circuits and exhibit the function assigned to each brain area (‘local circuit’ viewpoint).

Focusing on the memory system, since the report of a patient H.M., who showed a profound and selective impairment in the formation of declarative (or explicit) memory (memory for facts and events) after bilateral surgical removal of the medial temporal lobe (hippocampus and adjacent regions; Scoville & Milner 1957), various studies of patients with brain lesions and their animal models have been conducted from the aspects of functional localization. These studies have suggested that long-term declarative memory is eventually stored in the temporal neocortical association area, which is also involved in sensory perception (Mishkin 1982; Miyashita 1993; but for another view, see Nadel & Moscovitch 1997). On the other hand, although patients with frontal lobe lesions do not have the severe amnesia typically observed in patients with medial temporal lobe lesions, they show impairments in memory for temporal context or temporal order, memory of the source of facts or events or metamemory (knowledge about one's own memory capabilities and knowledge about strategies that can aid memory; Petrides 2000). From these observations, it is suggested that memorizing and remembering are orchestrated by a global network that consists of the medial temporal lobe, the temporal and frontal cortices.

Most of our knowledge on the mechanisms of local circuits that support memory come from experimentation in lower mammals or even invertebrates. Accumulating evidence suggests that neural circuits are formed through developmental processes, and remain plastic even in adulthood, exhibiting modification in synaptic efficacy as well as morphological change following learning (Bailey & Kandel 1993). In the primary somatosensory cortex of adult monkeys, peripheral denervation has been shown to produce a reorganization of the cortical topographic map in the deafferented region (Buonomano & Merzenich 1998). Such cortical representational plasticity has also been reported to occur in the primary auditory and visual cortices (Buonomano & Merzenich 1998). While some molecular mechanisms (e.g. basal forebrain cholinergic system) have been reported to be involved in the formation of these non-declarative memories (Blake et al. 2002), studies on hippocampal synaptic plasticity and spatial memory in rodents have provided us with rich sources of knowledge on hippocampal local circuit functions (Leutgeb et al. 2006; Morris 2006). For example, together with many other intracellular signalling molecules related to synaptic plasticity, neurotrophins have been implicated as effector molecules that contribute to morphological aspects of synaptic plasticity. Studies in slice preparations and in in vivo animal preparations have shown that expression and release of neurotrophins are modulated by neuronal activity, and the presence of neurotrophins can alter the structure of dendrites and spines in the mammalian CNS and induce synaptic plasticity (McAllister et al. 1999; Poo 2001). Interestingly, recent studies have shown that, in humans, verbal episodic (event) memory is affected by the Val66Met polymorphism of brain-derived neurotrophic factor (BDNF) gene (Egan et al. 2003; Hariri et al. 2003). However, it is difficult to test how the polymorphism of BDNF gene affects the neural network of declarative memory and eventually the behavioural performance. Clearly, it is quite important to use an animal model for declarative memory, which allows detailed analyses of both global network and local circuits. We have been proposing to use the memory of a visual object as such a model (Miyashita 1993; Miyashita & Hayashi 2000; Miyashita 2004) on the basis of discovery of memory neurons that encode engrams of associative memory of visual objects in the anterior part of the inferior temporal (IT) cortex (Miyashita 1988; Sakai & Miyashita 1991).

In this paper, we review the wealth of studies on the neural mechanism of declarative memory focusing on the functions of the IT cortex, which serves as the visual long-term memory storehouse, and further discuss some new methodological innovations that would improve our understanding of the memory system from the perspectives of global network and local circuit.

2. Neural representation of visual associative memory in the monkey IT cortex

(a) Long-term memory of paired associates

The neuronal correlates of visual associative long-term memory were first identified by Miyashita (1988) and by Sakai & Miyashita (1991) in the monkey IT cortex. They conducted single-unit recordings while monkeys performed the pair-association (PA) task, a well-known neuropsychological test used for the assessment of learning of associative relationships between paired words or figures. In the PA task, meaningless computer-generated pictures were sorted randomly into pairs. We refer to each member of a pair as a paired associate (figure 1a). The monkeys were trained to memorize combinations of paired associates. In each trial, a cue stimulus was presented, and after a delay period the monkey obtained fruit juice as a reward for correctly choosing the paired associate of the cue.

Figure 1.

(a) Examples of the paired associates used in the PA task. (b) Lateral view of a macaque brain. V4, visual area 4; TEO, area TEO; TE, area TE; A36, area 36. (c) Schematic view representing the hierarchical structure of the IT cortex that consists of two subdivisions; A36 in the limbic cortex and TE in the neocortex. V1, primary visual area. (d) An example of a neuron showing pair-coding activity. Raster displays and PSTHs (peristimulus time histograms) in the optimal (optimal, thick black line) and pair (pair, thick grey line) trials. The trials were aligned at the cue onset. The thin black line denotes the averaged responses in the other trials (other). The horizontal black bar indicates the cue presentation period. (e) Mean discharge rates during the cue period (60–320 ms from the cue onset) for each cue presentation from the neuron shown in (d). Twelve pairs of cue stimuli are labelled on the horizontal axis. The open and filled bars in pair 4, for example, refer to the responses to stimulus 4 and 4′, respectively. Error bars denote s.e.m. (b) is adapted from Naya et al. (2001). (c–e) are adapted from Naya et al. (2003a).

Two types of task-related neuron were found in the IT cortex. In one type of neuron, the strongest and the second strongest responses during the cue period were ascribed to particular paired pictures (‘pair-coding neuron’; figure 1d,e). The other type of neuron, which had the strongest response to optimal stimulus during the cue period, exhibited stimulus-selective activity during the delay period when the paired associate of the optimal stimulus was presented as a cue. This single-unit recording identified two mnemonic properties of IT neurons. First, IT neurons can acquire stimulus selectivity for visual patterns through associative learning in adulthood. Second, the activity of IT neurons can link the representations of temporally associated but geometrically unrelated stimuli. These properties were confirmed by later studies (Logotheteis et al. 1995; Kobatake et al. 1998; Erickson & Desimone 1999). This work opened the door to neurobiological investigations of a cortical semantic-like memory network by reducing a complex network into its elementary associative links between two objects, and then by seeking the neural mechanisms underlying such elementary associative links.

(b) Forward processing of long-term associative memory

The IT cortex consists of two cytoarchitectonically distinct but mutually interconnected areas: area TE (TE) and area 36 (A36; Suzuki & Amaral 1994; Saleem & Tanaka 1996; figure 1b,c). TE is a unimodal neocortex and is located at the final stage of the ventral visual pathway, which processes object vision (Tanaka 1996). A36 is a limbic polymodal association area and a component of the medial temporal lobe system, which is involved in the formation of the declarative memory (Zola-Morgan & Squire 1990; Higuchi & Miyashita 1996). Naya et al. (2003a) found that association between the representations of semantically linked objects proceeds from TE to A36. The pair-coding responses in these two areas were compared using the correlation coefficient for each neuron between the mean firing rate during the cue period to one stimulus and the mean firing rate during the cue period to the paired associate of that stimulus (pair-coding index; PCI). This analysis showed that the distributions of the PCIs for all of the cue-selective neurons shifted to positive values in both areas, and that the PCIs for A36 neurons were significantly higher than those for TE neurons (figure 2a). The percentage of neurons that showed significantly positive PCIs was significantly higher in A36 (33%) than in TE (4.9%). This means that although neurons in both areas acquired stimulus selectivity through associative learning, the effect of the associative learning was engraved more intensely on neuronal representation in A36 than in TE. Other properties, a selectivity index (STI) or a tuning index (TNI), cannot explain the difference in the pair-coding responses between the two areas (figure 2b,c). Taken together, it was concluded that the association between the representation of the paired associates proceeds forward through the anatomical hierarchy of the IT cortex, from TE to A36.

Figure 2.

Response correlation with paired associates and general response properties of cue-selective neurons in A36 and TE. (a) Cumulative frequency histograms of PCIs (correlation coefficient) for A36 (n=76, median =0.51, grey) and TE (n=347, median =0.14, black) neurons. PCIs for A36 neurons were significantly higher than those for TE neurons (†p<0.001; Kolmogorov–Smirnov test). (b,c) Cumulative frequency histograms of the selectivity indices (STIs, R² statistic from the ANOVA table, an estimate of how much of the variance in firing rate can be accounted for by the factor of stimulus; b) and tuning indices (TNIs, kurtosis, a measure of the sharpness of the stimulus selectivity; c) for A36 (grey) and TE (black) neurons. Neither the STIs (p>0.99) nor the TNIs (p=0.41) differed significantly between A36 and TE neurons. Adapted from Naya et al. (2003a).

(c) Circuit reorganization during formation of associative memory

It has long been hypothesized that memory engrams of declarative knowledge, as illustrated by the emergence of pair-coding neurons, develop with structural and functional reorganization of neural circuits in the association cortices (Squire & Zola-Morgan 1991; Miyashita 1993). This reorganization of neural circuits could be achieved through a cellular programme of gene expression leading to increased protein synthesis and then to alterations in synaptic connections (Bailey & Kandel 1993). This hypothetical framework has been primarily investigated in invertebrates and lower mammals, in which it is difficult to examine the organization of semantic memory.

However, this hypothesis was tested in molecular biological studies in monkeys (Tokuyama et al. 2000, 2002), showing that upregulation of mRNA encoding proteins involved in structural reorganization occurs during the formation of PA memory. In these studies, mRNA quantitation using RT-PCR was combined with the use of split-brain monkeys. This approach enabled the comparison of mRNA expression between the two hemispheres (PA and control hemispheres) within the same monkey, thereby eliminating experimental confounds by genetic and cognitive variations between individuals (supplementary figure 1a). mRNA encoding BDNF (supplementary figure 1b) and an immediate-early gene, zif268, were found to be increased in A36. BDNF mRNA- and zif268 mRNA-positive cells accumulated as a patch-like cluster in A36, extending for at least 0.4 mm along the anterior–posterior axis (supplementary figure 1c–i). Since zif268 encodes a transcription factor, its expression could trigger a cascade of gene activation that leads to the cellular events underlying neuronal reorganization. Thus, analysis of the formation of PA memory has provided the first evidence supporting the hypothesis that BDNF contributes to the reorganization of neural networks, and that perhaps this reorganization is initiated by zif268.

The location of the focal patch expressing BDNF approximates the location of aggregates of pair-coding neurons detected by single-unit recordings (referred to as a ‘hot spot’; Naya et al. 2003a; Yoshida et al. 2003). A combined anatomical–physiological analysis showed that structural reorganization does occur at hot spots and that the fibre terminals of stimulus-selective neurons in TE that project to A36 are retracted from areas outside of the hot spot in A36 but remain within the hot spot (Yoshida et al. 2003). We suggest that BDNF expression may induce axonal and synaptic reorganization in the hot spot in A36, and that such a reorganization of local circuits is detected electrophysiologically as a change in neuronal stimulus selectivity, typically as the emergence of pair-coding neurons.

3. Retrieval signal for visual associative memory in the monkey IT cortex

(a) Delay-period activity in TE and A36

As mentioned above, one population of IT neurons shows stimulus-selective activity during the delay period in the PA task. Previous studies have suggested that delay-period activities in the IT cortex reflect either a cue stimulus itself, that is, perception-derived information from the early visual areas during delayed matching-to-sample (DMS) or PA tasks (Miyashita & Chang 1988; Yakovlev et al. 1998; Erickson & Desimone 1999), or a sought target, that is, information retrieved from long-term memory during PA tasks (Sakai & Miyashita 1991; Naya et al. 1996; Erickson & Desimone 1999). We refer to the former and latter as ‘cue-holding’ and ‘target-recall’ activities, respectively. How are these two types of delay activities represented in TE and A36? Naya et al. (2003b) conducted single-unit recordings to examine the signal contents of the delay-period activities between TE and A36 using the PA task. PA tasks enable to detect the difference in the signal contents of delay-period activities owing to its advantage that the cue stimulus is different from the target stimulus, whereas DMS tasks do not.

In neurons showing cue-holding activity, one stimulus elicited the strongest cue-period response and maintained the highest tonic activity throughout the delay period (figure 3a,b). On the other hand, in neurons with target-recall activity, the peak response elicited by the cue-optimal stimulus was attenuated after cue presentation. When the pair stimulus was used as a cue, these neurons started to discharge in the middle of delay period and maintained their tonic activity (figure 3c,d).

Figure 3.

Examples of delay-selective neurons from A36 (a,b) and TE (c,d). (a,c) For the raster displays and PSTHs, trials using the cue-optimal stimulus as a cue (black) and those using the paired associate as a cue (grey) were aligned at the cue onset. Responses during the periods within the grey boxes (cue period, 60–320 ms; delay period, 1320–2320 ms) were examined. (a) Black, 4′: cue optimal; grey, 4: paired associate. (c) Black, 4: cue optimal; grey, 4′: paired associate. (b,d) Mean discharge rates during (i) the cue and (ii) delay periods are shown in polar plots for each cue presentation. The responses to stimuli and to their paired associates are indicated by radial lines. The discharge rates were normalized based on the maximum values for each period (b, cue period, 44.0 Hz at stimulus 4′, delay period, 18.6 Hz at stimulus 4′; d, cue period, 48.7 Hz at stimulus 4, delay period, 18.7 Hz at stimulus 4′). (e) Cumulative frequency histograms of the cue-holding index (CHI, black) and the pair-recall index (PRI, grey) for delay-selective neurons in (i) A36 (n=38) and (ii) TE (n=70). The PRIs (median=0.54) were significantly higher than the CHIs (median=0.23) for delay-selective neurons in TE (^*p<0.001, Kolmogorov–Smirnov test), but not in A36 (p=0.78, PRI median=0.46, CHI median=0.44). Delay-selective neurons showed significantly positive CHIs and PRIs in both areas. Adapted from Naya et al. (2003b).

Cue-holding and target-recall activities were characterized using the partial correlation coefficients of delay-period activity for each cue stimulus with the cue-period responses to that stimulus (cue-holding index; CHI) and those to its paired associate (pair-recall index; PRI). In A36, the PRIs for the delay-selective neurons were not different from the CHIs, whereas in TE the PRIs were significantly higher than the CHIs (figure 3e). Moreover, the CHIs in TE were significantly lower than those in A36, while the PRIs were not significantly different between the two areas.

The finding that A36 showed high values for both CHI and PRI could be due to single neurons that had high CHI and PRI. This possibility was examined by calculating the correlation coefficient between CHI and PRI values of single neurons in A36. It was found that CHI and PRI values of the A36 neurons negatively correlated (r=−0.53; p<0.001), which indicated that high CHI and PRI values in A36 were supported by a set of neurons, some with high CHI and low PRI and some others with low CHI and high PRI.

These results indicate that TE mostly represents the sought target that is retrieved from long-term memory, while A36 retains both the cue stimulus and the sought target.

(b) Distractor-resistant active maintenance of associative mnemonic signal

In another study focusing on delay activity with the aim of determining the effect of persistence of internal representations, Takeda et al. (2005) conducted single-unit recordings while monkeys performed a sequential-type PA task in which distractor stimuli interrupted the delay period between cue and target stimuli (supplementary figure 2a).

During trials in which the cue-optimal stimulus was presented (optimal trials), the delay-selective neurons showing cue-holding activity discharged strongly during the first delay period. This sustained discharge was attenuated when a distractor was presented. During trials in which the paired associate of the optimal stimulus was presented as a cue stimulus (optimal-pair trials), neuronal discharge was weak throughout the fixation, cue and delay periods (electronic supplementary material, figure 2b,c). On the other hand, for another type of delay-selective neuron showing target-recall activity, the optimal stimulus did not elicit high discharges through the delay period, although the discharge during the cue period was high. In optimal-pair trials, visual responses were not observed during the cue period, but the firing rate increased during the first delay period. When a distractor stimulus was presented during the first test period, the neuronal discharge during the second delay period dropped transiently but then rapidly recovered to a level comparable with that seen during the first delay period. This indicates that the selectivity during the delay period persisted even after distractor presentation (electronic supplementary material, figure 2d,e).

Neuronal activity during the delay period was analysed using multiple regression analysis to evaluate the effects of an intervening visual distractor on cue-holding and target-recall activities (electronic supplementary material, figure 2f,g). This analysis showed that target stimulus information signalled by delay activity is more resistant to the distractor stimulus than cue stimulus information, and suggests that long-term memory-derived information about the target stimulus needed to solve the task is actively maintained in IT neurons, whereas visually derived information tends to be updated when a new visual stimulus is presented irrespective of behavioural relevance.

4. Activation of memory representations

There appear to be two types of memory retrieval processes: automatic and active. Sometimes no effort is required to recall, whereas at other times one must make an effort to recall successfully. We refer to the former process as automatic retrieval and to the latter process as active retrieval (Petrides 2000; Fletcher & Henson 2001). Long-term memory can be retrieved from the IT cortex by either of these two processes. In this section, we suggest that automatic retrieval is supported by the signal created within the network of the IT cortex, whereas active retrieval is supported by the top-down signal that runs from the prefrontal cortex to the IT cortex.

(a) Automatic retrieval: backward spreading of memory retrieval signal in the IT cortex

During a PA task, groups of neurons in both TE and A36 show target-recall activity specified by a cue stimulus (Naya et al. 2001). One stimulus elicited the strongest response during the cue period, and the latencies of the visual response for the TE neuron were shorter than those in A36 (figure 4a(i),b(i)). In trials in which the paired associate of the cue-optimal stimulus was used as a cue, the neurons exhibited the highest target-recall delay activity among the stimuli. The onset of target-recall activity in TE neurons was later than that in A36 neurons (figure 4a(ii),b(ii)).

Figure 4.

Neuronal activity related to memory retrieval during the PA task, as shown by a single cell in (a) A36 and one in (b) TE. (a,b) For the raster displays, PSTHs were aligned at the cue onset in trials with (i) the cue-optimal stimulus as a cue and in trials with (ii) its paired associate as a cue. In the PSTHs, black lines indicate responses to (i) the cue-optimal stimulus or (ii) its paired associate, and grey lines indicate mean responses to all stimuli. (c,d) Temporal dynamics of averaged PRI(t) for the population of stimulus-selective neurons (A36, n=45; TE, n=69). Mean values of PRI(t) were plotted every 100 ms for (c) A36 neurons and (d) TE neurons (filled circles, total; open diamond, monkey 1; open squares, monkey 2; open triangles, monkey 3). Thick lines indicate the best-fit Weibull functions for the population-averaged PRI(t) in both areas. Thin lines show the same, but for neurons whose PRI(t) increased above the 5% significance level. Adapted from Naya et al. (2001).

The time-course of the target-recall activity of each neuron was examined using the partial correlation coefficients (PRI) relating instantaneous firing rates at time t for each cue stimulus calculated with the visual responses to its paired associate (figure 4c,d). The PRI(t) for the A36 neurons began to increase within the cue period and developed with a rapid time course. On the other hand, the PRI(t) for the TE neurons increased slowly and reached a plateau at the middle of the delay period. Apparently, although the visual signal reached TE before it reached A36, memory retrieval signals appeared first in A36 before TE neurons were gradually recruited to represent the sought target. One interpretation of this finding is that mnemonic information that was extracted from long-term storage spreads backward from A36 to TE, although it is not a logical requirement. Another interpretation that cannot be excluded is that the two areas receive the information from other areas, such as the prefrontal cortex, with different time delays. It remains to be clarified whether the memory-retrieval signal that TE neurons represent originates from frontotemporal signal, from backward signal or from both sources.

(b) Active retrieval: top-down signal from the prefrontal cortex in executive control of memory retrieval

A clinical study highlights active retrieval in humans and provides a clue to an experimental model (Sidtis et al. 1981). An epileptic patient with a selective posterior callosotomy (partial disconnection of the commissural fibres) was presented a word in his left visual field. He could not read the word, although he claimed to ‘see’ its image in mind. He was eventually able to answer the word using inferential strategies based on his mental image. This observation suggests that his right hemisphere was transmitting to his left hemisphere, through the commissural fibre, semantic information about the stimulus, but not visual, phonetic or lexical information about the stimulus. After the callosum was completely sectioned, semantic information was no longer transferred from the right hemisphere to the left one.

Hasegawa et al. (1998) combined this posterior-split-brain paradigm with the PA task in monkeys. In the posterior-split-brain monkey, in which only the anterior corpus callosum remained intact and other commissural fibres were surgically transected, the cortex received bottom-up visual information from the contralateral visual field only. In this paradigm, long-term memory acquired through PA learning did not transfer interhemispherically via the anterior corpus callosum. Nonetheless, when the visual cue was presented to one hemisphere, the anterior callosum could instruct the other hemisphere to retrieve the correct stimulus specified by the cue. Thus, although visual long-term memory is stored in the temporal cortex, memory retrieval is under the executive control of the prefrontal cortex.

Direct proof of the existence of a top-down signal from the prefrontal cortex to the temporal cortex and its contribution to the active retrieval process was provided by single-unit recordings from the IT cortex of posterior-split-brain monkeys (Tomita et al. 1999). It was found that a considerable number of IT neurons received top-down signals from the prefrontal cortex as well as bottom-up signals from the retina (figure 5). The onset latency of the neuronal responses in the top-down condition was significantly longer than that in the bottom-up condition, reflecting the multisynaptic conduction delay within the prefrontal cortex. After the transection of the remaining anterior corpus callosum (‘full-split’), the top-down responses were abolished. This study suggested that the top-down signals conveyed a categorical feature of the stimulus rather than a physical feature. This view was supported by a report that prefrontal neurons showed responses in stimulus categorization tasks (Freedman et al. 2001) and in working memory tasks (Fuster 1997).

Figure 5.

(a,b) Experimental design and (c) neuronal activity in top-down condition. (a) Bottom-up condition. Visual stimuli (cue and choice pictures) were presented in the hemifield contralateral to the recording site (‘electrode’) in the IT cortex. Monkeys had to choose the correct picture specified by the cue. Bottom-up sensory signals (black arrow) would be detected in this condition. (b) Top-down condition. The cue was presented in the hemifield ipsilateral to the recording site, whereas the choice was presented contralaterally. In posterior-split-brain monkeys, sensory signals do not reach visual areas in the opposite hemisphere. In this condition, only top-down signals (grey arrow) can activate IT neurons through feedback connections from the prefrontal cortex. CC, corpus callosum. (c) Activity of a single IT neuron (top-down, grey; bottom-up, black). Raster displays and PSTHs were aligned at the cue onset. In the PSTHs, thick lines show responses to the optimal cue, whereas thin lines show responses to a null cue. The onset of the top-down response (arrowhead) was later than that of the bottom-up response (double arrowhead). Adapted from Tomita et al. (1999).

Neuroimaging studies in humans further confirmed that the prefrontal cortex plays a key role in the active retrieval process. Activation of the prefrontal cortex during memory retrieval is widely observed in functional neuroimaging studies using various psychological paradigms, including recognition tests, word-stem tasks, word-fragment tasks, paired associate tasks and free recall (Buckner & Wheeler 2001).

5. Cognitive control of memory system–human fMRI study

What type of cognitive process in the prefrontal cortex sends the top-down signal? Neuropsychological studies have reported that patients with focal frontal cortex lesions are impaired in tasks such as memory for temporal order, source memory or metamemory (Fuster 1997; Petrides 2000). Although there have been many functional imaging studies on the prefrontal contributions to controlled memory retrieval, only a relatively small number of reports have directly investigated the above types of tasks. In this section, we describe example studies using the memory for temporal order and feeling-of-knowing (FOK) tasks.

(a) Memory for temporal order

Neuropsychological studies in humans have reported that damage to the lateral prefrontal cortex impairs temporal-order retrieval and that the effect of damage is greater in retrieving the temporal order of past events than in retrieving the past events themselves (Shimamura et al. 1990; Milner et al. 1991). Konishi et al. (2002) identified the neural correlates of temporal-order retrieval in humans during a recency judgement using event-related fMRI. In this paradigm, after a list of words was presented, subjects were simultaneously presented with two words in the studied list and were required to choose which word had been presented more recently. Two types of such retrieval trials, with varied (high and low) levels of load for temporal-order retrieval, were intermixed and compared. The intraparadigm comparison of high versus low demand trials revealed brain regions with activation that was modulated on the basis of demand for temporal-order retrieval. Differential fMRI signal increase was detected in multiple regions, including bilateral middle lateral prefrontal areas (Brodmann's cytoarchitectonic area 9, BA 9), left inferior lateral prefrontal areas (BA 45/44), left anterior prefrontal areas (BA 10/46) and bilateral medial temporal areas (BA 28/35). By contrast, no significant signal decrease was observed across the brain, indicating that there was no specific brain region that was recruited during low demand trials compared with high demand trials. These data suggest a detailed network of regions engaged during temporal-order retrieval.

(b) Metamemory

Metamemory requires the execution of an extensive retrieval process and, at the same time, supervises the retrieval process. Kikyo et al. (2002) identified brain areas related to a metamemory system in humans using FOK paradigm. FOK is a subjective sense of knowing an item or a word before recalling it and is a well-established tool for assessing metamemory system (figure 6a). Event-related fMRI with parametric analysis revealed multiple frontal regions that showed stronger activity when subjects had a greater FOK, including the bilateral inferior frontal gyri (BA 47), left middle frontal gyrus (BA 46/9), left frontopolar area (BA 10) and anterior cingulate cortex extending to the supplementary motor area (BA 32/24/6; figure 6b). Among these FOK regions, the subset in the bilateral inferior frontal gyri was not recruited for successful recall processes. This suggests that these regions have specific roles in the human metamemory system. One of the FOK regions was located in the anterior part of the left frontopolar area. In some literature, this area has been regarded as part of the memory areas in the anterior prefrontal cortex and is thought to be related to retrieval strategy and/or ‘third level of executive control’ (Fletcher & Henson 2001).

Figure 6.

(a) Experimental procedures in the FOK tasks. Subjects were required to recall word answers to general knowledge questions, such as ‘Who first conquered Mt Everest?’ during fMRI scans. By pressing buttons, they indicated whether or not they had recalled the target words or not. Then they wrote their recalled answers to the questions or they judged their degree of FOK for the questions they could not recall the answer to, on a scale of 1–3: 3=‘I definitely could recall the answer if given hints or more time’; 2=‘I probably would recognize the answer’; and 1=‘I definitely did not know’. Each trial was sorted into trial type (recalled, FOK3, FOK2 and FOK1) according to the participant's judgement and was subjected to event-related fMRI analysis. (b) Regions showing greater activity when the subjects had a greater FOK. A and B, inferior frontal gyrus (BA 47); C, anterior cingulate cortex/supplementary motor area (BA 32/24/6); D and E, middle frontal gyrus (BA 10, BA 46/9). Adapted from Kikyo et al. (2002).

6. methodological innovations

(a) fMRI of monkeys: bridging the gap

As described above, fMRI studies in humans have been used to explore high-level cognitive functions. On the other hand, most of the detailed knowledge of the anatomy and cellular basis of the cerebral cortex have come from studies in monkeys. However, attempts to compare and integrate the information provided by invasive studies in monkeys and that provided by non-invasive human studies have not been straightforward owing to the differences in species and in techniques. One direct way to bridge the gap is to use the same methods to study both humans and monkeys. For example, fMRI can bridge the gap by enabling a direct comparison of the functional organization of the brains of monkeys and humans (Hayashi et al. 1999; Logothetis et al. 1999, 2001; Vanduffel et al. 2001; Nakahara et al. 2007). Using this approach, Nakahara et al. (2002) conducted the first time comparative fMRI study for high-level cognition. In this study, they carried out event-related fMRI in humans and monkeys using a 1.5 T MRI while subjects performed a modified Wisconsin Card Sorting Test that required behavioural flexibility in the form of cognitive set shifting. fMRI scans revealed transient activation related to cognitive set shifting in the rostral bank of the inferior ramus of the arcuate sulcus in monkeys, and the posterior part of the bilateral inferior frontal sulcus in humans. This experiment revealed that functional homologues were located in cytoarchitectonically equivalent regions in the posterior part of ventrolateral prefrontal cortex.

We successfully extended these comparative fMRI studies into a higher spatial resolution one with a high-field (4.7 T) MRI system. Koyama et al. (2004) conducted a comparative fMRI study while monkeys and humans performed identical visually guided saccade tasks. Multiple activation areas related to saccade eye movement were found in the parietal and frontal cortices in both species (figure 7). They examined the selectivity to saccade direction at each activation focus, thereby inferring functional correspondence between the two species. In the parietal cortex, the dorsal lateral intraparietal area in monkeys and the posterior part of the superior parietal lobule in humans exhibited the highest selectivity. In the frontal cortex, the highest selectivity was found in the frontal eye field (FEF) in monkeys and at the intersection of the precentral sulcus and the superior frontal sulcus in humans. The saccade-related area extended from the FEF to the premotor area (BA 6) in monkeys. This suggests that the apparent discrepancy in location between monkey FEF (BA 8, identified by electrophysiological and microstimulation methods; Schall 1997) and human FEF (BA 6, suggested by functional imaging; Paus 1996) partly arose from the methodological differences. Such comparative imaging between humans and monkeys has the potential to provide significant new insights into the evolution of cognition in primates.

Figure 7.

Comparison of saccade-related activity in monkeys and humans. (a,c) Monkey and (b,d) human brain images are cut to show activity buried in the sulci in the frontal (a,b) and posterior parietal (c,d) cortices. Asterisks and crosses indicate the regions showing the highest selectivity to the contraversive saccades in the frontal and parietal cortices, respectively. FEF, frontal eye field; LIPd, dorsal lateral intraparietal; PrCS/SFS, intersection of the precentral sulcus and the superior frontal sulcus; SPL, superior parietal lobule; as, arcuate sulcus; ips, intraparietal sulcus; prcs, precentral sulcus; sfs, superior frontal sulcus; post, posterior. Adapted from Koyama et al. (2004).

While comparative imaging studies are important, we can further combine fMRI with various invasive methods. Obviously, this combination is only possible in experimental animals. The following (i)–(iii) are some typical examples: (i) Simultaneous fMRI and electrophysiological recordings help to elucidate the relationship between neural activities and the BOLD (blood oxygenation level-dependent) signal (Logothetis et al. 2001). (ii) Monkey fMRI can also serve as a navigation tool to target microelectrodes. Researchers can identify multiple responsive regions using fMRI at the whole brain scale and then investigate the electrical activities of neurons with high spatio-temporal resolution using microelectrode recordings (Sawamura et al. 2006; Tsao et al. 2006). (iii) The combination of monkey fMRI and microstimulation (Tolias et al. 2005) or cortical reversible inactivation with the local injection of drugs could provide ways to explore functional organization of the global brain-wide network. Microstimulation can activate restricted populations of neurons and in some cases induce particular behaviour, whereas cortical inactivation of restricted areas can induce deficit in behaviour. These combined techniques could investigate not only the causal links between neural activities and behaviours but also the functional connectivity in the global network because one can detect changes in activity of the regions that have connections with a stimulated or inactivated site.

(b) New electrophysiological approaches: investigation of local circuit

The electrophysiological studies discussed in previous sections pertain to the activity of single neurons. Recently, several attempts have been conducted to improve our understanding of the relationship between cognitive processes and neural activity in the context of the connection among neighbouring neurons within an area. We introduce two approaches as examples: MRI-based localization of recording sites within the cortex and simultaneous recording of multiple neurons.

(i) MRI-based localization of recording sites within the cortex

The primate cerebral cortex, approximately 2–3 mm in thickness, typically consists of six layers that characterize the input and output connections of a given cortical area with other areas or local connections within a given area (Felleman & Van Essen 1991). The precise localization of recorded neurons in the cortex, particularly in the direction perpendicular to the cortical layers, would provide important clues for understanding recorded neuronal activity in relation to underlying connections between and within cortical areas.

In conventional electrode-localizing methods, recording sites are marked in brain tissue in several ways, such as electrolytic lesions at the tip of the recording microelectrode, and are subsequently detected in histological sections. Although these methods provide definite locations of recording sites, they are often inadequate in chronic recordings from behaving primates, because the locations of recorded sites are unknown until long-term recording experiments are completed. Recently, Matsui et al. (2007) developed an MRI method for the precise in vivo localization of cortical recording sites. In this method, the susceptibility-induced effect thickens the appearance of the microelectrode and enhances the detectability of microelectrode tip. Using a 4.7 T MRI, they localized recording sites within anatomical MR images of the monkey cerebral cortex (in-plane resolution, 150 μm; figure 8). Furthermore, they compared the locations of microelectrode tips determined using MRI with corresponding electrolytic lesion marks in histological sections. The localization accuracy fell significantly within plus or minus one voxel (figure 8h,i). This MRI-based method could help to infer the relationship between recorded neuronal activities and cortical neuronal connections in behaving monkeys.

Figure 8.

MRI-based localization of the microelectrode tip within the monkey cortex. (a) Schematic of (i) a lateral view and (ii) a plane of the brain with the microelectrode inserted. The microelectrode (blue line) was inserted at a 90° angle to the superior temporal sulcus and the angle between the microelectrode and the static magnetic field B₀ was set at more than 60°. The red-framed area indicates the position of the area presented in (b–d). LS, lateral sulcus. STS, superior temporal sulcus. (b) Magnetic resonance image of the brain with the inserted microelectrode. Inversion recovery fast spin echo sequence was used (in-plane resolution, 150 μm). Histological sections stained for (c) myelin and with (d) Nissl stain corresponding to the magnetic resonance image in (b). (e–g) Enlarged images from the framed areas in (b–d). The cortical location of the microelectrode tip on the magnetic resonance image matches well with that of the lesion mark (arrowheads) on the histological section. The histological sections were corrected for shrinkage. (h) Cortical locations of the microelectrode tips determined by histology (y-axis) were plotted against those determined by MRI (x-axis; n=16). Distances from the ventral border of the cortex along the recording tracks were plotted. A linear regression line between them fitted well (slope, 0.98; r² =0.87; p<0.0001). (i) Distributions of localization accuracy that was normalized against the voxel size. Mean ±s.d. was 0.13±0.97 voxel, which fell significantly within plus or minus one voxel (equivalence test, p<0.05). Adapted from Matsui et al. (2007).

(ii) Simultaneous recording of multiple neurons

Another way to investigate the connection among the neighbouring neurons is to record the activities of multiple neurons simultaneously. By recording simultaneously, we can understand the interactions between neurons and obtain insight into the network properties of neural processing. An example of such attempts can be found in a study on visual object recognition. Hirabayashi & Miyashita (2005) conducted simultaneous extracellular recordings using a multichannel electrode from pairs of IT neurons during a visual discrimination task between face-like objects (FOs) and their corresponding non-FOs (NFOs). They found that although the firing rates of neurons did not show bias depending on the feature configuration, discharges of cell pairs elicited by FOs were more strongly correlated than those elicited by NFOs. This study suggests that feature configuration within a whole object can be reflected in the spike correlation among a population of neurons in the IT cortex.

Such simultaneous recording methods and cross-correlation analysis help us to understand interactions between neurons in the network that could not be predicted in single-unit recording studies. These methods can also be extended from studying neurons within one cortical area to studying neurons between areas, providing information about the brain-wide network.

7. conclusions

The research introduced in this paper has clarified that associative long-term memory is stored in the IT cortex and is recalled by reactivation of the neural representation. The findings have ranged even to the structure of the local circuit and the flow of signal at the global, inter-area level. IT neurons can acquire stimulus selectivity through learning and come to link representations of associated stimuli. This process has been suggested to be induced by reorganization of local neural circuits. From the global network viewpoint, the representation of PA memory proceeds forward from TE to A36 following the anatomical hierarchy. When remembering, two types of retrieval signal reach the cortical representations. One spreads backward from A36 to TE, whereas the other runs from the prefrontal cortex to the temporal cortex as the top-down signal.

The spatially patchy arrangements of the pair-coding neurons in TE and A36 suggest the existence of orderly organized local circuits in each region related to the processing of the associative memory. It is undoubtedly valuable to study how the neurons are arranged and interact mutually in these local circuits. The simultaneous recording techniques or the method for in vivo localization of electrode position will extend our knowledge of these local circuits.

fMRI studies using memory tasks, such as memory for temporal order, source memory or metamemory, are important for investigating the role of the prefrontal cortex in the memory system. However, comparing findings from human fMRI studies and findings from monkey neurophysiological studies is not straightforward owing to species and methodological differences. Therefore, in order to bridge the gap, it will be indispensable to conduct monkey fMRI studies using these tasks. Monkey fMRI also has an advantage in its capability to be combined with invasive methods. Combining these experiments with, for instance, a reversible inactivation with a local drug injection will help us to understand, from the global network perspective, the causal relationship between the activated cortical areas and cognitive processes. Knowledge on the cortical interactions in the global network and the neuronal interactions in the local circuits will advance our understanding of the neural mechanism underlying human cognitive functions.