Recognition memory and the medial temporal lobe: a new perspective

Abstract

Recognition memory is widely viewed as consisting of two components, recollection and familiarity, which have been proposed to be dependent on the hippocampus and the adjacent perirhinal cortex, respectively. Here, we propose an alternative perspective: we suggest that the methods traditionally used to separate recollection from familiarity instead separate strong memories from weak memories. A review of work with humans, monkeys and rodents finds evidence for familiarity signals (as well as recollection signals) in the hippocampus and recollection signals (as well as familiarity signals) in the perirhinal cortex. We also indicate ways in which the functions of the medial temporal lobe structures are different, and suggest that these structures work together in a cooperative and complementary way.

Declarative memory refers to the capacity to consciously remember the past and depends on the integrity of the medial temporal lobe (comprising the hippocampus, the dentate gyrus and the subicular complex, together with the entorhinal, perirhinal and parahippocampal cortices, which lie along the adjacent parahippocampal gyrus). One of the most widely studied examples of declarative memory is recognition — the ability to judge a recently encountered item as having been presented previously. Recognition memory is widely viewed as consisting of two components: recollection and familiarity^1,2. Recollection involves remembering specific contextual details about a prior learning episode; familiarity involves simply knowing that an item was presented, without having available any additional information about the learning episode.

Interest in this distinction greatly increased when Brown and Aggleton³ proposed a neuroanatomical basis for these two processes. Their proposal was that recollection depends on the hippocampus, whereas familiarity depends on the adjacent perirhinal cortex. Since that time, others have elaborated on this idea^4–6, and it has become the basis for the design and analysis of a good deal of experimental work. However, alternative formulations have also been advanced about the nature of recognition memory and its anatomical foundations^7,8, and a consensus has not yet emerged.

Here, we review studies of humans, monkeys and rodents that have investigated the functional organization of the medial temporal lobe using a variety of methods: lesions, single-unit activity and neuroimaging. The general organizational principles of the medial temporal lobe are similar in these species⁹, and it is reasonable to suppose that what is learned about medial temporal lobe function in one species has relevance to the others. Although many of the studies have been interpreted as providing support for the neuroanatomical separation of recollection and familiarity in the medial temporal lobe, in this Review we develop an alternative perspective. We suggest that there has been a rush to judgment about the anatomical substrate of recognition memory, and that the available findings can be more simply interpreted in terms of strong and weak memories. We conclude that the structures of the medial temporal lobe operate in a more cooperative fashion than has been envisioned in recent discussions of recollection versus familiarity and the hippocampus versus perirhinal cortex.

Studies of memory impairment

Recall and recognition deficits

Perhaps the most straightforward way to test the divided-labour account of medial temporal lobe function is to assess the degree to which recall and recognition are impaired in patients with circumscribed damage to the hippocampus. Recall is generally thought to depend solely on recollection, whereas recognition is thought to depend on both recollection and familiarity^1,2. Accordingly, if the hippocampus selectively supports recollection, then hippocampal damage should impair recall more than recognition. Although a few single-case studies have been advanced in support of this idea^10–13, group studies have provided strong evidence against it. One study was carried out with 56 patients who had suffered a brief period of hypoxia and who were thought to have lesions limited to the hippocampus, though radiological data were not available¹⁴. Although recall was initially reported to be more impaired than recognition, this conclusion depended entirely on the aberrant performance of a single one of the 55 control subjects¹⁵. This individual’s recognition memory score was more than 4.4 standard deviations below the mean score of the controls, which markedly increased the variability of the control scores. Once the obvious outlier was excluded from the analysis, recall and recognition were similarly impaired. Indeed, the P value that resulted from comparing the recall and recognition deficits in the patients shifted from 0.027 to >0.50.

A different study¹⁶ involved six patients with evidence of bilateral hippocampal damage and normal parahippocampal gyrus volumes. Again, recall and recognition were similarly impaired — a result that should not be observed if recollection is exclusively dependent on the hippocampus. A third study¹⁷ involved three patients with damage limited to the hippocampus and two patients with damage that extended into the parahippocampal gyrus. Recognition performance in patients and controls was first equated by manipulating the participants’ exposure time to the items to be remembered. Under these conditions, the recall scores of the patients matched those of the controls across three different retention intervals (30 seconds, 2 minutes and 10 minutes): that is, recall and recognition were similarly impaired. This result strongly suggests that the hippocampus is important for both recollection and familiarity.

Prior suggestions that hippocampal lesions disproportionately affect recall were based on a few single-case studies^10–13 and on the aforementioned group study that was compromised by the inclusion of an outlier¹⁴. When that group study (without the outlier) is considered along with the two other available group studies^16,17, a consistent picture emerges in which recall and recognition are similarly impaired in patients with hippocampal lesions.

Source memory and associative memory

Source memory and associative recognition procedures have also been used to assess the status of recollection and familiarity in patients with hippocampal lesions. These procedures have also been characterized as providing tests of ‘relational memory’ — a term originally used as an alternate term for declarative memory^18,19, but which has recently tended to refer more specifically to recollection^5,20,21. In a source-memory procedure, studied items are presented from different sources (for example, the top or bottom of a screen). On the subsequent recognition test, subjects are first asked for an old–new decision and then asked to indicate the item’s source. The old–new decision is assumed to be based on both recollection and familiarity, but the source judgment is assumed to rely only on recollection. In an associative recognition task, subjects are typically presented with pairs of items and later asked to discriminate intact pairs from recombined pairs. Because the items of both pairs are familiar, successful discrimination between the intact and recombined pairs presumably depends only on recollection.

In a study of source memory, five patients with lesions thought to be limited to the hippocampus (based on quantitative analysis of magnetic resonance images) were impaired in both old–new decisions and source judgments²². When controls and patients were matched on old–new memory performance (by using longer study lists for the controls), source memory performance was also comparable for the two groups. Thus, as with the recall and recognition studies that were discussed above, this result suggests that recollection and familiarity are similarly impaired in patients with hippocampal lesions. Another study that used an associative recognition procedure arrived at similar conclusions²³. However, other associative recognition studies have reported that recollection is differentially impaired after hippocampal lesions (for example, see REF. 24), and still others have produced both outcomes by manipulating seemingly minor details of the experimental procedure²⁵. Further work is needed to clarify the effects of hippocampal lesions on associative-recognition performance.

Delayed non-matching to sample and novel object recognition

The effects of selective hippocampal lesions have also been explored using tasks of recognition memory where single items must be remembered independently of any context, and where one might therefore suppose that performance depends substantially on familiarity. The most widely used tasks are delayed non-matching to sample and novel object recognition (BOX 1). Studies of humans, monkeys and rodents with selective hippocampal lesions document impairment in these tasks. The impairment is more severe when the hippocampal damage occurs together with damage to adjacent cortices, including perirhinal cortex²⁶. Nonetheless, these studies indicate that the ability to remember a single item across a delay of more than just a few minutes depends substantially on the hippocampus, even when the task has no overtly associative or contextual component. The table in BOX 1 lists the considerable evidence for this conclusion, but also identifies instances in which no impairment was detected. Important factors that might explain the discrepancies include variability in lesion size or delay interval, along with the possibility (which applies to many of the behavioural assays that are given to experimental animals) that animals can approach some tasks in fundamentally different ways (for example, by using either declarative or non-declarative memory). Nevertheless, these data show that the effects of hippocampal lesions on these tasks should not be minimized^4,5,27.

Box 1 | tasks used to assess recognition memory across species.

The effects of selective hippocampal lesions have often been explored in animals using tasks that require recognition memory. The most widely used tasks in this regard are delayed non-matching to sample tasks and novel object recognition tasks. The delayed non-matching to sample task requires that the subjects first be trained to learn the non-matching rule, whereas the novel object recognition task simply exploits an animal’s innate preference for novelty. The table lists 24 studies in which performance was impaired by hippocampal lesions and 7 studies in which performance was unimpaired.

Delayed non-matching to sample

A sample object is presented and then, after a delay that ranges from a few seconds to a few minutes, the sample object is presented again together with a new object. A choice of the new object is rewarded.

Novel object recognition

Two identical objects are presented for visual inspection (in the case of humans or monkeys) or exploration (in the case of rodents). After a delay that ranges from a few seconds to several hours, the original object is presented again together with a new (novel) object. Normally humans and other animals preferentially look at (or explore) the novel object, indicating that they remember the original object.

Species	Studies in which hippocampal lesions impaired performance	Studies in which hippocampal lesions did not impair performance
Delayed non-matching to sample
Humans	97
Monkeys	98–102	103
Rats	104–107	108,109
Novel object recognition
Humans	110,111
Monkeys	101,102
Rats or mice	112–115,116*,117–119,120‡,121	122–124,125§

^*Impairment when the lesion was made after the familiarization phase but not when the lesion was made before the familiarization phase.

^‡Complicated by the fact that the lesion group explored the objects less than the control group during the initial familiarization phase.

^§No impairment under standard conditions but impairment when the test box was novel.

Analysis of the receiver operating characteristic

A new approach to investigating the neuroanatomical basis of recollection and familiarity involves analysing the receiver operating characteristic (ROC) in recognition-memory tasks. An ROC is a plot of the hit rate (the rate at which old items are correctly identified as old) versus the false-alarm rate (the rate at which new items are incorrectly identified as old) for decisions made with different levels of confidence (FIG. 1). Analyses of ROC curves from recognition-memory tasks have historically been associated with signal-detection theory²⁸. As illustrated in FIG. 1, signal-detection theory holds that each old item (or target) or new item ( foil) is associated with a particular memory strength, which reflects the degree of certainty that an item did or did not appear on a recently presented list. Figure 1 shows that the distributions of memory strength for targets and foils overlap. An item is declared to be old if its memory strength exceeds some criterion value; otherwise it is declared to be new. According to traditional signal-detection theory, recognition decisions are based on a single strength variable, but the theory is also fully compatible with two-component views of recognition on the assumption that recollection and familiarity are both continuous processes that combine to determine the memory strength of a test item^8,29.

Figure 1. Signal-detection theory and the receiver operating characteristic.

aThe signal-detection representation¹²⁸ of a strong memory condition. The targets (old items) and foils (new items) in a recognition memory test are presumed to have varying degrees of memory strength (the subjective certainty that an item was or was not previously presented), and the mean and variance of the target distribution are greater than those of the foil distribution. A test item that generates a memory strength that exceeds a criterion value (indicated by the vertical line labelled c) is declared to be ‘old’. Otherwise, the test item is declared to be ‘new’. Confidence ratings (indicated by vertical dashed lines) of one to six range from ‘sure new’ to ‘sure old’. Items with memory strength to the left of the left-most vertical line are given ‘high confidence new’ responses. Items with memory strength to the right of the right-most vertical line are given ‘high confidence old’ responses. The right-hand panel shows a receiver operating characteristic (ROC) curve constructed from a subject’s confidence ratings. Five pairs of hit and false-alarm rates are computed from the six-point confidence rating scale. The left-most point represents the hit and false-alarm rates for targets and foils that receive a rating of 6. The second point represents the proportion of targets and foils that receive a rating of either 5 or 6, and so on. As is almost always the case in strong memory conditions¹²⁹, the resulting ROC curve in this example is asymmetrical. b. The signal-detection representation of a weak memory condition, in which the means and variances of the target and foil distributions are more similar and the ROC curve is more symmetrical.

A more recently formulated two-component theory that also describes ROC data reasonably well holds that recollection occurs only when the strength of an item in memory exceeds a high threshold and that familiarity is a signal-detection process³⁰. That is, recollection is conceptualized as a discontinuous variable and is assumed to occur with a discrete probability. Items that are strongly associated with the experimental context during learning have a higher probability of being recollected than items that are encoded in isolation. As illustrated in FIG. 2, when an item generates recollection of its prior occurrence, the subject is assumed to make a ‘high confidence old’ decision. Alternatively, when an item fails to generate recollection, the subject is assumed to make an old–new decision based only on familiarity, with the level of confidence determined by the item’s degree of familiarity. Thus, the high threshold/signal-detection model assumes that individual recognition decisions are based either on recollection or on familiarity, not on a combination of the two. In this model, the target and foil distributions are assumed to have equal variance.

Figure 2. High threshold/signal-detection theory and the receiver operating characteristic.

aThe high threshold/signal-detection representation³⁰ of a strong memory condition that involves both recollection and familiarity. Recollection occurs with some discrete probability (R; in this example, R = 0.5). When a test item generates recollection, a ‘high confidence old’ decision (which is equivalent to a confidence rating of six on a six-point scale) is made. When a test item fails to generate recollection (a situation that occurs with probability equal to 1 − R), the decision is based on familiarity. Decisions based on familiarity are characterized by a signal-detection model in which the targets (the old items) and foils (the new items) are presumed to have different average levels of familiarity but equivalent variances. A test item that generates a familiarity value exceeding a criterion value (indicated by the solid vertical line labelled c) is declared to be ‘old’. Otherwise, the test item is declared to be ‘new’. Thus, whereas most confidence ratings of six are based on recollection in this example (the 50% of the targets that are recollected receive this rating), a few additional ratings of six are based on familiarity. The right-hand panel shows the predicted asymmetrical ROC curve, which is similar to the asymmetrical ROC curve predicted by the traditional signal-detection model when memory is strong (FIG. 1a). b. The high threshold/signal-detection representation of a condition in which recollection is not involved (that is, in which R = 0) but familiarity can be used to discriminate targets and foils. The right-hand panel shows the predicted symmetrical ROC curve, which is similar to the symmetrical ROC curve predicted by the traditional signal-detection model when memory is weak (FIG. 1b). Although the high-threshold/signal-detection model fits ROC data reasonably well, recent findings favour the traditional signal-detection model³⁴–³⁷.

A number of studies that used the high-threshold/signal-detection model to interpret ROC data have concluded that hippocampal lesions selectively impair recollection^14,31–33. The key finding is that the asymmetry of the ROC curve is greater for control subjects than for patients with hippocampal lesions. According to the high threshold/signal-detection model, familiarity-based responses (those given in the absence of recollection) would yield a symmetrical ROC curve, because the target and foil familiarity distributions have equal variance. Any recollection-based responses are assumed to support decisions made with high confidence, and those additional high-confidence decisions distort the shape of the ROC curve (making it asymmetrical)³⁰. Accordingly, the more symmetrical ROC curve produced by amnesic patients is thought to indicate that their recognition performance is based more on familiarity than is the recognition performance of controls. Indeed, the familiarity-based performance of amnesic patients is often estimated to be unimpaired in these studies.

In contrast to this view, traditional signal detection theory, which has in recent years been strongly supported over the high threshold/signal-detection model^34–37, suggests that a symmetrical ROC curve reflects weak memory rather than the absence of recollection (FIG. 1). Further, signal-detection theory holds that an asymmetrical ROC curve implies only that the target and foil distributions have unequal variance, which is generally a sign of a strong memory. It does not imply that recognition is supported by recollection.

The idea that the target distribution typically has greater variance than the foil distribution is easily understood. The targets can be thought of as foils that have had memory strength added to them as a result of having appeared on a study list. The target distribution would have the same variance as the foil distribution if each item on the list had the same amount of strength added to it during study. However, it is more plausible to suppose that the amount of added strength differs across items. In this case, the target distribution would shift to the right, its variance would increase, and the ROC curve would become more asymmetrical. Typical ROC curves are asymmetrical, and they become more asymmetrical as overall memory strength increases³⁸ (FIG. 3). These considerations imply that the symmetrical ROC curves produced by patients with hippocampal lesions simply reflect the fact that their memories are weaker (in both recollection and familiarity).

Figure 3. Roc data as a function of memory strength.

In the study depicted, young adults (in groups of 19–24) studied 50-item word lists and then took a recognition-memory test involving the 50 old words and 50 new words after one of five retention intervals. The ROC curve was asymmetrical after the shortest retention interval (1 hour) and became more symmetrical as the interval grew and performance decreased (from 83% correct at 1 hour to 53% correct at 8 weeks). Performance was significantly above chance after all retention intervals. These findings differ from a previous study in rats³² which suggested that the ROC curve might be linear after a relatively long retention interval (75 minutes). The ROC data illustrated here are better fit by a curvilinear function based on signal-detection theory. Figure reproduced, with permission, from REF. 41 © (2006) Elsevier Science.

If the traditional signal-detection interpretation of ROC data is correct, then the alternative high threshold/signal-detection model³⁰ misidentifies strong and weak memories as memories that are based on recollection and familiarity, respectively. Although it is likely that, on average, recollection contributes greater strength to an item’s memory than does familiarity, the defining characteristic of recollection and familiarity is not their relative strength. Indeed, one can experience a strong sense of familiarity even in the absence of recollection¹. Further, although strong memories are often accompanied by recollection, considerable evidence indicates that some degree of recollection often accompanies weaker memories as well^36,39,40.

These differing interpretations of the ROC evidence were tested in a recent study⁴¹ of recognition memory in patients with selective hippocampal lesions. The ROC curve for the patients was closer to the diagonal line (reflecting weaker memory; see FIG. 1b) and was more symmetrical than the ROC curve of matched controls, thereby replicating prior results. Notably, however, when the memory strength of the patients was increased to a level near that of controls (by using a shorter study list), the patient ROC curve became just as asymmetrical as the control ROC curve. The signal-detection model^8,29 and the high threshold/signal-detection model³⁰ offer compatible interpretations of this result.

According to the signal-detection account, the symmetrical ROC curve for the patients in the weak memory condition reflects the fact that the target and foil distributions have equal variance (as in FIG. 1b), precisely because memory is weak (in both recollection and familiarity). The asymmetrical ROC curve for the patients in the strong memory condition reflects the fact that the target distribution has greater variance than the foil distribution (as in FIG. 1a), precisely because memory is strong (in both recollection and familiarity). This result suggests that recollection and familiarity combine in typical fashion in the strong memory condition. That is, both recollection and familiarity are operative in the presence of hippocampal lesions.

According to the high threshold/signal-detection model³⁰, the fact that the patient ROC curve in the strong memory condition was as asymmetrical as that of the controls implies that recollection and familiarity were the same for both groups (FIG. 2a). Quantitative mRI analysis performed on five of the six patients indicated an average bilateral reduction in hippocampal volume of 44%. On the basis of two other patients with similar bilateral hippocampal volume loss, for whom detailed postmortem neurohistological information was obtained⁴², this degree of volume loss reflects nearly complete loss of hippocampal neurons. If the five patients in the study also had complete loss of hippocampal neurons, then the high threshold/signal-detection model would predict a symmetrical ROC curve even in the strong memory condition. Further, even if some degree of hippocampal function was retained in the patients, and if the hippocampus selectively supports recollection, then the ROC curve in the strong memory condition should still have been less asymmetrical (reflecting partially impaired recollection) than the ROC curve of controls. Instead, the patient and control ROC curves were equally asymmetrical. Thus, according to both models, the ROC curves that were observed for the patients suggest that the component processes of recognition memory, including recollection, are operative in the absence of the hippocampus.

This finding differs from an earlier study in rats³². In agreement with studies of memory-impaired patients^14,41, the ROC curve calculated for rats with hippocampal lesions was symmetrical (reflecting either weak memory or familiarity-based responding, depending on which theory is used to interpret the result). However, in contrast to the just-discussed study in patients⁴¹, the ROC curve for control rats with weakened memory (following a long retention interval) was nearly linear. The linear ROC, which is inconsistent with signal-detection theory and which the high threshold/signal-detection model interprets to reflect purely recollection-based responding³⁰, was the critical result that led to the conclusion that the hippocampus is necessary for recollection to occur. Yet there is reason to view the linear old–new ROC cautiously, as it has never been observed in the human literature (see FIG. 3).

Analysis of remember–know judgments

Other studies have used the remember–know procedure to investigate the neuroanatomical basis of recollection and familiarity. In this procedure, subjects are asked to judge whether an item is old or new, and then for each item judged to be old they are asked whether they remember the item (recollection) or simply know that the item has been presented (familiarity). Several studies using this procedure have concluded that hippocampal lesions impair recollection but impair familiarity to either a lesser degree or not at all^{12–14,31,43,44}. However, these conclusions depend entirely on the assumption that subjective judgments of ‘remember’ and ‘know’ do indeed reflect recollection and familiarity, respectively. This assumption is based on the finding that behavioural manipulations thought to selectively influence recollection often affect only ‘remember’ responses, whereas manipulations thought to selectively influence familiarity affect only ‘know’ responses (for a review, see REF. 45).

Signal-detection theory interprets remember–know judgments differently ^46–48 and readily accounts for these same findings (FIG. 4). Specifically, a large reduction in ‘remember’ responses, together with little or no reduction in ‘know’ responses, happens naturally when strong memories become weak memories (for example, as time passes after learning). This occurs because ‘remember’ responses are associated with items in the right-hand tail of the target distribution (the strong memories that fall above some high decision criterion), whereas ‘know’ responses reflect a segment of the target distribution that falls below the high criterion but above some lower criterion. As the mean of the target distribution decreases (that is, as memory weakens), the proportion of targets that fall above the high criterion decreases rapidly, whereas the segment of the target distribution that falls between the high and low criteria remains largely unchanged. Accordingly, a differential loss of ‘remember’ responses should not imply a specific deficit in recollection. Indeed, there is considerable evidence that remember–know judgments index memory strength and are not reliable markers of qualitatively different processes, such as recollection and familiarity^46–50.

Figure 4. The signal-detection interpretation of remember–know judgments.

aA strong memory condition. ‘Remember’ judgments are made for items that exceed a high memory-strength criterion (labelled R), whereas ‘know’ judgments are made for items that exceed a lower criterion (labelled K) but not the high criterion. Items that fall below the K criterion are judged new. The ‘remember’ hit rate is the proportion of the target distribution that exceeds the R criterion (representing strong memories), and the ‘know’ hit rate is the proportion of the target distribution that falls between the K and R criteria (representing weak memories). b. In a weaker memory condition (as might occur, for example, after less extensive training), both criteria shift to the left, with the K criterion remaining approximately midway between the means of the target and foil distributions (its typical location for an unbiased subject). As a result, the ‘remember’ hit rate decreases, and the ‘remember’ false-alarm rate increases. This illustration explains why memory-impaired patients with hippocampal lesions often have a high ‘remember’ false-alarm rate compared with controls¹⁶. In contrast to the ‘remember’ hit rate, the ‘know’ hit rate can actually increase in a weak memory condition, as illustrated in this example. If estimates of recollection and familiarity were derived from data like these (as they often are), such estimates would suggest that recollection is greatly reduced in the weak memory condition whereas familiarity is relatively unaffected. However, a simpler explanation of data like these is that memory is weaker overall.

Single-unit recording and neuroimaging

Additional evidence that bears on the neuroanatomical basis of recollection and familiarity is provided by studies that have recorded neural activity in the hippocampus and perirhinal cortex during learning and retrieval. Two different techniques have been used: single-unit neurophysiology and functional MRI (fmRI).

Neural activity in the hippocampus

There is strong support for the idea that the hippocampus has a role in associative recollection. For example, in one study, monkeys learned rapidly to form arbitrary associations between complex visual scenes and one of four spatial locations⁵¹. Of 89 hippocampal cells that responded selectively to scenes, 25 signalled new learning with changes in firing rate that closely paralleled behavioural learning. Similar results have come from human neuro-imaging studies that used source-memory procedures to identify neural correlates of recollection. Typically, items that are recognized as old together with their correct source information (which requires recollection) are associated with increased activity in the hippocampus, both at encoding^22,52–54 and retrieval^55–57, relative to the activity associated with misses, or relative to the activity associated with correct item judgments when the source information is incorrect.

Other evidence comes from studies that made use of the idea that recollection-based decisions can be identified by ‘remember’ responses or by ‘old’ decisions made with high confidence^30,58. Several studies have found that hippocampal activity is increased for such responses, relative to either ‘know’ responses or to ‘old’ decisions made with low confidence (or, in some cases, relative to misses)^59–64.

Although there is broad agreement about the role of the hippocampus in recollection, its role in familiarity is debated. Some studies appear to support the idea that familiarity does not involve the hippocampus. For example, in neuroimaging studies of source memory in which brain activity was recorded during encoding, items that are later correctly recognized as ‘old’ but with incorrect source information (that is, responses that are assumed to be familiarity-based) are often not associated with increased hippocampal activity relative to items that are later missed (for examples, see REFS ^52,53, but see REF. 22 for an exception). It is generally assumed that such responses are based on familiarity, because the task-relevant source information was not recollected (although one cannot exclude the possibility that some other source information about the item was recollected). In addition, several remember–know studies have found that hippocampal activity at encoding for items that will later be given ‘know’ responses often fails to exceed the activity associated with items that will later be missed (for examples, see REFS ^59,65, but see REF. 66 for an exception). A similar pattern is evident for ‘know’ responses (or for decisions made with low confidence) at retrieval^60–64,67. If these recognition responses are presumed to reflect familiarity-based decisions, such a result would imply that the hippocampus is not involved in familiarity^4,5.

However, as described above, we propose that the methods intended to separate recollection-based decisions from familiarity-based decisions, including remember–know judgments, high or low confidence ratings and the presence or absence of source recollection, instead separate strong memories from weak memories. We suggest that strong memories are associated with increased hippocampal activity, regardless of whether they reflect strong familiarity, strong recollection or a combination of the two. Further, weak memories are often not associated with detectably increased activity in the hippocampus, regardless of whether such memories reflect weak familiarity, weak recollection or a weak combination of the two. Accordingly, the failure to detect increased hippocampal activity for items that are recognized only in association with a ‘know’ response, or only with low confidence or without correct source information could have more to do with the failure to detect weak memory than with the presence or absence of recollection. An implication of this view is that the typical relationship between memory strength and neural activity in the hippocampus (as measured by fmRI) is nonlinear (FIG. 5).

Figure 5. Characteristic nonlinear relationships between fMri activity and memory strength.

aIn the hippocampus, the relationship between functional MRI (fMRI) activity and memory strength at encoding is such that there is often a relatively steep increase in activity at the high end of the memory-strength scale (with high strength indicated by ‘remember’ responses or by hits with correct source information)⁵²,⁵³,⁶⁵. Thus, the slope of the fMRI response is steeper when memory is strong than when it is weak. We suggest that this relationship reflects nonlinear properties of the measurement scale that arise for reasons unrelated to the distinction between recollection and familiarity. The same relationship has been observed even for purely recollection-based tasks⁶⁸. b. In the perirhinal cortex, the relationship between fMRI activity and memory strength at encoding is such that there is often a relatively steep increase in activity when memories are weak, but a more shallow increase when memories are stronger⁵²–⁵⁴. This nonlinear pattern has also been observed for purely recollection-based tasks⁸². c. In the hippocampus, the relationship between fMRI activity and memory strength at retrieval is generally the same as is observed at encoding⁶⁰,⁶³,⁶⁴. d. In the perirhinal cortex, the relationship between fMRI activity and memory strength at retrieval is such that there is often a decrease in activity as memory strength increases⁶⁰,⁶³,⁷⁸. This pattern holds true for item-based memory tasks and may be indicative of novelty detection. The same pattern has also been observed in the anterior hippocampus (not shown)⁶⁰. Lastly, unlike item-based memory tasks, the relationship between fMRI activity and memory strength in perirhinal cortex tends to be positively sloped for recollection-based associative memory tasks (not shown)⁸²,⁸³.

A neuroimaging study of associative learning illustrates this point⁶⁸. While being scanned, participants learned face–name associations and then were shown a studied face along with two possible choices: the correct name and another name that had been presented earlier with a different face. Because the two names were equally familiar, it is reasonable to assume that successful performance depended on recollection. Participants were first asked to choose the correct name and then to indicate high or low confidence in their decision. Accuracy was higher for high-confidence decisions (approximately 78% correct) than for low-confidence decisions (approximately 58% correct); however, both kinds of decision were significantly more accurate than would be expected by chance. Thus, behavioural accuracy could be detected in both conditions, even though the low-confidence decisions would have included more random noise owing to guessing than the high-confidence decisions. By contrast, only the high-confidence correct decisions were associated with increased hippocampal activity (relative to incorrect decisions), as measured by fmRI. Thus, fmRI does not readily detect weak memory in the hippocampus, even when memory is based on recollection and can be detected by behavioural measures.

Similarly, another study used a variant of the remember–know procedure and found at retrieval that the activity associated with strong recollection (which was designated by the subject as ‘remember 2’) was elevated relative to the activity associated with ‘know’ responses in a region within and adjacent to the hippocampus⁶⁷. However, the activity associated with weaker recollection (which was designated by the subject as ‘remember 1’) did not differ from the activity associated with ‘know’ responses. These findings are consistent with the idea that the hippocampal activity associated with relatively weak memory — whether it be weak recollection or weak familiarity — is hard to detect with fmRI. Single-unit recording methods might be more suited to detecting such activity.

In a recent study ⁶⁹, the activity of single hippocampal neurons was recorded using depth electrodes in epileptic patients who were being evaluated for surgery. The patients saw a sequence of twelve visual images, each of which was presented in one of four quadrants on a computer screen. About 30 minutes later, the patients made old–new decisions for the 12 studied images and 12 new images and were also asked to make a source memory judgment — specifically, to recollect the spatial location (the quadrant) in which the image had been presented. Two kinds of neurons were identified in the hippocampus: neurons that signalled novelty by increasing their firing to new items, and other neurons that signalled prior occurrence by increasing their firing to old items (other hippocampal neurons have been found that decrease their firing in response to either new or old items⁷⁰). Importantly, the neurons that responded to prior occurrence exhibited increased firing even on trials when the recollection of spatial location failed, and even across test sessions when spatial recollection performance was no better than chance. Thus, successful recollection of spatial location was not required for hippocampal neurons to exhibit item recognition. These results provide direct evidence that the hippocampus is involved in item recognition even when recollection is absent for a major feature of the task.

Single-unit data from monkeys and rats underscore what has been learned from humans. The hippocampus does sometimes signal stimulus-specific information, particularly in associative-memory tasks⁵¹. Nevertheless, in single-item recognition tasks, hippocampal neurons tend not to signal specific information about the stimuli, but rather to signal whether a stimulus is familiar or not⁷¹. In one study in monkeys⁷², 34% of hippocampal cells responded differently depending on whether the stimulus was familiar or novel. Similarly, in rats⁷³, 13% of responsive hippocampal cells responded differentially to familiar and novel stimuli. This response pattern has been referred to as an ‘abstracted’ recognition signal, because what is signalled is a prior occurrence, not specific information about the nature of the familiar event.

Other work shows that, during memory tasks, hippocampal neurons also signal multiple relevant aspects of the task structure. For example, in one notable study⁷⁴, rats performed an olfactory recognition task in which the odour stimuli could appear at several different locations on an open platform. Some hippocampal neurons (14%) conveyed the ‘abstracted’ recognition signal, others (20%) signalled prior occurrence or novelty in conjunction with information about the location in which the odour had been presented, and still others signalled a specific location (34%) or a specific odour (11%). Many other neurons (29%) responded when a stimulus was approached, regardless of which odour or location was involved. These data show that the hippocampus signals virtually all aspects of an ongoing behavioural episode, including whether a stimulus is familiar or novel.

Neural activity in the perirhinal cortex

Studies in monkeys show that single neurons in the inferotemporal cortex (a region that includes perirhinal cortex) respond frequently to a visual stimulus when it is first presented and then less frequently as the stimulus becomes more familiar⁷⁵. These responses are stimulus selective^75,76, and they can persist for more than 24 hours⁷⁷. Accordingly, such responses are thought to provide a suitable index of familiarity. The fact that perirhinal neurons tend to respond to a repeated item in a stimulus-selective fashion, whereas hippocampal neurons tend to convey a non-selective (abstracted) recognition signal suggests that the two regions make distinct, and potentially complementary, contributions to recognition performance.

Several fmRI studies have shown a similar effect in human perirhinal cortex. Specifically, activity at retrieval is inversely related to memory strength — that is, activity is strongest for novel items and weakest for old items that are correctly recognized^{57,60,63,78–80}. In one study ⁶⁰, the same relationship between memory strength and neural activity was found in the anterior hippocampus. These results are consistent with the idea that the perirhinal cortex (and perhaps the hippocampus as well) responds to item familiarity, which might be signalled in part by a reduced novelty response⁸¹. It is also possible that the detection of novelty is distinct from the detection of familiarity⁶⁰. Indeed, one could suppose that familiarity is signalled by an increased neuronal firing rate that is associated with old stimuli, as was observed for neurons in the human hippocampus⁶⁹, whereas the detection of novelty is associated with an increased firing rate to new stimuli, as is commonly observed for neurons in perirhinal cortex³.

There has been less agreement about whether the perirhinal cortex also has a role in recollection. Evidence against the idea comes mainly from fmRI studies. For example, activity in the perirhinal cortex at encoding is often no different for items that will later be strongly remembered than for items that will later be weakly remembered but nevertheless exceeds the activity associated with items that will later not be remembered^22,52–54. Such findings have often been taken to suggest that perirhinal activity influences the subsequent familiarity of an item but does not additionally contribute to its later recollection.

However, as discussed above, these studies confound memory strength with the presence or absence of recollection. An alternative interpretation of these results holds that the relationship between memory strength and activity in the perirhinal cortex is nonlinear (that is, that the fmRI signal is relatively insensitive to changes in memory strength at the high end of the scale), not that the perirhinal cortex is uninvolved in the recollection process (FIG. 5). For example, in one study⁵⁴ the level of perirhinal activity at encoding predicted higher levels of confidence in subsequent ‘old’ decisions, but only up to a confidence rating of five on a six-point scale. The high threshold/signal-detection model³⁰, which was used to interpret these data, holds that confidence ratings as high as five are based entirely on familiarity, because recollection reliably yields a confidence rating of six. However, considerable evidence shows that confidence ratings of 4 and 5 are associated with lower degrees of source recollection, not with the absence of source recollection^8,36,40. Thus, although there is evidence that activity in perirhinal cortex does not always distinguish between items that will be strongly remembered and items that will be weakly remembered, there is a simple explanation for such a finding, based on memory strength, which is unrelated to the distinction between recollection and familiarity.

A more direct way to evaluate the role of the perirhinal cortex in recollection is to use an associative-learning procedure where the familiarity of the test items provides no information and where it is reasonable to assume that successful performance is based on recollection. In one study of this kind⁸², volunteers learned arbitrary associations between kaleidoscopic images and spatial locations on a computer screen (similar to an earlier study that was carried out with monkeys⁵¹). Activity in right perirhinal cortex, as well as bilaterally in the hippocampus and parahippocampal cortex, increased as accuracy improved on the task. Interestingly, a nonlinear relationship between memory strength and neural activity was observed in the perirhinal cortex. Thus, even in this purely recollection-based task, perirhinal activity increased rapidly early in learning and then levelled off as learning continued and memory strength increased. The authors point out that the nonlinear relationship between neural activity and memory strength might simply reflect the fact that the blood-oxygen-level-dependent (BOlD) signal for the perirhinal cortex is not linearly related to memory strength. Similarly, our view is that a monotonic relationship between memory strength and neural activity, as measured by fmRI, is meaningful, but that differences in the nonlinear shape of that function across different brain structures should not imply qualitative differences in the memory processes that they subserve (BOX 2). This nonlinearity is not always observed (for example, in one study⁸³ it was observed at encoding but not at retrieval), but it is observed often enough to suggest that the fmRI signal in perirhinal cortex is relatively insensitive to variations in memory strength at the high end of the scale.

Box 2 | Interpreting nonlinear response functions.

Functional MRI (fMRI) blood-oxygen-level-dependent (BOLD) signals correlate with neural activity in the brain, but the precise nature of the relationship is not well understood. Accordingly, attaching theoretical significance to different nonlinear shapes of otherwise monotonically comparable functions (for example, two increasing functions, such as in FIG. 5a,b) is problematic. Different nonlinearities in the hippocampus and the perirhinal cortex would imply different functionality only under the untested assumption that the relationship between the underlying neural activity and the fMRI signal is the same for both structures. Caution on this point has been expressed previously, on the grounds that the BOLD response might depend nonlinearly on the neural signal, and further that this nonlinearity might differ across brain regions, even when the regions lie in close proximity^126,127. By contrast, a qualitative difference in the relationship between the BOLD signal and memory strength, such as an increasing function in the hippocampus and a decreasing function in the perirhinal cortex (FIG. 5c,d), is likely to be meaningful. In this case, an increasing function would seem to indicate that higher neural activity is associated with stronger memory, whereas a decreasing function would seem to indicate that higher neural activity is associated with increased novelty.

Numerous other imaging studies have also observed activity in medial temporal lobe structures, including perirhinal cortex, that is related to successful associative recollection (for examples, see REFS ^83–87, but see REF. 88 for a possible exception). These studies are advantaged by not relying on any particular cognitive model to identify recollection-based responses. In addition, they do not confound the presence or absence of recollection with the strength of memory.

Other evidence of a role for the perirhinal cortex in associative recollection comes from single-unit studies in monkeys. In one study ⁸⁹, monkeys learned a pair-association task. Altogether, the monkeys learned 12 stimulus–stimulus associations (that is, there were 24 stimuli in all), with the individual stimuli of each pair serving as the cue stimulus on some trials and as the correct stimulus on other trials. Neurons in both perirhinal cortex (14% of 510 neurons) and the immediately adjacent unimodal visual area TE (19% of 1,858 neurons) responded selectively to at least 1 of the 24 stimuli during the cue period. Crucially, many of those same neurons were also preferentially activated by the cue’s paired associate. The percentage of these ‘pair coding’ neurons was much higher in perirhinal cortex (33% of cue-selective neurons) than in area TE (4.9% of cue-selective neurons). This finding suggests that perirhinal cortex is specifically involved in the process of associative encoding, perhaps as the result of convergence onto perirhinal neurons from separate area TE neurons that encode the two visual stimuli. Importantly, a subsequent study found that ‘pair-coding’ neurons in perirhinal cortex (as well as in area TE) develop progressively over the course of several hours and in parallel with behavioural learning⁹⁰. The fact that many of the neurons in perirhinal cortex exhibited pair-coding responses as soon as they exhibited stimulus selectivity suggests that the associative-memory signals originate in the perirhinal cortex without feedback from other areas⁸⁹.

In another study⁹¹, monkeys learned arbitrary associations between complex scenes and one of four spatial locations on a computer screen (as in the study discussed earlier⁵¹). About 30% of scene-selective cells in perirhinal cortex signalled learning by changing their firing rate in parallel with behavioural learning (for comparison, in the hippocampus, the percentage was 28%). These findings demonstrate the existence of associative-learning signals in perirhinal cortex even in tasks that are learned relatively rapidly. The results are also consistent with lesion studies showing that damage to the perirhinal cortex in monkeys impairs the learning of new associations^92,93. Thus, although it is true that the perirhinal cortex is involved in familiarity responses, it is also clearly involved in associative recollection.

Conclusions

The idea that directly adjacent and interconnected structures of the medial temporal lobe selectively subserve recollection and familiarity has generated a great deal of apparently contradictory research in recent years. Part of the dispute is traceable to the fact that a number of neuroimaging studies and studies of human amnesia, which appear to support a divided-labour account of medial temporal lobe function, have depended for their interpretation on the validity of controversial cognitive models. In this article we have presented a different perspective. We suggest that the methods that have typically been used to distinguish between recollection and familiarity instead distinguish between strong memories and weak memories. Further, nonlinear fmRI signals in the medial temporal lobe have often been taken to indicate qualitative distinctions between memory processes. However, this interpretation depends on the assumption that the fmRI signal is linearly related to memory strength. A simpler possibility is that nonlinearities indicate that the fmRI signal does not provide a linear measurement scale. Studies that are not dependent on particular models, such as direct comparisons of recall versus recognition deficits in patients with hippocampal lesions, and single-unit studies of recollection and familiarity, find that both the hippocampus and the perirhinal cortex are associated with recollection and familiarity.

Although the function of medial temporal lobe structures cannot be sharply dichotomized according to recollection and familiarity, this does not imply that these structures have the same function. For example, neurons in perirhinal cortex and the hippocampus differ in their response to novel stimuli and familiar stimuli. Neurons in the perirhinal cortex tend to signal novelty and then return to baseline as an item becomes more familiar⁷⁷. Neurons in the hippocampus are more responsive to familiar stimuli, sometimes increasing their firing rate and sometimes decreasing their firing rate below baseline^69,70. What these differences imply about the functional organization of the medial temporal lobe is not yet clear, but they are not related in any obvious way to the distinction between recollection and familiarity.

Another difference between the hippocampus and the perirhinal cortex lies in the degree to which the two structures code information in stimulus-specific or more abstract forms. Whereas perirhinal neurons often respond in a stimulus-selective manner^75,76, the neurons of the hippocampus are less stimulus-selective and are more likely to signal prior occurrence, regardless of which stimulus is presented^69,71,94. Although these signals in the hippocampus are abstract, they nevertheless appear to be directly involved in both recollection and familiarity, and do not merely reflect encoding of the outcome of recognition experiences, as recently suggested⁵. The evidence that these signals are directly involved in recollection and familiarity comes from tests of the effect of hippocampal lesions on tasks that depend on these processes. As discussed earlier, hippocampal lesions impair recognition (which is partially based on familiarity) to the same extent that they impair recall (which is based on recollection).

Our Review emphasizes that recollection and familiarity signals are evident in both the perirhinal cortex and the hippocampus. In place of approaches that focus on these constructs, we suggest an alternative approach to understanding the function of medial temporal lobe structures. Underwood⁹⁵ argued that memory is best understood as a collection of specific stimulus attributes. Thus, spatial location, frequency, recency, modality, semantic class and stimulus associations are all attributes of an episode thought to collectively form the memory trace. Single-unit recording studies suggest that a similarly broad range of attributes is in fact encoded by the neurons of the medial temporal lobe⁹⁶. Single-unit studies further suggest that these attributes are associated by neurons of the perirhinal cortex (for example, pair-coding neurons⁸⁹) as well as by the hippocampus (for example, neurons that encode conjunctions⁷⁴). These considerations suggest that searching for the anatomical correlates of recollection and familiarity in the medial temporal lobe might be a less fruitful way to understand its organization than investigating how the hippocampus, the perirhinal cortex and other medial temporal lobe structures participate in encoding and retrieving the attributes of memory.

Recognition

The ability to distinguish a previously presented stimulus from one that was not previously presented.

Recall

The ability to remember a previously presented stimulus in the absence of that stimulus.

Hypoxia

A condition in which there is insufficient oxygen in blood or tissue.

Source memory

The recollection of information about how, when or where a particular stimulus was presented (that is, its source).

Associative recognition

The ability to distinguish a pair of stimuli that have previously been presented together from another pair of stimuli whose items were also previously presented, but as part of different pairs.

Old–new decision

The decision as to whether a test item in a recognition test has been presented before. If the subject thinks it has, the item is termed ‘old’; if not, it is termed ‘new’.

Target

An item on a recognition memory test that appeared on a list presented earlier (that is, an ‘old’ item).

Foil

An item on a recognition memory test that did not appear on a list presented earlier (that is, a ‘new’ item).

Single-unit neurophysiology

A method used to measure the activity of individual neurons in awake, behaving animals. This method has excellent spatial and temporal resolution but can survey the activity of relatively small numbers of neurons.

Functional MRI

(fMRI). An imaging technique that measures changes in haemoglobin oxygenation as blood flows to functioning areas of the brain.