Working memory, long-term memory, and medial temporal lobe function

Abstract

Early studies of memory-impaired patients with medial temporal lobe (MTL) damage led to the view that the hippocampus and related MTL structures are involved in the formation of long-term memory and that immediate memory and working memory are independent of these structures. This traditional idea has recently been revisited. Impaired performance in patients with MTL lesions on tasks with short retention intervals, or no retention interval, and neuroimaging findings with similar tasks have been interpreted to mean that the MTL is sometimes needed for working memory and possibly even for visual perception itself. We present a reappraisal of this interpretation. Our main conclusion is that, if the material to be learned exceeds working memory capacity, if the material is difficult to rehearse, or if attention is diverted, performance depends on long-term memory even when the retention interval is brief. This fundamental notion is better captured by the terms subspan memory and supraspan memory than by the terms short-term memory and long-term memory. We propose methods for determining when performance on short-delay tasks must depend on long-term (supraspan) memory and suggest that MTL lesions impair performance only when immediate memory and working memory are insufficient to support performance. In neuroimaging studies, MTL activity during encoding is influenced by the memory load and correlates positively with long-term retention of the material that was presented. The most parsimonious and consistent interpretation of all the data is that subspan memoranda are supported by immediate memory and working memory and are independent of the MTL.

The distinction between immediate memory (or working memory) and long-term memory has been fundamental to understanding how the brain has organized its memory functions (Atkinson and Shiffrin 1968; Baddeley and Warrington 1970; Milner 1972; Squire 2009). Immediate memory refers to the limited amount of information that can be held in mind when material is presented for learning. Working memory refers to the capacity to maintain this limited amount of information through active rehearsal, usually across a relatively short time interval (Baddeley and Hitch 1974). Long-term memory refers to what can be recalled from the past when the information to be learned no longer occupies the current stream of thought, either because immediate memory capacity was exceeded or because attention was diverted from the memoranda.

Early studies of memory-impaired patients with medial temporal lobe (MTL) damage found immediate memory and working memory (sometimes referred to as short-term memory) to be intact, despite markedly impaired performance on tasks of long-term memory (Drachman and Arbit 1966; Baddeley and Warrington 1970; Milner 1972). Thus, patients with damage to MTL structures (the hippocampus and the adjacent entorhinal, perirhinal, and parahippocampal cortices) demonstrated intact immediate memory for strings of digits, words, tones, and nonsense visual patterns and shapes (Drachman and Arbit 1966; Wickelgren 1968; Baddeley and Warrington 1970; Milner 1972; Cave and Squire 1992), as well as an intact ability to maintain a limited amount of information in memory by rehearsal, even for several minutes (Milner 1972). Accordingly, the view that developed was that MTL structures are involved in the formation of long-term memory and that immediate memory and working memory are independent of these structures.

This traditional idea has recently been revisited. A number of studies have reported impaired performance after MTL damage on tasks involving delays as short as a few seconds. In addition, several neuroimaging studies have reported MTL activation during short-delay tasks involving various kinds of visual material. These findings have led to debate about the concepts of immediate memory and working memory and raised the possibility that the MTL, in addition to its established role in forming long-term memory, is needed for at least some kinds of working memory (e.g., for the active maintenance of novel visual objects or the relations between items). This view has been presented in two comprehensive reviews (Ranganath and Blumenfeld 2005; Graham et al. 2010). The latter review also proposed that the MTL (specifically, the perirhinal cortex) is needed even for certain kinds of visual perception. Here, we present a reappraisal of these issues. A major challenge is determining when a task actually depends on only working memory (or perception) and when a task imposes a sufficiently large burden on memory such that the task requirements exceed working memory capacity.

When does a task depend on working memory?

Working memory cannot be operationally defined in terms of any particular retention interval. Instead, working memory involves the process of active maintenance of a limited amount of information. The key factors that determine whether working memory is sufficient to support performance, or whether performance must also depend on long-term memory, are the amount of information that can be held in mind and how amenable this information is to active rehearsal. If the capacity of working memory is exceeded, or if material cannot be effectively maintained by rehearsal (as can be the case for nonverbal material), performance must depend at least in part on long-term memory, even at short retention intervals.

Long-term memory is also needed to support performance as soon as attention is diverted, even when the amount of material to be learned is limited and even when it is amenable to rehearsal. Attention can be diverted either through “the passage of time (with its endogenous and exogenous distractions) or a purposely induced distraction” (Drachman and Arbit 1966, p. 58). Because the probability that attention will be diverted increases with the amount of time that has passed since learning, long-term memory is often needed to support performance on tasks involving long delays. Similarly, because a short retention interval reduces the probability that attention will be diverted, working memory is sometimes sufficient to support performance in tasks involving short delays so long as the amount of information to be maintained is not too great. Still, a limited amount of information may be held in mind indefinitely if attention is continuously directed toward the memorandum. At the same time, the same information can be lost from working memory even after a short interval if attention is diverted (Drachman and Arbit 1966; Milner 1972).

William James's (1890) distinction between primary memory (immediate memory) and secondary memory (long-term memory) did not emphasize the learning-test interval as an important factor. He distinguished a limited-capacity, impermanent memory system from a large-capacity, permanent storage system:

“[A]n object of primary memory … never was lost … [but] comes to us as belonging to the rearward portion of the present space of time, and not to the genuine past.”

An object of secondary memory, by contrast, “ … is one which has been absent from consciousness altogether, and now revives anew. It is brought back, recalled, fished up, so to speak, from a reservoir in which, with countless other objects, it lay buried and lost from view.” (James 1890, pp. 646–647.)

Drachman and Arbit's (1966) later treatment of short-term and long-term memory echoed this emphasis on capacity and did not favor any particular retention interval:

“Short-term” memory … deals only with subspan memoranda, evanescently, as long as the subject's attention is directed towards the memorandum. Recall following redirection of attention (i.e., by sufficient distraction or delay) depends upon a more permanent storage mechanism. By contrast, “long-term” memory (storage) deals both with supraspan memoranda held for long or short intervals and with subspan memoranda recalled following the redirection of attention.” (Drachman and Arbit 1966, p. 59.)

The idea that long-term memory may be needed to support performance even when memory is tested immediately following learning of new material might seem counterintuitive. The terms subspan and supraspan material are perhaps more helpful than the terms immediate memory and long-term memory. Consider the following example: when presented with ten words and then asked to recall them, memory-impaired patients recall fewer words than controls, even if memory is probed immediately after learning. Patients recall fewer items than controls because the ten words exceed what can be held in mind. Ten words are not subspan material. The point is that long-term memory sometimes benefits performance even when memory is tested immediately after or within seconds of learning (see also Baddeley et al. 2010, 2011; Brady et al. 2011).

Impaired short-term retention of visual information after MTL damage

In several recent studies, patients with bilateral MTL damage were found to be impaired at remembering visual information across delays as short as a few seconds. Thus, impairments have been noted on tasks involving novel objects or patterns (Aggleton et al. 1992; Buffalo et al. 1998; Holdstock et al. 2000), faces (Nichols et al. 2006; Olson et al. 2006a; Ezzyat and Olson 2008), colored squares (Olson et al. 2006a), topographical scenes (Hartley et al. 2007), and tasks requiring retention of the relations between items (Hannula et al. 2006; Olson et al. 2006b). The majority of these impairments were observed in delayed match-to-sample tasks or change-detection tasks where the delays were 4 sec or longer (Aggleton et al. 1992; Buffalo et al. 1998; Holdstock et al. 2000; Nichols et al. 2006; Olson et al. 2006a,b). Impaired performance has also been noted in two continuous recognition tests where the inter-stimulus-interval was only 3 sec (Hannula et al. 2006) and (less consistently) in tasks where the delays were as short as 1–2 sec (Olson et al. 2006b, Experiment 2; Hartley et al. 2007; Ezzyat and Olson 2008, in one of two tasks). In most of these studies, the findings were interpreted to mean that the MTL is needed for certain kinds of working memory.

The key question is whether these findings suggest a role for the MTL in working memory or whether they reflect instances where performance is supported in part by long-term memory (even though the study-test interval is quite brief).

Retention of novel visual objects and patterns

Earlier reviews of short-term retention of visual information in memory-impaired patients (Ranganath and Blumenfeld 2005; Graham et al. 2010) highlighted findings from five studies where retention of novel, complex, and difficult-to-verbalize visual material was impaired after delays of 6–10 sec (Fig. 1 of Ranganath and Blumenfeld 2005; reprinted as Fig. 9 of Graham et al. 2010). Three of these studies involved patients with presumed or confirmed bilateral MTL damage (Aggleton et al. 1992; Buffalo et al. 1998; Holdstock et al. 2000). The other two studies are less informative about MTL function. One involved unilateral surgical lesions of the MTL, but no neuroimaging data were presented to describe the lateral extent of the lesions (Owen et al 1995). In addition, the performance of the patients at the shortest delay was only modestly disadvantaged (<10%), and it is unclear that there was a significant impairment. The other study involved a mixed group of patients and was not restricted to MTL function (Holdstock et al. 1995). The performance of the patients in these five studies, in comparison to controls, was interpreted to mean that MTL damage impaired working memory for novel visual objects. We consider the first three studies and suggest a different interpretation of the data.

Figure 1.

Short-term retention of novel visual objects in memory-impaired patients with: (A) presumed MTL damage; (B,C) confirmed bilateral MTL damage; and (D) memory impairment from damage other than the MTL (AMN denotes amnesia). (A) Participants studied a detail of an abstract painting for 10 sec and then, after a delay of 10, 30, or 90 sec, decided which of two patterns they had seen previously. (B) Participants studied four kaleidoscope designs (1 sec each) with a 1-sec inter-stimulus interval. After a variable delay (0–2 sec, 6–10 sec, or 25–40 sec), they decided (yes or no) whether or not a test stimulus matched one of the images just presented. (C,D) Participants studied a monochrome abstract pattern and then, after unfilled delays of 0–5 sec or filled delays of 10–30 sec, indicated from an array of 14 patterns which pattern they had seen previously. Participants included four patients with confirmed MTL damage (C) and five different patients with mixed etiologies and memory impairment from damage other than the MTL (D). Unfortunately, two earlier reviews (Ranganath and Blumenfeld 2005; Graham et al. 2010) presented the data from the five patients with mixed etiologies (shown here in D) and mistakenly labeled the patients as MTL patients. (Panels A,B,D adapted from Ranganath and Blumenfeld 2005 [with permission from Elsevier © 2005]; panel C adapted from Holdstock et al. 2000 [with permission from Elsevier © 2000].)

In the first study (Aggleton et al. 1992), information about the localization and extent of the lesions (caused by viral encephalitis) was not available. It is, therefore, difficult to make firm conclusions from these data about the MTL. In any case, it is noteworthy that the patients performed well at the 10-sec delay (our Fig. 1A). Even though the patients as a group performed numerically worse than controls at the 10-sec delay, the original report emphasized that only one of the post-encephalitic patients made any errors at this delay (Aggleton et al. 1992). Thus, the patients were, as a group, impaired only after delays of 30 sec or longer (i.e., under conditions where performance most likely depended on both working memory and long-term memory).

In the second study (Buffalo et al. 1998), two patients with confirmed MTL damage exhibited intact performance at the shortest retention delay and impaired performance at the longer delays (Fig. 1B). At the shortest delay, 0–2 sec elapsed between the last of four study items and the test item. Note, however, that because the four study items were presented serially (1-sec presentation time/item) with a 1-sec inter-stimulus-interval, the delay between the first study item and test was actually 6–8 sec, even in the 0–2-sec delay condition (average delay for all four study items at the shortest delay = 3.8 sec). Similarly, at the intermediate delay (6–8 sec), where patients first exhibited impaired performance, the average delay for the four study items was 11 sec.

In the third study (Holdstock et al. 2000), four MTL patients exhibited fully intact performance at the three short delays (Fig. 1C), and were impaired only at the longer delays (during which active maintenance was disrupted by a filler task). Unfortunately, Ranganath and Blumenfeld's (2005) review mistakenly illustrated not the performance of the four patients with MTL damage, but the performance of five different patients from the same study who had mixed etiologies and no evidence of damage to the MTL (our Fig. 1D). This error, which was reproduced in the more recent review by Graham et al. (2010), may have contributed to the impression that the MTL is needed for working memory of novel visual objects because, in Figure 1D, the patients performed poorly at short delays. However, the discussion should have been based on the data presented in Figure 1C, and these data would have suggested a different interpretation.

In the three studies just reviewed, which tested retention of novel visual objects after MTL damage, performance was intact at the shortest delay(s) and impaired only at longer delays. Intact performance at the shortest delays indicates that immediate memory was intact. Given the limits of immediate memory capacity (typically only three to four simple visual objects can be maintained even for a delay as short as 1 sec) (Luck and Vogel 1997; Cowan 2001), the striking finding summarized in Figure 1 is not that the patients were impaired at study-test delays of 6–30 sec but that they exhibited intact retention of complex visual information at delays of 0–10 sec.

It is also worth emphasizing that intact performance at short delays after hippocampal lesions or larger MTL lesions, and impaired performance at longer delays, has been well-demonstrated in monkeys (Overman et al. 1990; Alvarez et al. 1994) and also in rats (Clark et al. 2001). The findings from humans, together with the findings from nonhuman primates and rats, provide no positive evidence for impaired working memory after MTL damage. Indeed, the findings are fully consistent with the traditional view that memory was impaired after delays of 6–30 sec because it is difficult to maintain difficult-to-verbalize material in working memory. Accordingly, we suggest that performance at these longer delays depends, in part, on long-term memory and that impaired performance at these longer delays reflects impaired long-term memory.

Nevertheless, it is possible in the case of delay-dependent memory impairments to propose an alternative perspective. Even when performance is intact at short delays, one could point to the next longest delay, where an impairment first appears, and propose that performance at that retention interval ordinarily depends on working memory. Accordingly, impaired performance at that same interval would reflect impaired working memory. While this line of reasoning is arbitrary and without background in the literature, there is no logical objection to it. The interpretation is, however, testable in any given instance, and later we describe a general method for deciding whether impaired performance after a brief delay depends on working memory or long-term memory (see Determining When Performance Depends on Long-Term Memory).

Retention of familiar visual items (single faces and colors)

MTL damage has also been found to impair retention after short delays of more familiar, concrete visual stimuli. In two studies where memory for a single face was probed using a change-detection task, patients with MTL damage exhibited impaired performance after delays of 4 sec (Olson et al. 2006a) and 7 sec (Nichols et al. 2006). The patients in the former study were also impaired on a change-detection task that required retention of three colored squares across delays of 4 or 8 sec (Olson et al. 2006a). In a third study (Ezzyat and Olson 2008), MTL damage impaired retention of a single morphed face across a delay of 1 sec (in a forced-choice test but not in a yes/no test) and after a delay of 8 sec (in the yes/no test but not in the forced-choice test).

In the study involving colored squares (Olson et al. 2006a), MTL patients and controls saw an array of three colored squares and then decided whether or not a designated square in a second array (presented after 4 or 8 sec) had the same color as the corresponding square in the first array. The poor patient performance in this task, as well as in a similar task requiring retention of one face for 4 sec, was interpreted as a visual working memory deficit and not as a result of being given supraspan material (i.e., a long-term memory deficit), because “most people can accurately remember four colors (Luck and Vogel 1997), or 1.5 faces (Eng et al. 2005)” (Olson et al. 2006a, p.1093).

It is true that previous research suggests that three or four colored squares can be maintained in visual working memory (e.g., Luck and Vogel 1997; Cowan 2001; Fukuda et al. 2010). However, these estimates of immediate memory capacity were all obtained from young adults. Similarly, Eng et al. (2005) obtained a capacity estimate of 1.1–1.5 faces (with memory display durations of 500–3000 msec) using a sample of Harvard undergraduates. The difficulty is that the appropriate comparison group for evaluating the memory capacity of MTL patients is a group of age-matched and education-matched individuals. Typically, such a group has a mean age >60 years and, on average, <16 years of education. Memory capacity is smaller for older adults than for undergraduate students (Jost et al. 2011). Furthermore, estimates of visual working memory capacity in change-detection tasks are typically obtained by assessing performance after delays of ∼1 sec (e.g., 900 msec in Luck and Vogel 1997; average delay of 1.1 sec [average of 300, 900, and 2000 msec] in Eng et al. 2005), and therefore do not provide suitable estimates of working memory capacity in tests given after delays of 4 sec or longer (as in the studies of Nichols et al. 2006; Olson et al. 2006a). Finally, it has been demonstrated that change-detection performance is limited both by the number of items that can be maintained in memory and by the similarity between sample and test stimuli (Awh et al. 2007). When the sample-test similarity is high, more visual detail must be maintained and the memory capacity is lower (see also Alvarez and Cavanagh 2004). It is, therefore, notable that in two of the studies involving faces, the inter-item similarity was high (Olson et al. 2006a; Ezzyat and Olson 2008).

These considerations make it difficult to rule out the possibility that, for the older participants in these studies, visual working memory capacity was exceeded even when the material involved three colored squares or one face. Note that patients were intact when task requirements were less demanding and likely within the limits of working memory capacity. For example, in the Nichols et al. (2006) study, patients exhibited intact change-detection performance for colored squares at a 1-sec delay. Our own recent study compared the performance of MTL patients and age-matched controls on a range of array sizes (one, two, three, four, and six colored squares) and a range of delays (1, 3, 4, and 8 sec) (Jeneson et al. 2011a). At the 1-sec delay, patients performed as well as controls at all array sizes. At the longer delays (average of the three longer delays), patients performed as well as controls for small array sizes (one and two items) and were impaired only for array sizes that could be expected to exceed memory capacity (three, four, and six items). It seems reasonable to suppose that long-term memory benefited control performance when the largest demands were made on memory (longer delays and larger array sizes). Indirect support for this idea comes from the finding that in Nichols et al. (2006) the best predictor of the patients' ability to remember a face for 7 sec was performance on standardized tests of long-term memory, suggesting that long-term memory supported performance on this task.

In the study by Ezzyat and Olson (2008), patients exhibited intact performance at the 1-sec delay in one of the two tests (in the forced-choice test but not in the yes/no test). Apart from the difficulty of knowing the capacity estimates of the highly similar faces used in this study, a difficulty also arises because the task did not involve trial-unique stimuli. The stimuli were faces selected from a series of faces in which one face was gradually morphed into another. Participants studied one face and then decided which of two faces from the same morph series more closely matched the just-seen face. The poor patient performance on this test after 1 sec was interpreted to reflect impaired working memory. However, the same faces reappeared several times during testing (16 study faces repeated six times each across 96 trials). This arrangement allowed for the possibility that healthy controls could gain an advantage over patients through learning. As discussed in the section on visual discrimination (below), there is precedence for such learning effects in controls (but not patients) in a similar task where perceptual judgments involved items that were repeated many times (Kim et al. 2011).

Retention of relational and spatial information

In two studies, MTL damage impaired retention of information about the relations between items or features, even at quite short delays (Hannula et al. 2006; Olson et al. 2006b). The first study used two continuous recognition tests to explore retention of object-in-scene information and scene-face associations across short and long lags in patients with hippocampal damage (Hannula et al. 2006). In the test of object-in-scene information (illustrated in Fig. 2), participants attempted to remember computer-generated scenes as well as the location of objects in each scene. Patients exhibited good memory for the scenes themselves (item information) but were impaired at remembering information about the location of objects within the scenes (relational information). In the test of scene-face associations, participants attempted to remember scene-face pairs (a single face superimposed on a scene for each study trial) and then decided, after a lag of one or nine items, which of three faces superimposed on a scene had been earlier associated with that scene. In both tests of relational information, patients performed worse than controls even at a lag of one item, i.e., when no stimuli intervened between study and test.

Figure 2.

(A) Repeated (match) and manipulated (nonmatch) test trials were interleaved systematically among a sequence of scenes. Test trials appeared either immediately after the corresponding scene had been presented (lag 1), five trials later (lag 5), or nine trials later (lag 9). The task for each trial was to decide whether the scene had appeared earlier in the series, and then, critically (in the case of a “yes” response), whether any items in the scene had changed location. Note that, even for tests at a lag of 1, participants had to try to hold in mind many previous scenes because they did not know whether the memory question would concern the most recently presented scene or a scene presented up to nine items earlier. (B) Two trials illustrating a lag of 1. Each scene was presented for a total of 20 sec. The scene was first presented alone for 5 sec. For the next 6 sec, the scene was presented along with an orienting question that drew the participant's attention to the item in the scene that would be moved or not moved (e.g., “Is the urn directly under the mirror?” [No]). Participants were not told that the orienting question identified the item that would be relevant to the memory decision. (Whenever a scene was presented a second time, the answer to the orienting question was always the same as it was when the scene was first presented. Accordingly, the answer to the orienting question did not provide information about whether the scene had been altered or not.) For the remaining 9 sec of the trial, the scene was accompanied by the two memory questions (“Have you seen this scene before?” and [if yes] “Have any items changed location?”). Note that 14 sec elapsed (3 + 5 + 6 sec) between the removal of a novel scene and the first (Old/New) memory question for the next scene (from Jeneson et al. 2011b).

The second study (Olson et al. 2006b) used a change-detection task to assess retention of objects, locations, and object-location conjunctions across delays of 1 and 8 sec in patients with MTL damage. Patients and controls studied three objects presented one at a time (1 sec per stimulus; 13 msec ISI) in one of nine possible locations in a 3 × 3 grid. Each object occupied a different square in the grid. In the feature condition, participants received one of two trial types: an object presented in the center square of the grid or a dot occupying one of the nine squares. They then decided whether or not the object had just been presented or, in the case of the dot cue, whether that particular location had been occupied by any of the three objects. Thus, feature trials required retention of objects and locations (but not the relations between them). In the conjunction condition, the test trial consisted of either an object-location combination that had been seen before (match trial) or an object-location recombination (mismatch trial, e.g., object A was presented in the location that had been occupied by object B during study). Thus, conjunction trials required retention of objects and locations plus the relations between them. Accordingly, and as one might expect, the relational (conjunction) condition in Olson et al. (2006b) was more difficult than the item (feature) condition. In two similar experiments, the results were that patients were intact in the feature condition and impaired in the conjunction condition. Moreover, in one experiment (Experiment 2), patients were impaired in the conjunction condition even when the study-test delay was as short as 1 sec (average delay of 2 sec for all three study items).

The selective impairments in retention of relational information in these two studies after short delays raised the possibility that the MTL is critical for retention of relational information, even when working memory is sufficient to support performance (Hannula et al. 2006; Olson et al. 2006b). Indeed, it was suggested that the distinction between memory for single items vs. memory for the relations among items might be more fundamental for understanding hippocampal function than the traditional distinction between working memory and long-term memory (Hannula et al. 2006; Olson et al. 2006b). An alternative possibility is that memory for relational information was impaired because the demands on memory were higher in the conditions assessing memory for relations than in the conditions assessing memory for items, and that these demands on memory exceeded visual working memory capacity (Shrager et al. 2008; Baddeley et al. 2010; Jeneson et al. 2011b).

In the case of Hannula et al. (2006), the relational memory question (which required maintenance of information about the items in each scene) involved a higher memory load than the item memory question (which required maintenance only of enough information to recognize the scene as familiar). In addition, the structure of the continuous recognition test meant that even at a lag of one item, participants still needed to try to hold in mind a number of previous scenes (up to nine), because the decision to identify each item as old or new sometimes depended on as many as nine previous items (Fig. 2). Moreover, even though the inter-stimulus interval was only 3 sec, the delay between the initial presentation of a study scene and the assessment of memory for object location was as long as 14 sec. In the second task which assessed memory for scene-face associations, participants also had to try to hold in mind a number of previous scene-face pairs even at the lag of 1, because they did not know whether the next trial would consist of a new study trial, a probe trial concerning the most recently presented scene-face pair, or a probe trial concerning a scene-face pair that had been presented up to nine trials earlier. It is therefore possible (as considered by Hannula et al. 2006; Baddeley et al. 2010) that these tasks depended on both working memory and long-term memory, even in the simplest condition (lag of 1).

We recently tested this idea in two experiments that differed in their demands on memory but that assessed retention of the same object-in-scene information (Jeneson et al. 2011b). In the first experiment, we used the same procedure as was used previously (Hannula et al. 2006) and obtained the same results. In the second experiment, we used a conventional test paradigm consisting of separate study-test trials that involved either a brief (3-sec) or a relatively long (14-sec) retention interval. In this case, participants were required to hold in mind only one scene at a time. If maintenance of object-in-scene information is critically dependent on the hippocampus, one would expect hippocampal damage to impair performance at the 3-sec retention interval as well as at the 14-sec retention interval. Instead, the patients exhibited fully intact memory for object-in-scene information when the retention delay was short (3 sec), and they exhibited impaired memory when the delay was long (14 sec). Thus, in this study, memory was impaired for object-in-scene information only when the task imposed relatively large demands on memory (because several scenes needed to be maintained [Experiment 1], or because the retention interval was long [Experiment 2]).

The study by Olson et al. (2006b) also raises questions about memory load and working memory capacity. The fact that healthy individuals can sometimes remember single features (e.g., the colors and shapes of objects) without remembering the relations between them (e.g., which color was bound to which shape) (Stefurak and Boynton 1986), suggests that retention of features plus conjunctions involves a greater memory load than remembering only the features themselves. Indeed, in studies directly comparing memory for features and memory for conjunctions, performance is typically poorer when individuals must distinguish combinations of features from recombinations of features than when they must identify single features (Mitchell et al. 2000; Olson et al. 2006b, Experiment 1; Wheeler and Treisman 2002; Treisman and Zhang 2006; Alvarez and Thompson 2009). One possibility is that detecting recombinations of features is more demanding than detecting changes in single features because recombination test trials interfere with maintenance (Alvarez and Thompson 2009) or retrieval (Wheeler and Treisman 2002) of the original feature combinations.

Given that conjunction trials are typically more difficult than single feature trials, the question remains whether the memory load in the conjunction condition in Olson et al. (2006b) was sufficiently large to exceed visual working memory capacity such that performance depended, in part, on long-term memory. Note that the serial presentation of study items meant that, on two-thirds of the trials, one or two sample images intervened between study and test. In addition, the impairment that was observed in both Experiments 1 and 2 was observed after a relatively long delay (average delay of 9 sec across all three study item positions). Both interference and delay increase the probability of distraction, and active maintenance can support good performance only so long as attention is continuously directed toward the memorandum. Note, too, that the impaired performance reported for patients at a 1-sec delay in Experiment 2 (but not in Experiment 1) occurred despite the fact that the patients performed the same in both experiments. Specifically, in Experiment 1, at the 1-sec delay, patients performed like controls. In Experiment 2, the controls unaccountably performed better than they did in Experiment 1 (even though the conjunction condition was identical in the two cases).

An additional study involving spatial information deserves mention (Hartley et al. 2007). A sample computer-generated landscape showing four hills was presented together with a four-alternative choice of landscapes (the same landscape as in the sample but depicted from a different viewpoint, and three foils that resembled the target but that depicted different landscapes). Participants tried to identify which of the four alternatives depicted the same landscape as in the target. In a second condition, a 2-sec delay intervened between sample and test. All five MTL patients were impaired after the 2-sec delay, and three of the five patients were impaired even in the matching task. The findings were interpreted to mean that the hippocampus is critically important for allocentric spatial processing (see also Bird and Burgess 2008). The question is whether the complexity of the landscapes meant that the task challenged the capacity of working memory. This issue is discussed further below, together with a proposed method for determining when the requirements of a task exceed what can be maintained within immediate memory or working memory.

Determining when performance depends on long-term memory

Across tasks involving a range of different procedures and visual materials (novel visual objects, faces, colors, and information about relations between items), MTL damage has been found to impair performance even at short delays and sometimes when relatively little material needs to be remembered. How should such findings be interpreted? Do they reflect either impaired immediate memory capacity or impaired working memory? Or do they reflect circumstances where working memory capacity has been exceeded such that performance depends, at least in part, on long-term memory? To make this determination, one needs new methods that are independent of any particular task or stimulus materials.

One method seems promising in cases where the retention interval is long enough (e.g., 8 sec) to allow distraction to be introduced during the retention interval (Shrager et al. 2008). In this approach, one assesses in the same task the effect of distraction on control performance as well as the effect of MTL damage on performance. It is assumed that distraction will be disruptive for controls whenever performance depends on maintaining information in working memory. In the first application of this method, controls (but not patients) were given either distraction or no distraction between study and test. Across tests involving names, faces, or object-location conjunctions (as in Olson et al. 2006b, discussed above), there was concordance between the performance of MTL patients and the effect on control performance of introducing distraction between study and test. Specifically, patients were intact in tasks where distraction disrupted control performance, suggesting that the patients were successful because they (like controls) could maintain information in working memory. In contrast, the patients were impaired in tasks where distraction minimally affected control performance. Controls presumably succeeded in the face of distraction in tasks where they were depending on long-term memory rather than working memory. And the patients failed in those same tasks because they could not successfully draw on long-term memory. These findings, which included data from a task like that used by Olson et al. (2006b), suggest that impaired performance was attributable to impaired long-term memory.

The method described above has the potential to disambiguate the interpretation in tasks where the retention interval is 8 sec or longer (enough time for distraction to be introduced). What method can be applied in the case of tasks where the retention interval is short, e.g., 1 sec? One promising approach emerges from a study of digit span in memory-impaired patients, including patient H.M. (Drachman and Arbit 1966). In that study, participants heard digit strings of increasing length. Each string was repeated until it was reported back correctly. Then, a new string of digits was presented that contained one digit more than the preceding string. Controls made their first errors with strings of eight digits, but, with repeated attempts at each string, they were eventually able to repeat back as many as 20 digits. In contrast, patients with MTL damage exhibited a sharp discontinuity in performance as the string length increased. For example, patient H.M. repeated back six digits without error (his premorbid digit span) but then could not succeed at seven digits even after 25 repetitions of the same digits (Fig. 3A).

Figure 3.

Intact working memory and impaired long-term memory. (A) The number of trials needed to correctly repeat back a string of digits as a function of string length. MTL patient H.M. succeeded at six digits in his first try but could not succeed at repeating back seven digits even after 25 attempts with the same string. (B) The number of trials needed to learn the locations of different numbers of objects for MTL patient G.P. and controls. G.P. succeeded easily with one, two, and three objects but could not reproduce the locations of four objects, even after 10 attempts with the same display. Note that in both cases, the patients failed at about the point when controls began to make their first errors (adapted from Squire and Wixted 2011 [with permission from Annual Reviews]).

We used this same method to assess the ability to remember object-location associations across a 1-sec delay (Jeneson et al. 2010). Patients with hippocampal damage and one patient with large MTL lesions (G.P.) performed well and similarly to controls when only a small number of object-location associations needed to be remembered, but the patients exhibited an abrupt decline in performance when as many as three or four object locations needed to be remembered. For example, patient G.P. reached criterion as quickly as controls for one, two, and three object-location associations (within one or two trials). Yet, when the set size was increased by only one additional object (set size 4), he failed to reach criterion even after 10 attempts with the same array of four objects (Fig. 3B). This pattern of performance is strikingly similar to the pattern of performance exhibited by patient H.M. on the digit task (Drachman and Arbit 1966). In the original study, the marked discontinuity in H.M.'s performance as he moved from six to seven digits was interpreted to mean that his immediate memory capacity was exceeded when seven digits were presented, and performance now depended on long-term memory. The abrupt discontinuity in performance that we observed in the object-location task suggests a similar interpretation, that is, an impairment was evident only when immediate memory capacity was exceeded. Interestingly, the capacity limit for patients corresponded to the point where controls first made errors. We suggest that controls made errors when the material to be remembered exceeded immediate memory capacity and thereby limited what could be maintained in working memory. This formulation leads to two predictions. First, if working memory is intact in MTL patients, performance should be intact in those task conditions where controls perform without error after brief delays. Second, the performance of MTL patients should become impaired as the task becomes more difficult and controls begin to make significant errors.

Visual discrimination tasks

The formulation just outlined, as derived from Jeneson et al. (2010), might also be applied to questions about the possible role of the perirhinal cortex (within the MTL) in certain tasks of complex visual perception. Recent findings in monkeys and in humans with perirhinal cortex lesions have been taken to indicate that perirhinal cortex is needed for visual discrimination tasks when the test items have a high degree of feature ambiguity (Bussey et al. 2002; Barense et al. 2005, 2007; Lee et al. 2005a,b; but see Clark et al. 2011). It has also been suggested that the hippocampus is needed when spatial processing is required, for example, in visual discriminations involving scenes (Lee et al. 2005a; Graham et al. 2006). The challenge is to decide whether impaired performance reflects impaired perception or impaired memory (for reviews of this growing literature, see Hampton 2005; Buckley and Gaffan 2006; Baxter 2009; Suzuki 2009; Graham et al. 2010).

In some studies in monkeys and also in humans (e.g., Barense et al. 2005; Graham et al. 2006), discrimination learning involved the same stimulus materials over multiple trials. In these cases, impaired performance could reflect a deficit, not in perception, but in associative learning or long-term memory for the repeated material (Hampton 2005; Suzuki 2009; Wixted and Squire 2011). In a recent study that explored this issue, patients with hippocampal lesions were impaired when the same comparison stimuli were used on every trial but were fully intact when the stimuli were unique to every trial (Kim et al. 2011).

Another way in which impaired memory could influence ostensibly perceptual tasks is that visual working memory capacity might be exceeded when numerous test stimuli are presented that are complex and have a high degree of feature ambiguity. Consider the oddity task illustrated in Figure 4 (left), which involved three levels of feature ambiguity. The finding was that patients with MTL lesions that included perirhinal cortex were intact in the minimum ambiguity condition and impaired in the intermediate and maximum ambiguity conditions. This result was interpreted to reflect a qualitative difference between the minimum condition (where only single features were relevant to the oddity discrimination) and the other two conditions (where conjunctions of features were relevant) (Barense et al. 2007; Graham et al. 2010).

Figure 4.

(Left) Controls and patients with MTL lesions that included damage to the perirhinal cortex decided which of seven novel objects (“fribbles”), simultaneously presented, did not have an identical match (i.e., each array consisted of three pairs of identical fribbles and one odd fribble). There were three levels of feature ambiguity (i.e., overlap) between the stimuli. In the minimum condition, features were unique to each fribble such that a single feature distinguished the odd fribble from the pairs. In the intermediate and maximum conditions, the features overlapped such that only a conjunction of features distinguished the odd fribble from the pairs. In each panel, the odd fribble is in the center of the bottom row (panels adapted from Barense et al. 2007 [with permission from Elsevier © 2007]). (Right) The percent error score at each level of feature ambiguity for MTL patients and controls. Note that the patients were intact when the controls made no errors but were impaired in two conditions when the controls made errors (adapted from Barense et al. 2007 [with permission from Elsevier © 2007]).

Yet, the data could also reflect quantitative differences in memory load across the three conditions. To illustrate this possibility, the data from Barense et al. (2007) have been redrawn in Figure 4 (right) to conform to Figure 3A,B. The idea is that, even though stimuli were presented simultaneously, information nonetheless must be retained in memory as attention shifts back and forth between the stimuli. In the minimum condition, only single features must be retained (and sometimes they are easily identified, as in Fig. 4, left, for the minimum condition). In contrast, in the other two conditions, conjunctions of features must be retained, and memory is required to keep track of the search process as items are identified as pairs. Indeed, the fact that controls exhibited errors only in these more difficult conditions (Fig. 4, right) suggests that working memory was not sufficient to support performance, that is, immediate memory capacity may have been exceeded in the two more difficult conditions. Accordingly, lesions of the perirhinal cortex might sometimes impair performance on tasks like these, not because of demands on perception but because of demands on memory.

To decide between these two interpretations (i.e., impaired perception vs. impaired memory), one could hold the level of feature-ambiguity constant while gradually increasing the demands on memory (from within capacity to above capacity). For example, one could present three-, five-, or seven-item arrays, each with intermediate feature ambiguity. If the degree of feature ambiguity is critical, then an impairment should be evident even when the memory load is small and controls make no errors. Alternatively, if memory load is critical, then an impairment should be evident only when the task is more difficult (e.g., with seven-item arrays but not with arrays of three or five items) and only at the stage where controls first exhibit errors. Note that this procedure could determine whether a supposed impairment in complex visual perception after MTL lesions should, instead, be attributed to a memory impairment. However, this procedure might not identify what component of memory is impaired (i.e., working memory or long-term memory).

Summary of the patient data

The preceding sections consider a number of studies in which patients with MTL lesions were impaired, either after quite brief retention intervals or in cases when there is no retention interval at all (e.g., judgments of simultaneously presented items). These studies have often been interpreted to mean that MTL lesions impair immediate memory (or working memory) and, in some circumstances, perception itself. We suggest an alternative perspective, namely, that most, if not all, of these studies in fact make a significant demand on long-term (or supraspan) memory. In some cases, controls have an opportunity to learn about the stimulus material as the task progresses, thereby gaining an advantage over memory-impaired patients. In other cases, the amount of test material presented likely exceeds what can be held within immediate memory. In this circumstance, controls gain an advantage by drawing on their intact capacity for long-term memory (also see Baddeley et al. 2010, 2011; Brady et al. 2011 for a similar point). We suggest two methods to help resolve the different interpretations, one suited for retention intervals of several seconds (Shrager et al. 2008) and another suited for shorter retention intervals (0–1 sec) (Jeneson et al. 2010).

The following sections consider data from neuroimaging studies which, like the patient data, have also figured prominently in discussions of the MTL and working memory.

MTL activity in imaging tasks involving short delays

MTL activity is not typically observed in imaging studies that assess activity during working memory tasks (e.g., Courtney et al. 1996, 1997; Cohen et al. 1997; Rypma et al. 1999; Cabeza and Nyberg 2000; Wager and Smith 2003; Todd and Marois 2004; Xu and Chun 2006). It is, therefore, interesting that some recent studies involving complex visual stimuli, such as faces and photographs of scenes, have reported MTL activity in association with short-delay recognition memory tasks (Mitchell et al. 2000; Ranganath and D'Esposito 2001; Stern et al. 2001; Schon et al. 2004, 2009, 2010; Ranganath et al. 2005; Nichols et al. 2006; Piekema et al. 2006, 2009; Axmacher et al. 2007; Hannula and Ranganath 2008; Olsen et al. 2009; Lee and Rudebeck 2010; Toepper et al. 2010). In this section, we consider findings like these and their possible interpretation.

MTL activity is influenced by memory load

In patient studies where MTL damage impaired performance after short retention delays, the task requirements often made substantial demands on long-term memory and exceeded what could be managed within working memory. It is, therefore, noteworthy that the majority of imaging studies where MTL activity was observed during a short-delay task also made large demands on memory. For example, in one study, participants were asked to form a mental image of the locations of four objects in a 3 × 3 grid, to mentally rotate the image 90°, and then to maintain the rotated image across an 11-sec delay (Hannula and Ranganath 2008). In another study, participants attempted to maintain three trial-unique face-face pairs, three trial-unique house-house pairs, or three trial-unique face-house pairs across an 8–20-sec delay period (Piekema et al. 2009). In other studies where MTL activity was observed during the maintenance of only one or two items, the tasks required the maintenance of complex visual items that are difficult to rehearse, such as faces (Ranganath and D'Esposito 2001; Nichols et al. 2006) or three-dimensional geometrical shapes (Ranganath et al. 2005). Thus, it is possible that the MTL activity in these studies occurred because the memory load exceeded immediate memory capacity, and performance depended in part on long-term memory.

Indeed, in studies that directly assessed the effect of memory load on MTL activity during short-delay tasks, MTL activity was enhanced for tasks involving greater demands on memory. Thus, Axmacher et al. (2007) noted increased activity in the left hippocampus during encoding and maintenance of four trial-unique faces, but not during the encoding or maintenance of one or two trial-unique faces (Fig. 5). In addition, activity in the left anterior hippocampus and subiculum was greater during retrieval of four novel scenes than during retrieval of two novel scenes (Schon et al. 2009). Subsequently, this effect of memory load was also observed in entorhinal cortex and perirhinal cortex during the delay period of the same task (Schon et al. 2010). In another study, enhanced MTL activity with greater memory load was observed when participants tried to remember complex rather than simple stimuli (Lee and Rudebeck 2010). In addition, for the complex stimuli, activity was greater when participants performed a two-back task than when they performed a one-back task. The two-back task (high memory load condition) required detection of stimulus repetitions that were separated by an intervening stimulus. The one-back task (low memory load condition) simply required detection of successive stimulus repetitions. These results were interpreted as reflecting a role for the MTL in complex spatial processing or perception as well as in working memory.

Figure 5.

Activation in left hippocampus during encoding (white bars) and maintenance (black bars) of one, two, or four faces. Activation increased as memory load increased. Brackets show SEM (adapted from Axmacher et al. 2007 [with permission from the Society for Neuroscience © 2007]).

Yet, there are other ways to understand such data. First, as already mentioned, it is possible that the high memory load conditions in these studies placed too great a burden on working memory, so that performance, in fact, depended in part on long-term memory. Second, activity detected at the time of stimulus presentation, or shortly thereafter, could reflect incidental encoding into long-term memory. Indeed, encoding into long-term memory is a relatively automatic and continuous process (for similar suggestions about neuroimaging findings obtained near the time of stimulus presentation, see Ryan and Cohen 2004; Zarahn et al. 2005; Olsen et al. 2009). In one study, MTL activity associated with the foils presented during a recognition test predicted subsequent performance on a second, surprise recognition test that assessed long-term retention of the foils (Stark and Okado 2003). There is, in fact, an abundant literature demonstrating that MTL activity at the time of learning can predict subsequent long-term memory performance (see Paller and Wagner 2002). The next section considers the relevance of this literature to neuroimaging studies of working memory.

MTL activity during and after learning as a predictor of long-term memory

As described earlier, Lee and Rudebeck (2010) found enhanced MTL activity when participants tried to remember complex stimuli rather than simple stimuli. In a separate behavioral study, they also found that subsequent recognition memory was better for complex images—the same condition that elicited the greatest MTL activity in the corresponding fMRI experiment when the material was studied (Fig. 6). This result raises the possibility that MTL activity observed during learning is related, not to working memory or other online processes, but to the formation of long-term memory.

Figure 6.

(A) Activity in right hippocampus during one-back trials (white bars) and two-back trials (black bars) with simple and complex spatial images (see text). A similar pattern of activity was observed in right parahippocampal cortex. (B) Eight individuals took the same test as in the fMRI experiment. After a 10-min filled delay, they then took a surprise test of long-term retention for the stimuli presented during the task. The patterns in A and B are not identical, but it is noteworthy that the different conditions of learning (simple vs. complex material; one-back vs. two-back testing) had similar effects on hippocampal activity during learning and on long-term behavioral memory. Brackets show SEM (from Lee and Rudebeck 2010 [with permission from the Massachusetts Institute of Technology © 2010]).

Three other studies obtained similar results. Thus, activity in the parahippocampal gyrus was associated with maintenance of novel photographs across a 10-sec delay (Schon et al. 2004), activity in the anterior hippocampus was associated with maintenance of complex geometrical shapes early during a 7–13-sec delay period (Ranganath et al. 2005), and activity in the hippocampus was associated with maintenance of a single face across a 7-sec delay (Nichols et al. 2006). In each of these cases, the MTL activity that occurred while maintaining information in memory was correlated with subsequent measures of long-term memory. In addition, in a fourth study, which required the encoding and maintenance of four face-house pairs, long-term memory success was predicted by hippocampal activity at the time of encoding (Bergmann et al. 2010). However, performance on memory tests that were interleaved during the encoding phase and that involved short retention intervals was not associated with MTL activity.

The finding of a correlation between activity during encoding and subsequent long-term memory does not, of course, exclude the possibility that the fMRI signal in such cases contains additional information related to working memory itself (Ranganath et al. 2005; Lee and Rudebeck 2010). Nonetheless, it is striking that the extent of MTL activity observed at the time of encoding is influenced by the demands on memory imposed by the task and also that the activity observed at the time of encoding correlates positively with long-term retention of the material that was presented. These observations provide only weak support for interpreting MTL activity during short-delay tasks as a reflection of the operation of working memory. Instead, this activity is more likely to reflect processes related to the formation of long-term memory.

If the MTL does not support working memory, what brain structures and brain systems are involved? A long tradition of work has identified the importance of prefrontal cortex and the cortical association areas that are involved in perceptual processing.

Cells in prefrontal cortex are maximally active during the delay portion of the delayed-response task (Fuster and Alexander 1971). This finding and much subsequent work linked the prefrontal cortex to what was initially termed short-term memory and, in later elaborations, working memory (Goldman-Rakic 1995; Fuster 2008). One view is that the prefrontal cortex supports working memory by directing attention to task-relevant sensory signals (Postle 2006). From this perspective, retention of information in working memory is supported by sustained activity in the various brain areas that process perceptual information (Jonides et al. 2005; Pasternak and Greenlee 2005; Postle 2006). For example, short-term retention (working memory) of visual stimuli was associated with sustained activity in inferotemporal cortex (Fuster and Jervey 1982). In addition, working memory for motion direction was associated with sustained activity in area MT (Bisley and Pasternak 2000; Bisley et al. 2004), and working memory for faces was associated with sustained activity in posterior fusiform gyrus (Ranganath et al. 2004). In studies of the capacity limit for visual working memory, activity in the intraparietal sulcus and regions of occipital cortex increased up to an array size of three or four visual objects and leveled off at the point where capacity was reached (Todd and Marois 2004, 2005; Vogel and Machizawa 2004; Xu and Chun 2006). Thus, working memory is a collection of temporary capacities intrinsic to “information processing” subsystems and are under top-down control by the prefrontal cortex.

Summary and conclusion

Recent neuropsychological and imaging literature has led to suggestions that the MTL may be important for immediate memory, working memory, and, perhaps, aspects of visual perception. This perspective challenges the historical view that these functions are independent of the MTL. Discussion of these ideas has often focused on a distinction between tasks with short retention intervals (a few seconds) and tasks with longer retention intervals. The terms “short-term memory” and “long-term memory” have commonly been used to make this distinction. Yet, questions about the possible role of the MTL in immediate memory and working memory do not turn on any particular retention interval. Instead, the important distinction is between tasks where the material to be learned and maintained is within the capacity of immediate memory (subspan material) and tasks where what is to be learned exceeds immediate memory capacity (supraspan material). When immediate memory capacity is exceeded, or when material must be retrieved following the redirection of attention, performance must depend on a stable memory store (“long-term memory”) that permits the organization and retrieval of large amounts of information. Immediate memory and working memory, in contrast, deal “only with subspan memoranda, evanescently, as long as the subject's attention is directed toward the memorandum” (Drachman and Arbit 1966, p. 59). These ideas are better captured by the terms “subspan memory” and “supraspan memory” than by the terms “short-term memory” and “long-term memory.”

A reappraisal of recent findings in light of the concepts of subspan and supraspan memory suggests a parsimonious and consistent perspective by which to understand the patient data as well as the neuroimaging data. Many of the tasks that have been used make a significant demand on long-term (or supraspan) memory. In tasks where working memory (or subspan memory) alone was sufficient to support performance, patients performed as well as controls regardless of the kind of material to be held in mind. The story that emerges is not that some kinds of working memory depend on the MTL, but rather that some kinds of short-delay tasks depend on long-term (supraspan) memory.