The role of the hippocampus in retaining relational information across short delays: The importance of memory load

Abstract

Patients with hippocampal damage are sometimes impaired at remembering information across delays as short as a few seconds. How are these impairments to be understood? One possibility is that retention of some kinds of information is critically dependent on the hippocampus, regardless of the retention interval and regardless of whether the task depends on working memory or long-term memory. Alternatively, retention may be dependent on the hippocampus only when the task involves a memory load large enough to exceed working memory capacity. To explore these possibilities, we assessed the performance of patients with hippocampal lesions on two tasks requiring retention of the same object-in-scene information across a brief delay. The tasks placed different demands on memory. In one task, which used a continuous recognition format, participants needed to try to hold up to nine scenes in mind, even when no scene intervened between the study scene and the corresponding test scene. Patients were impaired in this condition. In a second task, using a conventional study-test format, participants needed to hold in mind only one scene at a time for either 3 or 14 sec. With this procedure, patients performed as well as controls after a 3-sec delay but were impaired after a 14-sec delay. We suggest that retention of object-in-scene information is dependent on the hippocampus only when working memory is insufficient to support performance (because memory load is high or the retention interval is long). In these circumstances performance depends, at least in part, on long-term memory.

Working memory refers to the ability to hold a limited amount of information actively in mind, usually across a brief time interval (Baddeley 2003). Early studies of memory-impaired patients with medial temporal lobe (MTL) damage, including the noted patient H.M., found this ability to be spared despite their severe impairment in long-term memory (Drachmann and Arbit 1966; Baddeley and Warrington 1970; Milner 1972; Cave and Squire 1992; Squire 2009). The principle that emerged from these investigations was that working memory (sometimes termed short-term memory) is independent of the hippocampus and adjacent MTL structures. It is therefore notable that a number of recent studies have reported that patients with MTL damage can be impaired at remembering information across quite brief time intervals (Hannula et al. 2006; Nichols et al. 2006; Olson et al. 2006a,b; Hartley et al. 2007; Kan et al. 2007; Piekema et al. 2007; Bird and Burgess 2008; Ezzyat and Olson 2008; Finke et al. 2008). In addition, functional magnetic resonance imaging (fMRI) studies have reported MTL activation during short-delay recognition memory tasks (Ranganath and D'Esposito 2001; Schon et al. 2004; Ranganath et al. 2005; Nichols et al. 2006; Piekema et al. 2006, 2010; Axmacher et al. 2007; Hannula and Ranganath 2008; Toepper et al. 2010). These observations have raised the possibility that working memory may sometimes depend on the MTL.

While the emerging fMRI literature does implicate a role for the MTL in some tasks involving brief retention intervals, it is less clear whether the findings implicate a role for the MTL in working memory. For example, it has been noted that MTL activity may reflect incidental encoding of or rehearsal of novel items in support of long-term memory rather than activity needed for active maintenance of information in working memory (e.g., Ryan and Cohen 2004; Zarahn et al. 2005; Olsen et al. 2009). If the fMRI findings can be understood without requiring a role of the MTL in working memory, how then should we understand the striking observations of impaired performance after MTL damage in tasks with delays as short as a few seconds?

In one study that explored recognition memory, patients with hippocampal damage performed well at remembering scenes, but were impaired at remembering information about the locations of objects in a scene. The impairment in object-location information was evident even at the briefest delay when no stimuli intervened between study and test (i.e., memory was impaired even at a lag of one item in a continuous recognition test format) (Hannula et al. 2006). One possibility is that maintenance of information about objects in a scene is critically dependent on the MTL, regardless of whether performance must span short or long delays and regardless of whether performance depends on working memory or long-term memory (Hannula et al. 2006). If so, the nature of the task (e.g., object-in-scene memory vs. scene-only memory) might be more fundamental for understanding hippocampal function than the classical distinction between working memory and long-term memory.

It is also possible that impaired memory performance across brief time intervals sometimes occurs because working memory capacity has been exceeded. Two important factors that influence working memory capacity are the amount of information that can be held in mind and how amenable this information is to active rehearsal. These considerations imply that the capacity of working memory can sometimes be exceeded, even at short retention intervals. For example, when presented with 10 word pairs and asked for immediate recall, memory-impaired patients recall many fewer word pairs than controls (zero or one pair vs. five or six pairs). Perhaps (as considered by Hannula et al. 2006), an impairment in object-in-scene memory at a lag of one item could mean that the capacity of working memory was exceeded. Indeed, even at a lag of one item, participants in the earlier study still needed to try to hold in mind a number of previous scenes (up to nine), because the decision to identify each item as old or new depended on as many as the previous nine items. In addition, there was a delay of 14 sec between the initial presentation of a study scene and the assessment of memory for object location.

We conducted two experiments. In the first experiment, we used the same procedure as was used previously (Hannula et al. 2006). In the second experiment, we used a conventional test paradigm consisting of successive study-test trials that involved either a brief (3-sec) or a relatively long (14-sec) retention interval. For this test procedure, participants were required to hold in mind only one scene at a time. If maintenance of object-in-scene information is critically dependent on the hippocampus, hippocampal damage should impair performance at the 3-sec retention interval, even in a conventional test paradigm.

Five memory-impaired patients participated (four men) (Table 1), all of whom have bilateral lesions thought to be limited to the hippocampus (CA fields, dentate gyrus, and subicular complex). Detailed descriptions of these patients are provided in Jeneson et al. (2010a). Estimates of medial temporal lobe damage were based on quantitative analysis of magnetic resonance (MR) images (Bayley et al. 2005; Gold and Squire 2005). Nine coronal magnetic resonance images from each patient, together with detailed descriptions of the lesions, are presented as supplemental material (Supplemental Fig. 1, available online). Nine controls also participated (eight males; mean age, 58.4 years; mean education, 14.0 years).

Table 1.

Characteristics of memory-impaired patients

WAIS-III is the Wechsler Adult Intelligence Scale-III and the WMS-R is the Wechsler Memory Scale-Revised. The WMS-R does not provide numerical scores for individuals who score <50. IQ scores for J.R.W. and R.S. are from the WAIS-R.

The procedure for Experiment 1 was based on an earlier study of memory for objects in scenes (Hannula et al. 2006, Experiment 1), as described in Figure 1. The stimuli consisted of 48 unique scenes (800 × 600 pixels) created using Punch! Home Design software (kindly provided by Dr. Hannula). Two versions of each scene were available (for a total of 96 stimuli): the original scene and a manipulated version of the scene in which the spatial relations among scene elements were changed (e.g., an urn was moved from the left side to the right side of the room) (Fig. 1). Participants completed two blocks of trials in a continuous recognition format. Each block consisted of 24 study trials (the original scenes) interleaved with 24 test trials (12 scenes that were repeated versions of the originally studied scenes and 12 that were manipulated versions of the scenes). There were two different pseudorandom orders of scenes for each block of 48 trials. For each participant, (1) an equal number of repeated and manipulated probe trials were presented at each lag, (2) the critical item was equally often at the left or right in the scene, and (3) at each lag, the orienting question was equally likely to be associated with a “yes” (or “no”) response.

Figure 1.

(A) Experiment 1. Repeated (match) and manipulated (nonmatch) test trials were interleaved systematically among study trials. Test trials appeared either immediately after the corresponding study trial (lag 1), five trials later (lag 5), or nine trials later (lag 9). The task for each scene was to decide whether the scene had appeared earlier in the series, and then, critically (in the case of a “yes” response), whether any items in the scene had changed location. Note that, even for tests at a lag of 1, participants had to try to hold in mind previous scenes because they did not know whether the memory question would concern the most recently presented scene or a scene presented up to nine items earlier. (B) Two trials from A illustrating a lag of 1. Each scene was presented for a total of 20 sec. The scene was first presented alone for 5 sec. For the next 6 sec the scene was presented along with an orienting question that drew the participant's attention to the item in the scene that will be moved or not moved (e.g., “Is the urn directly under the mirror?” [No]). Participants were not told that the orienting question identified the item that would be relevant to the memory decision. (Whenever a scene was presented a second time, the answer to the orienting question was always the same as it was when the scene was first presented. Accordingly, the answer to the orienting question did not provide information about whether the scene had been altered or not.) For the remaining 9 sec of the trial the scene was accompanied by the two memory questions (“Have you seen this scene before?” and “Have any items changed location?”). Note that 14 sec elapsed (3 + 5 + 6 sec) between the removal of a novel scene and the first (Old/New) memory question for the next scene.

The patients were modestly impaired at answering the first yes/no question about whether a scene had been presented earlier in the block (Table 2). Overall, the patients scored 88.3 ± 3.5% correct at classifying the scene as old or new, and the controls scored 99.6 ± 0.2% correct (t_(4.0) = 3.26, P = 0.03, unequal variance t-test). The result was the same when the data were analyzed using d′ scores (2.8 vs. 4.1 for patients and controls, respectively; t_(4.0) = 6.13, P = 0.004, unequal variance t-test).

Table 2.

Performance on tests of scene memory and location memory in Experiment 1

Mean percent hits (and SEM) at each lag, overall percent correct rejections (CR) across lags, and CRs at each lag for location memory. CRs cannot be calculated for scene memory because novel items are presented only once and have no “lag.” Overall percent correct is mean percent correct across lags (overall hit rate + overall CR rate/2).

^aSignificantly poorer than controls, P < 0.05.

The patients were also impaired at answering the second yes/no question about whether any items in the scene had changed location (Table 2; Fig. 2A).⁸ An analysis of variance (ANOVA) for lag (1, 5, or 9) and group (patients with hippocampal lesions vs. controls) revealed an effect of group (F_(1,12) = 26.2, P < 0.001) as well as an effect of lag (F_(2,24) = 6.9, P = 0.004) but no lag–group interaction (F_(2,24) = 1.9, P = 0.17). The hippocampal patients were impaired at all three lags (ts₍₁₂₎ > 2.8, Ps < 0.02). The result was the same when the data were analyzed using d′ scores (patients, 1.60, 0.93, and 0.49; controls, 2.52, 2.10, and 2.20, for lags of 1, 5, and 9, respectively; all ts₍₁₂₎ > 2.6, Ps < 0.03).

Figure 2.

(A) Experiment 1. Performance of memory-impaired patients with hippocampal lesions (gray bars, n = 5) and matched controls (white bars, n = 9) on a test of memory for spatial relations among items in a scene. The patients were impaired at all lags. Nevertheless, they did score above chance at a lag of 1 and at a lag of 5 (ts₍₄₎ > 3.3, Ps < 0.02), and they scored marginally above chance at a lag of 9 (t₍₄₎ = 2.30, P = 0.08). Error bars indicate SEM. (B) Experiment 2. Performance of memory-impaired patients with hippocampal lesions (gray bars, n = 5) and matched controls (white bars, n = 9). The patients performed well when memory was tested after a short (3-sec) delay, but they were impaired when memory was tested after a longer (14-sec) delay. Despite the marked impairment, patient performance was above chance at the 14-sec delay (t₍₄₎ = 8.8, P < 0.001). Error bars indicate SEM.

Experiment 2 was based on Experiment 1, but used a more conventional procedure to test memory for scenes (Fig. 3). The stimuli consisted of the same 48 scenes as in Experiment 1 plus 48 new scenes created by us using the same software as in Experiment 1. Ninety-six manipulated versions of each scene were also used, 48 from Experiment 1 and 48 new scenes created by us. Participants completed eight test blocks in a single session (12 study-test trials/block). The study-test delay was the same (3 or 14 sec) for all 12 trials in a block, and the delay alternated from block to block. The scene presented after the delay was the same as the studied scene on half of the trials and was altered on half of the trials. Two different versions of the test were available so that, across participants, each scene was equally likely to be repeated or manipulated. In addition, across participants, each scene was equally likely to be tested after a 3- and 14-sec delay. The interval between Experiment 1 and Experiment 2 was at least 6 wk (mean = 22.6 wk). Experiment 1 always preceded Experiment 2.

Figure 3.

Experiment 2. A sample study-test trial. Participants first studied the scene to the left for 5 sec and then were asked an orienting question (“Is the plant on the table?” [No]). After viewing the scene for a total of 10 sec, a delay of 3 or 14 sec intervened before either a matching or nonmatching scene was presented together with a memory question (“Have any items changed location?”). Only this question was asked. In the nonmatching scene illustrated here, the plant has moved from left to right.

The patients performed well at the 3-sec delay when deciding whether any items had changed location, but they were impaired at answering the same question after the 14-sec delay (Fig. 2B) (3-sec delay: t_(4.1) = 1.3, P = 0.28, unequal variance t-test; 14 sec delay: t_(5.0) = 3.9, P = 0.01, unequal variance t-test). As in Experiment 1, trials were scored only when the participant gave a correct response to the orienting question (both hippocampal patients and controls scored 97.1% correct).

The fact that the group means differed at all at the brief (3-sec) delay was due to one patient who scored 85.0% correct. The mean score of the other four patients was 96.3% (controls = 97.4% correct). The results were the same when the data were analyzed using d′ scores (at 3 sec, patients 3.3, controls 3.7, t₍₁₂₎ = 1.5, P = 0.15; at 14 sec, patients 2.1, controls 3.4, t₍₁₂₎ = 4.4, P < 0.001).

Experiment 1 replicated the results of an earlier study that used the same continuous recognition procedure (Hannula et al. 2006). The impairment observed with this procedure raised the possibility that the hippocampus is sometimes critical for maintaining object-in-scene information, regardless of whether the task depends on working memory or long-term memory (Hannula et al. 2006; see Olson et al. 2006b; Finke et al. 2008 for similar suggestions about other kinds of tasks). An alternative possibility, also considered by the authors of the earlier study (Hannula et al. 2006), is that impaired memory for object-in-scene information across brief time intervals sometimes occurs because performance under those conditions depends on long-term memory (also see Ryan and Cohen 2004 for a similar suggestion).

We reasoned that the continuous recognition procedure used in the earlier study (Hannula et al. 2006) and in Experiment 1 might indeed depend on long-term memory because of the requirement that, even for memory decisions made after a lag of 1 item, participants nonetheless needed to try to be holding in mind as many as nine earlier items. That is, participants were not only holding in mind information about the immediately preceding item, they also needed to try to hold in mind information about the previous nine items. To test this idea, we constructed a test (Experiment 2) in which the same object-in-scene information needed to be retained over the same brief delay. However, in this case only a single item was presented. Although patients were impaired at a brief delay in Experiment 1, they performed well in Experiment 2. The key difference in the two conditions was that in Experiment 1 participants needed to try to carry a memory load involving the previous nine items, but in Experiment 2 they needed to remember only a single item.

Note that object-in-scene memory (in Experiment 1) was impaired, but that scene memory itself (in Experiment 1) was less impaired. It seems reasonable to suppose that the burden on working memory of maintaining object-in-scene information was greater than the burden of maintaining scene-only information. In the case of object-in-scene memory, it was necessary to remember specific details about each scene, whereas scene-only memory required only that participants remember the general appearance of each scene.

It is notable that visual working memory is quite limited in capacity (Luck and Vogel 1997; Cowan 2001; Wheeler and Treisman 2002; Alvarez and Cavanagh 2004). Typically, in healthy young adults, only three to four simple visual objects can be maintained (Cowan 2001; Fukuda et al. 2010). Accordingly, in Experiment 1, when as many as nine items involving complex scenes needed to be held in mind for successful performance, it is reasonable to suppose that the task exceeded working memory capacity and that performance depended, at least in part, on long-term memory.

In several studies, patients with bilateral medial temporal lobe damage have been found to be impaired at remembering visual information over brief delays where no stimuli intervene between study and test. In some cases, impairments were observed after a delay as short as 1 or 2 sec. Specifically, impairments have been noted in memory tasks for three object–location associations after 1 and 8 sec (Olson et al. 2006b), for topographical scene information after 2 sec (Hartley et al. 2007), and for a single face after 1 sec (Ezzyat and Olson 2008, in a forced-choice task). The question of interest is whether the impairments found after short delays reflect impaired working memory or, if working memory capacity has been exceeded, an impairment in long-term memory.

This issue has been addressed in the case of retention intervals of 8 sec or longer (Shrager et al. 2008) as well as in the case of a retention interval as short as 1 sec (Jeneson et al. 2010b). Shrager et al. (2008) found concordance between the performance of patients with medial temporal lobe damage and the effect on control performance of distraction between study and test. It was assumed that distraction would be disruptive when performance depended on maintaining information in working memory. The finding was that the patients were intact on tasks in which distraction disrupted control performance, suggesting that the patients were successful when they could maintain information in working memory. In contrast, the patients were impaired on tasks in which distraction minimally affected control performance, suggesting that the patients failed when performance depended significantly on long-term memory.

To explore memory for object–location associations across delays as short as 1 sec, Jeneson et al. (2010b) drew on a method suggested by the classic study of digit span in patient H.M. (Drachman and Arbit 1966). The task required participants to maintain up to seven object–location associations across a 1-sec delay. Patients with medial temporal lobe damage performed similarly to controls when only a small number of object–location associations needed to be remembered, but they exhibited an abrupt decline in performance when as many as three to four object locations needed to be remembered. The marked discontinuity in patient performance as they moved from smaller to larger set sizes occurred at about the same set size that first produced errors in controls. Presumably, controls began making errors at this point because the material now exceeded their working memory capacity. These findings suggested that maintenance of relational information in working memory is intact in patients with medial temporal lobe damage. The patients were impaired only when the task exceeded working memory capacity such that long-term memory now benefitted performance. This method may have a useful application to other reports of impaired performance after short retention intervals.

The results of the current study are consistent with the findings of Shrager et al. (2008) and Jeneson et al. (2010b). The challenge has been to understand the impairment associated with medial temporal lobe lesions that can be observed in some tasks after brief delays. It has been suggested that the ability to form new associations is an important factor, regardless of the retention interval (Ranganath and Blumenfeld 2005; Hannula et al. 2006; Olson et al. 2006b; Finke et al. 2008). Memory load is also an important factor. A question then is whether the medial temporal lobe is required in some tasks regardless of how much material needs to be maintained (i.e., memory load), or whether the medial temporal lobe is required only when the memory load is large enough (or the retention interval long enough) such that long-term memory now benefits performance. We reasoned that if retention of object-in-scene information is critically dependent on the medial temporal lobe, then hippocampal damage should impair performance even when the memory load is minimal (and the retention interval is short). We assessed the performance of patients with hippocampal lesions on two tests requiring retention of the same information across a brief delay. The patients were intact on the test where the memory load was minimal and the retention interval was short, but they were impaired on the test where the memory load was greater (Experiment 1) or when the retention interval was long (Experiment 2). We suggest that retention of object-in-scene information is dependent on the hippocampus only when working memory is not sufficient to support performance, and performance depends, at least in part, on long-term memory.