Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Feb 1.
Published in final edited form as: Hippocampus. 2016 Dec 30;27(2):184–193. doi: 10.1002/hipo.22682

Visual short-term memory for high resolution associations is impaired in patients with medial temporal lobe damage

Joshua D Koen 1,2, Alyssa A Borders 3, Michael T Petzold 3, Andrew P Yonelinas 3,4
PMCID: PMC5235973  NIHMSID: NIHMS830338  PMID: 27859914

Abstract

The medial temporal lobe (MTL) plays a critical role in episodic long-term memory, but whether the MTL is necessary for visual short-term memory is controversial. Some studies have indicated that MTL damage disrupts visual short-term memory performance whereas other studies have failed to find such evidence. To account for these mixed results, it has been proposed that the hippocampus is critical in supporting short-term memory for high resolution complex bindings, while the cortex is sufficient to support simple, low resolution bindings. This hypothesis was tested in the current study by assessing visual short-term memory in patients with damage to the MTL and controls for high resolution and low resolution object-location and object-color associations. In the location tests, participants encoded sets of two or four objects in different locations on the screen. After each set, participants performed a two-alternative forced-choice task in which they were required to discriminate the object in the target location from the object in a high or low resolution lure location (i.e., the object locations were very close or far away from the target location, respectively). Similarly, in the color tests, participants were presented with sets of two or four objects in a different color and, after each set, were required to discriminate the object in the target color from the object in a high or low resolution lure color (i.e., the lure color was very similar or very different, respectively, to the studied color). The patients were significantly impaired in visual short-term memory, but importantly, they were more impaired for high resolution object-location and object-color bindings. The results are consistent with the proposal that the hippocampus plays a critical role in forming and maintaining complex, high resolution bindings.

Keywords: Working memory, Hippocampus, Relational memory, Precision, Binding

The medial temporal lobe (MTL), particularly the hippocampus and surrounding parahippocampal gyrus, plays a critical role in episodic long-term memory (Aggleton & Brown, 2006; Eichenbaum, Yonelinas, & Ranganath, 2007; Squire & Zola-Morgan, 1991; Yonelinas, Aly, Wang, & Koen, 2010), but whether the MTL is necessary for short-term memory is controversial. Early neuropsychological studies demonstrated preserved short-term memory in amnesic patients with MTL damage on tasks such as digit span (e.g., Scoville & Milner, 1957) and recency effects in free recall of word lists (e.g., Baddeley & Warrington, 1970). This early work, along with subsequent research (e.g., Holdstock, Gutnikov, Gaffan, & Mayes, 2000; Warrington, 1981; Warrington & Baddeley, 1974), led to the conclusion that MTL damage does not impair short-term memory. However, the above conclusion has been challenged by recent findings showing that, under certain circumstances, the MTL plays an important role in short-term memory (Esfahani-Bayerl et al., 2016; Hannula & Ranganath, 2008; Hartley et al., 2007; Olson, Moore, Stark, & Chatterjee, 2006; Olson, Page, Moore, Chatterjee, & Verfaellie, 2006; Ranganath & Blumenfeld, 2005; Ryan & Cohen, 2004; Warren, Duff, Tranel, & Cohen, 2010). For example, Olson and colleagues (2006a; 2006b) found that patients with focal hippocampal damage showed impaired short-term memory for object-location associations, but normal short-term memory for individual objects. Such findings have led to the proposal that the MTL, particularly the hippocampus, is critical for short-term memory for associative or relational information.

There are however a number of studies that failed to find short-term memory impairments for associative information in MTL patients (Baddeley, Allen, & Vargha-Khadem, 2010; Jeneson, Mauldin, Hopkins, & Squire, 2011; Jeneson, Mauldin, & Squire, 2010; Jeneson & Squire, 2012; Jeneson, Wixted, Hopkins, & Squire, 2012; Shrager, Levy, Hopkins, & Squire, 2008), which suggests that a purely associative deficit account does not sufficiently capture the conditions in which the MTL is involved in short-term memory. To reconcile the disparate findings that hippocampal or MTL damage can affect both long-term and short-term memory, Yonelinas (2013) proposed the hippocampus supports long- and short-term memory, as well as perception, when a task requires the formation or maintenance of complex, high resolution bindings. According to this hypothesis, associations between an item and other features can vary continuously along two dimensions: complexity and resolution. The complexity of an association varies from simple bindings, such as binding one item to one color, to complex bindings, such as binding multiple items with multiple features together. Moreover, the resolution of the representation can also vary from low (e.g., orange vs. blue) to high (e.g., navy blue vs. royal blue). The theory proposed by Yonelinas (2013) predicts that the hippocampus will be critical for associations that involve high resolution representations (e.g., precise color or location) whereas the hippocampus will be less involved for associations that depend on low resolution representations (e.g., orange vs. blue or left vs. right).

The purpose of the present experiment was to test the prediction that the MTL plays a pivotal role in visual short-term memory for high resolution associations. In the present study, we tested four patients with MTL damage and 12 age- and education-matched controls using a visual short-term memory task that varied the resolution requirements of associative information (see Figure 1). Participants were presented sets of trial-unique objects that varied in location or color. After a brief retention interval, participants were shown each object in the prior set and asked to retrieve the object-location or object-color association with a two-alternative forced choice task. For the location condition, the object appeared in the correct (target) location and in a different (lure) location. For the color condition, the object appeared in both the target color as well as a lure color. The resolution required to make the memory judgment was controlled by varying how similar the lure detail was to the target detail. We predicted that MTL patients would be impaired for high resolution, but not low resolution, associations. An important feature of the high resolution complex binding hypothesis is that the hippocampus supports short-term memory for high resolution associations regardless of the type of associative detail (e.g., location and color). Thus, this account predicts that hippocampal damage will impair high resolution associations for both location and color details. However, given that the hippocampus plays a pivotal role in representing spatial information (e.g., Burgess, 2002; Lee et al., 2005; O'Keefe & Nadel, 1978), it could be the case that the hippocampus supports high resolution bindings only for location. If this latter account is accurate, then we would expect our patients to show selective impairments for high resolution object-location, but not high resolution object-color bindings.

Figure 1.

Figure 1

Open in a new tab

Depiction of the experimental task. There were a location and color perception phases, as well as a four visual short-term memory (STM) phases that crossed resolution (high vs. low) with feature type (color vs. location). The phases were completed in order from A to F. In the perception tasks, participants were presented with a single object followed immediately by a dynamic mask, which was then followed by a two-alternative forced-choice decision in which participants had to select the object in the correct location or color. The procedure was similar for the STM blocks, with the exception that there were multiple objects in each set, no dynamic mask, and retention interval between the last item in the set and the first test probe. The change in the lure location or color was always more pronounced for the low resolution (set size of 4) relative to the high resolution (set size of 2) blocks.

There are two additional aspects of our design that bear introduction. First, as depicted in Figure 1, we selected a smaller set size for the high resolution conditions (set size = 2) to ensure that performance was not lower than that of the low resolution conditions (set size = 4). The reason for this was to avoid interpretive difficulties that could arise if the patients were found to be more impaired in the harder of the two conditions (i.e., the high resolution conditions). To foreshadow our results, this manipulation was successful. In fact, the healthy control participants performed slightly better in the high than the low resolution condition. This same pattern of performance (high resolution > low resolution) with the above set sizes was also observed in a behavioral pilot study in young college students conducted prior to the experiment reported here.

Second, given that the high-resolution binding hypothesis asserts that the hippocampus is critical for binding of high-resolution information within an event, it was important rule out the possibility that the patients simply had a deficit in their ability to accurately discriminate between the specific high resolution colors or locations that we used in the current experiment. To do so, we included a perception control task that was identical to the short-term memory test except that on each trial only a single object was presented, and this was followed by a very brief delay with a backward mask (102 ms). Participants were then given a two-alternative forced-choice test for the location and color that was just presented (see Figure 1A)1. Unlike the short-term memory tasks in which the color or location information had to be bound to a specific trial-unique object, the perceptual control condition only required the high resolution feature (i.e., color or location) to be perceived (and remembered).

Method

This study was approved by the Institutional Review Board at the University of California, Davis. All participants provided informed consent prior to participation and were monetarily compensated for their time.

Participants

The participants in the present study comprised 4 patients with MTL damage resulting from various etiologies (see below) and 12 healthy controls that were matched on age, [t(14) = .062, p = .951], and education, [t(14) = .522, p = .610]. The average age and years of education for the patients was 47 years (range: 30–62) and 16 years (range: 12–19), respectively. The average age and years of education for healthy controls was 48 years (range: 27–71) and 16 years (range: 13–19), respectively.

Patients

Table 1 shows the neuropsychological characteristics of each patient included in this study, and Figure 2 shows representative coronal slices illustrating the MTL damage in each patient. Each patient was administered the Shipley (Shipley, 1940), WMS-R (Wechsler, 1987), and Doors and Peoples Test (Baddeley, Emslie, Nimmo-Smith, & Thames Valley Test, 1994). The Shipley was used to estimate WAIS-R IQ (Zachary, Crumpton, & Spiegel, 1985).

Table 1.

Patient Demographics and Neuropsychological Scores

WAIS-R WMS-R (z-score) D & P
Patient Damage Age Edu Est IQ Verbal Visual Gen. Attn. Delay %
1001 HC 56 16 110 −0.87 −1.00 −1.00 1.60 −0.47 25%
1003 HC 62 12 112 −1.80 −0.27 −1.53 0.07 −2.20 1%
1005 MTL 31 19 110 −0.07 1.07 0.33 0.27 −0.40 5%
1007 MTL 43 18 106 0.87 −0.87 0.13 1.20 −0.07 10%
Open in a new tab

Notes. The individual scores are normalized scores and percentiles

Abbreviations: HC, hippocampus; PHG, parahippocampal gyrus; MTL, medial temporal lobe; Edu, education in years; D&P, Doors and People Test.

WAIS-R IQ was estimated from the Shipley (Shipley, 1940; Zachary et al., 1985).

Figure 2.

Figure 2

Open in a new tab

MRI scans for a representative healthy control and the four patients in the present study. Two patients had selective hippocampal damage (1001, 1003), and two patients had damage to the hippocampus and surrounding parahippocampal gyrus (1005, 1007).

Patient 1001 suffered from Hashimoto encephalopathy, and exhibited abnormal necrotic cavities on the left hippocampus and similar but less pronounced cavities on the right hippocampus. This patient’s cavities had a rounded shape and resembled the pathologic cavities consistent with individuals who suffered hypoxia-related CA1 necrosis (Nakada, Kwee, Fujii, & Knight, 2005). No damage was visible in any other brain region.

Patient 1003 had limbic encephalitis that resulted in bilateral hippocampal damage with no apparent damage to other regions including the surrounding MTL cortex. Grey matter volume estimates indicated that the left and right hippocampi were reduced in volume, but no other MTL structures showed significant volume reduction (estimates of MTL volumes are reported as Patient 2 in Aly, Ranganath, & Yonelinas, 2013).

Patient 1005 had a traumatic brain injury due to a car accident and suffered bilateral damage to the MTL that included damage to the perirhinal cortex, parahippocampal cortex and hippocampus. Data for the regional volumes of the different MTL regions for this patient is reported in Kolarik and colleagues (2016).

Patient 1007 suffered from viral encephalitis, resulting in encephalomalacia and extensive volume loss in the right temporal lobe and the right orbitofrontal cortex. The damage to the right temporal lobe included damage to the perirhinal cortex, the parahippocampal cortex and the hippocampus.

Materials

The materials comprised 298 black and white clip art images of objects obtained from various Internet sources (see Figure 1). All of the images were resized to 150×150 pixels, and the white background of each image outside of the object was made transparent. A total of 288 object images were assigned to be the critical stimuli in the short-term memory task, 8 objects were assigned to be practice items in the short-term memory tasks, one image was assigned to the location perception phase, and one image was assigned to the color perception phase. Note that the two images for the perception condition were resized to be 100×100 pixels. The images used as the critical stimuli, practice stimuli, and perception stimuli were the same across participants. The 288 critical objects for the short-term memory task were randomly divided into four groups of 72 objects for the low resolution location, high resolution location, and low resolution color conditions, and 70 objects to the high resolution color conditions for each participant. We intended to have 72 objects in the high resolution color condition, but an error in the stimulus presentation script that was noticed after completion of the study resulted in having only 70 objects in the high resolution color condition. The locations and colors for each object in the short-term memory phase and the trials in the perception phase were randomly determined for each participant. All stimuli were presented to participants on a gray background using PsychoPy Version 1.75.01 software (Peirce, 2007) on a 24 inch Apple iMac computer.

Location Trials

In the low resolution location condition, an object could appear in one of 8 possible locations (in pixels on the horizontal plane): −700, −500, −300, −100, 100, 300, 500, and 700. Note that the location refers to the center of each image. Objects appeared in one of the above locations during the encoding period. During the test probe, each object appeared in two locations; one was identical to the encoding location, and the other was the lure location that was 2 positions (i.e., 400 total pixels of horizontal distance) to the left or right of the encoding location. This approach allowed for 12 possible encoding/lure location pairings: −700/−300, −300/−700, −500/−100, −100/−500, −300/100, 100/−300, −100/300, 300/−100, 100/500, 500/100, 300/700, and 700/300. The twelve pairings were randomly assigned to each object with the constraint that each pairing was used equally often. This resulted in 6 trials of each target/lure pairing. After assigning the locations, the 72 objects were randomly divided into 18 sets of 4 objects. Within each group, the order of the encoding and test presentations was randomly determined for each set of 4 objects.

The objects in the high resolution location condition were allowed to appear in any location between −700 to 700 pixels. Each object was assigned a random location for the encoding period from the range described above (with 1 pixel increments), and the lure location was determined as 150 pixels to the left or right of the encoding location. Before randomly determining the encoding location, 50% of the trials were assigned to have the lure appear left (−150 horizontal pixels) of the encoding location with the other 50% assigned to have the lure appear to the right (+150 horizontal pixels) of the encoding location. However, the direction of the lure location was switched from left to right (or vice versa) if the lure location was outside of the range of possible locations. For example, if the encoding location was within 150 pixels of the left endpoint (e.g., −600 pixels), and the lure location was assigned to appear to the left of the encoding location, which would have resulted in a location of −750 pixels, then the lure location was switched to the right of the target location (e.g., −450 pixels). This was done to avoid completely novel lure locations. After the encoding and lure locations were assigned, the 72 objects were randomly divided into 36 sets of 2 objects, and the ordering of the encoding and test presentations were randomly determined for each of the 36 sets.

Color Trials

To create the colored objects, all of the white pixels in the object were replaced with a color determined using Hue-Saturation-Value (HSV) coordinate system. The saturation and value coordinates were constant for each image and both were set at .75. Thus, the only parameter that varied between the colors was the hue which ranged from 0°–359°.

In the low resolution color condition, the objects appeared in one of 8 possible colors (in hue values): 0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°. Objects appeared in one of the above colors during the encoding period, and the lure color was determined by shifting the encoding hue ±90°. The 12 encoding/lure color pairings were (in hue values): 0°/90°, 90°/0°, 45°/135°, 135°/45°, 90°/180°, 180°/90°, 135°/225°, 225°/135°, 180°/270°, 270°/180°, 225°/315°, and 315°/225°. The twelve pairings were randomly assigned to each object with the constraint that each pairing was used equally often, resulting in 6 trials of each target/lure pairing. Similar to the low resolution location trials, the 72 objects assigned to the low resolution location condition were randomly divided into 18 sets of 4 objects, and the order of the encoding and test presentations was randomly determined for each set of 4 objects.

The objects in the high resolution condition could be assigned a hue value between 0° and 359°. Each object was assigned an encoding color in the range just described (in 1° increments), and the lure color was determined to be ±25° of the encoding color. Half of the trials were assigned to have the color change by +25° and the other half were assigned to have the color change by −25°. However, if the lure location crossed the 360° value, the direction of the change was reversed. For example, if the encoding color of an objects was assigned a hue of 5°, and the lure color was determined by changing the color by −25° (resulting in a hue of 340°), then the direction of the change in hue was reversed. The reason for this was to have the high resolution color condition mimic the high resolution location condition. Finally, the 70 trials in the high resolution color were condition randomly divided into 35 sets of 2 objects, and, for each group, the ordering of the encoding and test presentations were randomized.

Practice Trials

The assignment of locations and colors for the 8 practice objects was nearly identical to that described above. However, unlike the critical stimuli, the 8 objects appeared in each of the four conditions outlined above. For both the location and color condition, the 8 objects were randomly assigned to 2 sets of 4 objects and 4 sets of 2 objects for the low and high resolution conditions, respectively.

Perception Trials

The target location for the object in the location perception condition was selected in the same fashion as described above for the high resolution location condition. However, the lure location was determined as ±90 horizontal pixels from the location condition. As with the high resolution location condition, if the lure location was outside of the acceptable range of locations, the direction of the lure location was switched. The target color for the object in the color perception was selected in the same manner as the high resolution color condition describe above. However, the lure color was determined by changing the hue of the encoding color by ±15. Again, if the change in the hue crossed 360°, the direction of the change was reversed. A total of 54 target/lure locations and colors were generated for the objects in the location and color perception condition. For each condition, six pairs were assigned as practice trials, and 48 were assigned to be critical trials.

In addition, 3 black-and-white and 3 color checkerboard images were created and used as backward masks in the perception condition. The images were 1600 × 400 pixels. The black-and-white checkerboards were created by randomly assigning pixels to be black or white. The color checkerboard images were created in a similar manner, but the pixels were randomly assigned a color using the same HSV parameters described above.

Procedure

Each participant completed the three main blocks of the experiment in the following order: (1) perception, (2) low resolution short-term memory (with a set size of 4), and (3) high resolution short-term memory (with a set size of 2). Within each of the three main blocks, the location condition occurred first and the color condition occurred second (see Figure 1). Note that for each of the six total phases participants first completed a short practice list before the critical list was presented.

Before each phase, participants were instructed that they would be shown a series of objects and their task was to remember the location or color each object was presented in. For the perception phases, a single object was initially presented for 102 ms and followed by a 102 ms dynamic mask (34 ms presentation of each of the three black-and-white or color mask images for the location and color phases, respectively). For the location condition, the initial object was presented in the target location along the horizontal axis centered along the vertical axis. The initial object in the color condition were presented centrally on the screen in the target color. Immediately after the dynamic mask, participants were administered a two-alternative forced-choice (2AFC) location/color decision where the object appeared in two locations or colors one the screen, with one being in the target location or color and the other being in a lure location or color. The words “left” and “right” appeared below the two objects on the screen. In the color condition the target and lure colored objects were presented −125 and 125 pixels left and right of the center of the screen, and the object in the target color appeared equally often on the left and right. Participants were instructed to select the object that was in the same location or color using the left and right arrow keys. It is important to point out that for the location condition the left and right responses did not correspond to the side of the screen the object was presented in, but the relative location of the two alternatives. The responses in the test period were self-paced. After a response was entered, there was a 500 ms fixation inter-trial interval (ITI) before the next trial. This procedure was completed until all of the perception trials in the location or color condition had been presented.

The four short-term memory phases were similar to the perception tasks described above with the exception that there was no dynamic mask and multiple objects were presented in the target location or color before the 2AFC test probes. Before presentation of the first object in a group, the word “study” appeared on the screen for 1000 ms to cue participants that they should remember the location or color of the objects. After a 100 ms fixation period, each object in the low (set size of 4) and high (set size of 2) resolution conditions were each presented for 350 ms followed by a 100 ms ITI. Following presentation of the last object, there was a 2000 ms delay (fixation) before the retrieval period. After the delay period, participants were cued with the word “test” for 1000 ms to inform participants that they would have to remember the location or color of the objects they just studied. Following a 500 ms fixation period, participants were tested on each object from the immediately preceding encoding session one at a time. The retrieval period for the short-term memory phases was identical to the perception phases, with the exception that there was a 250 ms ITI between successive trials in the retrieval period. After the final object of a set was tested, there was a 2000 ms fixation period before the encoding period for the next set of objects began. The encoding-retrieval cycle above was completed until all of the sets of objects were presented to participants.

Results

The dependent variable of interest was the proportion correct on the 2AFC judgment for the perception and short-term memory blocks (see Table 2). To test whether patients and controls differed in their ability to perceive and differentiate the stimulus feature dimensions used in the short-term memory tasks (i.e., color and location), we examined performance in the perceptual control tasks using a 2 (Group: Control, Patient) × 2 (Feature: Location, Color) mixed ANOVA. There was a significant main effect of Feature, indicating that accuracy was higher for the color feature relative to the location feature [F(1,14) = 5.2439 MSe = .011, ηp2=.272, p = .038]. Importantly, there was not a main effect of group, [F(1,14) = .142, MSe = .020, ηp2=.010, p = .712], nor a group by feature interaction [F(1,14) = .119, MSe = .011, ηp2=.008, p = .736]. This result suggests that, on average, patients and controls were similar at perceiving location and color information.

Table 2.

Proportion correct on the short-term memory task calculated on all test probes (top panel) or only the first test probe (bottom panel).

Location Color
Low Resolution High Resolution Perception Low Resolution High Resolution Perception
Accuracy on All Test Probes
Controls .72 (.03) .84 (.03) .65 (.04) .74 (.01) .80 (.02) .76 (.03)
Patients .67 (.09) .63 (.16) .64 (.10) .70 (.03) .73 (.04) .72 (.04)
1001 .76 .72 .52 .74 .80 .67
1003 .63 .65 .60 .61 .60 .65
1005 .85 .94 .94 .72 .80 .81
1007 .44 .19 .50 .74 .71 .77
Accuracy on First Test Probe
Controls .76 (.04) .83 (.03) - .76 (.03) .82 (.02) -
Patients .68 (.13) .61(.16) - .75 (.05) .74 (.03) -
1001 .83 .72 - .67 .77 -
1003 .56 .56 - .72 .69 -
1005 .94 .97 - .89 .80 -
1007 .39 .19 - .72 .72 -
Open in a new tab

Note. There was only one object per set for the perception task, thus accuracy is identical for all test probes and first test probes. The standard error of the mean is provided in parentheses.

To test our primary hypothesis that patients with MTL damage would show larger impairments in short-term memory for tasks that requires high resolution bindings, we conducted a 2 (Group: Control, Patient) × 2 (Feature: Location, Color) × 2 (Resolution: Low, High) mixed ANOVA on accuracy data from the four short-term memory blocks. There was a main effect of Group showing that patients (M = .683, SE = .038) performed worse than controls (M = .777, SE = .022), [F(1,14) = 4.653, MSe = .023, ηp2=.249, p = .049] and a main effect of Resolution showing that performance in the high resolution condition (M = .751, SE = .028) was higher than performance in the low resolution condition (M = .710, SE = .017), [F(1,14) = 5.444, MSe = .004, ηp2=.280, p = .035]. Importantly, the above main effects were moderated by a significant interaction, [F(1,14) = 6.999, MSe = .004, ηp2=.333, p = .019]. Post-hoc contrasts revealed that MTL patients had larger impairments in visual short-term memory for high resolution associations (MTL Patients: M = .680, SE = .049; Healthy Controls: M = .821, SE = .028), [F(1,14) = 6.120, MSe = .039, ηp2=.304, p = .027], relative to low resolution associations (MTL Patients: M = .686, SE = .030; Healthy Controls: M = .734, SE = .017), [F(1,14) = 1.915, MSe = .014, ηp2=.120, p = .188] (see Figure 3). No other effects were significant at p < .05, [Main Effect of Feature: F(1,14) = .5578, MSe = .021, ηp2=.038, p = .468; Feature by Group Interaction: F(1,14) = .778, MSe = .021, ηp2=.053, p = .393; Feature by Resolution Interaction: F(1,14) = .103, MSe = .003, ηp2=.007, p = .753; Group by Feature by Resolution interaction: F(1,14) = 4.239, MSe = .003, ηp2=.232, p = .059].

Figure 3.

Figure 3

Open in a new tab

Results depicting the reliable interaction between of Group (patient vs. control) and Resolution (low vs. high) on visual short-term memory accuracy. The figure plots the proportion correct for the high and low resolution shot-term memory tasks for patients and controls collapsed across material (location or color). The interaction was driven by larger visual short-term memory impairments in MTL patients relative to controls for high resolution (p < .05) but not low resolution (p = .19) associations. Open symbols depict the performance of the two patients with selective hippocampal damage, whereas close symbols depict the performance of the two patients with extensive MTL damage. Error bars reflect ±1 standard error of the mean.

In order to assess whether the patient deficit in the high resolution short-term memory condition was related to interference from multiple test probes, we examined performance on the first test item only (see bottom panel of Table 2). Importantly, as in the overall analysis, the patients showed a significant impairment in visual short-term memory for high resolution associations (Patients: M = .677, SE = .092; Controls: M = .823, SE = .017), [F(1,14) = 6.347, MSe = .01, ηp2=.312, p = .025], but not for low resolution associations (Patients: M = .715, SE = .078; Controls: M = .762, SE = .029), [F(1,14) = .487, MSe = .013, ηp2=.034, p = .497].

Discussion

The purpose of this study was to test the hypothesis that the hippocampus plays an important role in visual short-term memory when high resolution associations are required (Yonelinas, 2013). To test this hypothesis, we assessed visual short-term memory for high and low resolution object-location and object-color associations in four patients with damage to the MTL and healthy controls. Relative to short-term memory for low resolution associations, short-term memory for high resolution associations was impaired to a greater extent in patients with MTL damage. Although it is challenging to determine exactly which MTL regions are critical for the observed deficits in the present study, the two patients in the current study with what appeared to be selective hippocampal lesions (patients 1001 and 1003) exhibited deficits that were comparable to the patients with more extensive MTL lesions. Thus, the current results suggest that hippocampal damage are sufficient to cause impairments in short-term memory for high resolution bindings. The findings from the current study add to the growing body of work showing that, in addition to long-term memory, the hippocampus plays a critical role in short-term memory and scene perception (e.g., Aly et al., 2013; Goodrich & Yonelinas, 2016; Graham, Barense, & Lee, 2010; Kolarik et al., 2016; Warren et al., 2010). Importantly, these results suggest that the contribution of the MTL, and likely the hippocampus, to short-term memory depends on the resolution of a binding that needs to be maintained over a short delay.

The current results suggest that task difficulty is unlikely to account for the finding that patients showed larger impairments for the high resolution condition, which was the condition with the highest performance in controls. As discussed previously, interpreting task performance impairments in patients is difficult if the impairment is in the most difficult condition. Critically, the patients showed the largest impairment in the easiest (high resolution), not the most difficult (low resolution), of the two conditions.

Furthermore, the impairment in the high resolution condition cannot be attributed simply to an inability to perceive or maintain the high resolution color or location information. Accuracy in the perceptual control task showed that the patients were able to make accurate discriminations about color and location when the pairs of stimuli were separated by brief delays (with a backwards mask) of 102 ms. Moreover, previous work has also shown that patients with MTL damage are not impaired at making high resolution perception judgments in an odd-man-out color discrimination task (Lee et al., 2005).

The current patients were found to be impaired when either high resolution color bindings or high resolution location bindings were required, indicating that their impairments were not limited to remembering spatial location information. This is consistent with models that propose that the role of the hippocampus is not limited to spatial information (e.g., Yonelinas, 2013). Nonetheless, there was a trend for the observed deficits in the high resolution condition to be numerically larger for location than for color (see Table 2), and we note that although the interaction between stimuli, resolution and patient group was not significant, it was close (p<.059). Thus, the hippocampus may be particularly important for the binding of high resolution location information.

Although it is difficult to make any strong claims about individual patients, there was one patient who did not seem to have a deficit in the high resolution condition (patient 1005). This finding was rather unexpected, and it is unclear why this patient was able to performed at such a high level. We could not identify anything unusual in their neuropsychological scores, with the exception that their visual score on the WMS-R (z score) was slightly high (z-score = 1.07) suggesting that they may be particularly good on visual memory tasks in general. Moreover, patient 1005 only performed poorly on neuropsychological test measures (i.e., the Delayed Score on the WMS-R and the Door’s and People test) that required memory for information after a delay much longer than the delay in the current short-term memory experiment performed. Consistent with this, Kolarik et al. (2016) recently reported a case study of patient 1005 that revealed this patient had impaired memory in a virtual Morris water maze task that required maintenance of information over a longer delay than used in the current short-term memory study.

In general, the patients were impaired when the task required high resolution bindings, but no impairments were observed when the task required low resolution bindings. Although we propose that the pattern of impairments was due to patients being unable to form high resolution associations, another logical possibility is that the patients were unable to take advantage of the small set size. However, we do not know of any data that suggests that hippocampal patients are more impaired with small set sizes than with larger set sizes, and in fact some data indicate that patients are not impaired with smaller set sizes relative to larger set sizes (e.g., Jeneson et al., 2010; Jeneson et al., 2012). In fact, prior findings that showed no short-term memory impairments in amnesics at low set sizes increase our confidence that the observed impairments are due to the high resolution requirement and not set size.

The current working memory conditions that required participants to retrieve high resolution bindings necessitated that participants have high resolution information to discriminate between highly similar targets and lures. One possible account of the deficits then is that the highly similar lures produced higher levels of interference between the targets and similar lures in the MTL patients relative to controls. Importantly, however, results from other studies that have used short term memory recall tests in which no test lures are provided, such as the color wheel task (Warren, Duff, Cohen, & Tranel, 2014; Zhang, Borders, & Yonelinas, 2012), have also indicated that hippocampal patients are impaired in short-term memory when they are required to provide high resolution color-binding information. Thus, the patients’ deficits cannot be explained solely as reflecting a problem overcoming interference from highly similar lures at time of test. Rather their deficits would seem to reflect a failure to form sufficiently distinct memory representations. Thus, the current work is in line with earlier work suggesting a role of the hippocampus in supporting pattern separation in order to overcome interference from similar memory representations (Kesner & Hopkins, 2006; Marr, 1971; McClelland, McNaughton, & O'Reilly, 1995; Shapiro & Olton, 1994; Yassa & Stark, 2011).

One potential limitation of the current study was that the high-resolution trials were presented after the low-resolution trials. We did this to avoid potential carryover effects whereby subjects may continue to use high resolution information in the low resolution conditions, and thus the low resolution condition might not reflect performance as measured in most previous short term memory studies. However, one inadvertent effect of this is that fatigue may have impacted task performance, and if the patients were more sensitive to fatigue than the controls they may perform more poorly on the high than low resolution conditions. The current study was relatively short in duration compared to other patient studies (i.e., it only took ~30–40 mins to complete), nonetheless fatigue effects are difficult to rule out. However, we don’t believe that fatigue could explain the patient deficits we observed because in another recent study in which the high resolution trials were mixed across the experimental session, the patients were also found to exhibit working memory impairments for high resolution working memory trials (Goodrich & Yonelinas, 2016).

In conclusion, the results from this study that the hippocampus supports short-term memory particularly for high resolution bindings (Yonelinas, 2013). These findings converge with recent research by Kolarik and colleagues (2016) showing that the hippocampus plays a critical role high resolution navigation (for related results in rodents, see e.g., Gilbert, Kesner, & DeCoteau, 1998; Hunsaker, Rosenberg, & Kesner, 2008; Kesner & Goodrich-Hunsaker, 2010; Talpos, McTighe, Dias, Saksida, & Bussey, 2010), and with results that have established hippocampal involvement in complex scene perception (e.g., Aly et al., 2013; Lee et al., 2005; Lee, Levi, Davies, Hodges, & Graham, 2007; for review, see Lee, Yeung, & Barense, 2012). For example, Aly and colleagues (2013) carried out corresponding patient and fMRI studies of a scene change detection paradigm in which they manipulated global aspects of the changed scenes. Accurate change detection in this paradigm relies on the ability to detect subtle differences in the manner in which the features of the scenes are related to one another (Aly & Yonelinas, 2012). Aly and Yonelinas (2013) found that patients with hippocampal damage were impaired in the scene change detection task and that the BOLD signal in the hippocampus tracked confidence in change detection. Taken together, these results suggest that the hippocampus supports perception, short-term memory, and long-term memory by representing high resolution bindings (Yonelinas, 2013). Future work is needed to determine the exact role the hippocampus plays in representing high resolution associations. Of particular relevance to the present study, it is unclear if the hippocampus is necessary for encoding, maintaining, or retrieving high resolution associations in visual short-term memory. Future studies that employ alternative methods, such as fMRI, will be useful in answering the above question as well as providing additional evidence for the hypothesis that the hippocampus supports short-term memory for complex, high resolution bindings.

Acknowledgments

This research was supported by National Institutes of Health grants (MH059352, MH083734, EY025999) awarded to APY, and a National Science Foundation Graduate Research Fellowship (1148897) awarded to JDK. JDK is currently supported by a Ruth L. Kirschstein National Research Service Award from the National Institute on Aging (AG049583).

Footnotes

1

We acknowledge that, due to the backwards mask, the perceptual control task might not be a pure measure of perception, but instead measure working memory on a very short time scale. The critical point however, is that this task allowed us to determine if patients were able to make accurate discriminations about the location and color features that were used in the critical short-term memory conditions.

References

  1. Aggleton JP, Brown M. Interleaving brain systems for episodic and recognition memory. Trends in Cognitive Sciences. 2006;10(10):455–463. doi: 10.1016/j.tics.2006.08.003. [DOI] [PubMed] [Google Scholar]
  2. Aly M, Ranganath C, Yonelinas AP. Detecting changes in scenes: the hippocampus is critical for strength-based perception. Neuron. 2013;78(6):1127–1137. doi: 10.1016/j.neuron.2013.04.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aly M, Yonelinas AP. Bridging Consciousness and Cognition in Memory and Perception: Evidence for Both State and Strength Processes. PLoS One. 2012;7(1):e30231. doi: 10.1371/journal.pone.0030231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Baddeley AD, Allen R, Vargha-Khadem F. Is the hippocampus necessary for visual and verbal binding in working memory? Neuropsychologia. 2010;48(4):1089–1095. doi: 10.1016/j.neuropsychologia.2009.12.009. doi: http://dx.doi.org/10.1016/j.neuropsychologia.2009.12.009. [DOI] [PubMed] [Google Scholar]
  5. Baddeley AD, Emslie H, Nimmo-Smith I, Thames Valley Test C. Doors and people: A test of visual and verbal recall and recognition. Bury St. Edmunds: Thames Valley Test Company; 1994. [Google Scholar]
  6. Baddeley AD, Warrington EK. Amnesia and the distinction between long- and short-term memory. Journal of Verbal Learning and Verbal Behavior. 1970;9(2):176–189. doi: http://dx.doi.org/10.1016/S0022-5371(70)80048-2. [Google Scholar]
  7. Burgess N. The hippocampus, space, and viewpoints in episodic memory. Q J Exp Psychol A. 2002;55(4):1057–1080. doi: 10.1080/02724980244000224. [DOI] [PubMed] [Google Scholar]
  8. Eichenbaum H, Yonelinas AP, Ranganath C. The medial temporal lobe and recognition memory. Annual Review of Neuroscience. 2007;30:123–152. doi: 10.1146/annurev.neuro.30.051606.094328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Esfahani-Bayerl N, Finke C, Braun M, Düzel E, Heekeren HR, Holtkamp M, Ploner CJ. Visuo-spatial memory deficits following medial temporal lobe damage: A comparison of three patient groups. Neuropsychologia. 2016;81:168–179. doi: 10.1016/j.neuropsychologia.2015.12.024. doi: http://dx.doi.org/10.1016/j.neuropsychologia.2015.12.024. [DOI] [PubMed] [Google Scholar]
  10. Gilbert PE, Kesner RP, DeCoteau WE. Memory for Spatial Location: Role of the Hippocampus in Mediating Spatial Pattern Separation. The Journal of Neuroscience. 1998;18(2):804–810. doi: 10.1523/JNEUROSCI.18-02-00804.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Goodrich RI, Yonelinas AP. The Medial Temporal Lobe Supports Sensing-Based Visual Working Memory. Neuropsychologia. 2016;89:485–494. doi: 10.1016/j.neuropsychologia.2016.07.011. doi: http://dx.doi.org/10.1016/j.neuropsychologia.2016.07.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Graham KS, Barense MD, Lee AC. Going beyond LTM in the MTL: a synthesis of neuropsychological and neuroimaging findings on the role of the medial temporal lobe in memory and perception. Neuropsychologia. 2010;48(4):831–853. doi: 10.1016/j.neuropsychologia.2010.01.001. [DOI] [PubMed] [Google Scholar]
  13. Hannula DE, Ranganath C. Medial Temporal Lobe Activity Predicts Successful Relational Memory Binding. The Journal of Neuroscience. 2008;28(1):116–124. doi: 10.1523/JNEUROSCI.3086-07.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Hartley T, Bird CM, Chan D, Cipolotti L, Husain M, Vargha-Khadem F, Burgess N. The hippocampus is required for short-term topographical memory in humans. Hippocampus. 2007;17(1):34–48. doi: 10.1002/hipo.20240. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Holdstock JS, Gutnikov SA, Gaffan D, Mayes AR. Perceptual and Mnemonic Matching-To-Sample in Humans: Contributions of The Hippocampus, Perirhinal and Other Medial Temporal Lobe Cortices. Cortex. 2000;36(3):301–322. doi: 10.1016/s0010-9452(08)70843-8. doi: http://dx.doi.org/10.1016/S0010-9452(08)70843-8. [DOI] [PubMed] [Google Scholar]
  16. Hunsaker MR, Rosenberg JS, Kesner RP. The role of the dentate gyrus, CA3a,b, and CA3c for detecting spatial and environmental novelty. Hippocampus. 2008;18(10):1064–1073. doi: 10.1002/hipo.20464. [DOI] [PubMed] [Google Scholar]
  17. Jeneson A, Mauldin KN, Hopkins RO, Squire LR. The role of the hippocampus in retaining relational information across short delays: The importance of memory load. Learning & Memory. 2011;18(5):301–305. doi: 10.1101/lm.2010711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Jeneson A, Mauldin KN, Squire LR. Intact working memory for relational information after medial temporal lobe damage. The Journal of Neuroscience. 2010;30(41):13624–13629. doi: 10.1523/JNEUROSCI.2895-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Jeneson A, Squire LR. Working memory, long-term memory, and medial temporal lobe function. Learning & Memory. 2012;19(1):15–25. doi: 10.1101/lm.024018.111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Jeneson A, Wixted JT, Hopkins RO, Squire LR. Visual Working Memory Capacity and the Medial Temporal Lobe. The Journal of Neuroscience. 2012;32(10):3584–3589. doi: 10.1523/JNEUROSCI.6444-11.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Kesner RP, Goodrich-Hunsaker NJ. Developing an animal model of human amnesia: The role of the hippocampus. Neuropsychologia. 2010;48(8):2290–2302. doi: 10.1016/j.neuropsychologia.2009.10.027. [DOI] [PubMed] [Google Scholar]
  22. Kesner RP, Hopkins RO. Mnemonic functions of the hippocampus: A comparison between animals and humans. Biological Psychology. 2006;73(1):3–18. doi: 10.1016/j.biopsycho.2006.01.004. [DOI] [PubMed] [Google Scholar]
  23. Kolarik BS, Shahlaie K, Hassan A, Borders AA, Kaufman KC, Gurkoff G, Ekstrom AD. Impairments in precision, rather than spatial strategy, characterize performance on the virtual Morris Water Maze: A case study. Neuropsychologia. 2016;80:90–101. doi: 10.1016/j.neuropsychologia.2015.11.013. doi: http://dx.doi.org/10.1016/j.neuropsychologia.2015.11.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Lee ACH, Buckley MJ, Pegman SJ, Spiers H, Scahill VL, Gaffan D, Graham KS. Specialization in the medial temporal lobe for processing of objects and scenes. Hippocampus. 2005;15(6):782–797. doi: 10.1002/hipo.20101. [DOI] [PubMed] [Google Scholar]
  25. Lee ACH, Levi N, Davies RR, Hodges JR, Graham KS. Differing profiles of face and scene discrimination deficits in semantic dementia and Alzheimer's disease. Neuropsychologia. 2007;45(9):2135–2146. doi: 10.1016/j.neuropsychologia.2007.01.010. doi: http://dx.doi.org/10.1016/j.neuropsychologia.2007.01.010. [DOI] [PubMed] [Google Scholar]
  26. Lee ACH, Yeung L-K, Barense MD. The hippocampus and visual perception. Frontiers in Human Neuroscience. 2012;6 doi: 10.3389/fnhum.2012.00091. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Marr D. Simple memory: A theory for archicortex. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences. 1971;B262:23–81. doi: 10.1098/rstb.1971.0078. [DOI] [PubMed] [Google Scholar]
  28. McClelland JL, McNaughton BL, O'Reilly RC. Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review. 1995;102(3):419–457. doi: 10.1037/0033-295X.102.3.419. doi: http://dx.doi.org/10.1037/0033-295X.102.3.419. [DOI] [PubMed] [Google Scholar]
  29. Nakada T, Kwee IL, Fujii Y, Knight RT. High-field, T2 reversed MRI of the hippocampus in transient global amnesia. Neurology. 2005;64(7):1170–1174. doi: 10.1212/01.WNL.0000156158.48587.EA. [DOI] [PubMed] [Google Scholar]
  30. O'Keefe J, Nadel L. The hippocampus as a cognitive map. Oxford, UK: Oxford University Press; 1978. [Google Scholar]
  31. Olson IR, Moore KS, Stark M, Chatterjee A. Visual Working Memory Is Impaired when the Medial Temporal Lobe Is Damaged. Journal of Cognitive Neuroscience. 2006;18(7):1087–1097. doi: 10.1162/jocn.2006.18.7.1087. [DOI] [PubMed] [Google Scholar]
  32. Olson IR, Page K, Moore KS, Chatterjee A, Verfaellie M. Working Memory for Conjunctions Relies on the Medial Temporal Lobe. The Journal of Neuroscience. 2006;26(17):4596–4601. doi: 10.1523/JNEUROSCI.1923-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Peirce JW. PsychoPy—Psychophysics software in Python. Journal of Neuroscience Methods. 2007;162(1–2):8–13. doi: 10.1016/j.jneumeth.2006.11.017. doi: http://dx.doi.org/10.1016/j.jneumeth.2006.11.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Ranganath C, Blumenfeld RS. Doubts about double dissociations between short- and long-term memory. Trends in Cognitive Sciences. 2005;9(8):374–380. doi: 10.1016/j.tics.2005.06.009. [DOI] [PubMed] [Google Scholar]
  35. Ryan JD, Cohen NJ. Processing and short-term retention of relational information in amnesia. Neuropsychologia. 2004;42(4):497–511. doi: 10.1016/j.neuropsychologia.2003.08.011. doi: http://dx.doi.org/10.1016/j.neuropsychologia.2003.08.011. [DOI] [PubMed] [Google Scholar]
  36. Scoville WB, Milner B. Loss of recent memory after bilateral hippocampal lesions. Journal of Neuropsychiatry and Clinical Neurosciences. 1957;12(1):103–113. doi: 10.1176/jnp.12.1.103. [DOI] [PubMed] [Google Scholar]
  37. Shapiro ML, Olton DS. Hippocampal function and interference. In: Schacter DL, Tulving E, editors. Memory Systems 1994. Cambridge, Massachusetts: The MIT Press; 1994. pp. 87–118. [Google Scholar]
  38. Shipley WC. A self-administering scale for measuring intellectual impairment and deterioration. Journal of Psychology. 1940;10:371–371. [Google Scholar]
  39. Shrager Y, Levy DA, Hopkins RO, Squire LR. Working Memory and the Organization of Brain Systems. The Journal of Neuroscience. 2008;28(18):4818–4822. doi: 10.1523/JNEUROSCI.0710-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Squire LR, Zola-Morgan S. The medial temporal lobe memory system. Science. 1991;253(5026):1380–1385. doi: 10.1126/science.1896849. [DOI] [PubMed] [Google Scholar]
  41. Talpos JC, McTighe SM, Dias R, Saksida LM, Bussey TJ. Trial-unique, delayed nonmatching-to-location (TUNL): a novel, highly hippocampus-dependent automated touchscreen test of location memory and pattern separation. Neurobiology of Learning and Memory. 2010;94(3):341–352. doi: 10.1016/j.nlm.2010.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Warren DE, Duff MC, Cohen NJ, Tranel D. Hippocampus contributes to the maintenance but not the quality of visual information over time. Learn Mem. 2014;22(1):6–10. doi: 10.1101/lm.037127.114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Warren DE, Duff MC, Tranel D, Cohen NJ. Medial temporal lobe damage impairs representation of simple stimuli. Frontiers in Human Neuroscience. 2010;4 doi: 10.3389/fnhum.2010.00035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Warrington EK. Neuropsychological evidence for multiple memory systems. Acta Neurologica Scandinavica. 1981;64(S89):13–19. doi: 10.1111/j.1600-0404.1981.tb02358.x. [DOI] [PubMed] [Google Scholar]
  45. Warrington EK, Baddeley AD. Amnesia and memory for visual location. Neuropsychologia. 1974;12(2):257–263. doi: 10.1016/0028-3932(74)90011-6. doi:http://dx.doi.org/10.1016/0028-3932(74)90011-6. [DOI] [PubMed] [Google Scholar]
  46. Wechsler D. Wechsler memory scale - revised (WMS-R) 1987 [Google Scholar]
  47. Yassa MA, Stark CE. Pattern separation in the hippocampus. Trends Neurosci. 2011;34(10):515–525. doi: 10.1016/j.tins.2011.06.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Yonelinas AP. The hippocampus supports high-resolution binding in the service of perception, working memory and long-term memory. Behavioural Brain Research. 2013;254:34–44. doi: 10.1016/j.bbr.2013.05.030. doi: http://dx.doi.org/10.1016/j.bbr.2013.05.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Yonelinas AP, Aly M, Wang W-C, Koen JD. Recollection and familiarity: examining controversial assumptions and new directions. Hippocampus. 2010;20(11):1178–1194. doi: 10.1002/hipo.20864. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Zachary RA, Crumpton E, Spiegel DE. Estimating WAIS-R IQ from the Shipley Institute of Living Scale. Journal of Clinical Psychology. 1985;41(4):532–540. doi: 10.1002/1097-4679(198511)41:6<820::aid-jclp2270410616>3.0.co;2-x. [DOI] [PubMed] [Google Scholar]
  51. Zhang W, Borders AA, Yonelinas AP. Cognitive Neuroscience Society. Chicago, IL: 2012. The influence of medial temporal damage on capacity and precision in visual working memory. [Google Scholar]

RESOURCES