Memory, scene construction, and the human hippocampus

Significance

It has been suggested that the primary role of the hippocampus is to construct spatial scenes and further that memory impairments following hippocampal damage may be attributed to spatial processing demands in memory tasks. Two types of tasks have contributed evidence to this perspective: boundary extension and scene imagination. Boundary extension refers to the phenomenon whereby a scene is remembered as having an expanded background. Imagination tasks ask participants to mentally construct scenes. We tested patients with hippocampal damage on both types of tasks. They were intact at both boundary extension and imagination, although they remembered the tasks poorly. These results support the traditional view that the human hippocampus is primarily important for memory.

Abstract

We evaluated two different perspectives about the function of the human hippocampus–one that emphasizes the importance of memory and another that emphasizes the importance of spatial processing and scene construction. We gave tests of boundary extension, scene construction, and memory to patients with lesions limited to the hippocampus or large lesions of the medial temporal lobe. The patients were intact on all of the spatial tasks and impaired on all of the memory tasks. We discuss earlier studies that associated performance on these spatial tasks to hippocampal function. Our results demonstrate the importance of medial temporal lobe structures for memory and raise doubts about the idea that these structures have a prominent role in spatial cognition.

Two traditions of work have influenced discussion about the function of the hippocampus (1). One tradition is based on work with memory-impaired patients and the idea that the hippocampus is important for a particular kind of memory (2, 3). The other tradition is based on work with rodents and the idea that the hippocampus is critical for spatial mapping (4). Its possible role in spatial processing has been recently explored in humans as well (5), and it has been proposed that the human hippocampus is essential for the ability to construct spatially coherent scenes (6, 7).

This view of hippocampal function has depended on evidence from two kinds of tasks: boundary extension and scene construction (6, 8). Boundary extension refers to the tendency to reconstruct a scene such that it has a larger background than was actually presented (9). In the Mullally et al. (8) study, memory-impaired patients exhibited boundary extension less strongly than controls. Scene construction refers to the ability to imagine and describe spatially coherent scenes. In two studies, memory-impaired patients made few references to space when visualizing and describing imagined scenes (6, 8).

It is unclear how to reconcile such findings with the view that the hippocampus chiefly supports memory functions. In particular, the idea that the construction and visualization of scenes involves the hippocampus seems at odds with the historic distinction between short-term (working) memory and long-term memory and the related idea that short-term memory is independent of the hippocampus (10–12). According to this perspective, hippocampal damage should not impair performance on spatial tasks, so long as testing puts no burden on long-term memory. In an attempt to clarify these issues, we gave tests of boundary extension, scene construction, and memory to patients with well-characterized lesions limited to the hippocampus or large lesions of the medial temporal lobe.

Results

Experiment 1.

Both controls and hippocampal patients exhibited boundary extension. That is, both groups drew the central objects such that they occupied less area than in the original photographs (Fig. 1 controls = 61.9% of the original area, hippocampal patients = 62.5% of the original area, P < 0.01). The extent of reduction was similar for the two groups [t₍₁₆₎ = 0.07]. Patient GP, who has large medial temporal lobe lesions, also drew the objects smaller than they had originally appeared (55.2% of the original area; Table 1). Despite the fact that the patients exhibited intact boundary extension, they did not reproduce the photographs as accurately as controls [control reproductions were rated 3.5 on a 1–5 scale; hippocampal patients, 2.5, P = 0.02; GP, 1.3; for the hippocampal patients, effect size (d) = 1.7]. This value for d is considered to reflect a large effect size. One can calculate that, if a similarly large effect size had been present for boundary extension, the probability of our having found an impairment in hippocampal patients (i.e., the statistical power) was 94%.

Fig. 1.

(Top) On each trial of Experiment 1, participants studied a photograph of a scene (Left) for 15 s and then were asked to draw the scene from memory. Shown here are sample drawings by a control (Middle) and a patient with damage thought to be limited to the hippocampus (Right). Boundary extension was measured by how much the foreground object in each drawing decreased in area compared with the foreground object in the photograph. (A) Both controls (CON, n = 12) and hippocampal patients (H, n = 6) drew the objects smaller than they had originally appeared. (B) Five independent raters evaluated the accuracy of the drawings on a 1–5 scale. Brackets indicate SEM.

Table 1.

Performance of GP

		Experiment 2
Experiment 1			a	b	Experiment 3
Percent of original area (%)	55.2	Closer (%)	0	50	EP	5.7
		Same (%)	100	21	SD	2.4
Accuracy (1 – 5)	1.3	Farther (%)	0	29	SPA	1.4
					TEA	1.3
Memory accuracy	65.0	Memory accuracy	50	56	Memory (no. of details)	3.8

Performance of patient GP, who has large medial temporal lobe lesions and virtually complete damage to the hippocampus bilaterally. In Experiment 1, like the hippocampal patients (Fig. 1 A and B), GP exhibited intact boundary extension (i.e., he drew the objects smaller than they had originally appeared) and also did not reproduce the photographs accurately. In Experiment 2a, GP always rated the second scene as the same as the first scene and was poor at remembering the scenes (compare with Fig. 2 A and B). In the second version of the same task (Experiment 2b), he performed like hippocampal patients (Fig. 2 C and D). That is, he exhibited intact boundary extension (i.e., he tended to rate the second scene as closer up more often than farther away in comparison with the first scene), and he was also poor at remembering the scenes. In Experiment 3, GP imagined what might come into view if he could see beyond the boundaries of photographs. His narratives were segmented into details and classified as belonging to one of four categories: entities present (EP), sensory descriptions (SD), spatial references (SPA), and thoughts/emotions/actions (TEA).

Patients were also impaired at remembering the central objects that they had seen in the 10 photographs (controls = 94.2% correct, hippocampal patients = 83.3% correct, P < 0.05; GP = 65% correct; chance = 50%).

Experiment 2a and 2b.

In Experiment 2a, controls and hippocampal patients behaved similarly (Fig. 2A). They rated the second scene as “the same” as the first scene on approximately half the trials (controls = 52.5%, hippocampal patients = 51.6%). Notably, both groups exhibited boundary extension, as evidenced by the fact that the second scene was rated “closer up” than the first scene more often than “farther away” (controls = 34.6% vs. 14.2%, P = 0.02; hippocampal patients = 37.9% vs. 5.8%, P = 0.01). The findings were the same when each of the five response categories were examined separately (“much closer,” “a little closer,” “same,” “a little farther,” “much farther”). The percentage of responses in each category was similar for the two groups (P > 0.10), and none of the mean scores differed by more than 9%. Patient GP rated all of the second scenes as the same as the first scenes (Table 1), perhaps because he used a conservative decision criterion (see below).

Fig. 2.

(A) The percentage of ratings (closer up, same, or farther away) provided by controls (CON) and hippocampal patients (H) on the Rapid Serial Visual Presentation task of Experiment 2a. On each of 24 trials, participants saw two identical scenes in sequence and rated the second scene as closer up, the same, or farther away compared with the first scene. Boundary extension was demonstrated when participants rated the second scene as closer up more often than farther away. (B) Recognition memory for the names of the objects depicted in the scenes. (C) In the second version of the same task (Experiment 2b), participants were told that the scenes were usually not identical. (D) Recognition memory for the names of the objects depicted in the scenes. Brackets indicate SEM.

Controls and patients also exhibited similar confidence in their judgments about the scenes (1–3 scale). Overall, the controls gave a mean rating of 2.1, and hippocampal patients gave a mean rating of 2.2. Patient GP, who rated all of the second scenes as the same, consistently gave a confidence rating of 3. Lastly, despite exhibiting boundary extension, the patients were impaired at remembering the scenes that had been presented (Fig. 2B) (controls = 78.3% correct, hippocampal patients = 66.4% correct, P < 0.05; GP = 50% correct; chance = 50%).

In Experiment 2b, we attempted to influence the decision criteria used by participants by telling them that the first and second scenes were “usually different.” Controls and patients again behaved similarly, and the boundary extension effect was stronger than in Experiment 2a (Fig. 2C). They rated the second scene as “closer” on more than half the trials. Moreover, they rated the second scene as closer up than the first scene more often than “farther away” (controls = 54.2% vs. 25.0%, P < 0.07; hippocampal patients = 68.1% vs. 7.6%, P < 0.01). The findings were the same when each of the five response categories were examined separately (much closer, a little closer, same, a little farther, much farther). The percent of responses in each category was similar for the two groups (P > 0.10), and none of the mean scores differed by more than 16.6%. Note that patient GP now exhibited boundary extension, rating the second scene closer up more often than farther away (Table 1; 50.0% vs. 29.2%).

As in Experiment 2a, controls and patients exhibited similar confidence in their judgments. Overall, the controls gave a mean rating of 2.0, hippocampal patients gave a mean rating of 2.3, and GP gave a mean rating of 2.5. Lastly, the patients were marginally impaired at remembering the scenes that had been presented (Fig. 2D) (controls = 75.8% correct, hippocampal patients = 65.1% correct, P < 0.07; GP = 56.0% correct; chance = 50.0%).

Another way to estimate the strength of boundary extension is to calculate the proportion of items endorsed as closer divided by the proportion of items endorsed as either closer or farther. By this measure, boundary extension in the patient group was, if anything, a little stronger than in controls (patients: Experiment 2a = 0.87, 2b = 0.9; controls: Experiment 2a = 0.71, 2b = 0.68; P < 0.09).

Experiment 3.

Controls and hippocampal patients exhibited intact scene construction. The two groups answered the initial four orienting questions without difficulty and then performed similarly when asked to imagine and describe what might come into view if they could see beyond the boundaries of the scene (Fig. 3 and Fig. S1) (for each of the four categories, P > 0.12). Controls and patients produced spatial details for almost all of the six scenes (controls = 5.4 scenes, patients = 5.8 scenes, P > 0.5; GP = 5 scenes). GP also performed like controls (Table 1 and Fig. 3). The single exception was that he produced fewer spatial details than either controls or hippocampal patients (P < 0.05). Nevertheless, one control produced fewer spatial details than GP, indicating that his performance was not outside the normal range.

Fig. 3.

(Left) One of six scenes presented in Experiment 3. On each trial, participants viewed a scene and were asked to imagine what might come into view if they could see beyond the boundaries of the scene. (Right) Mean number of details per scene in each of four categories as provided by controls (CON) and hippocampal patients (H). The descriptions were segmented into details and classified as belonging to one of four categories. EP, entities present; SD, sensory descriptions; SPA, spatial references; TEA, thoughts/emotions/ actions.

Table 2 indicates that controls, hippocampal patients, and GP also performed similarly with respect to how well they could visualize the imagined scene, how vividly they imagined it, and, after the scene was removed, how they rated its spatial coherence. In contrast, the patients were impaired at remembering the scene itself. Specifically, when they tried to describe the scene they had just viewed, the hippocampal patients and GP produced fewer details than controls (Table 2; P < 0.05). Interestingly, despite this deficiency, the hippocampal patients, with the exception of a single patient on one trial, were able to recall the central object in each scene (the controls always succeeded at recall). However, GP was unable to recall the central object in any scene. Indeed, his responses to the memory question were stereotyped, and he repeated the same phrases in all six trials (e.g., he recalled the “openness and greenery” in the scenes). Considering the severity of GP’s amnesia, his responses to questions about a recently presented scene should not be considered reliable.

Table 2.

Results of Experiment 3

Measure	CON (n = 8)	H (n = 4)	GP
Visualization (Max. 6)	5.5	5.5	6
Vividness rating (1–3)	2.3	2.7	2.8
Spatial Coherence Index (−4 to +8)	5.2	4.8	6.8
Memory (no. of details)	12.4	5.8	3.8

In Experiment 3, participants viewed six scenes, and in each case, imagined what might come into view if they could see beyond the boundaries of the scene. Afterward, participants were asked (yes or no) whether they had been able to visualize the imagined scene (visualization). Participants also rated the vividness of the imagined scene (1 = low, 3 = high). The scene was then removed from view. Participants next saw 12 statements (Spatial Coherence Index) and endorsed as many as accurately described what they had imagined. Eight of the statements reflected good spatial coherence (scored as +1), and four statements reflected poor spatial coherence (scored as −1). Lastly, participants were asked to describe the scene they had viewed. The controls, the hippocampal patients, and GP performed similarly on the first three measures. By contrast, patients were markedly impaired at recalling the scene that had been presented.

Discussion

In three experiments, we tested aspects of spatial cognition and spatial memory in six patients with hippocampal lesions. In Experiment 1 (Drawing Task), participants saw photographs of scenes and then reproduced them from memory. Controls and patients performed similarly, drawing the central object in each scene so that it occupied less area than in the original photograph (i.e., both groups exhibited the phenomenon of boundary extension). At the same time, the reproductions of the patients were less accurate and contained less detail than the control reproductions (Fig. 1). We also calculated that there was a probability of 0.94 of having found an effect on boundary extension, if an effect had been present and if the effect of hippocampal lesions on boundary extension had been as large as the effect on memory. We tested for an effect size of this magnitude because, if the hippocampus were primarily concerned with constructing scenes and not primarily concerned with memory, as proposed (7), it seems reasonable to expect that the effect of a hippocampal lesion on scene construction should have been as large as the effect on memory.

In Experiment 2a (Rapid Serial Visual Presentation task), two identical scenes were presented briefly in sequence, and participants (unaware that the scenes were identical) decided whether the second scene depicted a closer up, same, or farther away view of the first scene. Again, controls and patients performed similarly, rating the second scene as closer up more often than as farther away (i.e., both groups exhibited boundary extension) (Fig. 2A). In this experiment, participants also exhibited a strong preference to rate the second scene as the same as the first scene. Experiment 2b attempted to reduce this preference by informing participants that the second scene was usually different from the first. In this case, controls and patients also performed similarly, exhibiting strong boundary extension (Fig. 2C). They also rated the second scene as the same as the first scene on fewer than one-third of the trials. As expected, patients were impaired at remembering the central objects in the scenes that had been presented (Fig. 2 B and D).

In Experiment 3 (Scene Imagination), participants viewed photographs of scenes and described what might come into view if they could see beyond the boundaries of the photographs. Controls and patients performed similarly, producing well-formed narratives with similar content, including references to spatial features of the imagined scenes (Fig. 3 and Fig. S1). The groups also rated the quality of their imagined scenes similarly (Table 2). At the same time, after each scene was removed from view, patients were impaired at describing the scene they had just viewed. Taken together, the findings suggest that the hippocampus is not needed for constructing scenes but is needed for remembering scenes.

Neurohistological analysis of patients similar to the ones studied here has suggested that the hippocampal volume loss documented in the present study is likely to reflect nearly complete loss of hippocampal neurons (13). However, it has also been suggested that some instances of intact performance in patients with hippocampal damage might result from functional residual tissue (14). Data for patient GP allowed this issue to be addressed, because GP has large medial temporal lobe lesions that virtually eliminated the hippocampus bilaterally. For the spatial tasks, GP performed like the hippocampal patients in all three experiments (Tables 1 and 2). In Experiment 1, he exhibited intact boundary extension (i.e., he drew the objects smaller than they had originally appeared). In Experiment 2b, he rated the second scene as closer up more often than farther away in comparison with the first scene. In Experiment 3, GP described what he could imagine outside the boundary of six photographs (Table 1). He produced fewer spatial details than hippocampal patients or controls (1.4 vs. 2.9 vs. 2.7 spatial details), although two of the eight controls scored as poorly or worse than he did. His overall score was similar to the score obtained by the hippocampal patients and controls (10.8 vs. 12.4 vs. 13.8 details).

The current results differ from the findings for seven memory-impaired patients in an earlier study, who were tested on similar tasks (8). In that study, patients exhibited boundary extension in both a Drawing Task and in a Rapid Serial Visual Presentation Task, but they exhibited the phenomenon less strongly than controls. In addition, in a test involving a single scene, the patients were impaired at imagining what might come into view if they could see beyond the boundaries of that scene. These impairments were attributed to hippocampal damage.

Several issues merit consideration. First, the Rapid Serial Visual Presentation Task (Experiment 2) is sensitive to decision criteria. That is, performance is influenced by a participant’s tendency to respond the same independently of perceptual experience. In our Experiment 2a, the controls and patients performed similarly to the patients in the earlier study, preferring to respond the same. By contrast, in Experiment 2b, with instructions that discouraged same responses, the two groups performed similarly to the controls in the earlier study. These results demonstrate that the expression of boundary extension, as measured by this task, is sensitive to shifts in a participant’s decision criterion. This factor was unexplored in the earlier study, and it is possible (although perhaps unlikely) that the controls and patients in the earlier study adopted different decision criteria.

Second, a question can be raised about the extent of hippocampal damage in the patients studied earlier. In our study, all of the hippocampal patients (with one exception) became amnesic as the result of anoxia or ischemia, and all had a similar degree of hippocampal volume loss (mean reduction = 43%; range = 33–49%). The seven patients in the earlier study had various etiologies and variable degrees of hippocampal volume loss (mean reduction = 32%). Notably, individual values were reported for two of the patients (75% and 71% volume reduction), which made it possible to calculate that the reduction in hippocampal volume for the remaining five patients was only 16%. This value represents quite a modest loss of hippocampal volume. In a recent study, this degree of hippocampal volume loss (19% in patients with mild cognitive impairment) was associated with only mild anterograde memory impairment (15). Indeed, only two patients in the earlier study (8) were reported to be impaired at recognition memory, suggesting rather adequate hippocampal function. Accordingly, it seems problematic to attribute deficits in such a patient group specifically to hippocampal damage. In short, our patients had substantial loss of hippocampal volume and substantial memory impairment but performed normally on tasks of scene construction. In contrast, the patients in the earlier study had only modest loss of hippocampal volume and modest memory impairment but were impaired on tasks of scene construction.

Third, we suggest that impaired performance in the earlier study (8) may reflect damage outside the hippocampus. Four of the seven patients in the earlier study became amnesic as the result of limbic encephalitis (two were of unknown etiology, and one had anoxia). In several reports, limbic encephalitis has been associated with persisting cognitive abnormalities outside the domain of memory (e.g., reduction in intelligence test scores, personality change, confabulation, confusion; refs. 16–22). These considerations suggest that limbic encephalitis is a complex disease and may not provide a sound basis for making inferences about the cognitive effects of focal hippocampal damage.

If boundary extension and scene construction do not depend specifically on the hippocampus, which brain regions are important? Neuroimaging studies have identified a large network of brain regions involved in scene construction, including hippocampus, parahippocampal gyrus, retrosplenial cortex, precuneus, posterior parietal cortex, angular gyrus, insula, posterior inferior temporal sulcus, anterior superior temporal sulcus, lateral and ventromedial prefrontal cortex, and lateral occipital cortex (14, 23–26). In one study, students viewed photographs of campus landmarks (25). The pattern of activity in the hippocampus corresponded to the real-world distance between landmarks shown on successive trials. In addition, other neuroimaging studies associated medial temporal lobe activity with performance on a task of boundary extension or on a task of mental imagery for scenes (23, 27, 28). Given the large number of brain regions active in these tasks, it is difficult to know the role of any particular region. In addition, it is difficult to rule out the idea that hippocampal activity in many tasks reflects the encoding of information into long-term memory (11, 29, 30). The present findings suggest that neither the hippocampus nor the parahippocampal gyrus (patient GP) is essential for the aspects of spatial cognition studied here.

In summary, we evaluated aspects of spatial cognition in memory-impaired patients with circumscribed hippocampal lesions or large medial temporal lobe lesions that eliminated the hippocampus. The patients performed normally on all of the spatial tasks. They exhibited the normal tendency to extend the boundary of scenes that they viewed, and they performed like controls when describing what might come into view if they could see beyond the boundary of scenes. At the same time, the patients were impaired at remembering the scenes that had been presented. These findings emphasize the importance of the human hippocampus for memory and raise doubts about its proposed role in scene construction.

Materials and Methods

Participants.

Seven memory-impaired patients participated, six with bilateral lesions thought to be limited to the hippocampus (CA fields, dentate gyrus, and subicular complex) and one with larger medial temporal lobe lesions (Table 3). Patients RS, GW, and DA became amnesic in 1998, 2001, and 2011, respectively, following a drug overdose and associated respiratory failure. Patient KE became amnesic in 2004 after an episode of ischemia associated with kidney failure and toxic shock syndrome. Patient LJ (the only female) became amnesic in 1988 during a 6-mo period with no known precipitating event. Her memory impairment has been stable since that time. Patient JRW became amnesic in 1990 following an anoxic episode associated with cardiac arrest.

Table 3.

Characteristics of memory-impaired patients

WAIS-III					WMS-R
Patient	Sex	Age, y	Education, y	WAIS-III IQ	Attention	Verbal	Visual	General	Delay
DA	M	31	12	95	104	90	91	90	56
KE	M	72	14	108	114	64	84	72	55
LJ	F	76	12	101	105	83	60	69	<50
GP	M	67	16	90	102	79	62	66	50
RS	M	57	12	99	99	85	81	82	<50
GW	M	54	12	108	105	67	86	70	<50
JRW	M	50	12	90	87	65	95	70	<50

WAIS-III, Wechsler Adult Intelligence Scale-III; WMS-R, Wechsler Memory Scale-Revised. The WMS-R does not provide numerical scores for individuals who score <50. IQ scores for RS and JRW are from the WAIS-Revised, and IQ score for DA is from the WAIS-IV.

Estimates of medial temporal lobe damage were based on quantitative analysis of magnetic resonance (MR) images from 19 age-matched, healthy males for KE, RS, GW, JRW, and GP; 8 younger healthy males for DA; and 11 age-matched, healthy females for patient LJ (31). KE, RS, GW, JRW, LJ, and DA have an average bilateral reduction in hippocampal volume of 49%, 33%, 48%, 44%, 46%, and 35%, respectively. All values are more than 2.9 SDs from the control mean. On the basis of two patients (LM and WH) with similar bilateral volume loss in the hippocampus for whom detailed postmortem neurohistological information was obtained (13), the degree of volume loss in these six patients may reflect nearly complete loss of hippocampal neurons. Volume estimates for the parahippocampal gyrus include temporopolar, perirhinal, entorhinal, and parahippocampal cortices. KE, RS, GW, JRW, LJ, and DA have an average bilateral reduction in the volume of parahippocampal gyrus of 11%, −5%, 10%, 12%, −17%, and −5%, respectively (all values within 2 SDs of the control mean). The minus values indicate volumes that were larger for a patient than for controls. The volumes for parahippocampal gyrus differ a little from volumes reported previously for these patients and are based on newly published, more detailed guidelines for identifying the caudal border of the gyrus (32).

One patient (GP) has severe memory impairment resulting from viral encephalitis. GP has demonstrated virtually no new learning since the onset of his amnesia, and during repeated testing over many weeks does not recognize that he has been tested before (33). GP has an average bilateral reduction in hippocampal volume of 96%. The volume of the parahippocampal gyrus is reduced by 94%. Eight coronal magnetic resonance images from each patient, together with detailed description of the lesions, can be found in Fig. S2.

For the seven patients, immediate and delayed (12 min) recall of a short prose passage averaged 4.0 and 0.5 segments, respectively (12 controls = 8.3 and 7.6) (34). Copy and reproduction (12-min delay) of the Rey–Osterrieth Complex Figure (35) averaged 27.8 and 3.8, respectively (8 controls = 30.3 and 20.6). Control scores for the figure are from Squire and Shimamura (36).

Twelve healthy volunteers also participated (four females; mean age = 63.9 y; mean education = 13.8 y; for patients, mean age = 58.1 y; mean education = 12.9 y). All procedures were approved by the Institutional Review Board at the University of California at San Diego, and participants gave written informed consent before participation.

Experiment 1: Drawing Task.

Stimuli.

The stimuli were two sets of 10 color photographs (15.2 cm × 15.2 cm) of everyday objects (e.g., a basketball) presented against simple natural backgrounds (e.g., wooden floor). On average, the object covered 30.2% of the entire area of the photograph. For each participant, one set of photographs served as stimuli in the drawing task and also as target items for the subsequent recognition memory task. The other set of photographs served as foils for the subsequent recognition memory task. The two sets were counterbalanced across participants, and list order was random for each participant.

Procedure.

On each trial, a single photograph was presented for 15 s. During presentation, participants were asked to pay close attention to the details of the photograph, including the size and the location of the central object. After presentation, the photograph was removed from view, and participants were immediately given a blank response sheet on which they were asked to draw the photograph that had just been presented in as much detail as possible. The response sheet was the same size as the photograph. Drawing was self-paced.

Immediately after completion of the drawing task (mean duration = 11 min), memory was tested for the 10 photographs. Twenty words (10 targets and 10 foils) were presented one at a time (e.g., boots), and participants responded (yes/no) according to whether they believed the word corresponded to a photograph that had been presented earlier.

Data analysis.

To measure boundary extension, the area of the central object was measured in the participant drawings and in the original photographs by using Photoshop CS6 (Adobe Systems). The area of each central object in the participant drawings was divided by its area in the original photographs and multiplied by 100. A score of 100% indicated that the central object had been drawn the same size as it had appeared in the photograph. Scores less than 100% indicated boundary extension. This measure of boundary extension was the same as was used by Mullally et al. (8).

The drawings were also scored according to how accurately participants reproduced the photographs. For each photograph, five independent raters (blind to participant identity) rated the accuracy of the drawings on a 1–5 scale. The accuracy score for each participant was the mean of the five ratings, averaged across the 10 drawings.

Experiment 2a: Rapid Serial Visual Presentation Task.

Stimuli.

The stimuli were two sets of 24 colored scenes depicting everyday objects presented against simple natural backgrounds. On average, the area of the central object covered 32.2% of the entire area of the scene. The scenes subtended a visual angle of 19.6° × 14.5° at a viewing distance of 60 cm. For each participant, one set of scenes served as stimuli and also as target items for the subsequent recognition memory task. The other set of scenes served as foils for the subsequent recognition memory task. The two sets were counterbalanced across participants, and list order was random for each participant.

Procedure.

Participants were instructed that on each trial they would see two scenes in sequence and that their task was to judge whether the second scene depicted a closer up, same, or farther away view of the first scene. On each trial, the first scene was presented on a laptop for 250 ms followed by a 250-ms dynamic mask (Fig. 1). The mask had a black-and-white static background with a circular icon (subtending a visual angle of 5.5° × 5.5°) in the center. The circular icon depicted a “happy face,” which changed to a second face after 150 ms. Following the mask, the second scene was presented and remained on the screen. After the second scene had been displayed for 1 s, response options were presented and remained on the screen until the participant rated the second scene as “much closer up,” “a little closer up,” the same, “a little farther away,” or “much farther away” in comparison with the first scene. Participants also rated their confidence for making the decision on a scale of “not sure,” “fairly sure,” and “very sure” (1–3 scale). Unbeknownst to participants, the second scene was always identical to the first scene. Thus, if participants had mentally extended the boundary of the first scene (i.e., if they remembered the central object as smaller than it was), then participants would perceive the second scene as closer up in comparison with the first scene. Six practice trials were given before presenting 24 test trials.

Immediately after completion of this task (mean duration = 5 min), memory was tested for the 24 scenes. Forty-eight words (24 targets and 24 foils) that identified the central objects in the scenes were presented one at a time, and participants responded (yes/no) according to whether they believed the corresponding photograph had been presented earlier.

Data analysis.

To measure boundary extension, the proportion of trials was calculated in which participants rated the test scenes as closer up (much closer up or a little closer up), the same, or farther away (a little farther away or much farther away). Boundary extension was demonstrated when participants rated the second scene as closer up than the first scene more often than they rated it as farther away.

Experiment 2b: Rapid Serial Visual Presentation Task.

In Experiment 2a, both patients and controls exhibited boundary extension, but the effect was weaker than reported previously for control participants (8). We reasoned that the phenomenon of boundary extension may be sensitive to a participant’s decision criterion. To test this idea, Experiment 2b (on average 113 d after Experiment 2a) followed the same procedure as Experiment 2a except that participants were told that “the second picture will usually be different from the first picture.”

Experiment 3: Scene Imagination.

Participants.

Five patients participated, four with bilateral lesions limited to the hippocampus (DA, KE, LJ, GW) and one with larger medial temporal lobe lesions (GP). Eight healthy volunteers also participated (two females; mean age = 66.9 y; mean education = 14.6 y).

Stimuli.

The stimuli were six color photographs (27.3 cm × 18.3 cm) of everyday objects presented against simple natural backgrounds (Fig. 4, Left). The photographs were presented one at a time and in a different order for each participant.

Fig. 4.

Sample trial for the Rapid Serial Visual Presentation task. On each trial of Experiment 2, two scenes were presented in sequence. Participants first saw a scene for 250 ms followed by a 250-ms mask. Participants then saw the second scene, which remained on the screen while participants rated whether it was much closer up, a little closer up, the same, a little farther away, or much farther away in comparison with the first scene. Unbeknownst to the participants, the first and second scenes were identical.

Procedure.

Test sessions were recorded for later transcription. For each scene, participants were first asked four orienting questions about the scene (i.e., “Can you describe the object in this picture?”, “Can you describe the background in this picture?”, “Can you describe the colors in this picture?”, “What sort of a place do you think this picture was taken in?”). Next, participants were asked to imagine and describe what might come into view if they could see beyond the boundaries of the scene (i.e., “If you were taking the picture and you took a few steps backwards, what else do you think would come into view?”). Probing was used to elicit more detail, although not to suggest specific content (e.g., “Can you tell me any more about what you might see if you had a larger view?”). Participants next reported whether they could visualize the imagined scene (yes or no) and also rated the vividness of the imagined scene (1 = low vividness, 2 = medium, 3 = high).

The scene was then removed from view, and participants rated their imagined scenes by using the Spatial Coherence Index (SCI, 6). The SCI consists of 12 plausible statements about the quality of the imagined scenes (e.g., It was quite fragmented; I saw the scene in color). Participants were instructed to endorse as many statements as accurately described what they had imagined. Eight of the statements reflected good spatial coherence (scored as +1), and four statements reflected poor spatial coherence (scored as −1). Thus, the SCI yielded a score from −4 to +8. The order of the statements was randomized for each trial. After the SCI (mean duration = 2 min), participants were asked to recall the central object in the scene they had just viewed and to describe what they could remember about the scene.

Data analysis.

A first rater, who was not blind to participant identity, scored the content of the narratives following methods described (8). Narratives for imagined scenes were segmented into details, which were then classified as belonging to one of four categories: entities present (EP), sensory descriptions (SD), spatial references (SPA), and thoughts/emotions/actions (TEA). Repeated details, irrelevant details, and tangential information were discarded. The narratives were also scored by a second rater who was blind to participant identity (interrater reliability = 0.96). Each participant’s score was the average of the score obtained by each rater. The narratives for each remembered scene was also segmented into details by two raters and averaged (interrater reliability = 0.95).