Experience-Dependent Eye Movements Reflect Hippocampus-Dependent (Aware) Memory

Abstract

We investigated the relationship between experience-dependent eye movements, hippocampus-dependent memory, and aware memory. We measured eye movements in young adults, older adults, and memory-impaired patients with damage to the medial temporal lobe as they viewed 120 novel scenes and 120 previously viewed scenes. Participants indicated if each scene was old or new and also gave a confidence rating for the memory judgment. Young adults and older adults explored old scenes less than they explored new scenes, but the patients did not. For the young and older adults, this effect was observed only when participants were aware of the scene's familiar or novel status. In a second experiment, young adults viewed scenes that were either new, had been viewed previously, or had been viewed previously but had been changed (i.e., an object within the scene was either added or removed). The only instructions were to pay attention to the scenes and view each scene as it appeared, and there was no expectation that memory would be tested. Directly after the first altered scene was presented, participants were asked to classify the scene as new, old, or old but changed. Participants who were aware of the manipulation preferentially viewed the changed region, but participants who were unaware did not. These findings suggest that experience-dependent eye movements reflect hippocampus-dependent (and aware) memory, even when participants have no expectation that memory is being tested; and they are consistent with the view that awareness of what is learned is a fundamental characteristic of hippocampus-dependent memory.

Introduction

Memory is composed of different abilities that depend on different brain systems (Schacter and Tulving, 1994; Eichenbaum and Cohen, 2001; Squire et al., 2004). Declarative memory supports the ability to remember facts and events and depends on the integrity of the medial temporal lobe. A key feature of declarative memory is thought to be that the acquired knowledge is accessible to awareness (Tulving and Schacter, 1990; Squire, 1992; Eichenbaum, 1997; Gabrieli, 1998). In contrast, nondeclarative memory is independent of the medial temporal lobe and can be expressed without awareness of what has been learned.

In some cases, the relationship between memory performance and awareness remains unclear. Consider, for example, recent findings concerning how eye movements are influenced by past experience. One notable finding is that individuals explore familiar material less than they explore novel material (Althoff and Cohen, 1999; Ryan et al., 2000; Smith et al., 2006; Hannula et al., 2007; Ryan et al., 2007; Sharot et al., 2008). What kind of memory underlies this effect? One possibility is that differential eye movements in response to familiar and novel material are supported by nondeclarative (hippocampus-independent) memory (Ryan et al., 2000). For example, it is know that due to priming, decisions about recently presented objects are faster than decisions about novel objects (Schacter et al., 1990; Cave and Squire, 1992). Perhaps differential eye movements in response to familiar and novel scenes are a manifestation of the phenomenon of priming. Alternatively, it is also true that after studying a set of scenes individuals can consciously recognize some of them as familiar and others as novel. Perhaps these differential eye movements depend on awareness and are also hippocampus-dependent.

Another notable finding about how eye movements are affected by past experience is that individuals view the region of a familiar scene in which a change has been introduced more than they view a matched region of another scene that has not been changed (Ryan et al., 2000; Smith et al., 2006). In our earlier study, this eye-movement effect occurred only when individuals were aware that a change had occurred in the scene and not when they were unaware of the change (Smith et al., 2006). Further, memory-impaired patients were impaired at classifying the scenes as changed or unchanged. Together, these findings suggested that the eye-movement effect relies on hippocampus-dependent, aware memory. In contrast, Ryan et al. (2000) reported that both aware and unaware individuals exhibited this eye-movement effect.

It has been suggested that task instructions and expectation might be important in determining the relationship between eye movements and awareness (Greene, 2007). In our earlier study, but not in the study by Ryan et al. (2000), participants were instructed that they should try to remember the scenes they saw, and they were expecting memory to be tested when they viewed scenes that had been altered. Perhaps if there were no expectation of a memory test, eye movements might be directed toward the altered regions of a scene even when individuals were unaware of the alteration.

We addressed these questions about eye movements, awareness, and hippocampus-dependent memory in two experiments. First, we asked what kind of memory is expressed when individuals view familiar and novel scenes differently (i.e., declarative, hippocampus-dependent memory or nondeclarative memory). We measured eye movements in young adults, older adults, and memory-impaired patients with medial temporal lobe damage as they viewed familiar and novel scenes. We also assessed the relationship between eye movements and awareness of which scenes were familiar and which scenes were novel. Second, we asked about the relationship between eye movements and awareness when participants were not informed that memory would be tested. Specifically, we measured eye movements as young adults viewed novel scenes, familiar scenes, and familiar scenes that had been altered, and we then determined whether they were aware or unaware that the scene had been altered.

Materials and Methods

Experiment 1

Participants

Young adults.

Sixteen undergraduates from the University of California, San Diego participated for course credit (9 female, 19.9 ± 0.3 years of age).

Older adults.

Thirteen volunteers (four female) served as controls for the memory-impaired patients. They averaged 59.6 ± 2.6 years of age (patients = 61.0 ± 4.7 years), and had 14.8 ± 0.7 years of education (patients = 13.8 ± 0.9 years). The controls averaged 8.0 and 6.8 segments for immediate and delayed (12 min) prose recall (Gilbert et al., 1968), respectively.

Memory-impaired patients.

Four patients participated (Table 1). G.P. is severely amnesic and has large lesions of the medial temporal lobe. Three other patients are moderately amnesic and have lesions limited to the hippocampus (K.E., L.J., and G.W.). G.P. became amnesic in 1987 due to viral encephalitis. K.E. became amnesic after an episode of ischemia associated with kidney failure and toxic shock syndrome. L.J. (the only female) became amnesic during a 6 month period in 1988 with no known precipitating event. Her memory impairment has remained stable since that time. G.W. became amnesic after a drug overdose and associated respiratory failure. For the four patients, immediate and delayed prose recall averaged 6.9 and 0.3 segments, respectively.

Table 1.

Characteristics of memory-impaired patients

Patient	Age (years)	Education (years)	WAIS-III IQ	WMS-R
				Attention	Verbal	Visual	General	Delay
K.E.	65	13.5	108	114	64	84	72	55
L.J.	70	12	101	105	83	60	69	<50
G.P.	61	16	98	102	79	62	66	<50
G.W.	48	12	108	105	67	86	70	<50

The Wechsler Adult Intelligence Scale-III (WAIS-III) and the Wechsler Memory Scale-Revised (WMS-R) yield mean scores of 100 in the normal population with an SD of 15. The WMS-R does not provide numerical scores for individuals who score <50.

Estimates of medial temporal lobe damage were based on quantitative analysis of MR images from 19 healthy males (for the male patients) and 11 females for patient L.J. (Gold and Squire, 2005). G.P. has a bilateral reduction in hippocampal volume of 96%. The volume of the parahippocampal gyrus (temporopolar, perirhinal, entorhinal, and parahippocampal cortices) is reduced by 93%. K.E., L.J., and G.W. have a bilateral reduction in hippocampal volume of 49, 46, and 48%, respectively (all values >2 SDs from the control mean). The volume of the parahippocampal gyrus is reduced by 17, −8, and 12%, respectively (all values within two SDs of the control mean).

Additional measurements, based on four controls for each patient, were performed for the insular cortex, fusiform gyrus, frontal lobes, lateral temporal lobes, parietal lobes, and occipital lobes (Bayley et al., 2005). The only volume reduction in these regions >2 SDs of the control mean was the bilateral insular cortex and the bilateral fusiform gyrus for G.P. (reduced in volume 65% and 49%, respectively). Nine coronal MR images from each patient appear in supplemental Figure 1 (available at www.jneurosci.org as supplemental material).

Apparatus

Eye movements were recorded at 30 Hz with a ViewPoint Eye Tracker (Arrington Research) and PC-60 software (version 2.8.3,416) for detecting pupillary position. A fixation was scored when >100 ms elapsed without a saccade. A saccade was defined as an eye movement >0.7° within 33 ms (∼¼ inch on the 20 inch computer monitor). Head motion and position were maintained with a bite bar, forehead rest, and chin rest. Viewing was binocular, though only movements of the left eye were tracked. The eye tracker was adjusted for each participant before the test session. Correction for head motion was performed before each test block and, when needed, during a block. A separate computer controlled image presentation and recorded behavioral responses using E-prime software (version 1.2.1.844; Psychology Software Tools). An external keyboard was used to record responses.

Materials and procedure

Before the experiment, the set of scenes that would serve as repeated (120 scenes) and novel (120 scenes) for each participant was selected randomly from a set of 240 scenes. Participants studied 120 color photographs of indoor and outdoor scenes as they were presented for 2 s each on a computer monitor. They were instructed to pay attention to the photographs so that they might be able to recognize them later. Next, eye movements were recorded as participants viewed 240 scenes across six test blocks (5 s per scene; 40 scenes per block). Within each block, half of the scenes were repeated from the study session and half were novel. The order of novel and repeated scenes presented in the test session was unique and in a mixed order for each participant. Immediately after viewing each scene, participants made a confidence judgment on a scale from 1 to 6 (1 = “definitely new,” 2 = “probably new,” 3 = “maybe new,” 4 = “maybe old,” 5 = “probably old,” and 6 = “definitely old”). No time limit was imposed for the confidence judgment, and all participants were encouraged to use the full range of confidence ratings. After each recognition judgment was made, a red crosshair appeared on a gray background for 5 s, and then the next scene was shown. Memory-impaired patients were reminded about the task and the confidence judgments before each block (and if necessary within a block). The young adults were given the test session 24 h after the study session, and the older adults and the memory-impaired patients were given the test session ∼5 min after the study session (the time needed to give the instructions and calibrate the eye tracking apparatus). The study test interval was longer for the young adults than for the older adults and the memory-impaired patients to obtain a sufficient number of errors (misses and false alarms) for eye movement analysis. The memory-impaired patients completed the same experiment a second time with a different set of 240 color photographs (1–7 months elapsed between tests). The memory scores and the eye movement data from the two test sessions were very similar and were averaged together for each patient.

Data analysis

Two eye-movement measures were taken to assess how participants viewed the scenes during the recognition memory test: (1) number of regions sampled, i.e., the number of different regions (out of 16) in which a fixation was detected (for this measure, each scene was divided into 16 equal-sized regions to form a 4 × 4 grid); (2) number of fixations during the 5 s viewing period.

Experiment 2

Participants

Fifty-one undergraduates from the University of California, San Diego participated for course credit (41 female, 20.1 ± 0.3 years of age).

Apparatus

The same apparatus was used as in Experiment 1. Correction for head motion was performed before each block and, when needed, during a block.

Materials and procedure

The design and the method for measuring eye movements were based on an earlier study of eye movements (Smith et al., 2006). The key difference was that in the present experiment participants were not informed that memory would be tested.

In each of three test blocks (1–3 min between blocks), color photographs of indoor and outdoor scenes were presented for 5 s each. After each scene presentation, a red crosshair appeared for 5 s on a gray background. Block 1 consisted of 24 novel scenes, block 2 consisted of 24 scenes (8 of which were novel and 16 of which were repeated from block 1), and block 3 consisted of 3 scenes. Of the 3 scenes, one was novel, one was repeated, and one was a manipulated version of a scene that had appeared in blocks 1 and 2. Half of the time, the manipulation consisted of adding an object to a scene presented in blocks 1 and 2, and half of the time the manipulation consisted of removing an object from a scene presented in blocks 1 and 2. For blocks 1 and 2, the order of the 24 scenes was randomized across participants. For block 3, the scenes were presented in one of two orders: (1) novel, repeated, and manipulated; or (2) repeated, novel, and manipulated. Across participants, these two orders occurred equally often. In this way, eye movements for a novel scene, a repeated scene, and a manipulated scene could be recorded before participants learned that their memory for the scenes was to be tested (see below).

All scenes were available in an original and a manipulated version. The scenes were counterbalanced across participants such that, for every 10 participants, the original and manipulated version of each scene served equally often in each of the three blocks and in the novel, repeated, and manipulated conditions. For each scene, a critical region was identified where the manipulation would occur whenever that scene was assigned to the manipulated condition. The manipulation was always introduced in block 3.

Before testing, participants were told that the purpose of the experiment was to learn how people look at pictures during full attention. They were instructed that it was important that they pay attention to each picture, that they remain alert, and that they look at the scene for the full time that it remained on the screen. Before blocks 2 and 3, participants were instructed that they would see more photos, and were reminded to keep their head still. There was no indication at any point that memory for the scenes was relevant to the experiment.

Directly after the first manipulated scene was presented in block 3 (and removed from view), the experiment was stopped, and participants were asked to indicate the status of the scene they had just seen: “New” (for a novel scene), “Identical” (for a repeated scene), or “Changed” (for a manipulated scene). When a scene was identified as changed, the scene was presented again, and participants were asked to describe the manipulation. For manipulated scenes not identified correctly (because they were labeled “New” or “Identical”), the scene was presented again, participants were told that the scene was manipulated, and they were asked to try to describe the manipulation.

Data analysis

Measures of interest for eye movements in response to novel, repeated, and manipulated scenes.

Three measures were used to assess how participants viewed the critical (manipulated) region of novel and repeated scenes during block 3 as well as how participants viewed the critical (unmanipulated) region of novel and repeated scenes: (1) proportion of fixations in the critical region, i.e., the number of fixations in the critical region divided by the total number of fixations; (2) proportion of viewing time in the critical region, i.e., the amount of time spent viewing the critical region divided by total viewing time (5 s); and (3) number of transitions into/out of the critical region, i.e., the number of times that participants transitioned from a fixation outside the critical region to a fixation inside the critical region or vice versa. For these measures, the critical region occupied one region in either a 4 × 4 grid of 16 equal-sized regions (43% of scenes) or a 3 × 3 grid of 9 equal-sized regions (57% of scenes). The number of regions (9 or 16) depended on the size and location of the manipulated object.

Two measures were used to assess viewing of novel and repeated scenes in block 2: (1) number of regions sampled and (2) number of fixations (see above, Experiment 1, Data analysis).

Designation of awareness or unawareness.

We assessed the relationship between awareness and eye movements made in response to the manipulated scene in a circumstance where participants did not know that their memory (and awareness) would be tested. Participants were designated as aware of a manipulated scene if they correctly recognized it as manipulated and also correctly described what the manipulation was [e.g., a man and his dog used to be on the right-hand side of the scene (see Fig. 4, bottom right panel)] or if they indicated the correct region of the scene that was manipulated. Participants were designated as unaware of a manipulated scene if they failed to identify it as manipulated and also could not describe correctly what the manipulation was (or indicate the correct region), even after being informed that a manipulation had been introduced. Participants were designated as having an intermediate level of awareness if they responded correctly to one of the two queries but not the other.

Figure 4.

Experiment 2. Eye movement traces and fixations (white diamonds) for one participant subsequently designated as aware (top panels) and for another participant designated as unaware (bottom panels) of the manipulation that occurred in block 3. In both cases, the scene was novel in block 1 (left panels), it was repeated in block 2 (center panels), and it was repeated in block 3 but with a manipulation introduced into the manipulated region (right panels). In block 3, the participant who was aware of the manipulation directed a greater proportion of her fixations to the manipulated region than the participant who was unaware of the manipulation. In each panel, the manipulated region is identified by a black square, but the square did not appear during the experiment (eye movements appeared to move beyond the scene when a participant blinked during the 5 s scene presentation).

Results

Experiment 1

Memory performance

Young adults and older adults exhibited good recognition memory for the scenes (73.6 ± 1.6% correct and 76.0 ± 2.2% correct, respectively) (Fig. 1). Memory-impaired patients were impaired relative to both of these groups (65.1 ± 4.4% correct, p values < 0.05) although they did perform above chance levels (50% correct, p < 0.05).

Figure 1.

Mean recognition memory accuracy for 120 old and 120 new scenes in Experiment 1. Young adults were tested 24 h after study, older adults were tested ∼ 5 min after study, and memory-impaired patients were also tested ∼ 5 min after study. Memory-impaired patients performed poorer than young and older adults but above the chance level of 50% correct. Error bars indicate SEM.

Confidence ratings were a reliable indicator of recognition accuracy. Each of the three groups exhibited better memory when they expressed more confidence in their old–new memory judgments (linear trend in accuracy across confidence ratings; p values < 0.05). Specifically, young adults scored 57.1 ± 1.4% correct when giving confidence ratings of 3 or 4 (maybe sure; 76.9 trials), 71.9 ± 2.2% correct when giving ratings of 2 or 5 (probably sure; 78.4 trials), and 90.8 ± 1.6% correct when giving confidence ratings of 1 or 6 (definitely sure; 80.9 trials). Older adults scored 54.9 ± 2.9%, 66.8 ± 2.5%, and 87.1 ± 2.2% correct, respectively, across these same three levels of confidence (48.9, 63.6, and 126.1 trials, respectively). Memory-impaired patients scored 52.2 ± 4.0%, 60.6 ± 4.0%, and 71.3 ± 6.0% correct across the three levels of confidence (42.4, 51.5, and 137.8 trials, respectively). Although both patients and older adults used the full range of memory confidence ratings (maybe sure, probably sure, and definitely sure), the memory-impaired patients had poorer memory than older adults, even when they indicated they were definitely sure (p < 0.01).

Eye movements associated with repeated and novel scenes

Young adults.

We first assessed viewing for all novel and all repeated scenes, regardless of recognition accuracy. Findings for the regions sampled measure appear in Figure 2. Findings for the fixation measure appear in the text. Young adults sampled fewer regions and made fewer fixations when scenes were repeated than when scenes were novel [for regions sampled, t₍₁₅₎ = 4.2, p < 0.001 (Fig. 2A, left); for fixations, t₍₁₅₎ = 2.4, p < 0.05]. Next, we assessed how confidence in the old–new judgments affected the viewing of novel and repeated scenes. All scenes were used in this analysis, regardless of whether their old–new judgment was correct. Participants viewed repeated scenes less than novel scenes only when they were confident in their old–new judgment. Specifically, for the number of regions sampled (Fig. 2A, center), repeated scenes were viewed less only when participants gave confidence ratings of 1, 2, 5, or 6 (probably sure or definitely sure; p values < 0.05), but not when they were guessing (ratings of 3 or 4 = maybe sure; p > 0.50). For the fixations measure, repeated scenes were viewed less only when participants gave confidence ratings of 1 or 6 (definitely sure, p < 0.05), but not when they gave confidence ratings of 2, 3, 4, or 5 (maybe sure or probably sure; p values > 0.20). Thus, when participants were guessing about whether a scene was repeated or novel, they viewed repeated and novel scenes similarly.

Figure 2.

Eye movements associated with novel and repeated scenes for young adults, older adults, and memory-impaired patients in Experiment 1. A, Left, Young adults sampled fewer regions of a scene when the scene was repeated (R) than when it was novel (N). Center, Young adults exhibited this effect only when they were confident in their old–new judgment (when they were probably sure or definitely sure). Right, Young adults also exhibited this effect only when their old–new judgment was correct and not when it was incorrect. B, Left, Older adults also sampled fewer regions of a scene when the scene was repeated than when it was novel. Center, Older adults exhibited this effect when they were highly confident of their old–new judgment (confidence ratings = definitely sure). Right, Like young adults, older adults exhibited this effect only when their old–new judgment (when they were definitely sure). C, Left, Memory-impaired patients viewed repeated scenes no differently than novel scenes. Center, For patients, viewing was not related to the confidence rating given for the old–new judgments. Right, For patients, viewing was also unrelated to whether the old–new judgment was correct or incorrect. Error bars indicate SE of the difference between viewing novel and repeated scenes. Asterisks indicate a significant difference between the viewing of novel and repeated scenes (p < 0.05).

It is important to emphasize that when the young adults were highly confident, they were quite accurate (91% correct) and when they were guessing they were inaccurate (57% correct). We suggest that when the young adults were highly confident and accurate, they were aware of which scenes were old and which were new. When they were guessing (and inaccurate), they were less aware.

Last, we assessed the effect of recognition memory accuracy on the viewing of novel and repeated scenes. Specifically, we examined separately the eye movements that occurred when novel and repeated scenes were correctly identified and the eye movements that occurred when novel and repeated scenes were incorrectly identified. We found that participants viewed repeated scenes less than novel scenes when the scenes were correctly identified as old or new [for the number of regions sampled (Fig. 2A, right) and for the number of fixations; p values < 0.01], but not when the scenes were incorrectly identified (for the number of regions sampled and for the number of fixations; p values > 0.20). Thus, young adults viewed repeated and novel scenes differently only when they judged accurately (and, we suggest, when they were aware) that the repeated scenes were repeated and the new scenes were new.

We noted that the mean confidence rating for correctly identified novel scenes was lower than the mean confidence rating for correctly identified repeated scenes (2.0 ± 0.06 vs 2.3 ± 0.05, respectively; p < 0.001). Furthermore, the mean confidence rating for incorrectly identified novel scenes was lower than the confidence rating for incorrectly identified repeated scenes (1.4 ± 0.05 vs 1.7 ± 0.07, respectively; p < 0.001). Thus, correct and incorrect responses were confounded with the confidence with which those responses were made. We therefore performed an additional analysis after equating confidence ratings for novel and repeated scenes that were correctly identified and for novel and repeated scenes that were incorrectly identified. To equate confidence ratings, we eliminated trials (but each participant always retained at least 10 novel and 10 repeated scenes). After these corrections, the mean confidence rating for correctly identified scenes was 2.3 (both novel and repeated), and the mean confidence rating for incorrectly identified scenes was 1.6 (both novel and repeated). The findings remained the same. Participants viewed repeated scenes less than novel scenes only when they correctly identified the scenes as novel or repeated (for the number of regions sampled and for the number of fixations; p values < 0.01).

Older adults.

The findings for older adults were similar to those for young adults. Older adults sampled significantly fewer regions and made numerically fewer fixations when scenes were repeated than when scenes were novel [for the number of regions sampled, t₍₁₂₎ = 5.2, p < 0.001 (Fig. 2B, left); for the number of fixations, t₍₁₂₎ = 1.5, p = 0.16]. When trials were sorted according to how confident participants were of their old–new judgments, participants viewed repeated scenes less than novel scenes only when they were highly confident of their old–new judgment (ratings of 1 or 6 = definitely sure). For the number of regions sampled, this difference between old and new scenes was highly significant (p < 0.001) (Fig. 2B, center). For the fixations measure, the difference was marginally significant (p = 0.051). In addition, participants viewed repeated scenes less than novel scenes only when the scenes were correctly identified as old or new [for regions sampled (Fig. 2B, right) and for the number of fixations; p values < 0.05], but not when the scenes were incorrectly identified (p values > 0.20). Thus, as was the case for the young adults, the older adults exhibited differential eye movements to old and new scenes only when they were highly confident and quite accurate in their judgments (87% correct) and not when they were guessing (and inaccurate, 55% correct). We suggest that being highly confident and accurate is an indication of being aware of which scenes were old and which were new.

Last, confidence ratings were equated for novel and repeated scenes using the same method that was used for young adults (see above). The findings remained the same. That is, participants viewed repeated scenes less than novel scenes only when they correctly identified the scenes as novel or repeated (for the number of regions sampled and for the number of fixations; p values <0.01). Thus, older adults, like young adults, viewed repeated and novel scenes differently only when they were aware of which scenes were repeated and which were novel.

Memory-impaired patients.

Unlike the two groups of healthy participants, memory-impaired patients viewed novel scenes and repeated scenes similarly [for the number of regions sampled, t₍₃₎ = 1.9, p = 0.15 (Fig. 2C, left); for the number of fixations, t₍₃₎ = 1.5, p > 0.20]. Furthermore, viewing was unrelated to how confident the patients were in their old–new judgments. That is, for both the number of regions sampled (Fig. 2C, center) and the number of fixations, novel and repeated scenes were viewed similarly even when the patients gave a high-confidence response (ratings of 1 or 6 = definitely sure; p values > 0.30). Note that even when the patients gave high-confidence responses, they were considerably less accurate than the young adults and older adults (71% correct vs 91% and 87% correct). Last, in contrast to the result for young and older adults, the viewing of novel and repeated scenes was unrelated to whether patients were correct or incorrect in identifying the scenes as novel or repeated [for the number of regions sampled, p = 0.16 (Fig. 2C, right) and for the number of fixations, p > 0.40].

To determine whether memory-impaired patients had any preexisting abnormalities in eye movements that might have influenced how their eye movements were affected by experience, we asked how the patients examined novel scenes in comparison to their age-matched, healthy controls [Fig. 2, compare B (left) and C (left)]. The patients sampled 5.23 ± 0.38 (mean ± SEM) regions of the novel scenes, and the older adults sampled 5.17 ± 0.21 regions [t₍₁₅₎ = 0.16, p > 0.80]. In addition, the patients made 13.4 ± 1.3 fixations, and the older adults made 13.2 ± 0.8 fixations [t₍₁₅₎ = 0.16, p > 0.80]. Thus, the failure of the patients to view novel and repeated scenes differently is unlikely to be related to abnormal eye movements per se.

Experiment 2

The three scenes presented in block 3 (the novel scene, the repeated scene, and the manipulated scene) provided the critical data. While viewing these scenes, participants did not yet know that their memory for the scenes would be tested. Therefore, eye movements in response to the three scenes in block 3 were free from any effect due to knowledge that a scene would be manipulated and that memory would be tested.

Awareness and eye movements in block 3

Twenty-six participants were aware of the manipulation that appeared in the manipulated scene in block 3, 11 were unaware, and 14 had an intermediate level of awareness. The critical comparison involved the repeated scene and the manipulated scene in block 3. For both these scenes, the area outside the critical region remained the same across blocks 1, 2, and 3. As a result, after balancing for which scene was presented, differences in how participants viewed the repeated scene and the manipulated scene should have been influenced only by what changed within the critical region.

Figure 3 shows that when participants were aware of a manipulation, they looked more at the manipulated (critical) region than at the unchanged critical region in the repeated scene. Specifically, they exhibited a greater proportion of fixations in the critical region (0.45 ± 0.05), a greater proportion of viewing time in the critical region (0.48 ± 0.06), and made more transitions into and out of the critical region (5.6 ± 0.6) than in the unchanged critical region of the repeated scene (0.29 ± 0.05, 0.29 ± 0.05, and 3.5 ± 0.6, respectively; p values < 0.05). In contrast, when participants were unaware of the manipulation, the proportion of fixations in the critical region (0.24 ± 0.07), the proportion of viewing time in the critical region (0.24 ± 0.07), and the number of transitions into and out of the critical region (3.0 ± 0.8) were about the same for the manipulated scene and for the repeated scene (0.27 ± 0.07, 0.29 ± 0.08, and 3.4 ± 1.2, respectively; p values > 0.40) (Fig. 3). Like those participants designated as unaware, participants that exhibited an intermediate level of awareness of the manipulation also looked at the critical region of the manipulated scene no differently than they looked at the unchanged critical region in the repeated scene (for the manipulated scene, the three measures were 0.29 ± 0.05, 0.30 ± 0.06, 3.7 ± 0.6; for the repeated scene, they were 0.27 ± 0.07, 0.29 ± 0.08, and 3.9 ± 0.9, respectively, p values > 0.80) (Fig. 3).

Figure 3.

Viewing of the manipulated (critical) region by participants in Experiment 2 who were designated as aware, unaware, or intermediately aware of the manipulation. The eye-movement data come from the three scenes that were presented in block 3, before participants learned that memory was relevant to the study. The scene was either novel (N), repeated from blocks 1 and 2 (R), or repeated from blocks 1 and 2 but with a manipulation (M). Only participants who were aware of the manipulation directed a higher proportion of their fixations to the manipulated region (top), spent more viewing time in the manipulated region (middle), and made more transitions into/out of the manipulated region of the scene (bottom) than in the case of the matched, unmanipulated (critical) regions of the novel and repeated scene. In contrast, participants who were designated as unaware or as having an intermediate level of awareness for the manipulation viewed the critical region of manipulated, novel, and repeated scenes in a similar way. Asterisks indicate a significant difference between the viewing of novel and repeated scenes (p < 0.05). Error bars indicate SEM.

Another way to make the same point is to note that, for all three measures, participants who were aware of the manipulation looked more at the critical region than participants who were unaware of the manipulation (0.45 vs 0.24, 0.48 vs 0.24, 5.6 vs 3.0; p values < 0.05) or participants who were intermediately aware (0.45 vs 0.29, 0.48 vs 0.30, 5.6 vs 3.7; p < 0.05, p < 0.06, p < 0.06) (Fig. 3). Figure 4 illustrates eye movement traces and fixations for a participant who was aware of the manipulation and a participant who was unaware of the manipulation.

Importantly, viewing of the to-be-manipulated region was similar for aware and unaware participants before block 3. Thus, in blocks 1 and 2, participants later designated as aware viewed the critical region of the to-be-manipulated scene to the same extent as participants later designated as unaware (proportion of fixations, 0.32 ± 0.03 vs 0.27 ± 0.06; proportion of viewing time, 0.33 ± 0.04 vs 0.27 ± 0.06; and number of transitions into/out of the critical region, 4.3 ± 0.49 vs 3.5 ± 0.64; p values > 0.30). Thus, the tendency for aware participants to view the critical region of the manipulated scene appeared only in block 3 and did not reflect some preexisting preference to look at this region of the scene.

The results remained the same even after we took steps to equate more exactly the viewing of the critical region during blocks 1 and 2. For each of the three eye movement measures, participants subsequently designated as aware were eliminated from the analysis until viewing in the critical region during blocks 1 and 2 was exactly equal for aware and unaware groups (4 or 5 participants out of the 26 aware participants were eliminated for each measure). The result was that for all three eye movement measures, aware participants in block 3 still viewed the critical region of the manipulated scene more than they viewed the critical region of the repeated scene (p values < 0.05). Furthermore, for all three measures, the aware participants still viewed the critical region of the manipulated scene more than did the unaware participants (p values < 0.05).

Eye movements in block 2

We also asked whether participants viewed repeated and novel scenes differently in block 2 at a time when they did not know that memory would be tested. We assessed viewing for all novel scenes (8 scenes) and all repeated scenes (16 scenes) in block 2. The finding was the same as in Experiment 1 when participants knew that memory was being tested. Thus, even when there was no indication that memory for the scenes was relevant to the experiment, participants sampled fewer regions and made fewer fixations when scenes were repeated than when scenes were novel [for regions sampled, t₍₅₀₎ = 3.0, p < 0.01; for fixations, t₍₅₀₎ = 3.6, p < 0.001].

We could not measure awareness for which scenes were repeated and which scenes were novel in block 2 because questions about the scenes at that stage would have defeated our plan for block 3; namely, to assess eye movements when there was no expectation that memory was being tested. We also could not measure the effect of awareness on viewing the single repeated and single novel scene in block 3, because there would have been few, if any, instances of unawareness. Only a total of 32 different scenes had been presented before block 3, and in our earlier study that used the same methods (Smith et al., 2006, Experiment 2), memory performance exceeded 90% correct in block 3. Thus, participants would almost always have been aware of the old–new status of the repeated and novel scene.

Discussion

In two experiments, we investigated the relationship between experience-dependent eye movements, awareness, and hippocampus-dependent memory. In Experiment 1, young adults and older adults viewed 120 novel scenes and 120 repeated scenes and made an old–new judgment for each scene together with a confidence rating. Both groups sampled fewer regions and made fewer fixations when viewing repeated scenes than when viewing novel scenes. The important finding was that the two groups exhibited these eye movement effects only when they were aware of the familiar or novel status of the scenes. First, they exhibited these effects only when they were confident (and accurate) in their old–new judgment and not when they were guessing (and inaccurate). Second, they exhibited these effects only when their old–new judgments were correct and not when they were incorrect. In contrast, memory-impaired patients failed to view the familiar and the novel scenes differently, and they were also impaired at remembering which scenes were familiar and which were novel. We suggest that the patients were insufficiently aware of which scenes were new and which were old. As a result, they did not exhibit differential eye movements to old and new scenes, even on occasions when they were confident (albeit relatively inaccurate) in their old–new judgments and even when their old–new judgments were correct.

These findings are the first to identify the importance of awareness when eye movements are directed toward novel scenes and familiar scenes. Other studies have reported, as we did, that familiar scenes are explored less than novel scenes (Althoff and Cohen, 1999; Ryan et al., 2000, 2007; Smith et al., 2006; Hannula et al., 2007; Sharot et al., 2008), but the relationship between this phenomenon and awareness of the familiar/novel status of the scenes has not previously been determined.

It is noteworthy that the memory-impaired patients explored familiar and novel scenes to the same degree and also performed poorly on a memory test for the scenes. In contrast, in one earlier study (Ryan et al., 2000; Experiment 4), memory-impaired patients explored familiar scenes less than novel scenes, just like control participants. Memory for the scenes was not tested. However, patients in that experiment studied many fewer scenes than the patients in our study (32 studied scenes vs 120 studied scenes). Memory-impaired patients can do well on simple recognition tests that involve short lists (e.g., Wais et al., 2006). It is possible that the patients tested by Ryan et al. (2000) had some declarative memory for the scenes they studied and subsequently viewed familiar and novel scenes differently because they could often recognize which scenes were familiar and which scenes were novel.

In Experiment 2, we asked whether the relationship between awareness and the tendency to direct eye movements toward the manipulated region of a scene depends on participants being given memory instructions and expecting their memory to be tested. This experiment addressed the suggestion that the link between hippocampus-dependent memory and awareness for what has been learned might not hold if participants had no expectation that memory would be tested (Greene, 2007). Thus, in Experiment 2, participants were not told to remember any scenes and, at the time eye movements were recorded, they had not been informed that memory would be tested. The finding was that participants who were aware that a change had been introduced looked more at the changed region of the manipulated scene than at a matched, unchanged region of a repeated scene. Participants designated as unaware or as having an intermediate level of awareness for the manipulation viewed the changed region of the manipulated scene and the matched region of the repeated scene similarly.

These results indicate that eye movements in response to manipulated scenes depend on awareness that a manipulation has occurred. Importantly, this result was independent of task instructions. That is, the same result was obtained when participants were given no indication that memory would be tested (Experiment 2) as was obtained previously when participants expected their memory for the scenes to be tested (Smith et al., 2006). This finding rules out the suggestion that the link we found between awareness and these eye movement effects may have occurred because participants expected that memory would be tested (Greene, 2007).

In Experiments 1 and 2, the tendency to explore familiar scenes less than novel scenes and the tendency to explore the manipulated region of a familiar scene were both related to aware memory for the scenes (see also Smith et al., 2006). Two earlier studies reached different conclusions; namely that experience-dependent eye movements can occur in the absence of awareness for the material being viewed (Ryan et al., 2000; Laeng et al., 2007). In one study (Laeng et al., 2007), two memory-impaired patients and their controls directed a disproportionate amount of viewing time toward the quadrant of a screen where an object had appeared recently (controls, 57%; patients, 36%; chance = 25%). In a separate memory test involving one of the patients, the patient pointed to the correct quadrant 56% of the time, which (because this test was structured differently than the eye movement test) was taken to be no different from chance performance (controls were not tested). These findings do not provide strong evidence for unaware eye movements. In the absence of comparisons between patients and controls on both measures, one does not know whether eye movements and pointing were impaired similarly in the patient group, or whether, as the authors concluded, the eye movement effects were differentially spared.

In a second experiment (Laeng et al., 2007), three memory-impaired patients exhibited larger pupillary diameters when new items were presented than when familiar items were presented, even though the patients were unable to recognize the items as novel or familiar in a separate test (controls were not tested). In the absence of control data, these results are difficult to interpret. If the pupillary effect were as large and as consistent in the patients as in controls, then this effect may be based on nondeclarative (and unaware) memory. However, if the pupillary effect were smaller or less consistent in the patients than in the controls, then nondeclarative memory need not be involved. Pupillary size and a recognition test may both be sensitive to declarative memory, but pupillary size may be a more sensitive measure than recognition (in the same sense that recognition tests can detect memory when recall tests fail altogether).

In another study (Ryan et al., 2000, Experiment 3) participants designated as unaware nonetheless viewed the changed region of a manipulated scene as much as aware participants. The basic design of this study was similar to the design of our experiments [Experiment 2 of the present study and Experiments 1 and 2 of our previous report (Smith et al., 2006)]. Two notable differences between this study and our experiments were (1) the way in which the scenes were manipulated in block 3; and (2) the instructions that participants received.

In our studies, scenes were manipulated by either adding or removing an object from the scene. In Ryan et al. (2000), scenes were manipulated by adding an object, removing an object, or shifting the left-right location of an object. Greene (2007) suggested that our method likely resulted in more awareness of the manipulations. Further, he suggested that a high level of awareness might predispose participants to engage a memory strategy during the experiment, thereby inhibiting the expression of unaware memory. In fact, participants in Ryan et al. (2000) were aware of more manipulations than participants in our experiment [Ryan et al. (2000), 57%; Smith et al. (2006), Experiment 1, 50%], probably because the instructions (see below) directed participants to the critical region and made that region particularly salient.

A second difference between the two studies was that the instructions were not the same. In the study by Ryan et al. (2000), participants were asked orienting questions in blocks 1 and 2 about the region of the scene that was to be manipulated, and they were asked the same orienting questions again in block 3 after the scene had been manipulated. Asking participants about the manipulated part of the scene would presumably have directed all participants to view this region of the scene in block 3, regardless of whether they were aware or unaware of the manipulation. At the same time, this unique procedure may not entirely explain why unaware participants preferentially viewed the manipulated region. Specifically, one would expect the orienting questions to direct viewing toward the critical region of all scenes, whether they were manipulated or not. Yet, both aware and unaware participants directed more viewing toward the critical region of manipulated scenes than the repeated scenes (but note that this difference between manipulated and repeated scenes was apparently not significant for either group alone).

In any case, Ryan et al. (2000) used an unusual method to study eye movements, i.e., instructing participants to look at the region of interest, and they concluded that participants tended to look at the manipulated region of a scene even when they were unaware of the manipulation. In our studies (the present study, Experiment 2; Smith et al., 2006), we found in three different conditions that participants looked at the manipulated region of a scene only when they were aware of the manipulation. Thus, the effect occurred when participants expected memory to be tested (Smith et al., 2006) as well as when there was no expectation of memory testing (the present study, Experiment 2). In addition, in our earlier study, the effect occurred both when memory was tested after all the scenes had been viewed (Experiment 1) and also when memory was tested immediately after viewing each scene (Experiment 2). Further work would be needed to explain why the findings of Ryan et al. (2000) were obtained in the specific conditions of their study. We suggest that for naturalistic, free-viewing conditions our findings provide an account of how memory affects eye movements in response to novel, familiar, and manipulated scenes.

In summary, we studied the effect of experience on eye movements. First, when individuals viewed a series of scenes, they explored familiar scenes less than novel scenes. This effect depended on aware, hippocampus-dependent memory. Second, when individuals viewed a familiar scene that had been manipulated, they preferentially explored the manipulated region. This effect also depended on aware memory, even when there was no expectation that memory would be tested. Thus, in a variety of conditions, experience-dependent eye movements reflect the kind of memory that depends on the hippocampus and that is accessible to conscious awareness for what has been learned. The findings support the conclusion that awareness of what is learned is a fundamental characteristic of hippocampus-dependent memory.