Experience-Dependent Eye Movements, Awareness, and Hippocampus-Dependent Memory

Abstract

We asked what kind of memory is operating when eye movements change as the result of experience. Participants viewed scenes that were either novel, repeated, or manipulated (i.e., a change was introduced in one region of the scene). Eye movements differed depending on the past viewing history of each scene. Participants made fewer fixations and sampled fewer regions when scenes were repeated than when scenes were novel. When scenes were altered, participants made more fixations in the altered region, spent more time looking at the altered region, and made more transitions into and out of the altered region than in unchanged (matched) regions in the repeated scenes. Importantly, these effects occurred only when individuals were aware that a change had occurred. Participants who were unaware that the scene had been altered looked at the changed scenes in the same way that they looked at repeated scenes. Thus, there was no indication that eye movements could reveal an unaware (unconscious) form of memory. Instead, eye movements reflected conscious memory of whether the scene was repeated or manipulated. The findings were the same when awareness was assessed after viewing all the scenes (experiment 1) and when awareness was assessed after each scene was presented (experiment 2). In experiment 3, memory-impaired patients with damage limited to the hippocampus were impaired at deciding whether scenes were novel, repeated, or manipulated. Thus, the ability to consciously recollect recent encounters with scenes reflects a form of hippocampus-dependent memory. The findings show that experience-dependent eye movements in response to altered scenes reflect conscious, declarative memory, and they support the link between aware memory, declarative memory, and hippocampus-dependent memory.

Introduction

Memory is not a unitary faculty of the mind but is composed of distinct abilities that depend on different brain systems (Schacter and Tulving, 1994; Eichenbaum and Cohen, 2001; Squire et al., 2004). One major system, declarative memory, supports the ability to remember facts and events and depends on the integrity of the hippocampus and related structures. Declarative memory is thought to be accompanied by knowledge or awareness of what has been learned, and the availability of learned material to conscious awareness has been considered one of its key features (Tulving and Schacter, 1990; Squire, 1992; Eichenbaum, 1997; Gabrieli, 1998).

In some cases, when behavior is changed by experience, it is unclear what kind of memory is being expressed. For example, when individuals view novel scenes, familiar scenes, or familiar scenes in which a change has been introduced, eye movements across the scenes differ depending on the past viewing history of each scene (Yarbus, 1967; Althoff and Cohen, 1999; Ryan et al., 2000). The question naturally arises as to what kind of memory is indexed by eye movements? On the one hand, eye movements could reflect conscious, declarative memory and correlate with other conventional measures of remembrance like recall and recognition (Manns et al., 2000). In support of this possibility, experience-dependent eye movements are often dependent on the hippocampus and related structures (McKee and Squire, 1993; Ryan et al., 2000; Zola et al., 2000; Nemanic et al., 2004). On the other hand, experience-dependent eye movements have been reported to occur in the absence of conscious memory for past events (Ryan et al., 2000). Such a finding would mean that eye movements can reveal the effects of past experience independent of an individual's conscious recollection of the experience. Furthermore, if these experience-dependent eye movements are hippocampus dependent, then such a finding would challenge the traditional idea that hippocampus-dependent memory is accompanied by conscious knowledge of what has been learned.

In two experiments, we measured eye movements as individuals viewed novel scenes, repeated scenes, and repeated scenes that had been altered. We also assessed whether participants were aware or unaware that a scene was novel, repeated, or altered. In a third experiment, we asked whether the ability to classify scenes as novel, repeated, or manipulated depends on the hippocampus. The results clarify the relationship between experience-dependent eye movements, awareness, and hippocampus-dependent memory.

Materials and Methods

Experiment 1

Participants

Twenty undergraduates participated for course credit (16 females, 21.2 ± 0.7 years of age).

Apparatus

Eye movements were recorded at 30 Hz with a ViewPoint eye tracker (Arrington Research, Scottsdale, AZ) and PC-60 software (version 2.7.1.55) for detecting pupillary position. A fixation was scored when >100 ms elapsed without a saccade. A saccade was defined as an eye movement >0.7° within 33 ms (∼0.25 inch on the 20 inch computer monitor). Head motion and position were maintained with a bite bar and forehead rest. Viewing was binocular, although only movements of the left eye were tracked. The eye tracker was adjusted for each participant before the test session. Correction for head motion was performed between blocks and, when needed, during a block. A separate computer controlled image presentation and recorded behavioral responses using E-prime software (version 1.1 SP3; Psychology Software Tools, Pittsburgh, PA). An external keyboard was used to record behavioral responses.

Materials and procedure

The experimental design and the method for measuring eye movements were based on a previous study of eye movements and experience-dependent behavior (Ryan et al., 2000). In each of three blocks, 24 color photographs of indoor and outdoor scenes were presented for 5 s each. After each image presentation, a red crosshair appeared for 5 s on a gray background. Three types of images were presented (Fig. 1). Eight images were novel in each of the three blocks (novel images). Eight images were novel in block 1 and were then repeated in blocks 2 and 3 (repeated images). Eight images were novel in block 1, repeated in block 2, and then manipulated in block 3 (manipulated images). Half of the manipulations consisted of adding an object to a previously studied image, and half consisted of removing an object from a previously studied image. Unlike Ryan et al. (2000), we did not manipulate the left-right position of an object within an image.

Figure 1.

Design of experiments 1, 2, and 3. In each of three blocks participants saw 24 color images (5 s/image). Eight images were novel in each block (top row), eight images were repeated in blocks 2 and 3 (middle row), and eight images were repeated in block 2 and then manipulated in block 3 (bottom row). In block 3 a manipulation occurred in the critical region of manipulated images. No change occurred in the critical region of repeated images. The critical regions are identified by black squares in the right panels, but these squares did not appear during testing. All images were available in an original and manipulated version. The images were counterbalanced across participants such that, for each group of 10 participants, each image served equally often in each of the three blocks and in the novel, repeated, and manipulated conditions.

All images were available in an original and a manipulated version. The images were counterbalanced across participants such that, for every 10 participants, the original or altered version of each image served equally often in each of the three blocks and in the novel, repeated, and manipulated conditions. For each image, a critical region was identified where the manipulation would occur whenever the image was assigned to the manipulated condition. In the case of manipulated images, the manipulation was introduced in the critical region in block 3. In the case of novel and repeated images, no manipulation was introduced in the critical region.

Before each block, participants were instructed to pay attention to the images so that they might be able to recognize them later. Directly after the third block, the images from block 3 were presented again one at a time for a memory test, and participants chose one of three responses to indicate the image's status in block 3: “it was new” (for novel images), “it was identical” (for repeated images), or “it was changed” (for manipulated images). For images identified as manipulated, participants were also asked to describe the manipulation. For manipulated images that were not identified correctly (because they were labeled “new” or “repeated”), participants were told that the image was manipulated and were asked to try to describe the manipulation. One to 3 min elapsed between blocks.

Data analysis

Measures of interest for eye movements in blocks 1 and 2 in response to novel and repeated images.

Two measures were used to assess how participants viewed the images during blocks 1 and 2: (1) number of fixations and (2) number of regions sampled [number of regions (of 16) in which fixations were detected]. For this measure, each image was divided into 16 equal-sized regions to form a 4 × 4 grid.

Measures of interest for eye movements in block 3 in response to novel, repeated, and manipulated images.

Three measures were used to assess how participants viewed the critical (manipulated) region of each manipulated image during block 3 as well as how participants viewed the critical (unmanipulated) region of novel and repeated images: (1) proportion of fixations in the critical region (the number of fixations in the manipulated region divided by the total number of fixations), (2) proportion of viewing time in the critical region (the amount of time spent viewing the manipulated region divided by total viewing time; 5 s), and (3) the number of transitions into/out of the critical region (number of times participants transitioned from a fixation outside the manipulated region to a fixation inside the manipulated region or vice versa). For these measures, the manipulated region occupied one region in either a 4 × 4 grid of 16 equal-sized regions (43% of images) or a 3 × 3 grid of nine equal-sized regions (57% of images). The number of regions (9 or 16) depended on the size and location of the manipulated object.

In addition, we measured how participants viewed the entire image, both inside and outside of the critical region. For this measure, we calculated the number of fixations and the number of regions sampled for novel, repeated, and manipulated images.

Designation of awareness or unawareness

Participants were generally aware of which images were novel and which ones were repeated. Accordingly, we did not explore the relationship between awareness and eye movements made in response to novel versus repeated images during blocks 2 or 3. Our focus instead was on the relationship between awareness and eye movements made in response to manipulated images. A participant was designated as aware of a manipulated image if he/she correctly recognized it as manipulated and also correctly described what the manipulation was (e.g., a man used to be standing inside the truck). A participant was designated as unaware of a manipulated image if he/she failed to identify it as manipulated and also could not describe correctly what the manipulation was, even after being informed that a manipulation had been introduced. A manipulated image was eliminated from the main analysis when these criteria did not allow us to classify a participant as aware or unaware of a manipulation (1.65 ± 0.24 images/participant out of eight manipulated images).

The small number of images that were excluded from this analysis were of two types: (1) images that participants designated as manipulated but the manipulation could not be described; (2) images that participants failed to designate as manipulated, but the manipulation was correctly described after participants were informed that a manipulation had been introduced. To determine whether excluding these images might influence the findings, the data were also analyzed by assigning these excluded images to either the aware or unaware category.

Experiment 2

Participants

Twenty undergraduates participated for course credit (eight females, 20.9 ± 0.8 years of age).

Apparatus

The apparatus was as in experiment 1.

Materials and procedure

These were the same as in experiment 1, except that the memory test was administered during block 3 instead of afterward. Before block 3, participants were instructed that they would see images that were novel, repeated, or manipulated. After each image was presented, participants indicated the status of the image by key press (“it was new,” “it was identical,” or “it was changed”). After each response, the crosshair was presented for 5 s as in experiment 1. After block 3 was completed, the eight manipulated images were presented one at a time. Participants were told that a manipulation had been introduced and were asked to describe the manipulation.

Data analysis

Data analysis was as in experiment 1.

Designation of awareness or unawareness

Designation of participants as aware or unaware was as in experiment 1. Manipulated images were eliminated from the main analysis when the awareness criteria did not allow us to classify a participant as aware or unaware of a manipulation (1.85 ± 0.28 images/participant out of eight manipulated images). One manipulated image was eliminated for one participant because of an equipment malfunction in block 3.

Experiment 3

Participants

Five memory-impaired patients were tested (four males and one female) (Table 1) with bilateral lesions thought to be limited to the hippocampus (dentate gyrus, CA fields, and subiculum). K.E. became amnesic in 2004 after an episode of ischemia associated with kidney failure and toxic shock syndrome. L.J. became amnesic in 1988 during a 6 month period with no known precipitating event. G.W. and R.S. became amnesic after a drug overdose and associated respiratory failure in 2001 and 1998, respectively. J.R.W. became amnesic in 1990 after an episode of cardiac arrest. For the five patients, immediate and delayed (12 min) prose recall (Gilbert et al., 1968) averaged 3.8 and 0.4 segments, respectively.

Table 1.

Characteristics of memory-impaired patients

Patient	Age (years)	Education (years)	WAIS-III IQ	WMS-R
				Attention	Verbal	Visual	General	Delay
K.E.	63	13.5	108	114	64	84	72	55
L.J.	67	12	101	105	83	60	69	<50
R.S.	45	12	99	99	85	81	82	<50
G.W.	45	12	108	105	67	86	70	<50
J.R.W.	38	12	90	87	65	95	70	<50

The Wechsler Adult Intelligence Scale-III (WAIS-III) and the Wechsler Memory Scale-Revised (WMS-R) yield mean scores of 100 in the normal population with an SD of 15. The WMS-R does not provide numerical scores for individuals who score <50. Intelligence quotient (IQ) scores for J.R.W. and R.S. are from the Wechsler Adult Intelligence Scale-Revised.

Estimates of medial temporal lobe damage were based on quantitative analysis of magnetic resonance images, compared with data for 19 controls (for K.E., R.S., G.W., and J.R.W.) or 11 controls (for the female, L.J.) (Gold and Squire, 2005). The volume of the full anterior–posterior length of the hippocampus and the parahippocampal gyrus were measured using criteria based on histological analysis of healthy brains (Amaral and Insausti, 1990; Insausti et al., 1998a,b). For each patient, the volumes of the hippocampus and parahippocampal gyrus were divided by the intracranial volume to correct for brain size. K.E., L.J., R.S., G.W., and J.R.W. have an average bilateral reduction in hippocampal volume of 49, 46, 33, 48, and 44%, respectively (all values >3.0 SDs below the control mean). In comparison, the volume of the parahippocampal gyrus (temporopolar cortex, perirhinal, entorhinal, and parahippocampal cortices) is reduced by 17, −8, 1, 12, and 6%, respectively (all values within 2 SDs of the control mean). Nine coronal magnetic resonance images from these patients are available as supplemental material to Wais et al. (2006). On the basis of two patients (L.M. and W.H.) with similar bilateral volume loss in the hippocampus for whom detailed postmortem neurohistological information was obtained (Rempel-Clower et al., 1996), this degree of volume loss likely reflects nearly complete loss of hippocampal neurons (Gold and Squire, 2005).

Additional measurements, based on four controls for each patient, were performed for the insular cortex, fusiform gyrus, frontal lobes, lateral temporal lobes, parietal lobes, and occipital lobes. The only volume reduction in these regions >1.3 SDs of the control mean was the parietal lobe for R.S. (Bayley et al., 2005).

Ten volunteers (seven men, three women) from the San Diego area served as controls. They averaged 60.2 ± 3.6 years of age (patients, 54.2 ± 4.8 years), and had 13.6 ± 0.6 years of education (patients, 12.4 ± 0.4 years). Their immediate and delayed prose recall averaged 6.7 and 5.4 segments, respectively.

Apparatus

Eye tracking equipment was not used because an insufficient number of patients were available to yield reliable eye-tracking data. A laptop computer with a 15 inch monitor was used to present the photographs and record behavioral responses.

Materials and procedure

These were as in experiment 2. Additionally, after each image in block 3 was classified as novel, repeated, or manipulated, participants gave a confidence rating ranging from 1 to 3 (1 indicated “not sure,” 2 indicated “somewhat sure,” and 3 indicated “very sure”).

Data analysis

Percent correct scores were calculated for novel, repeated, and manipulated images.

Results

Experiment 1

Eye movements in blocks 1 and 2

In block 2, images were either different from what had been presented in block 1 (novel; n = 8), or they were repeated from block 1 (n = 16). Participants made fewer fixations and sampled marginally fewer regions in block 2 when images were repeated than when images were novel (for fixations, t₍₁₉₎ = 3.0, p < 0.01; for regions sampled, t₍₁₉₎ = 1.8, p = 0.10) (Fig. 2). A result of these effects was that fixations in block 2 for repeated images were longer than fixations for novel images (t₍₁₉₎ = 2.3; p < 0.05). Another way to make the same point is to note that participants looked as much at the novel images in block 2 as they looked at the novel images in block 1 (for fixations, t₍₁₉₎ = 1.2; for regions sampled, t₍₁₉₎ = 0.3, p values >0.20). In contrast, participants looked at the repeated images in block 2 less than they looked at the same images when they appeared in block 1 (for fixations, t₍₁₉₎ = 3.8; for regions sampled, t₍₁₉₎ = 3.0, p values <0.01)

Figure 2.

Experiment 1. Participants (n = 20) made fewer fixations and sampled fewer regions when an image was repeated in block 2 than when an image was novel. A, The number of fixations made per image during two successive presentations of images (5 s/image). B, The number of different regions sampled within a 4 × 4 grid superimposed on each image. Error bars indicate SEM.

Eye movements in block 3

Memory test.

Participants were accurate at identifying novel, repeated, and manipulated images (88.4 ± 3.2, 88.1 ± 2.6, and 50.4 ± 5.6% correct, respectively; all values above the chance level of 33%, p values <0.01). On average, participants were designated as aware of 3.45 ± 0.48 manipulated images, and they were designated as unaware of 2.90 ± 0.41 manipulated images. Three participants had no images designated as unaware, and one participant had no images designated as aware.

Eye movements in the critical region of manipulated and repeated images.

The critical comparison is between repeated images and manipulated images. For both repeated images and manipulated images, the regions outside the critical region are the same in blocks 2 and 3. As a result, after balancing for which image is presented, differences in how participants view repeated and manipulated images should be influenced only by what has changed in the critical region. In contrast, for novel images, regions both inside and outside of the critical region change from block 2 to block 3. In that case, differences in how participants view the critical region in novel and manipulated images could be influenced by changes inside the critical region, as well as by changes outside the critical region. What follows, therefore, is a comparison of eye movements in response to manipulated and repeated images.

Figure 3 shows that when participants were aware of a manipulation, they looked more at the manipulated critical region than at the unchanged critical region in the repeated images. Specifically, they exhibited a greater proportion of fixations in the critical region (0.46 ± 0.04), a greater proportion of viewing time in the critical region (0.47 ± 0.04), and made more transitions into and out of the critical region (6.6 ± 0.5) than in the unchanged critical region of repeated images (0.26 ± 0.01, 0.27 ± 0.2, and 3.9 ± 0.2, respectively; p values <0.01). In contrast, when participants were unaware of a manipulation, the proportion of fixations in the critical region (0.25 ± 0.03), the proportion of viewing time in the critical region (0.25 ± 0.03), and the number of transitions into and out of the critical region (3.8 ± 0.4) were the same for manipulated images and for repeated images (p values >0.80) (Fig. 3). In addition, for all three measures, participants looked more at the critical region of images when they were aware of manipulations compared with when they were unaware of manipulations (p values <0.01). Figure 4 illustrates eye movement traces and fixations to indicate how awareness of a manipulation affected viewing.

Figure 3.

Experiment 1 (block 3). In block 3, images were either novel, repeated from blocks 1 and 2, or repeated from blocks 1 and 2 but with a manipulation. When images were presented, participants were instructed to study them but were not informed that images might repeat or be altered. Participants who were subsequently designated as aware of a manipulation (and not participants who were designated unaware) looked at the manipulated (critical) regions more than matched, unmanipulated (critical) regions in the novel or repeated images. A, The proportion of fixations in the critical region for novel images, repeated images, and manipulated images. B, The proportion of viewing time in the critical region for novel images, repeated images, and manipulated images. C, The number of transitions into and out of the critical region for novel images, repeated images, and manipulated images. Error bars indicate SEM.

Figure 4.

Experiment 1. Eye movement traces (black lines) and fixations (diamonds) for four participants in block 3. Eye movement data are for the 5 s that the image was presented. A, A participant for whom the image was novel. B, A participant for whom the image was repeated. C, A participant for whom a repeated image was manipulated and who was aware of the manipulation. D, A participant for whom a repeated image was manipulated and who was unaware of the manipulation. For participants in C and D, the manipulated region contained a man with a dolly when it was presented in blocks 1 and 2 (see Fig. 1). The participant who was aware of the manipulation (C) exhibited a greater proportion of her fixations within the manipulated region than the participant who was unaware of the manipulation (D) or participants who had never seen a different version of the image (A, B). In each panel, the critical region is identified by a black square, but the square did not appear during testing.

These effects of awareness on eye movements did not depend on the criteria used to designate participants as aware or unaware of the manipulated images. Specifically, the 1.65 images/participant that could not be readily classified (and that were excluded from the main analysis) were now assigned to either the aware or unaware category. The results were virtually unchanged. When these images were assigned to the aware category, the proportion of fixations in the critical region, the proportion of viewing time in the critical region, and the number of transitions into and out of the critical region were decreased by only 0.06, 0.06, and 0.8, respectively; all values remained different from the corresponding values for repeated images (p values <0.001). When these images were assigned to the unaware category, these same measures decreased by only 0.003, 0.003, and 0.2, respectively; all values remained similar to the corresponding values for repeated images (p values >0.50).

Equating prior viewing of the manipulated region

We considered the possibility that awareness of the manipulations during block 3 might have been influenced by the extent to which participants looked at those regions during blocks 1 and 2. If so, eye movements during block 3 might be related more to the previous viewing history for each image, and less to what participants noticed during block 3 itself. Accordingly, we equated the amount of viewing in the critical region during blocks 1 and 2 for images that were later designated as aware or unaware (i.e., we equated for all three measures of interest: the proportion of fixations in the critical region, the proportion of viewing time in the critical region, and the number of transitions into/out of the critical region). To do this, we eliminated 1.1 ± 0.3 images/participant among those images that were subsequently designated as aware and 0.7 ± 0.2 images/participant among those images that were subsequently designated as unaware. This procedure resulted in eliminating 27.9% of the data that were used in Figure 3.

An analysis of these data indicated that, even when viewing before block 3 was equated for participants who would be designated as aware or unaware, it remained the case that when participants were aware of manipulations in block 3, they looked more at the manipulated region of the manipulated images than they looked at matched regions of the repeated images. In contrast, when participants were unaware of manipulations, they viewed the manipulated regions of the manipulated images and matched regions of the repeated images to the same extent. Specifically, for participants designated as aware, the proportion of fixations in the critical region, the proportion of viewing time in the critical region, and the number of transitions into and out of the critical region were 0.45 ± 0.04, 0.46 ± 0.04, 6.2 ± 0.2, respectively (all values different from matched regions in the repeated images; p values <0.001) (compare Fig. 3), whereas for participants designated as unaware these measures were 0.24 ± 0.03, 0.25 ± 0.03, and 3.8 ± 0.2, respectively (all values were the same as for matched regions in repeated images; p values >0.70) (compare Fig. 3).

Viewing associated with the entire image

The foregoing analyses indicated that participants who were unaware of manipulations in the images looked at the manipulated critical region no differently than they looked at unchanged critical regions in the repeated images (Fig. 3). We next asked how these same participants viewed the entire image, both inside and outside of the critical region. Perhaps such an analysis could yield evidence that the introduction of a manipulation can affect eye movements even when participants are unaware of the manipulation. However, we found that these participants viewed manipulated and repeated images similarly. Specifically, manipulated images that were not identified by participants (i.e., the participants were unaware of the manipulation) were fixated as much as repeated images (14.9 ± 0.5 vs 15.6 ± 0.4 fixations, respectively). Furthermore, the number of regions sampled was the same for these manipulated images as for repeated images (6.2 ± 0.3 vs 5.7 ± 0.2, respectively; p values >0.10). Thus, we could find no effect of manipulated images on eye movements if participants were unaware that a manipulation had been introduced. The results were virtually identical when alternative criteria, as described above, were used to designate participants as aware or unaware.

Experiment 2

Eye movements in blocks 1 and 2

As in experiment 1, participants made fewer fixations and sampled fewer regions in block 2 when images were repeated than when images were novel (for fixations, (t₍₁₉₎ = 4.0, p < 0.001; for regions sampled, (t₍₁₉₎ = 3.0; p = 0.01) (Fig. 5). In block 2, fixations for repeated images tended to be longer than fixations for novel images, but in experiment 2 this finding did not reach significance (t₍₁₉₎ = 1.5; p > 0.10). Furthermore, participants looked as much at the novel images in block 2 as they looked at the novel images in block 1 (for fixations, t₍₁₉₎ = −0.3; for regions sampled, t₍₁₉₎ = 0.9, p values >0.30). In contrast, participants looked at the repeated images in block 2 less than they looked at the same images when they appeared in block 1 (for fixations, t₍₁₉₎ = 2.6; for regions sampled, t₍₁₉₎ = 3.9, p values <0.01)

Figure 5.

Experiment 2. As in experiment 1, participants in experiment 2 (n = 20) made fewer fixations and sampled fewer regions when an image was repeated in block 2 than when an image was novel. A, The number of fixations made per image during two successive presentations of images (5 s/image). B, The number of different regions sampled within a 4 × 4 grid superimposed on each image. Error bars indicate SEM.

Eye movements in block 3

Memory test.

Participants were accurate at identifying novel, repeated, and manipulated images (94. 4 ± 2.3, 91.3 ± 2.2, and 65.6 ± 3.3% correct, respectively; all values above the chance level of 33%; p values <0.001). On average, participants were designated as aware of 4.00 ± 0.36 manipulated images, and they were designated as unaware of 2.10 ± 0.27 manipulated images. One participant had no images designated as unaware.

Eye movements in the critical region of manipulated and repeated images.

As in experiment 1, when participants were aware of a manipulation, they looked more at the manipulated critical regions than at unchanged critical regions in the repeated images (the proportion of fixations in the critical regions was 0.44 ± 0.03 vs 0.27 ± 0.01 for manipulated vs repeated images; the proportion of viewing time in the critical region was 0.45 ± 0.04 vs 0.28 ± 0.01; and the number of transitions into/out of the critical region was 6.3 ± 0.5 vs 4.0 ± 0.2; p values <0.001) (Fig. 6). In contrast, when participants were unaware of a manipulation, the proportion of fixations in the critical region (0.31 ± 0.03) and the proportion of viewing time in the critical region (0.31 ± 0.03) were the same for manipulated images and repeated images (p values >0.40). Participants who were unaware of a manipulation made marginally more transitions into and out of the critical region (4.9 ± 0.5) than they did in the matched regions of repeated images (p = 0.09). In addition, for all three measures, participants looked more at the critical region of images when they were aware of the manipulation than when they were unaware (p < 0.05, p < 0.05, and p = 0.08, respectively).

Figure 6.

Experiment 2 (block 3). In block 3, images were either novel, repeated from blocks 1 and 2, or repeated from blocks 1 and 2 but with a manipulation. While the images were presented, participants designated them as novel, repeated, or manipulated. A, The proportion of fixations in the critical region for novel images, repeated images, and manipulated images. B, The proportion of viewing time in the critical region for novel images, repeated images, and manipulated images. C, The number of transitions into and out of the critical region for novel images, repeated images, and manipulated images. Error bars indicate SEM.

As in experiment 1, these effects of awareness on eye movements did not depend on the criteria used to designate participants as aware or unaware of the manipulated images. Specifically, the 1.85 images/participant that could not be readily classified (and that were excluded from the main analysis) were now assigned to either the aware or unaware category. The results were nearly identical. When these images were assigned to the aware category, the proportion of fixations in the critical region, the proportion of viewing time in the critical region, and the number of transitions into and out of the critical region were decreased by only 0.04, 0.04, and 0.4, respectively; all values remained different from the corresponding values for repeated images (p values <0.001). When these images were assigned to the unaware category, these same measures decreased by only 0.01, 0.00, and 0.1, respectively; all values remained similar to the corresponding values for repeated images (p values >0.05).

Viewing associated with the entire image

As in experiment 1, we found that participants designated as unaware of a manipulation viewed the manipulated images no differently than they viewed repeated images. Participants who were unaware of a manipulation exhibited a similar number of fixations (15.8 ± 0.6 vs 15.7 ± 0.4, respectively) and sampled a similar number of regions in manipulated images as in repeated images (6.1 ± 0.3 vs 6.0 ± 0.2, respectively) (p values >0.60). The results were virtually the same when alternative criteria, as considered previously, were used to designate participants as aware or unaware of the manipulated images.

Experiment 3

The patients were impaired on the recognition memory test given in block 3 (Fig. 7). Overall, the patients scored 60.8 ± 7.3% correct at classifying the images as novel, repeated, or manipulated, and the controls scored 75.0 ± 2.7% correct (p < 0.05). Both groups performed above the chance level of 33.3% correct (p values <0.05). The confidence ratings given after classifying each image indicated that controls were more confident in their correct responses than in their incorrect responses (2.84 ± 0.06 vs 2.51 ± 0.12; p < 0.01), as is commonly found in tests of recognition memory (Reed et al., 1997). Memory-impaired patients failed to exhibit this effect (2.54 ± 0.12 vs 2.41 ± 0.14; p > 0.20). Accordingly, the ability to accurately and confidently classify the images depends on declarative, hippocampus-dependent memory.

Figure 7.

Experiment 3. Performance of memory-impaired patients (AMN; n = 5) and matched controls (CON; n = 10) during block 3. A, The patients were impaired at classifying the images as novel, repeated, or manipulated (p < 0.05). B, Confidence ratings given by memory-impaired patients and matched controls for images classified correctly (CORR) and incorrectly (INC) in block 3. Controls exhibited more confidence in their correct responses than in their incorrect responses (p < 0.01), but the patients did not (p > 0.20).

Discussion

In three experiments we investigated how eye movements are affected by experience. In experiments 1 and 2, we asked how eye movements change as a function of the viewing history of images. In both experiments, participants made fewer fixations and sampled fewer regions when viewing familiar compared with novel images. In addition, fixations for familiar images were longer than fixations for novel images (p < 0.05 for experiments 1 and 2 combined). These results indicate that simple repetition of an image is sufficient to change viewing behavior.

In the first two experiments, we also measured awareness for changes that were made to previously presented images. In experiment 1, awareness was determined after all the images had been presented. When participants were aware that images had been manipulated, they looked more at the changed (critical) region in the manipulated images than at unchanged (critical) regions in repeated images. In contrast, when participants were unaware of a manipulation, their viewing of critical regions was the same for manipulated images and repeated images.

In experiment 2, awareness of which images were changed was assessed immediately after viewing each image, rather than after all images had been presented. The results were nearly the same as in experiment 1. Awareness that an image had been changed dictated how the critical region was viewed. Specifically, participants looked more at the changed (critical) region of manipulated images than at the unchanged (critical) region of repeated images only when participants were aware of the change. There was no indication that eye movements could reveal an unaware (unconscious) form of memory. Rather, eye movements correlated closely with aware (conscious) memory (i.e., with declarative memory for what was new and what was familiar).

In experiment 3, memory-impaired patients with damage limited to the hippocampus were impaired at deciding whether images were novel, repeated, or manipulated. Furthermore, unlike controls, the patients exhibited the same level of confidence for their correct and their incorrect responses. Thus, the ability to classify images correctly and confidently depends on hippocampus-dependent, declarative memory. We did not measure eye movements in the patients because many more individuals would be needed to obtain reliable data (in experiments 1 and 2, n = 20). In any case, the principal question in our study concerned the relationship between experience-dependent eye movements and awareness.

It is worth emphasizing that our results were the same across different methods of measuring awareness. In experiment 1, awareness was assessed after the completion of block 3. In experiment 2, awareness was assessed after each image was presented during block 3. In addition, two supplementary methods were used to designate participants as aware or unaware of the changes that occurred in manipulated images (see Materials and Methods, Designation of awareness or unawareness). The results were the same in all these cases.

We also performed two additional analyses. First, our findings remained the same when eye movements were measured across the entire image, not just within the critical region of the image where the manipulation occurred. That is, participants who were unaware of a manipulation in an image viewed the whole of that image no differently than they viewed repeated images. Second, the results were the same when we equated how much viewing had been directed toward the critical region of an image before block 3. That is, even when aware and unaware participants had studied the critical region to the same extent, only participants who were aware that an image changed in block 3 tended to look especially at the manipulated region of the image.

A previous study also investigated memory-related changes in eye movements and their relationship to awareness (Ryan et al., 2000). This study was similar to the present one in a number of ways. Both studies involved three successive blocks of images, both studies presented novel, repeated, or manipulated images in block 3, and the studies used the same measures to assess eye movements. The two studies obtained the same findings with respect to how familiar and novel images are viewed. That is, participants made fewer fixations and sampled fewer regions of the image when they viewed familiar images than when they viewed novel images. Similar findings were also reported for viewing famous (familiar) and nonfamous (novel) faces (Althoff and Cohen, 1999).

Unlike the findings for novel and familiar images, the relationship between eye movements and awareness was different in the two studies. In the previous study by Ryan et al. (2000), eye movements in the manipulated region were measured as in our study. Yet, in contrast to our findings, participants who were designated unaware of a manipulation viewed the critical region as much as participants who were aware of the manipulation (as measured by the proportion of fixations in the critical region and the proportion of viewing time in the critical region). Furthermore, for the third measure (the number of transitions into/out of the critical region), eye movements were related to the manipulated region only for participants who were unaware of the manipulation and not for participants who were aware. In our study, eye movements were related to the manipulated region only for participants who were aware of the manipulation, and this same result was obtained for all three eye-movement measures.

There is a methodological difference between our study and the previous one (Ryan et al., 2000) with respect to what participants were asked as they viewed the images. In our study, participants were asked to attend to the images so that they might be able to recognize them later. In contrast, Ryan et al. (2000) asked participants to respond “yes” or “no” to a specific question about the image each time an image was presented. The question always pertained to the relationship between objects inside and outside of the critical region. For example, for an image in which girls were located in the critical region, the question was asked, “are there any girls next to the bridge?” These questions were apparently meant to direct attention to relationships among the objects in each image, but the questions presumably affected eye movements. That is, participants needed to direct their eye movements to the critical regions to answer the questions. Indeed, in a separate experiment in the same study, no orienting questions were asked, and eye movements directed at the critical region were markedly reduced for all image types (novel, repeated, and manipulated). Unfortunately, the relationship between eye movements and awareness of the manipulations was not measured in this case.

Because all participants likely moved their eyes to the manipulated region to answer the questions, they would have accumulated fixations and viewing time within the region. In that case, one could anticipate the result that was obtained; namely, that both aware and unaware participants viewed the critical region similarly for the two measures that indexed viewing in the critical region (the proportion of fixations in the critical region and the proportion of viewing time). In short, the orienting questions in the study by Ryan et al. (2000) may have complicated the measurement of eye movements. Their participants were invited to inspect the very part of the image that would subsequently be manipulated and that would later be queried about in a memory test. Eye movements in our study were naturalistic in the sense that participants freely viewed the entire scene, and eye movements were not guided to the critical region.

Our findings show that the kind of memory that controls eye movements in the presence of manipulated images depends on awareness of the manipulation (experiments 1 and 2) and that memory for the manipulations themselves is hippocampus-dependent (experiment 3). One of the findings from the previous study by Ryan et al. (2000) for six memory-impaired patients is compatible with this conclusion. Their patients, like our unaware participants, viewed the manipulated regions of images no more than matched regions of repeated images. Thus, memory for the images depends on the brain structures damaged in amnesia, and these structures are needed for eye movements to be directed to the manipulated (critical) region.

Access to awareness about what has been learned has been considered a key feature of hippocampus-dependent (declarative) memory (Tulving and Schacter, 1990; Eichenbaum, 1997; Gabrieli, 1998; Squire et al., 2004). The present findings for experience-dependent eye movements and awareness are consistent with this view. It is therefore of interest that recent studies involving three other tests raised questions about this link between awareness and hippocampus-dependent memory.

First, in a study of context effects in perceptual learning, memory-impaired patients were impaired on the task, although task knowledge was not accessible to awareness (Chun and Phelps, 1999). Specifically, healthy controls searched repeated displays faster than new displays but could not report which displays were repeated. These results raised the possibility that learning can be dependent on the hippocampus yet not accessible to awareness. Subsequently, it was found that patients with damage limited to the hippocampus (dentate gyrus, hippocampus proper, and subiculum) acquired the task as well as controls (Manns and Squire, 2001). In the previous study by Chun and Phelps (1999), two patients had damage from encephalitis that included the medial temporal lobe as well as atrophy of the surrounding temporal lobe (magnetic resonance imaging data were not available for the other two patients). Thus, it appears that damage beyond the hippocampus is needed to impair perceptual learning in this task, and the available data do not contradict the idea that hippocampus-dependent memory is accessible to conscious recollection. Another study using the same task found that midazolam affected perceptual learning, particularly the learning of repeated displays (Park et al., 2004). However, the effects of midazolam are not limited to the hippocampus (Taguchi et al., 1989; Veselis et al., 1997), and they also extend to hippocampus-independent cognitive functions (Polster et al., 1993; Hirshman et al., 1999; Fisher et al., 2006). Accordingly, it is difficult to reach conclusions about hippocampus-dependent memory and awareness in studies involving midazolam.

A second task relevant to the present discussion is transitive inference. In this task, overlapping pairs of items are first trained (e.g., A+B−, B+C−, C+D−, D+E−, where + and − indicate the correct and incorrect choices). Participants who later choose B over D are said to exhibit transitive inference. Successful transitive inference is hippocampus dependent in rats and monkeys (Dusek and Eichenbaum, 1997; Buckmaster et al., 2004). In humans, successful performance is related to awareness. That is, participants who become aware of the hierarchical relationship among the stimuli perform well, but those who do not become aware perform poorly (Martin and Alsop, 2004; Frank et al., 2005; Smith and Squire, 2005). One previous report was potentially at odds with this finding, because task awareness did not correlate with successful performance (Greene et al., 2001). Yet, scores were rather high on both the awareness measure and the transitive inference measure, and there may have been too little variability for a correlation to yield a meaningful value. Overall, the data make a rather strong case that awareness is critical for high levels of performance on tasks of transitive inference (for additional discussion, see Smith and Squire, 2005).

Last, the role of awareness in hippocampus-dependent learning has been explored in the context of delay classical eyeblink conditioning. It is widely agreed that delay classical eyeblink conditioning is independent of the hippocampus (Daum et al., 1989; Gabrieli et al., 1995) and also independent of awareness when a single conditioned stimulus (CS) is used (Papka et al., 1997; Manns et al., 2001). When two CSs are used (a CS+ that is followed by the unconditioned stimulus and a CS− that is not), eyeblink conditioning is still hippocampus-independent (Clark and Squire, 1998; Carrillo et al., 2001). Yet there have been reports that this version of the task (differential delay eyeblink conditioning) nevertheless depends on awareness (Nelson and Ross, 1974; Knuttinen et al., 2001). A recent study (Smith et al., 2005) re-explored this issue in four separate experiments that incorporated many of the methods from these previous studies. Awareness of the stimulus contingencies was unrelated to successful eyeblink conditioning in all four experiments. This finding supports the view that when eyeblink conditioning is hippocampus-independent, the conditioning is unrelated to awareness.

In summary, we have studied the effects of experience on eye movements and the relationship of eye movements to awareness and to hippocampus-dependent memory. When individuals searched visual displays, eye movements reflected aware (conscious) memory of what was familiar and what was different about the images. Furthermore, the ability to identify what was new and what was different depended on the integrity of the hippocampus. We also reviewed the results from three other tasks that have been used to explore the link between hippocampus-dependent memory and awareness of what has been learned (perceptual learning, transitive inference, and differential eyeblink conditioning). The available data support the conclusion that awareness of what is learned is a fundamental characteristic of hippocampus-dependent (declarative) memory.