When eye movements express memory for old and new scenes in the absence of awareness and independent of hippocampus

Abstract

Eye movements can reflect memory. For example, participants make fewer fixations and sample fewer regions when viewing old versus new scenes (the repetition effect). It is unclear whether the repetition effect requires that participants have knowledge (awareness) of the old–new status of the scenes or if it can occur independent of knowledge about old–new status. It is also unclear whether the repetition effect is hippocampus-dependent or hippocampus-independent. A complication is that testing conscious memory for the scenes might interfere with the expression of unconscious (unaware), experience-dependent eye movements. In experiment 1, 75 volunteers freely viewed old and new scenes without knowledge that memory for the scenes would later be tested. Participants then made memory judgments and confidence judgments for each scene during a surprise recognition memory test. Participants exhibited the repetition effect regardless of the accuracy or confidence associated with their memory judgments (i.e., the repetition effect was independent of their awareness of the old–new status of each scene). In experiment 2, five memory-impaired patients with medial temporal lobe damage and six controls also viewed old and new scenes without expectation of memory testing. Both groups exhibited the repetition effect, even though the patients were impaired at recognizing which scenes were old and which were new. Thus, when participants viewed scenes without expectation of memory testing, eye movements associated with old and new scenes reflected unconscious, hippocampus-independent memory. These findings are consistent with the formulation that, when memory is expressed independent of awareness, memory is hippocampus-independent.

Declarative memory affords the capacity for conscious recollection of facts and events and depends on the integrity of the hippocampus and related medial temporal lobe structures (Squire and Zola-Morgan 1991; Eichenbaum and Cohen 2001). Declarative memory can be contrasted with a collection of nondeclarative (and unconscious) memory abilities, such as skills, habits, and priming, which are expressed through performance rather than conscious recollection and that are dependent on other brain systems (Squire 1992; Schacter and Tulving 1994; Gabrieli 1998). In this formulation, conscious knowledge (awareness) of what has been learned is a key feature of declarative memory and always accompanies the learning of tasks that depend on the hippocampus and related structures (Clark and Squire 1998). However, recent work identifies possible exceptions to this association between hippocampus-dependent memory and awareness. Specifically, some studies suggest that, under some circumstances, learning can depend on the hippocampus but not be accompanied by conscious awareness of what has been learned (Hannula et al. 2010; Henke 2010; Hannula and Greene 2012).

A fruitful area of work where these ideas have been explored involves experience-dependent eye movements. For example, it is well established that individuals explore scenes or faces differently depending on whether the material is novel or familiar (i.e., the repetition effect) (Althoff and Cohen 1999; Ryan et al. 2000, 2007; Smith et al. 2006; Smith and Squire 2008). The finding is that participants make fewer fixations, sample fewer regions, and make longer fixations when viewing previously presented items compared with novel items. Two key questions are whether this eye movement effect is dependent on the hippocampus and whether this eye movement effect depends on awareness of which items are novel or familiar. A finding that an eye movement effect is hippocampus-dependent, but occurs regardless of awareness for the status of the items (novel or familiar), would count against the idea that hippocampus-dependent memory always implies aware memory.

In an earlier study, we found that the repetition effect was hippocampus-dependent and also reflected declarative (aware) memory (Smith and Squire 2008). That is, memory-impaired patients did not exhibit the repetition effect, and they were also impaired at recognizing which scenes were familiar or novel. In addition, healthy individuals exhibited the repetition effect only when they had conscious knowledge (awareness) of which scenes were old or new. Thus, they exhibited the repetition effect when their old/new judgments were accurate (not when they were inaccurate), and when they were confident in their judgments (not when they were guessing). However, these findings were obtained in circumstances when participants expected to be queried about their memory after they viewed each scene. This is important because expectation of memory testing might interfere with the expression of unaware (unconscious) memory, and it has been proposed that unaware memory is more likely to be observed outside the context of a memory test (Schacter et al. 1989; Greene 2007; Hannula and Greene 2012).

Note that our use of the terms “unaware” and “unconscious” memory refers to a circumstance where behavior distinguishes between old and new items regardless whether participants have conscious knowledge about the old–new status of the items.

The repetition effect has not been well studied when participants have no expectation of memory testing. In two studies (Ryan et al. 2000; Olsen et al. 2016), memory-impaired patients and controls exhibited the repetition effect when there was no mention of memory testing. These studies raised the possibility that the repetition effect might be hippocampus-independent in some circumstances.

We studied the relationship between the repetition effect, the hippocampus, and conscious awareness following the recommendation (Greene 2007) that testing of eye movements and awareness should occur when there is no expectation of memory testing (experiment 1). Experiment 1 also included a comparison condition in which participants were informed that memory would be tested. Last, with memory-impaired patients we determined whether the repetition effect is dependent or independent of the hippocampus when there was no indication that memory would be tested (experiment 2).

Results

Experiment 1

Memory accuracy

Memory was tested in two ways. For the first memory test, participants viewed repeated scenes (set A) intermixed with new scenes (set C) with no expectation that memory was relevant (phase 2, Fig. 1) . Then, in phase 3 participants were shown the same scenes again and asked to judge whether each scene had been old or new when the scene was presented in phase 2. Participants scored 85.4 ± 1.2% correct (d′ = 2.4 ± 0.1). In this way it was possible to estimate whether or not each scene was recognized in phase 2 while eye movements were recorded. The second memory test occurred in phase 4 (Fig. 1). Participants viewed repeated scenes (set B) intermixed with novel scenes (set D) and made old/new memory judgments after viewing each scene. Participants were a little more accurate in phase 4 (92.1 ± 0.8% correct, d′ = 3.0 ± 0.1) than in phase 3 (Ps < 0.001).

Figure 1.

The four phases of Experiment 1. In phase 1, participants viewed the 20 scenes from set A, intermixed with the 20 scenes from set B. In phase 2, participants again viewed the scenes from set A, intermixed with the 20 new scenes from set C. In phase 3, the same 40 scenes from phase 2 were presented again, and participants made old/new recognition judgments on a 1–6 confidence scale, deciding whether the scenes had been old or new when presented in phase 2. In phase 4, participants viewed the 20 scenes from set B, intermixed with 20 new scenes from set D and made old/new recognition judgments. Thus, phase 2 provided measures of eye movements in response to old and new scenes while participants simply viewed the scenes with no expectation of memory testing. Phase 3 provided memory scores for the scenes from phase 2, so that the eye movements in phase 2 could be evaluated separately depending on whether scenes in phase 2 had been recognized as familiar. Phase 4 provided concurrent measures of eye movements and memory in response to old and new scenes.

The difference in performance between phases 3 and 4 was due specifically to the difference in scores for the new items. There was a higher false alarm rate in phase 3 than phase 4 (i.e., identifying a novel item as repeated; 14.7 ± 2.0% versus 3.5 ± 0.6% false alarm rates, respectively, P < 0.001). This difficulty in identifying novel items as novel in phase 3 likely reflects the fact that none of the scenes presented in phase 3 was truly novel. Instead, after viewing all the scenes from phase 2, participants then decided in phase 3 whether each scene had been novel when it had been seen in phase 2. Unlike the memory scores for new items, memory scores associated with old items (hit rates and miss rates) were similar in phases 3 and 4 (85.5 ± 1.6% versus 88.7 ± 1.3% hit rates, respectively, P > 0.10; miss rate = 100 minus hit rate). Thus, the ability of participants to identify which repeated items had been studied earlier (i.e., the hit rate and miss rate) was not affected by being asked to make memory judgments in phase 3 for items presented in phase 2. This finding of a similar hit rate for phases 3 and 4 is important because our principal analysis involved just two comparisons in phase 2: eye movements in response to novel scenes were compared with eye movements in response to repeated scenes that were correctly identified as old (hits); and eye movement data for novel scenes were compared with eye movement data for repeated scenes that were incorrectly identified as new (misses). The hits allowed us to measure eye movements in association with aware memory, and the misses allowed us to measure eye movements in association with unaware memory.

Eye movements recorded when there was no expectation of memory testing (phase 2)

During phase 2, participants viewed repeated scenes (set A) intermixed with novel scenes (set C) with no expectation of memory testing (see Fig. 1). In phase 2, participants viewed novel and repeated scenes differently (i.e., they exhibited the repetition effect). Specifically, they made fewer fixations (P < 0.001), sampled fewer regions (P < 0.001), and made longer fixations (P < 0.001) when scenes were repeated than when scenes were novel (13.3 ± 0.2 versus 13.7 ± 0.2 fixations; 5.7 ± 0.1 versus 5.9 ± 0.1 regions sampled, and 250 ± 6 msec versus 239 ± 4 msec duration, respectively).

We then asked how the repetition effect was related to whether or not the scenes were recognized as familiar (using the memory judgments from phase 3). Data analysis was based on the 58 participants who obtained at least one hit (mean = 16.3 ± 0.3 hits) and at least one miss (mean = 3.7 ± 0.3 misses) (17 participants had no misses). We found that the 58 participants exhibited the repetition effect regardless whether they recognized scenes as familiar (hits) or whether they did not recognize scenes as familiar (misses). That is, compared with novel scenes, participants made fewer fixations (Ps < 0.01), sampled fewer regions (Ps < 0.01), and made longer fixations (Ps < 0.05) for both hits and misses (Fig. 2A–C, respectively). The main findings were the same when the analysis was restricted to participants who obtained at least two hits and at least two misses.

Figure 2.

Experiment 1: phases 2 and 3. Measures of eye movements in response to 20 repeated scenes (black bars) and 20 novel scenes (white bars) when participants simply viewed the scenes and had no expectation of memory testing (phase 2). Recognition memory judgments for the same scenes were obtained directly after eye movements were recorded (phase 3). Participants viewed repeated and novel scenes differently (i.e., they exhibited the repetition effect) regardless of the accuracy and confidence of their memory judgments (asterisks indicate differential viewing of novel and repeated scenes, P < 0.05). Thus, participants made fewer fixations, sampled fewer regions, and made longer fixations for repeated scenes than for novel scenes regardless whether their old/new decisions for repeated scenes were correct (Hit) or incorrect (Miss) (A–C, respectively). Similarly, the repetition effect occurred regardless whether memory judgments for repeated scenes were associated with high or low confidence (D–F, respectively). Although the repetition effect was observed regardless of the accuracy and confidence associated with memory judgments, the effect was numerically larger when participants were incorrect (i.e., Miss) or when they expressed low confidence than when they were correct (i.e., Hit) or expressed high confidence. These differences (between the two black bars) were marginally significant for panels A, B, and F (Ps < 0.10) and were significant for panels D and E (Ps < 0.05; see Discussion). Data are presented for 58 participants (A–C) and for 64 participants (D–F; see Results). Error bars indicate standard error of the difference between viewing novel and repeated scenes. In each panel, the means of the novel conditions are identical (i.e., the two white bars). However, the standard error of the difference score is not the same for each pair of white and black bars, because it was computed separately each time novel and repeated scenes were compared.

We next asked how the repetition effect related to whether memory judgments for the repeated scenes were associated with high confidence or low confidence (using the memory judgments obtained in phase 3). High-confidence responses were confidence ratings of definitely sure (i.e., ratings of 1 or 6). Low-confidence responses were confidence ratings of maybe sure or probably sure (i.e., ratings of 2, 3, 4, or 5). All repeated scenes were used for this analysis, regardless of whether the old–new memory judgment was correct. As is typically the case, accuracy was substantially higher for the scenes associated with high confidence (92.6 ± 1.2% correct) than for the scenes associated with low confidence (69.1 ± 2.2% correct). Importantly, we found that participants exhibited the repetition effect regardless whether they had high confidence or low confidence in their memory judgments. Thus, they made fewer fixations (Ps < 0.01), sampled fewer regions (Ps < 0.05), and made longer fixations (Ps < 0.05) compared with novel scenes for both high-confidence and low-confidence judgments (Fig. 2D–F, respectively). This analysis was carried out for the 64 participants who had at least one repeated scene associated with high confidence (mean = 14.2 ± 0.4 responses) and at least one repeated scene associated with low confidence (mean = 5.0 ± 0.4 responses) (11 participants had either no low-confidence trials or no high-confidence trials). The main findings were the same when the analysis was restricted to participants who had at least two of each of these. In summary, when participants viewed scenes with no expectation that memory would be tested, differential viewing of repeated and novel scenes was observed regardless whether memory judgments were correct or incorrect and regardless whether memory judgments were made with high or low confidence.

We noted that the repetition effect was numerically larger for misses than for hits and larger for low-confidence judgments than for high-confidence judgments (Fig. 2A through 2F). The difference between these conditions was statistically significant in panels 2D and 2E (Ps < 0.05).

Eye movements recorded when memory was tested concurrently (phase 4)

During phase 4, participants viewed repeated scenes (set B) intermixed with novel scenes (set D) and made old/new memory judgments for each scene after each scene was presented (see Fig. 1). When participants expected that memory would be tested, they viewed old and new images differently (i.e., they exhibited the repetition effect). That is, they made fewer fixations (P < 0.001), sampled fewer regions (P < 0.001), and made longer fixations (P < 0.001) when scenes were repeated than when scenes were novel (13.2 ± 0.2 versus 13.7 ± 0.2 fixations; 5.3 ± 0.1 versus 5.8 ± 0.1 regions sampled, and 252 ± 6 msec versus 232 ± 4 msec durations, respectively).

Next, we examined this repetition effect in relation to memory judgments. In contrast to the findings from phase 2, the repetition effect was observed only when participants identified repeated scenes correctly (i.e., for hits) and only when memory judgments were associated with high confidence. Specifically, compared with novel scenes participants made fewer fixations, sampled fewer regions, and made longer fixations only for hits (Fig. 3A–C, respectively; Ps < 0.001), and only for high-confidence responses (Fig. 3D–F, respectively; Ps < 0.001). Accuracy was higher for repeated scenes associated with high confidence (95.8 ± 0.6% correct) than for scenes associated with low confidence (70.3 ± 3.5% correct). These analyses were carried out for the 59 participants who obtained at least one hit (mean = 16.9 hits ± 0.3) and at least one miss (mean = 3.1 misses ± 0.3) and for the 60 participants who had at least one high-confidence response (mean = 15.8 ± 0.4 responses) and at least one low-confidence (mean = 3.3 ± 0.3 responses) response to repeated scenes (16 participants had no miss trials, and 15 participants had either no high-confidence trials or no low-confidence trials). The main findings were the same when the analyses were restricted to participants who had at least two of each of these trials.

Figure 3.

Experiment 1: phase 4. Measures of eye movements in response to 20 repeated scenes (black bars) and 20 novel scenes (white bars) when memory was tested concurrently. In contrast to the findings from phase 2 (Fig. 2), participants viewed novel and repeated scenes differently (i.e., they exhibited the repetition effect) only when memory judgments were correct and only when memory judgments were associated with high confidence. Specifically, participants made fewer fixations (A), sampled fewer regions (B), and made longer fixations (C) only when their old/new decision for repeated scenes was correct (Hit) and not when their decision was incorrect (Miss). Similarly, these effects occurred only when participants expressed high confidence in their memory judgments for repeated scenes and not when they expressed low confidence (D–F). Data are presented for 59 participants (A–C) and for 60 participants (D–F) (see Results). Asterisks indicate differential viewing of novel and repeated scenes (Ps < 0.05). Error bars indicate standard error of the difference between viewing novel and repeated scenes. In each panel, the means of the novel conditions are identical (i.e., the two white bars). However, the standard error of the difference score is not the same for each pair of white and black bars, because it was computed separately each time novel and repeated scenes were compared.

Experiment 2

Eye movements recorded when there was no expectation of memory testing (phase 2; Sessions 1–10)

For each of 10 sessions, patients and controls viewed repeated scenes (set A) intermixed with novel scenes (set B) with no expectation of memory testing (see Fig. 4). Different sets of scenes were presented in each session, and the data were combined across sessions. Both controls and patients viewed novel and repeated scenes differently (i.e., both groups exhibited the repetition effect). Specifically, both groups exhibited fewer fixations, sampled fewer regions, and made longer fixations for repeated scenes than for novel scenes (Fig. 5A–C, respectively, Ps < 0.05). For number of fixations and fixation duration, the repetition effect was marginally significant for the patients (P = 0.088 and P = 0.074, respectively). Notably, the size of the repetition effect (novel versus repeated) was similar for controls and patients. For number of fixations, the difference between fixations associated with novel and repeated scenes (mean ± SD) was 0.70 ± 0.25 for controls and 0.42 ± 0.19 for patients (P > 0.40). For number of regions sampled, the novel-repeated difference was 0.39 ± 0.08 for controls and 0.30 ± 0.10 for patients (P > 0.40). For duration of fixations, the novel-repeated difference was −18.6 ± 6.5 msec for controls and −15.1 ± 6.3 msec for patients (P > 0.70). In summary, patients with hippocampal lesions exhibited the repetition effect, and the size of the effect was similar in patients and controls.

Figure 4.

The 12 sessions of Experiment 2. Sessions 1 through 10 provided measures of eye movements in response to repeated and novel scenes for patients and controls. Each session involved two phases. In phase 1, participants viewed 20 scenes (Set A) with no expectation that memory would be tested. In phase 2, they viewed 20 repeated scenes (Set A) intermixed with 20 novel scenes (Set B), again with no mention of memory testing. This sequence was repeated nine additional times using different scenes for each session (Set C through Set T). Sessions 11 and 12 provided memory scores for repeated and novel scenes. Each session involved two phases. In phase 1, participants viewed 20 scenes (Set U) with no expectation that memory would be tested. In phase 2, they viewed 20 repeated scenes (Set U) intermixed with 20 novel scenes (Set V) and made recognition memory judgments. This sequence was repeated again on a different visit using different scenes (Set W and Set X).

Figure 5.

Experiment 2. Measures of eye movements in memory-impaired patients and controls in response to repeated scenes (black bars) and novel scenes (white bars) when participants simply viewed the scenes and had no expectation of memory testing. In each of 10 test sessions, participants viewed 20 scenes and then viewed 20 repeated scenes intermixed with 20 novel scenes (total = 400 scenes). Both patients and controls made fewer fixations (A), sampled fewer regions (B), and made longer fixations (C) for repeated scenes than for novel scenes (i.e., both groups exhibited the repetition effect). Asterisks indicate differential viewing of novel and repeated scenes (Ps < 0.05; †Ps < 0.09). MTL = 5 patients with medial temporal lobe lesions; CON = 6 controls. Error bars indicate standard error of the difference between the scores for novel and repeated scenes that was used to test for the repetition effect.

Figure 5 also shows that the patients exhibited numerically fewer fixations, fewer regions sampled, and longer fixation durations than controls for both novel scenes and repeated scenes. These numerical differences were not significant (Ps > 0.30). The numerical differences between groups appear more substantial than they actually are because the error bars show the standard error of the difference between novel and repeated scenes (a within-participant measure) and not the standard error of the mean (a between-participant measure).

Note that data for eye movements in response to novel and repeated scenes, under conditions when patients and controls did expect that memory would be tested, were not collected in the present study. These data were reported in an earlier study (Smith and Squire 2008).

Memory accuracy (phase 2; sessions 11 and 12)

For session 11, participants viewed repeated scenes (set U) intermixed with novel scenes (set V) and made old/new memory judgments after viewing each scene (Fig. 4). The same procedure was repeated in session 12 with different scenes. Memory scores and confidence ratings obtained from test sessions 11 and 12 were combined. These measures provided an estimate of memory accuracy and confidence for sessions 1–10, where participants also viewed 20 repeated scenes intermixed with 20 novel scenes but where memory was not measured. Patients exhibited impaired memory and low confidence when distinguishing between repeated and novel scenes (Fig. 6). The controls scored 95.8 ± 1.9% correct (d′ = 3.4 ± 0.2) and patients scored 82.3 ± 4.4% correct (d′ = 2.3 ± 0.4), Ps < 0.05. In addition, patients were less confident in their memory decisions than controls (2.5 ± 0.1 versus 2.8 ± 0.1, respectively; P < 0.05). Thus, the ability to discriminate effectively between 20 repeated scenes and 20 novel scenes depends on the integrity of the hippocampus.

Figure 6.

Experiment 2. Mean recognition memory accuracy (left) and confidence (right) in memory-impaired patients (gray bars) and controls (white bars). In each of two tests, participants viewed 20 scenes and then viewed 20 repeated scenes intermixed with 20 novel scenes. After viewing each scene, participants indicated whether the scene was old or new and made a confidence judgment (1 = definitely new, 2 = probably new, 3 = maybe new, 4 = maybe old, 5 = probably old, and 6 = definitely old). The patients were impaired relative to controls at identifying which scenes were old and which were new (left). Patients also expressed lower confidence in their memory judgments than controls (right). To obtain a common measure of confidence for novel and repeated items the 1–6 scale was converted to a 1–3 scale (1 = maybe sure, 2 = probably sure, 3 = definitely sure). Asterisks indicate differential viewing of novel and repeated scenes (Ps < 0.05). MTL = 5 patients with medial temporal lobe lesions; CON = 6 controls. Error bars indicate standard error of the mean.

Dissociation between memory accuracy and the repetition effect for patients and controls

We asked whether the relationship between the size of the repetition effect (novel minus repeated; Fig. 5) and memory scores (d′) was different for patients and controls. For each of the three eye-tracking measures, we carried out a Group (Patient/Control) × Performance Measure (Repetition Effect/Memory Score), two-factor, mixed-design ANOVA. Behavior across the two performance measures was different for patients versus controls for the number of regions sampled and for fixation duration (interaction terms, Ps < 0.05). The effect was marginally significant for the number of fixations (P < 0.06). That is, patients and controls behaved differently across the two performance measures. The patients and controls exhibited similar repetition effects for old and new scenes but different levels of conscious knowledge for which scenes were old and which were new.

Discussion

Individuals view previously presented scenes differently than novel scenes (the repetition effect). Specifically, when viewing familiar scenes, individuals make fewer fixations, sample fewer regions, and make longer fixations than when viewing novel scenes. The first finding of the present study was that this effect occurred regardless whether participants recognized which scenes were old and which were new and regardless whether memory judgments were made with high or low confidence (experiment 1) (Fig. 2). In other words, these eye movement effects were independent of participant knowledge about the old–new status of the scenes and, in that sense, reflected unconscious (unaware) memory. The second finding was that memory-impaired patients with hippocampal lesions viewed novel and repeated scenes differently, just as healthy participants did, even though they were impaired at recognizing which scenes were old and which were new (experiment 2) (Figs. 5, 6). Thus, the repetition effect reflected unaware memory and was also hippocampus-independent, supporting the idea that unaware memory is a feature of hippocampus-independent memory. It is important to emphasize that the findings just described occurred in free-viewing conditions when there was no indication that participants needed to decide the old–new status of the scenes.

One might suppose that the presence of the repetition effect in patients when there is no expectation of memory testing might be related, not to expectation of testing, but to the accuracy with which patients recognize novel and repeated scenes (high accuracy = repetition effect; low accuracy = no repetition effect). However, the data do not support this idea. First, in experiment 1 there was no correlation between recognition accuracy and measures of the repetition effect (rs < 0.13, Ps > 0.30; also see Olsen et al., 2016). Second, the size of the repetition effect was similar in patients and controls, despite the fact that patients and controls differed in their ability to recognize scenes as old or new. Although there were too few patients to compute a reliable correlation between scores on the recognition test and the size of the repetition effect, the two patients who obtained the highest and lowest accuracy scores were not the same patients who demonstrated the largest and smallest repetition effects. In fact, the patient who obtained the highest score on the recognition test exhibited the smallest repetition effect. Finally, in two earlier studies, memory-impaired patients exhibited the repetition effect for faces, even though they performed poorly on a subsequent memory test (Althoff et al. 1999; Olsen et al. 2016).

Greene (2007) had suggested that an expectation of memory testing might interfere with the expression of unaware (unconscious) memory and that expectation is an important variable when investigating aware and unaware memory. Our findings confirmed this idea. The relationship between the repetition effect and awareness was dramatically different depending on whether participants knew that memory would be tested. When participants did not expect that memory would be tested, the repetition effect was observed regardless of awareness of the old–new status of the scenes (Fig. 2). In contrast, when participants expected that memory would be tested, the repetition effect was exhibited only when they were aware of which scenes were old and which were new and when they had high confidence in their decisions (Fig. 3). This latter finding replicates our earlier study in which eye movements were recorded in the context of memory testing, and the repetition effect depended on awareness (Smith and Squire 2008). The finding that expectation of memory testing can affect eye movements is reminiscent of earlier, classical work (Yarbus 1967) showing that task instructions affect eye movements (e.g., participants sampled different regions of a scene depending on which questions they were asked about the scene). The current study (experiment 1) shows that expectation of memory testing is also an important variable affecting eye movements.

In an earlier study, when eye movements were recorded in the context of memory testing (Smith and Squire 2008), we demonstrated for healthy individuals that the repetition effect depended on conscious awareness of the old–new status of the scenes. We also demonstrated that the repetition effect was hippocampus-dependent (i.e., patients failed to exhibit the repetition effect). Thus, this earlier work, together with the current work, shows that knowledge of memory testing can affect whether experience-dependent eye movements depend on awareness for what has been learned and whether experience-dependent eye movements are hippocampus-dependent.

It is notable that when participants did not expect that memory would be tested, the repetition effect was numerically larger (and statistically larger in some cases) when participants made low-confidence memory judgments compared with when they made high-confidence memory judgments (Fig. 2D–F). The repetition effect was also larger when participants did not recognize an item as old (misses) compared with when they correctly recognized an item as old (hits) (Fig. 2A–C). One possibility is that, when participants view scenes naturally with no task demands, the automatic and unconscious repetition effect is diminished if participants have a strong sense that a scene has been encountered previously. A belief that an item was presented earlier might result in eye movements intended to confirm that impression, and these eye movements might be different than eye movements that scan an apparently novel scene.

It is useful to compare the repetition effect to the preferential viewing effect that is associated with the visual paired-comparison (VPC) task (Fagan 1970). In the repetition effect, participants sample a scene more fully when it is novel than when it is familiar. In the typical VPC task, participants see two identical scenes and then see a new scene paired with an old scene. Participants tend to spend more time viewing the novel scene than the repeated scene. The repetition effect, as reported here, is hippocampus-independent and occurs regardless whether participants recognize that a scene is old or new. In contrast, the preferential looking effect is related to measures of declarative (aware) memory (Manns et al. 2000). Furthermore, this effect is diminished after hippocampal damage in both humans (McKee and Squire 1993) and nonhuman primates (Pascalis and Bachevalier 1999; Zola et al. 2000), and it is diminished in a rodent version of the VPC task as well (Clark et al. 2000). It is unclear why in free viewing conditions the repetition effect and the preferential looking effect are fundamentally different. However, the tasks are quite distinct. For example, in tests of the preferential viewing effect, participants can view one item or the other, but in tests of the repetition effect participants maintain viewing of a single stimulus.

There is another kind of experience-dependent eye movement effect that has been taken to reflect unaware memory. Participants studied a list of faces (Hannula et al. 2012), Chinese characters (Chen and Lee 2015), or face–scene pairs (Hannula and Ranganath 2009), and then took a three-alternative recognition test while eye movements were recorded. Participants tended to look at the selected item (i.e., the item judged to be old) more than they looked at the other items. Nevertheless, regardless whether the overt behavioral choice was correct or incorrect, eye movements differentiated the old items from the new items. Specifically, a greater proportion of viewing time was directed at the selected item when the choice was correct than when the choice was incorrect.

Another possibility worth exploring is that viewing time in these studies reflected conscious memory, which was guided by the level of confidence assigned to each recognition judgment. Confidence judgments are typically higher in association with correct choices than with incorrect choices (Wixted et al. 2015). Accordingly, on correct trials, eye movements may have been strongly directed toward the selected item because confidence was high. On incorrect trials, confidence was lower, and this uncertainty led participants to sample the other possible choices. Indeed, in one of the studies the level of confidence associated with the correct item (on trials that were correct) predicted how much viewing was directed to that item (Chen and Lee 2015).

The available findings make it clear that experience-dependent eye movements can reflect aware or unaware memory depending on the task and the instructions. For example, in the present study the repetition effect reflected either aware or unaware memory, depending on whether participants were informed that memory would be tested. In contrast, in earlier work, the phenomenon whereby participants preferentially explore the altered portion of a repeated scene reflected aware, hippocampus-dependent memory, regardless whether memory testing was expected (Smith et al. 2006; Smith and Squire 2008; but see Ryan et al. 2000). In addition, for the preferential viewing effect as identified in VPC tasks, participants do not expect memory testing, and performance reflects aware, hippocampus-dependent memory. Thus, experience-dependent eye movement effects can reflect hippocampus-dependent (aware) memory or hippocampus-independent (unaware) memory, and expectation of memory testing can be an important factor.

To identify the relationship between awareness and the hippocampus in experience-dependent eye movements, it is important to determine (1) whether the effect reflects conscious or unconscious memory; (2) whether the effect is hippocampus-dependent or hippocampus independent; and (3) whether an expectation of memory testing affects the aware–unaware status of the effect. We followed this approach. When there was no expectation of memory testing, the repetition effect reflected unaware memory and was independent of the hippocampus (experiments 1 and 2). When participants expected that memory would be tested, the repetition effect reflected aware memory and was hippocampus-dependent (Smith and Squire 2008). These findings, taken together, are consistent with the idea that aware memory implies hippocampus-dependent memory and that unaware memory implies hippocampus-independent memory.

Materials and Methods

Experiment 1

Participants

Seventy-five undergraduates from the University of California, San Diego participated for course credit (49 female, 21.3 ± 0.3 yr of age [mean ± SEM]).

Apparatus

Eye movements were recorded at 30 Hz with a ViewPoint Eye Tracker (Arrington Research, Inc.) and PC-60 software (version 2.8.4.536) for detecting pupillary position. A fixation was scored when more than 100 msec elapsed without a saccade. A saccade was defined as an eye movement greater than 0.7° within 33 msec (approximately one-fourth inch on the 20-inch computer monitor). Head motion and position were maintained with a bite bar, forehead rest, and chin rest. Viewing was binocular, though only movements of the left eye were tracked. The eye tracker was adjusted for each participant prior to the study. Correction for head motion was performed before each test phase and, when needed, during a phase. A separate computer controlled image presentation and recorded behavioral responses using E-prime software (version 2.0.8.90; Psychology Software Tools). An external keyboard was used to record responses.

Materials

Four sets of 20 color photographs of indoor/outdoor scenes served as stimuli (total = 80 scenes) and across participants these sets were equally likely to serve as stimulus sets A, B, C, or D, respectively (Fig. 1).

Procedure

The objective was to test the relationship between the repetition effect and awareness for which scenes were old and which were new under conditions when participants were not informed that memory would be tested. Determining this relationship is not straightforward. Once memory is mentioned, the strategy used to view the stimuli (and the corresponding eye movements) might change on subsequent trials. Yet, unless memory is tested, one cannot assess participant knowledge of which particular scenes are old or new. To achieve our objective, participants were first told that they would be seeing pictures and that the purpose of the study was to learn how people look at pictures during full attention. They were instructed that it was important that they pay attention to each picture, that they remain alert, and that they look at the picture for the full time that it remained on the screen. There were four phases in the experiment (Fig. 1). All scenes were presented for 5 sec.

In phase 1, 40 scenes were presented (Set A and Set B). In phase 2, 20 repeated scenes (Set A) were presented intermixed with 20 novel scenes (Set C). Note that in phases 1 and 2, eye movements in response to scenes were recorded when participants were simply viewing the scenes with no expectation of memory testing. In phase 3, memory was tested for the repeated and novel scenes from phase 2 (Set A and Set C). Memory testing in phase 3 provided an estimate of whether the old scenes in phase 2 could have been recognized as familiar in phase 2 (if memory had been tested in phase 2). Specifically, in phase 3 participants made old/new recognition judgments for the scenes from phase 2 by indicating on a six-point confidence scale whether the scenes had been old or new when they were presented in phase 2 (1 = definitely new, 2 = probably new, 3 = maybe new, 4 = maybe old, 5 = probably old, and 6 = definitely old). Note that the method used to obtain memory scores in phase 3 resembles a source memory judgment because participants made a judgment about when the scenes were encountered (i.e., Was the scene old/new when you saw it earlier?). In phase 4, eye movements in response to repeated scenes (Set B) and novel scenes (Set D) were recorded, and memory judgments were made after viewing each scene. In each phase, before each scene was presented, participants viewed a red cross (3-sec duration) to monitor head motion. The intervals between the phases were 30 sec (phase 1–2), 5 min (phase 2–3), and 30 sec (phase 3–4). The time between the beginning of memory testing in phase 3 and the beginning of memory testing in phase 4 was about 5 min.

Data analysis

Three eye movement measures (calculated using Matlab [Mathworks, Inc.]) assessed how participants viewed the repeated scenes and new scenes (i.e., in phases 2 and 4): (a) number of fixations during the 5-sec viewing period; (b) number of regions sampled, that is, the number of different regions (out of 16) in which a fixation was detected (for this measure, each scene was divided into 16 equal-sized regions to form a 4 × 4 grid); and (c) the mean duration for all fixations during the 5-sec viewing period. These are the same measures used in our earlier studies of eye movements (Smith et al. 2006; Smith and Squire 2008). All data are reported as mean ± SEM. Differential viewing of novel and repeated scenes (i.e., the repetition effect) was tested using paired t-tests. Eye movements in phases 1 and 3 were not analyzed.

In a few trials, a smaller number of fixations were detected than would be expected, either because the pupillary position could not be maintained by the ViewPoint software or because head movement resulted in eye movements directed outside of the area where eye movements could be measured. To identify such trials, we examined the distribution of the number of fixations associated with each of 20 novel scenes from 71 pilot participants (1420 scenes total) who were not informed that memory was relevant. The distribution was noticeably bimodal with a peak at 15 fixations and another, smaller peak at 5 fixations (overall mean = 13.0 fixations, SD = 4.0 fixations). Accordingly, in all experiments reported here, trials where participants made 5 or fewer fixations were eliminated from further analysis (i.e., ≥2 SD below the mean). On average, 4.1 trials were eliminated for each participant in experiment 1 (5.1% of each participant's trials from phases 2 and 4).

Experiment 2

Participants

Memory-impaired patients

Five patients participated (Table 1), four with bilateral lesions limited to the hippocampus (DA, KE, LJ, and GW) and one with larger medial temporal lobe lesions (GP). DA and GW became amnesic following a drug overdose and associated respiratory failure in 2001 and 2011, respectively. KE became amnesic after an episode of ischemia associated with kidney failure and toxic shock syndrome. LJ (the only female) became amnesic during a 6-m period in 1988 with no known precipitating event. Her memory impairment has remained stable since that time. GP became amnesic in 1987 due to viral encephalitis.

Table 1.

Characteristics of memory-impaired patients

Estimates of medial temporal lobe damage were based on quantitative analysis of magnetic resonance images from 19 age-matched, healthy males for patients KE, GP, and GW, and 11 age-matched healthy females for patient LJ (Gold and Squire 2005) and 8 younger healthy males for DA (also see Knutson et al. 2013 for eight coronal magnetic resonance images and detailed descriptions of the lesions). DA, KE, LJ, and GW have an average bilateral reduction in hippocampal volume of 35%, 49%, 46%, and 48%, respectively (all values more than 2.9 SDs below the control mean). In comparison, the volume of the parahippocampal gyrus (temporopolar cortex, perirhinal, entorhinal, and parahippocampal cortices) is reduced by −5%, 11%, −17%, and 10%, respectively (all values within 2 SDs of the control mean). The negative values indicate instances where the volume was larger for a patient than for controls.

GP has average bilateral reductions in hippocampal and in parahippocampal gyrus volume of 96% and 94%, respectively. In addition, GP has 50% reduction in fusiform gyrus bilaterally and 80% reduction of left insular cortex (Bayley et al. 2005). Based on detailed neurohistology from a patient who had similar etiology and damage as GP (Insausti et al. 2013), we recognize that loss of afferent and efferent connections to and from the medial temporal lobe in GP could have resulted in abnormal tissue in areas beyond what was detected in the magnetic resonance images.

Controls

Six volunteers (two females) served as controls for the memory-impaired patients. They averaged 59.8 ± 6.1 yr of age (patients = 61.2 ± 8.2 yr [mean ± SEM]), and had 13.7 ± 0.7 yr of education (patients = 13.1 ± 0.9 yr [mean ± SEM]).

Materials

Twenty-four sets of 20 color photographs of indoor/outdoor scenes (Set A through Set X) served as stimuli (total = 480 scenes). Two sets of scenes were presented in each of 12 sessions across six visits. Within each session, each of the two sets of scenes was equally likely to serve as repeated scenes or novel scenes.

Procedure and data analysis

Given the small number of patients available (N = 5), multiple tests were given to obtain reliable estimates of eye movements (10 sessions) and memory (2 sessions). For session 1, participants viewed 20 scenes (Set A) (instead of 40 scenes as in Experiment 1) with no expectation that memory would be tested (Fig. 4, phase 1). After 30 sec, they viewed 20 repeated scenes (Set A) intermixed with 20 novel scenes (Set B), again with no mention of memory testing (phase 2). This sequence was repeated across nine more sessions in five different visits using different novel and repeated scenes in each session (Set C through Set T). These 10 test sessions were scheduled across 3.8 mo ± 18 d. Eye movements were collected during these 10 sessions, and the data were combined across sessions. Then, immediately after all the eye movement data were collected (i.e., after session 10), memory was tested (Session 11). Participants first viewed 20 new scenes (Set U, phase 1). Then after 5 min (i.e., the time that it took to provide instructions for the memory test and load the test program), they viewed 20 repeated scenes (Set U) intermixed with 20 novel scenes (Set V, phase 2). After viewing each scene, participants made recognition memory judgments using the six-point confidence scale. Finally, memory was tested one additional time (session 12) 13.7 mo ± 25 d later using a different set of novel and repeated scenes (study 20 scenes [Set W], test with 40 scenes [Set W and Set X]), and the memory scores were combined. Thus, the first 10 sessions provided measures of eye movements in response to old and new scenes when participants did not expect that memory would be tested. Test sessions 11 and 12 provided measures of memory only (and eye movements were not recorded). Two controls were unavailable for test session 12, and their memory scores were based on one test session instead of two.

On average, 4.7 trials were eliminated for each participant in experiment 2 (2.4% of each participant's trials) due to head movement or loss of pupillary position (see experiment 1). Differential viewing of novel and repeated scenes (i.e., the repetition effect) was tested using paired t-tests.

Competing interest statement

The authors declare no competing financial interests.