The Eyes Know: Eye Movements as a Veridical Index of Memory

Abstract

In two experiments, we examined whether observers’ eye movements distinguish studied faces from highly similar novel faces. Participants’ eye movements were monitored while they viewed three-face displays. Target-present displays contained a studied face and two morphed faces that were visually similar to it; target-absent displays contained three morphed faces that were visually similar to a studied, but not tested, face. On each trial in a test session, participants were instructed to choose the studied face if it was present or a random face if it was not and then to indicate whether the chosen face was studied. Whereas manipulating visual similarity in target-absent displays influenced the rate of false endorsements of nonstudied items as studied, eye movements proved impervious to this manipulation. Studied faces were viewed disproportionately from 1,000 to 2,000 ms after display onset and from 1,000 to 500 ms before explicit identification. Early viewing also distinguished studied faces from faces incorrectly endorsed as studied. Our findings show that eye movements provide a relatively pure index of past experience that is uninfluenced by explicit response strategies, and suggest that eye movement measures may be of practical use in applied settings.

Everyday experience suggests that memory is subject to distortion and inaccuracy, an intuition that has been confirmed empirically (Bartlett, 1932; Loftus, Miller, & Burns, 1978). Under most circumstances, these mistakes are merely annoying or embarrassing—as when, for instance, people misremember the details of a casual conversation with a coworker. However, this everyday annoyance can also have grave consequences, including wrongful convictions in cases of mistaken eyewitness identification (Wells et al., 1998). Empirical investigations have shown that false memories can be endorsed with considerable subjective certainty (Loftus, Donders, Hoffman, & Schooler, 1989) and that participants may mistakenly remember having seen items that resemble a studied target in arrays from which the target is absent (Wells, 1993).

Consequently, identifying sensitive measures of memory that do not hinge on the accuracy or reliability of explicit verbal reports would have considerable significance. More generally, identifying measures that have the potential to represent past experience in the absence of, or despite inaccuracies in, overt responses would be useful in clinical settings and in investigations of populations in which reliable assessments of memory are unlikely or impossible to obtain with more traditional approaches (e.g., explicit recall or recognition; Luck & Gold, 2008).

The use of eye movement measures constitutes a promising alternative approach to assessing memory, and, unlike traditional methods, measuring eye movements requires no special instruction or potentially complicated response mappings. Furthermore, eye movements can be recorded together with overt behavioral responses; such contemporaneous recording may allow researchers to identify circumstances in which explicit reports of memory and implicit effects of memory (e.g., on patterns of viewing) are dissociated.

The sensitivity of eye movement measures to memory has been documented in several investigations (see Hannula et al., 2010). Collectively, these studies have shown that participants make fewer fixations on and sample fewer distinct regions of previously studied items, compared with novel items (an effect of stimulus repetition), and that when looking at previously studied scenes, participants tend to look disproportionately at regions in which the relations among scene elements have been altered (e.g., Althoff & Cohen, 1999; Ryan, Althoff, Whitlow, & Cohen 2000; Smith, Hopkins, & Squire, 2008). Critically, these latter viewing effects have been documented even for cases in which participants failed to explicitly detect the manipulation (e.g., Beck, Peterson, & Angelone, 2007; Henderson & Hollingworth, 2003; Ryan & Cohen, 2004; Ryan et al., 2000), which suggests that eye movements might provide more information about past experience than is available for conscious report.

In the research reported here, we focused on a related but novel question: whether eye movements distinguish actually studied materials from materials incorrectly endorsed as studied. As indicated by the literature we have just reviewed, such a demonstration would have potential real-world significance and strong potential for use in investigations conducted with clinical populations, preverbal infants, and animals.

In two experiments, participants were shown displays consisting of three faces each while their eye movements were recorded. Half of the displays (target-present displays) contained a studied face and two faces morphed to be visually similar to that face; the other displays (target-absent displays) contained three faces morphed to be visually similar to a studied face that was not seen in the test phase. On each trial, participants attempted to identify the studied face in the display, selected one of the three faces (making an arbitrary selection if they felt none of the faces had been studied), and indicated whether the chosen face had in fact been studied.

We systematically manipulated the visual similarity of the morphed faces to the studied targets and evaluated the influence of visual similarity between morphs and targets on explicit recognition and eye movement behavior. We expected that when the similarity between morphs and targets was high, participants’ recognition of studied faces in target-present displays would be poor and false recognition of novel (morphed) faces in target-absent displays would increase. Our primary question, however, was whether the manipulation of visual similarity would also influence the effects of memory on eye movement behavior. Drawing on past research showing rapid, obligatory viewing directed at studied materials (Hannula, Ryan, Tranel, & Cohen, 2007; Ryan, Hannula, & Cohen, 2007), we expected that eye movements would be impervious to manipulations of visual similarity and would distinguish studied faces from novel faces even when explicit recognition did not.

Experiment 1

Experiment 1 was designed to examine whether manipulations of visual similarity would influence overt recognition of studied faces or lead to incorrect endorsement of novel faces as studied. We also examined whether measures of eye movement behavior would be dissociated from overt responses, providing a more veridical index of past experience.

Method

Participants

Twenty-seven University of Illinois students participated in the experiment in return for course credit; 3 of these participants were excluded from further testing because their eye position could not be reliably calibrated.

Stimuli and design

Stimuli used in this experiment included 36 sets of 3 male faces each (108 faces in 36 triplets); the hair color, facial expression, and photographic perspective of the faces in each triplet were matched. One face from each triplet was designated as the target (i.e., the face that would be presented during the study phase). For each triplet, the target face was morphed with the other two faces at two levels of visual similarity to create four morphed faces that were used in the test phase. Faces were morphed with Morph Version 2.5 software (Gryphon Software, San Diego, CA). (For more details on how stimuli were created, see the Supplemental Material available online.) Morphed faces were either 20% of the target face and 80% of one of the nontarget faces (20% morphs) or 40% of the target face and 60% of one of the nontarget faces (40% morphs). The two nontarget members of each triplet were morphed together at 50% each to create a foil that was also used in the test phase (see Fig. 1a). The resulting set of stimuli consisted of 216 faces, comprising 36 targets, 72 faces at each level of morphing (20%, 40%), and 36 foils.

Fig. 1.

Illustration of materials and trial sequences. Stimuli (a) included triplets of three original faces (face A, face B, and face C). The target face (face A) was morphed with face B and with face C at two levels of visual similarity to create 20% morphs (20% of face A and 80% of face B or C) and 40% morphs (40% of face A and 60% of face B or C). Faces B and C, which were not presented in the experiment, were morphed together to create a foil face used in target-absent test displays. On each study trial, a target face was presented for 5 s; on each test trial, a three-face display was presented for 10 s (b). Four types of displays were presented during test trials (two types are illustrated here). Target-absent displays contained a foil face and two 20% morphs (shown) or 40% morphs (not shown) that corresponded to a studied target face not presented during the test block. Target-present displays contained a previously studied target face and the two corresponding 20% morphs (not shown) or 40% morphs (shown). All test trials were initiated by the experimenter after the participant fixated a centrally located crosshair. On each trial, participants attempted to identify the studied face in the display, selected one of the three faces (the target if they believed it was present and an arbitrary face if they believed the target was not present), and subsequently indicated whether the chosen face had in fact been studied.

Each target face was presented once in each of five study blocks, with presentation order independently randomized within each block. In a subsequent test block, 36 three-face displays (18 target-present displays and 18 target-absent displays) were presented. In each target-present display, a studied target was presented with two corresponding morphs that had not been seen during study. In each target-absent display, a foil face was presented with two faces that had been morphed with one of the studied (but not tested) targets. Visual similarity of the two morphed faces within each display was always matched (i.e., both faces were either 20% morphs or 40% morphs; see Fig. 1b).

Counterbalancing ensured that each set of faces (i.e., the target or foil and morphs) was shown equally often in every type of display (target-present with 20% morphs, target-present with 40% morphs, target-absent with 20% morphs, target-absent with 40% morphs) across participants. All four types of displays were presented equally often and in random order during the test block, and targets and foils appeared equally often in all three spatial positions (i.e., left, right, and bottom) across trials.

Procedure

Eye position was recorded at 60 Hz with an ASL remote eye tracker (Applied Science Laboratories, Bedford, MA). After informed consent was obtained, and prior to each block, eye position was calibrated using a 3 × 3 spatial array. The experimenter initiated each trial after the participant fixated a centrally located crosshair.

On each of 36 study trials within a given block, a target face was presented for 5 s (Fig. 1b). The experimenters instructed participants to commit each face to memory and emphasized that they should pay close attention to the details of each face because distractors used in the recognition test would bear a strong resemblance to the studied faces.

On each of 36 test trials, a three-face display was presented for 10 s while the participant’s eye position was recorded (Fig. 1b). Participants were instructed to press a button to identify the studied face when a studied face was presented in the test display; they were also told that some displays would not contain a studied face, in which case they should press a button to select a face at random. Immediately after the offset of the test display, participants indicated verbally whether the face they selected had been studied.

Eye movement measures

For every test trial, the proportion of total viewing time directed to a region of interest (ROI) was evaluated using two measures of eye movement behavior (i.e., time-course and response-locked measures). The ROI for target-present displays was the location (i.e., left, right, or bottom) occupied by the correctly identified target,¹ and the ROI for target-absent displays was the location occupied by the (arbitrarily) selected face. See the Supplemental Material for more information about eye movement measures.

To examine the influence of memory and visual similarity on eye movements directed to the ROIs, we binned data from individual trials within each display type (target-present displays with 20% morphs, target-present displays with 40% morphs, target-absent displays with 20% morphs, and target-absent displays with 40% morphs). We used time-course measures to examine the proportion of total viewing time directed to the ROIs in successive 1,000-ms time bins (starting with the onset of the test display) and to determine whether and when effects of prior exposure emerged in eye movement behavior. We used response-locked measures to examine the proportion of viewing time directed to the ROIs in successive 500-ms time bins beginning 2 s before participants explicitly selected a face and to determine whether participants’ eye movements prior to explicit responses distinguished studied targets from selected nonstudied faces (for details, see the Supplemental Material).

Statistical analyses

Omnibus analyses of variance (ANOVAs) were calculated using the Greenhouse-Geisser adjustment to the degrees of freedom. The corrected p value and the Greenhouse-Geisser epsilon (ε) are reported for F tests with more than 1 degree of freedom in the numerator. Post hoc comparisons were Bonferroni corrected. For more details on statistical analyses, see the Supplemental Material.

Results and discussion

Behavior

Despite high levels of visual similarity among faces in target-present displays, participants readily identified studied targets and correctly endorsed them as studied (see Table 1), a result contrary to our prediction. Furthermore, and in compliance with instructions, participants selected a face on 96.76% (SD = 5.41%) of the target-absent trials. As predicted, the rate of false alarms (i.e., incorrect endorsements of the selected face as studied) was higher when target-absent displays contained 40% morphs than when they contained 20% morphs, an effect of visual similarity on explicit responses (see Table 2). This effect was driven by differential false alarm rates when participants selected one of the two morphs; the false alarm rates for 20%-morph and 40%-morph displays did not differ significantly when the foil face was selected.

Table 1.

Percentage of Targets That Were Selected and Endorsed as Studied and Associated Statistical Comparisons

Experiment and measure	20%-morph displays	40%-morph displays	t(23)	p
Experiment 1
Selection of target	93.52 (11.78)	92.13 (12.03)	0.57	.57
Endorsement of target as studied	95.66 (8.07)	93.48 (10.22)	0.86	.40
Experiment 2
Selection of target	90.74 (10.70)	87.50 (13.23)	1.43	.17
Endorsement of target as studied	95.21 (7.63)	95.17 (9.73)	0.02	.99

Note: Statistical tests were performed to determine whether or not there were significant differences in the rate of correct identification of targets or in the rate of endorsement of targets as studied between 20%-morph and 40%-morph displays. Standard deviations are shown in parentheses.

Table 2.

Percentage of Selected Faces in Target-Absent Displays That Were Incorrectly Endorsed as Studied and Associated Statistical Comparisons

Experiment and measure	20%-morph displays	40%-morph displays	t(23)	p
Experiment 1
Morphs and foils	19.44 (17.72)	33.51 (16.23)	3.58	.002
Morphs only	15.79 (18.25)	33.09 (20.71)	4.56	< .001
Experiment 2
Morphs and foils	33.80 (22.10)	43.81 (17.08)	1.65	.11
Morphs only	31.40 (28.88)	49.39 (20.26)	2.59	.02

Note: Statistical tests were performed to determine whether or not there were significant differences in the rate of incorrect endorsements of selected nonstudied faces as studied between target-absent displays containing 20% morphs and target-absent displays containing 40% morphs. Results are shown for all of the target-absent trials on which participants selected either a morph or a foil and for all of the target-absent trials on which participants selected one of the two morphs, but not the foil. Standard deviations are shown in parentheses.

Evaluation of response times (RTs) showed that faces were selected more quickly from target-present displays than from target-absent displays, F(1, 23) = 230.93, p < .001 (see Table 3), but there was no difference in RTs as a function of visual similarity (20%-morph displays vs. 40%-morph displays), and there was no significant interaction between display type and visual similarity (ps ≥ .15).

Table 3.

Mean Response Times (in Milliseconds) for Identification of Targets (Target-Present Trials) and Selection of Nonstudied Faces (Target-Absent Trials) in 20%- and 40%-Morph Displays

Type of display	Experiment 1	Experiment 2
Target-present displays
20% morphs	3,316 (873)	3,382 (1,149)
40% morphs	3,558 (887)	3,578 (1,133)
Target-absent displays
20% morphs	5,661 (919)	5,998 (1,248)
40% morphs	5,583 (928)	5,667 (1,205)

Note: Standard deviations are shown in parentheses.

Eye movements

Having confirmed that the visual-similarity manipulation influenced explicit behavioral responses, we next examined the impact of visual similarity on eye movement behavior.

Time-course measures: visual similarity

We used a repeated measures ANOVA to examine viewing time as a function of face type (correctly identified target face in target-present display, selected face in target-absent display), visual similarity (20%-morph displays, 40%-morph displays), and time bin (0–1,000 ms, 1,000–2,000 ms, etc.). Results (Fig. 2a) showed that participants spent more time looking at studied targets in target-present displays than at selected nonstudied faces in target-absent displays, F(1, 23) = 28.29, p < .001, and that patterns of viewing changed with time, F(9, 207) = 10.67, p < .001, ε = .60. Disproportionate viewing of targets relative to selected nonstudied faces was marginally more robust early in the trials—Face Type × Time Bin interaction: F(9, 207) = 2.13, p = .06, ε = .61. However, disproportionate viewing of targets was not influenced by manipulations of visual similarity, p = .42, ε = .53. No other main effects or interactions were significant (ps ≥ .21). Post hoc comparisons indicated that disproportionate viewing of targets was evident in the bin from 1,000 to 2,000 ms after the onset of the three-face displays, ts(23) ≥ 3.33, ps ≤ .02, and that at no point were there significant differences in viewing time between targets in 20%-morph displays and targets in 40%-morph displays (ps ≥ .30) or between selected faces in 20%-morph displays and 40%-morph displays (ps ≥ .52; see Fig. 2a).

Fig. 2.

Proportion of total viewing time directed to regions of interest as a function of time bin. The graphs on the left show results from time-course analyses for Experiments 1 (a) and 2 (c). The proportion of total viewing time directed to correctly identified targets in target-present displays and to selected faces in target-absent displays is plotted for each successive 1,000-ms time bin (0–1,000 ms, 1,000–2,000 ms, 2,000–3,000 ms, etc.). Time 0 is the onset of the test display. The graphs on the right show results from response-locked analyses for Experiments 1 (b) and 2 (d). The proportion of total viewing time directed to correctly identified targets in target-present displays and to selected faces in target-absent displays is plotted for each successive 500-ms time bin, from 2,000 ms prior to the behavioral response (preresponse) until 2,000 ms after the behavioral response (postresponse). Time 0 is the time of the button-press response and is indicated here with a vertical dashed line. All graphs show results separately for 20%-morph trials and 40%-morph trials. Error bars indicate standard errors of the mean.

This analysis controlled for influences of explicit responding on eye movements, and the results suggest an influence of memory for studied faces on viewing behavior. However, because targets were identified more quickly than nonstudied faces were selected, the disproportionate viewing of targets relative to selected nonstudied faces may have been an artifact of differences in the time participants required to make a selection. We conducted response-locked analyses to test for this possibility.

Response-locked measures: visual similarity

We used a repeated measures ANOVA to examine preresponse viewing as a function of face type (correctly identified target face in target-present display, selected face in target-absent display), visual similarity (20%-morph displays, 40%-morph displays), and time bin (2,000–1,500 ms before response, 1,500–1,000 ms before response, 1,000–500 ms before response, 500–0 ms before response).

Results (Fig. 2b) showed that targets were viewed disproportionately before overt responses were made, F(1, 23) = 8.27, p = .009. The magnitude of this effect changed in the time leading up to the response—Face Type × Time Bin interaction: F(3, 69) = 4.49, p = .01, ε = .81. However, this effect was not influenced by visual similarity, p = .14, ε = .90. Differences in viewing times for studied targets (in target-present displays with 40% morphs) and selected nonstudied faces (in target-absent displays with 20% morphs) were evident in the bin from 1,500 to 1,000 ms before responses, t(23) = 3.333, p = .02; by 1,000 to 500 ms before responses, target viewing for both 20%- and 40%-morph displays exceeded viewing of selected faces in target-absent displays with 20% morphs, ts(23) ≥ 3.95, ps ≤ .004. Viewing-time differences between targets and selected faces in target-absent displays with 40% morphs were eliminated by correction for multiple comparisons (ps ≥ .35). Consistent with our time-course analyses, our response-locked analyses revealed no significant differences between viewing times for targets in target-present displays containing 20% morphs and viewing times for targets in target-present displays containing 40% morphs or between viewing times for selected nonstudied faces in target-absent displays containing 20% morphs and viewing times for selected nonstudied faces in target-absent displays containing 40% morphs in any time bin (ps ≥ .07; see Fig. 2b). These results, which were replicated in an additional control experiment (for more details, see the Supplemental Material), confirm the effect of memory on eye movement behavior and indicate that differences in RTs did not contribute to the reported results.²

Experiment 2

In Experiment 1, eye movements distinguished studied targets from selected nonstudied faces and, in contrast to behavioral responses, were unaffected by manipulations of visual similarity. Experiment 2 was conducted to replicate this effect, and was also designed to address a question that is arguably more compelling—namely, whether or not eye movements accurately distinguish studied faces from faces incorrectly endorsed as studied. In order to address this question, we compared selected faces incorrectly endorsed as studied (false alarms), selected faces correctly discounted as not studied (correct rejections), and targets correctly endorsed as studied (hits). We could not make these comparisons in Experiment 1 because the number of false alarms was insufficient for analysis. Therefore, to increase the rate of false alarms, we reduced the number of study exposures from five to three in Experiment 2. We predicted that preresponse viewing would reflect past experience (i.e., viewing times for hits would exceed viewing times for false alarms and correct rejections), but that postresponse viewing would reflect the explicit decision (i.e., viewing times for hits and false alarms would exceed viewing times for correct rejections).

Method

Participants

Twenty-nine students from the University of California, Davis, participated in the experiment in return for course credit. Five participants were excluded from the reported analyses because they had fewer than four false alarms for target-absent displays.

Stimuli, design, and procedure

The stimuli and procedure were as described for Experiment 1, with the exception that the number of study exposures was reduced to three.

Eye movement measures

In Experiment 2, we used the same ROIs (i.e., the locations occupied by correctly identified targets in target-present displays and by selected nonstudied faces in target-absent displays) and eye movement measures used in Experiment 1 to examine viewing behavior, but individual trials were binned in two ways. Analyses that examined influences of visual similarity used the same binning strategy that was described earlier, to allow comparison with Experiment 1. Analyses that examined whether or not eye movements provide a veridical index of memory, even when explicit reports are incorrect, involved grouping trials on the basis of participants’ subjective impressions of the status of a chosen face (i.e., false alarm, correct rejection, or hit). For this analysis, data from 20%-morph trials and 40%-morph trials were collapsed to ensure adequate bin sizes for evaluating the effects of memory on eye movement behavior.

Results and discussion

Behavior

Studied faces in target-present displays were readily identified and correctly endorsed as studied, and performance did not differ as a function of visual similarity (see Table 1); these results are consistent with those from Experiment 1. Participants complied with instructions by selecting faces on 96.05% (SD = 6.45) of the target-absent trials. On an average of 73.8% (SD = 10.6) of these trials—more often than would be expected by chance, t(23) = 3.14, p = .005—the selected face was one of the two faces that had been morphed with a studied (but not tested) target.

When judging their selection of a nonstudied face from target-absent displays, participants erroneously endorsed that face as studied on approximately one third of the trials. Although there was no significant difference in the rate of false alarms for 20%-morph and 40%-morph displays when we collapsed the data for selected foils and selected morphs, there was a significant difference when selected morphs were evaluated separately (i.e., when data for target-absent trials in which foils were selected were excluded from analysis): As in Experiment 1, participants were more likely to incorrectly endorse 40% morphs as studied than to endorse 20% morphs as studied, a result that suggests an influence of visual similarity on the accuracy of explicit responses (see Table 2).

Results for RTs in Experiment 1 were also replicated in Experiment 2. Participants identified studied targets more quickly than they selected nonstudied faces, F(1, 23) = 130.25, p < .001 (see Table 3). In Experiment 2, however, this main effect was qualified by a significant interaction with visual similarity, F(1, 23) = 4.17, p = .05. RTs were marginally faster for target-absent displays with 40% morphs than for target-absent displays with 20% morphs (uncorrected p = .07). Further analysis revealed that RTs were shorter for hits (M = 3,357.19 ms, SD = 1,018.16) than for either false alarms (M = 5,660.66 ms, SD = 1,291.12) or correct rejections (M = 5,986.50 ms, SD = 1,298.52), ts(23) ≥ .11.24, ps < .001. RTs for false alarms and correct rejections did not differ, p > .05.

Eye movements

Time-course measures: visual similarity

Results for the effect of visual similarity on viewing time in Experiment 1 were replicated in Experiment 2. More time was spent viewing studied targets than viewing selected nonstudied faces, F(1, 23) = 45.04, p < .001, regardless of visual similarity, p = .88, ε = .66; this effect was retained throughout the entire time course of the trials, p = .12, ε = .58. Disproportionate viewing of targets relative to selected nonstudied faces was evident in the 0- to 1,000-ms bin for all comparisons, ts(23) ≥ 2.89, ps ≤ .05, with one exception: Viewing time in this bin did not differ significantly between targets in target-present displays with 20% morphs and selected nonstudied faces in target-absent displays with 20% morphs; for these displays, disproportionate viewing of targets was significant beginning in the 1,000- to 2,000-ms bin, t(23) = 5.55, p < .001. There were no viewing-time differences between targets in 20%-morph displays and targets in 40%-morph displays, nor were there viewing-time differences between selected nonstudied faces in 20%-morph displays and selected nonstudied faces in 40%-morph displays, at any point during the trials (ps ≥ .24; see Fig. 2c).

Response-locked measures: visual similarity

Results for response-locked measures in Experiment 1 were also replicated in Experiment 2. Before responding, participants looked disproportionately at studied targets, F(1, 23) = 16.94, p < .001. This effect was most robust close in time to response execution—Face Type × Time Bin interaction, F(3, 69) = 11.99, p < .001, ε = .78. However, disproportionate target viewing was not influenced by visual similarity, p = .15, ε = .81. As in Experiment 1, significantly disproportionate viewing of targets relative to selected nonstudied faces was evident by 1,000 to 500 ms before responses, ts(23) ≥ 4.25, ps ≤ .005, and there were no preresponse differences between viewing times for targets in 20%- and 40%-morph displays or between viewing times for selected nonstudied faces in 20%-morph and 40%-morph displays (ps ≥ .36; see Fig. 2d).

Time-course measures: veridical versus false memory

Findings from both experiments showed that participants looked disproportionately at studied target faces relative to selected nonstudied faces, and that the time course of this effect was impervious to manipulations of visual similarity. This result constitutes strong evidence for an effect of memory on eye movement behavior, but it remains unclear whether participants looked disproportionately at any faces that were subsequently endorsed as studied, even when such endorsements were erroneous.

To test for this possibility, we conducted a 3 (selected-face type: hits, false alarms, correct rejections) × 10 (time bin: 0–1,000 ms, etc.) repeated measures ANOVA. Results showed significant main effects of selected-face type, F(2, 46) = 29.16, p < .001, ε = .68, and time bin, F(9, 207) = 14.37, p < .001, ε = .51, and a significant interaction of these factors, F(18, 414) = 3.36, p = .001, ε = .44. Disproportionate viewing of hits relative to both false alarms and correct rejections was evident in the bin 2,000 through 3,000 ms after the onset of the three-face displays, ts(23) ≥ 3.19, ps ≤ .01; viewing times for false alarms and correct rejections were not significantly different within this time bin, t(23) = 1.08, p = .87 (see Fig. 3a).

Fig. 3.

Results from Experiment 2. The graph in (a) shows the proportion of total viewing time directed to correctly identified targets (hits), selected nonstudied faces incorrectly endorsed as studied (false alarms), and selected nonstudied faces correctly discounted as not studied (correct rejections) for each successive 1,000-ms time bin (0–1,000 ms, 1,000–2,000 ms, 2,000–3,000 ms, etc.; Time 0 is the onset of the test display) of the 10,000-ms test trials. The graph in (b) shows the proportion of total viewing time directed to hits, false alarms, and correct rejections for each successive 500-ms time bin, from 2,000 ms prior to the behavioral response (indicated by the vertical dashed line at Time 0) until 2,000 ms after the behavioral response. Error bars indicate standard errors of the mean.

Response-locked measures: veridical versus false memory

We expected that preresponse viewing would reflect rapid, obligatory retrieval of previously studied information from memory and would distinguish hits from correct rejections and false alarms. We also expected that patterns of viewing subsequent to responses would reflect participants’ decision-making process and would distinguish hits and false alarms from correct rejections.

Results (see Fig. 3b) showed significant differences in viewing as a function of selected-face type, F(2, 46) = 10.05, p = .001, ε = .84, and response window (prereponse vs. postresponse), F(1, 23) = 7.07, p = .01, as well as a significant three-way interaction of selected-face type, response window, and time bin, F(6, 138) = 2.59, p = .04, ε= .68. Disproportionate viewing of hits relative to correct rejections was evident 1,500 through 1,000 ms before responses, t(23) = 2.66, p = .04, and disproportionate viewing of hits relative to both correct rejections and false alarms was evident 1,000 through 500 ms before responses, ts(23) ≥ 4.72, ps ≤ .001. There were no differences between preresponse viewing times for correct rejections and false alarms (ps ≥ .22). Evaluation of postresponse viewing times showed no differences between hits and false alarms (ps ≥ 1.12). However, postresponse viewing times were marginally greater for false alarms than for correct rejections within the 0- to 500-ms time bin, t(23) = 2.24, p = .11, and significantly greater for false alarms than for correct rejections by the 1,000- to 1,500-ms time bin, t(23) = 2.59, p = .05. These results confirm our predictions and constitute strong preliminary evidence that eye movements reflect veridical experience even when overt behavior does not.

Discussion

Results from two experiments demonstrated several dissociations between overt behavior and eye movements, such that eye movements better represented past experience. For example, visual similarity between morphs and targets influenced behavioral responses (i.e., participants were more likely to mistakenly identify a 40% morph as studied than to identify a 20% morph as studied) but not eye movements. Furthermore, viewing of selected nonstudied faces and targets was not graded as a function of visual similarity: In both experiments, there were no differences in the proportion of time spent viewing selected faces in target-absent displays that contained 20% morphs and selected faces in target-absent displays that contained 40% morphs, nor were there any differences in viewing times between correctly identified targets in the 20%-morph displays and correctly identified targets in the 40%-morph displays.

Whereas behavioral performance may be influenced by similarity between distractors and studied targets, and explicit choices may be based on the outcome of a comparison process (Lindsay & Wells, 1985; Wells, 1984, 1993), this seems not to be the case for eye movements, which consistently distinguished studied faces from distractors within 2 s after the onset of test displays. These early viewing effects seem to reflect a relatively pure index of past experience that is uninfluenced by explicit response strategies or motivations (see also Chanon & Hopfinger, 2008; Hannula et al., 2007; Holm, Eriksson, & Andersson, 2008; Richmond & Nelson, 2009; Ryan et al., 2007); such effects suggest the potential utility of eye movement measures in applied settings and in investigations conducted with populations whose performance may be misleading when testing is limited to explicit measures of memory (Luck & Gold, 2008).

Perhaps our most interesting finding is that viewing time distinguished studied faces from faces mistakenly identified as studied. Effects of veridical experience on eye movement behavior were evident 1,000 to 500 ms before a face was explicitly selected; all significant effects of memory on eye movement behavior in Experiments 1 and 2 and in our previous work (Hannula et al., 2007) emerged within this same time frame. The consistency of these viewing effects across conditions and studies is striking. Moreover, the results of Experiment 2 are the first to demonstrate that eye movements distinguish previously seen materials from novel materials incorrectly endorsed as studied. This result complements previous findings showing that eye movements are disproportionately drawn to altered regions of scenes even when participants are unaware of such manipulations (Ryan et al., 2000).

From a theoretical perspective, the results of Experiment 2 seem consistent with a recently proposed two-stage model of conscious recollection (Moscovitch, 2008). According to this model, information is initially retrieved quickly and obligatorily from memory, perhaps without awareness, and may contribute to performance on tasks in which learned information is expressed indirectly (e.g., via eye movements). During the second, slower stage of processing, retrieved information becomes consciously accessible and can influence explicit responses. The findings from Experiment 2 were consistent with the model in that they showed disproportionate preresponse viewing of faces that were studied and that therefore matched stored representations in memory. These early effects on eye movement behavior may have reflected automatic memory retrieval and may have been used to garner evidence to support eventual (conscious) endorsement of faces as studied. After responses had been made, eye movements reflected explicit choices, a result consistent with the second stage proposed by the model. More studies are required to determine what drives differences in viewing before and after explicit responses and to examine the alignment of eye movement data with subjective responses in later stages of processing.

In sum, in two experiments we found that the effects of prior exposure on eye movements are different from, and sometimes more veridical than, the effects of prior exposure on explicit judgments. Such results make it tempting to propose applying the measures used in our experiments to real-life identification situations (e.g., police lineups). However, because the reported results are based on averaging across many similar identification events, further investigations are necessary to examine patterns of viewing, behavioral responses, and confidence judgments under testing conditions resembling, for instance, those experienced by actual eyewitnesses. Future work could also examine whether and how search strategies change when the target face is not readily detected in this paradigm and, by extension, the effect of changes in strategy on eye movement behavior.