Hippocampal Contribution to Implicit Configuration Memory Expressed via Eye Movements During Scene Exploration

Abstract

Although hippocampus unequivocally supports explicit/ declarative memory, fewer findings have demonstrated its role in implicit expressions of memory. We tested for hippocampal contributions to an implicit expression of configural/relational memory for complex scenes using eye-movement tracking during functional magnetic resonance imaging (fMRI) scanning. Participants studied scenes and were later tested using scenes that resembled study scenes in their overall feature configuration but comprised different elements. These configurally similar scenes were used to limit explicit memory, and were intermixed with new scenes that did not resemble studied scenes. Scene configuration memory was expressed through eye movements reflecting exploration overlap (EO), which is the viewing of the same scene locations at both study and test. EO reliably discriminated similar study-test scene pairs from study-new scene pairs, was reliably greater for similarity-based recognition hits than for misses, and correlated with hippocampal fMRI activity. In contrast, subjects could not reliably discriminate similar from new scenes by overt judgments, although ratings of familiarity were slightly higher for similar than new scenes. Hippocampal fMRI correlates of this weak explicit memory were distinct from EO-related activity. These findings collectively suggest that EO was an implicit expression of scene configuration memory associated with hippocampal activity. Visual exploration can therefore reflect implicit hippocampal-related memory processing that can be observed in eye-movement behavior during naturalistic scene viewing.

INTRODUCTION

Lesions of hippocampus and surrounding medial temporal lobe (MTL) disrupt performance on explicit/direct tests of long-term memory such as recognition and recall more than on implicit/indirect tests such as priming (Penfield and Milner, 1958; Milner, 1962; Cohen and Squire, 1980; Shimamura and Squire 1987). This dissociation evidence has motivated the influential view that MTL is the seat of explicit memory, whereas other regions support implicit memory (Squire and Zola-Morgan, 1991; Gabrieli et al., 1995; Henson, 2003; Squire et al., 2004). However, it is difficult given only lesion evidence to determine contributions from MTL to implicit memory in healthy individuals (Ryals and Voss, forthcoming). Indeed, neuroanatomical distinctions between explicit and implicit memory may not be rigid (Dew and Cabeza, 2011), and a number of recent accounts of hippocampus emphasize its role in relational and/or associative aspects of memory irrespective of explicit awareness of memory processing (e.g., Ryan and Cohen, 2004; Greene et al., 2007; Henke, 2010; Hannula and Greene, 2012; Voss et al., 2012; Shohamy and Turk-Browne, 2013; Wang et al., 2014).

MTL activity identified using functional magnetic resonance imaging (fMRI) during recognition memory for complex visual scenes is typically assumed to reflect explicit memory processing (e.g., Henson, 2005; Montaldi et al., 2006), however some evidence also suggests that scene learning and discrimination are selectively disrupted in hippocampal amnesics using implicit/indirect tasks (e.g., Graham et al., 2006; Mundy et al., 2013). Complex scenes include stable configurations of elements, and relational processing of these configurations is highly relevant for scene memory (Aly et al., 2013). Indeed, scene recognition memory may derive from the match between the configurations of elements viewed during test with the configuration of elements of previously viewed scenes (Ryals et al., 2013). The hippocampus has been shown to be critically involved in differentiating similar from novel visual displays in contextual cueing paradigms (e.g., Chun and Phelps, 1999; Greene et el., 2007; Manelis and Reder, 2012; Geisbrecht et al., 2013) and in relational processing of complex scene features (e.g., Barense et al., 2010; Maguire and Mullaly, 2013). However, it is unclear if this relational processing of complex scene configuration information can occur without awareness as an implicit expression of memory.

Some findings emphasize hippocampal binding of distinct elements into coherent relational representations irrespective of explicit awareness (Cohen and Eichenbaum, 1993; Eichenbaum et al., 2007; Olsen et al., 2012). For instance, Ryan et al. (2000) demonstrated a relational memory effect whereby subjects viewed areas of scenes for which relationships among elements had been manipulated. This viewing effect occurred in the absence of conscious awareness of the scene manipulations, thus suggesting an implicit expression of relational memory in viewing behavior. Furthermore, individuals with amnesia due to bilateral hippocampal damage failed to exhibit increased viewing of relational manipulations, therefore suggesting a necessary role for hippocampus in implicit expressions of scene relational memory. In another study, Hannula and Ranganath (2009) identified hippocampal activity related to eye-movement correlates of relational memory for face-scene pairings even when explicit memory performance failed (i.e., when recognition responses were incorrect).

Although previous findings have linked hippocampus with eye-movement expressions of implicit relational memory (e.g., Ryan et al., 2000; Hannula and Ranganath, 2009), these studies have involved memory for relations among individual elements (i.e., an item presented within a scene that is reconfigured in Ryan et al., 2000; the specific pairing between faces and scenes in Hannula and Ranganath, 2009). In contrast, memory for complex stimuli such as scenes likely involves processing of the overall configuration of relations among elements, such as the relative locations of objects, surfaces, boundaries, and other information (Aly et al., 2013; Ryals et al., 2013). It is unknown whether eye-movement expressions of relational memory can capture memory processing of this configural information, whether such eye-movement memory expressions occur implicitly, or whether they are related to hippocampus.

To address these unknowns, we used concurrent fMRI and eye-movement tracking during a novel scene recognition task in order to test relationships among eye-movement measures of scene configuration memory, explicit judgments of scene memory, and hippocampal activity. Scenes viewed during test were not identical to studied scenes, but instead had feature configurations similar to those in corresponding studied scenes (i.e., configural similarity). This served to reduce explicit memory (which would be high if instead identical scenes were repeated at test, Aly et al., 2013). Novel scenes that were not similar to studied scenes were also presented during test. Eye-movement expressions of scene configuration memory were quantified as the tendency to view similar features at both study and test, or visual “exploration overlap” (EO). fMRI correlates of EO were compared to those of objective and subjective explicit recognition to test the relationship between hippocampal correlates of hypothesized implicit measures (EO) and explicit/direct measures (judgments) of scene configuration memory. We hypothesized that hippocampal correlates of EO would be distinct from activity associated with explicit/direct measures to the extent that hippocampal activity can reflect implicit expression of scene configuration memory.

Notably, the memory test in the current experiment was “direct” in the sense that subjects made overt judgments regarding whether each item was previously studied or was new (in contrast to “indirect” test formats that measure memory covertly, Schacter, 1990). Nonetheless, we hypothesized that implicit memory processing of scene configurations could have occurred and have been indexed by EO viewing behavior. Indeed, it is well established that explicit and implicit memory processing can occur during tests that are both direct and indirect in format, contrary to the notion that direct tests measure only explicit memory processing and indirect tests measure only implicit memory processing (Henke 2010; Dew and Cabeza 2011, 2012; Ryan 2012; Voss et al., 2012). We reasoned that if explicit memory were to fail in the sense that subjects did not reliably discriminate similar from new scenes using overt judgments, yet nonetheless EO reliably discriminated these categories, then EO would represent an implicit expression of memory that occurred irrespective of explicit memory (as in Hannula and Ranganath, 2009). Furthermore, we sought converging neural evidence of this distinct implicit processing associated with EO by testing whether fMRI correlates of EO were distinct from those of processing related to explicit/direct memory measures.

MATERIALS AND METHODS

Subjects

Twenty-seven participants were recruited from the Chicago metropolitan area (ages 18–35 yr; 18 females). Data from six individuals were discarded (four for failing to complete the study and two for failure to obtain accurate eye-tracking calibration). Results are thus reported for N=21 (ages 21–35 yr; mean age 28 yr; 14 females). All participants were right-handed, had normal or corrected-to-normal vision, did not report neurological or psychiatric disorders, and did not report the current use of psychoactive drugs. All participants gave written, informed consent and were remunerated for their participation. The Institutional Review Board at Northwestern University approved the study protocol.

Scene Stimuli

Stimuli were static images of 90 nameable color scenes originally developed for the virtual-reality study by Cleary et al. (2012). Scene images were created using Sims2^™ software, which allowed uniform placement of structural features (e.g. walls, floors, ceilings, architecture, landscape terrains, and horizons) as well as individual elements (e.g., chairs, plants, light fixtures) using a geometric grid. This grid allowed for precise matching between scene element positions in space and simultaneous variation of the nature of these elements. We achieved configural similarity for matched pairs by carefully maintaining the angles of walls, horizons, the placement of objects on the grid, and the camera vantage point in each image. When constructing scenes, we specifically avoided replicating individual elements across scenes when creating configurally similar study-test pairs.

Experiment Procedures

Participants were given an overview of the procedures before beginning the experiment. Printed examples of study scenes, similar test scenes, and novel test scenes were reviewed, and participants practiced using the response box while in the MRI scanner. Following in-scanner eye-tracker calibration, participants viewed detailed computerized instructions prior to beginning the experiment.

The experiment comprised three study-test blocks performed during fMRI scanning. Each study session included 12 novel scenes. Individually presented scenes were preceded immediately by a descriptive name (i.e., “train station”, “pool”) presented centrally for 750 ms. The scene was then presented for eight seconds. Participants were then prompted to make a prospective judgment of encoding (JOE) rating using a 4-point scale to indicate how well they thought they had learned the scene, ranging from low confidence to high confidence (Dunlosky et al., 2003). Subjects were given 6 s to make the JOE. A variable intertrial interval (2–36 s, mean=7.63 s) followed the JOE response period.

After each study session, participants viewed instructions for 42 s before beginning the test. During test, participants viewed 24 novel scenes individually. Half of the test scenes resembled studied scenes from the previous study session in terms of configuration (i.e., “configural similarity”), and half were dissimilar (i.e., “new”; Fig. 1A). Examples of configurally similar study/test scene stimuli are displayed in Figure 1B. Each test scene was presented for 8 s. Participants were then prompted to rate the test scene familiarity using a 4-point scale ranging from low to high familiarity. Subjects were instructed to base familiarity ratings on how strongly they felt that a test scene resembled one that had been viewed previously during study, and they were given 6 s to make the familiarity rating. Participants were then prompted to make a yes/no response to indicate whether they remembered studying a specific similar scene during the preceding study session. Subjects were given 6 s to make the yes/no judgment. A variable intertrial interval (2–36 s, mean=9 s) separated the yes/no judgment from the following test scene. Each scene (during study and test) subtended ~18.8° vertical and ~23.1° horizontal of visual angle.

FIGURE 1.

Trial structure and stimuli. (A) Example study trials (top) and test trials (bottom). (B) Study/test scene stimuli examples. Yellow arrows indicate configurally similar scene pairs. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Following the final study-test block and a subsequent structural MRI scan (see below), participants were removed from the scanner and then given an old/new recognition test. For this test, participants viewed 36 scenes that had appeared either during the study or test sessions during scanning intermixed with 26 novel scenes (foils), and were asked to make self-paced yes/no recognition judgments. This final test took no more than 5 min to complete.

Eye Tracking Methods and Exploration Overlap Calculation

Eye movements were recorded during study and test using an Eyelink 1000 remote system (SR Research, Ontario, Canada) at a sampling rate of 500 Hz. The eye-tracking camera was focused on the right eye via the head coil-attached mirror used to view the projection screen. Continuous eye-movement records were transformed into a time series of fixations, saccades, and blinks. Motion (0.15°), velocity (30°/s), and acceleration (8000°/s²) thresholds were used to identify saccades. Periods when the pupil was missing from the image were classified as blinks. Saccade- and blink-free periods were categorized as fixations. The duration and time course of fixations for study and test scenes were imported and analyzed with custom scripts in Matlab (The MathWorks). Eye-tracking data were successfully obtained from the 21 participants that contributed to the fMRI analysis. One study-test block was discarded from each of three participants due to poor calibration.

EO is a trial-level measure of the extent to which individuals viewed similar regions of a studied scene and the corresponding similar test scene, thus providing an index of configuration memory. For each subject, EO was calculated for each study-test scene pair by first determining the duration of fixations during test that were within a critical distance (“spotlight radius”) of the fixations that occurred during study (i.e., overlapped in space between test and study), then taking the ratio of this overlapped duration to the total duration of fixations during study and test. The two initial central fixations were discarded to reduce the influence of central fixation during the pre-stimulus period on the EO measure.

It is important to note that large numbers of fixations could result in EO that is unrelated to scene configural memory. That is, simply viewing many scene locations in a random manner during study and test would produce high EO values having nothing to do with memory. Figure 2A depicts results from a simulation showing that EO values depend on the number of fixations at study and test when fixation placements are completely random, averaged over 500 simulated trials. In order to control for the effects of number of fixations on EO, all EO analyses were normalized (z-scored) by the mean and SD of the null distribution of EO given random fixation derived from simulation. Thus, EO values above 0 represent fixation overlap that occurred more than would have been expected by random chance given the number of fixations that occurred for a given scene at both study and test.

FIGURE 2.

Calculation of exploration overlap (EO). (A) EO values as a function of number of fixations at study and test, derived from simulation. Values were used to normalize EO by the number of study and test fixations for all analyses. (B) t Values of discriminability of study-similar scene pairs from study-new scene pairs using EO values as a function of spotlight radius used in the calculation of EO. Maximum discriminability based on EO occurred when EO was calculated using a radius of 65 pixels (~3° of visual angle). *P < 0.005; **P < 0.0005; ***P < 0.0002. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Because EO values depend on spotlight size, which should be related to (but do not entirely reflect) the size of the foveal area, we tested a range of radius values (15–150 pixels) in order to determine the radius that produced EO values that most accurately reflected configuration processing. Study-similar scene pairs (i.e., studied scenes and their matching configurally similar scenes at test) shared high configural similarity and would be expected to elicit high EO to the extent that the EO value given a specific spotlight radius reflected viewing of configural similarity. In contrast, study-new scene pairs (i.e., studied scenes and randomly selected new scenes at test) did not have high configural similarity and would be expected to elicit low EO values. We therefore calculated the ability to discriminate study-similar pairs from study-new pairs given EO values for each possible study-new combination across a large range of spotlight radii while simultaneously controlling for the number of fixations.

As shown in Figure 2B, EO was significantly higher for study-similar scene pairs compared to study-new scene pairs for a wide range of radii (25–115 pixels), but there was peak discrimination at 65 pixels. All analyses of EO therefore used normalized EO values calculated with a spotlight radius of 65 pixels (~3° of visual angle) unless otherwise reported.

MRI Data Acquisition and Analysis

MRI scanning used a Siemens 3 T Trio scanner with a 32-channel head coil. Head movement was minimized using foam padding. Visual stimuli were back-projected onto a screen and viewed through a mirror attached to the head coil. The screen resolution was 1024 × 768 pixels, with a screen refresh rate of 60 Hz, and viewed at an eye-to-screen distance of approximately 64 cm.

Whole-brain BOLD EPI was collected during study and test sessions [repetition time (TR)=2000 ms, echo time (TE)=20 ms, acquisition voxel size=1.70 × 1.70 × 3 mm³, field of view (FOV)=22 cm, flip angle=80°]. Images were collected and the session began after the scanner reached steady state. All images were acquired in interleaved fashion in the axial plane beginning at the top of the brain. Randomized interstimulus intervals (ISIs) for study and test blocks (see above) were optimized in order to maximize estimation of event-related signals for each condition using the Optseq toolbox (Dale, 1999). A structural MRI was acquired to provide anatomical localization (MPRAGE T₁-weighted scans, TR=2400 ms, TE=3.16 ms, voxel size=1 mm³, FOV=25.6 cm, flip angle=8°, 176 sagittal slices).

Functional and structural MRI data were analyzed using the AFNI software package (Cox, 1996). Preprocessing steps included volume registration through time (motion correction), slice-timing correction, functional/structural coregistration, stereotactic transformation using the Montreal Neurologic Institute (MNI) 305 template, resampling to 1.5 × 1.5 × 1.5 mm³, and spatial smoothing with a 4-mm FWHM Gaussian kernel. Event-related activity for each condition was modeled using a deconvolution approach within a general linear model (GLM), with a regressor corresponding to scene viewing generated by convolving a boxcar function of eight-second “on” periods locked to stimulus onsets with a canonical hemodynamic response function. Nuisance variables included T₀ and T₁* components of the MR signal and six-parameter movement estimates. To estimate fMRI activity related to trial-by-trial measures of EO, event onsets were amplitude modulated by the EO value for each trial (i.e., parametric analysis). This identified activity that linearly varied with trial-by-trial variability in EO independent from variance accounted for by the main effects of condition.

Regions exhibiting significant activity at the group level were identified via random-effects analysis. The primary analyses were performed to test hypotheses regarding the involvement of MTL structures, and a combined voxel-wise threshold (P<0.005) and spatial-extent threshold (10 contiguous supra-threshold voxels) was used for MTL regions, as in other targeted analyses of MTL activity (e.g., Witmann et al., 2005; Maass et al., 2014). We additionally used AFNI 3dClustSim to evaluate significance of MTL clusters. We used a bilateral hippocampal mask (9,852 mm³) defined via the Desai atlas of Desikan and colleagues (2006), with a FWHM parameter of 3.5 mm, determined by averaging the estimated smoothness of the GLM residuals within the mask from each subject (using AFNI 3dFWHM). These significance values are listed with all reported fMRI activity estimates. For the exploratory whole-brain analyses, a threshold of 30 contiguous supra-threshold voxels (101 mm³) was used to obtain a combined corrected threshold of P<0.05, as determined in the same manner as for the MTL analysis but using a whole-brain mask.

RESULTS

Recognition Memory Performance

Subjects were not able to reliably discriminate similar scenes from dissimilar (“new”) scenes during test using yes/no judgments. Rates of “yes” responses to similar items (hits) did not differ reliably from rates of “no” responses (misses), [t(20)=1.48, P=0.15], indicating unsuccessful overt similarity-based recognition (Fig. 3A). Furthermore, discrimination sensitivity (d′) calculated using signal detection methods (Macmillan and Creelman, 2005) indicated that similar/new recognition did not significantly differ from zero [Mean d′ =0.11, t(20)=1.61, P=0.12, C=–0.07 (a minimal liberal response bias)]. Explicit yes/no recognition of similar scenes thus failed on average.

FIGURE 3.

Behavioral and functional magnetic resonance imaging correlates of explicit memory. (A) Mean proportion of yes/no recognition responses and d′ estimate for discrimination of similar from new scenes. (B) Mean confidence/familiarity ratings for similar versus new scenes. (C) Medial temporal lobe activity identified for the Objective Explicit Memory contrast of Hits versus Misses. *P < 0.001; **P < 0.0001. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Familiarity ratings indicated that minimal similar/new discrimination did occur based on subjective confidence. Participants provided higher familiarity ratings when making “yes” judgments than when making “no” judgments, irrespective of whether the scenes were actually similar or new (Fig. 3B), indicating that familiarity was not a strong contributor to accurate similar/new discrimination. However, familiarity ratings differed slightly yet significantly for similarity hits versus false alarms [t(20)=3.96, P < 0.001, Cohen’s d=0.50; Fig. 2B]. This suggests a minimal degree of explicit familiarity for scene similarity information (replicating previous findings of familiarity-based discrimination of scene configuration similarity using similar testing procedures; Ryals et al., 2013) that did not robustly produce above-chance yes/no discrimination.

fMRI Correlates of Explicit and Implicit Recognition Memory in MTL Regions

To identify fMRI correlates of minimal explicit memory within MTL regions of interest, we compared fMRI activity related to hits versus misses for similar scenes at test. These two categories differed slightly yet significantly in familiarity ratings despite chance-level yes/no decisions (see above). This contrast identified greater activity of bilateral hippocampus body (bordering on parahippocampal cortex) as well as left parahippocampal cortex (Fig. 3A and Table 1). Although explicit memory differences between conditions used in this contrast were minimal, these findings are consistent with well-established associations between MTL regions and explicit memory, and particularly with the association between parahippocampal cortex and scene memory.

TABLE 1.

Summary of Functional Magnetic Resonance Activations for Primary Conditions of Interest Within Medial Temporal Lobe

	MNI centroid coordinates			BA area(s)	Volume (mm3)	Peak t-value
	X (mm)	Y (mm)	Z (mm)
Explicit memory (hits vs. misses)
L. Hippocampus	−17	−26	−9	−	240	3.93
L. Parahipp gyrus/cingulate	−5	−40	3	29	213	3.30
R. Hippocampus	16	−26	−9	−	152	3.82
R. Parahipp gyrus	38	−34	−15	35/36	54	3.86
Explicit memory (hits vs. correct rejections)
L Hippocampus	−14	−30	−16	−	54	3.49
Misses vs. correct rejections (negative effect)
L. Parahipp cortex	−19	−29	−18	35/36	50	−4.69
EO
R. Hippocampus	24	−21	−25	−	54	3.26

Effects are positive unless otherwise noted. Peak centroid t-values are listed. All clusters are corrected for multiple comparisons at P<0.05.

BA, Brodmann area; EO, exploration overlap; MNI, Montreal Neurologic Institute.

In order to provide comprehensive assessment of possible explicit memory fMRI correlates for comparison with fMRI correlates of EO (see below), we also performed an analysis of activity associated with hits for similar scenes versus with correct rejections of new scenes. This analysis identified a small area of activity in the left hippocampal body (Table 1).

Previous studies have identified neural correlates of implicit memory for scenes by comparing missed old scenes to new scenes (i.e., repetition with unsuccessful old/new discrimination), and have identified relative activity reductions in parahippocampal cortex for this comparison (e.g., Yi and Chun, 2005; Sayres and Grill-Spector, 2006; Xu et al., 2007). We, therefore, compared activity estimates for similar test scenes classified as new (misses) to new scenes classified as new (correct rejections). This comparison identified relatively reduced activity of left parahippocampal cortex for similar misses (Table 1). This finding is consistent with previous reports of implicit repetition effects for scenes.

EO and its fMRI Correlate in MTL Regions

Study scenes and corresponding similar scenes at test shared configural similarity and would be expected to elicit high EO values to the extent that EO successfully captured viewing of configural similarity. Indeed, EO values indicated significant viewing of overlapping regions of study scenes and corresponding similar scenes at test (mean raw EO=0.39, SD=0.08; t(20)=22.30, P<0.0001, Cohen’s d=2.13 vs. 0). As indicated above, EO values were calculated controlling for the number of fixations at study and test, as more fixations could nonspecifically inflate EO values (Fig. 2A). Therefore, EO occurred significantly more than what should have occurred by chance given random fixations. Further, as described above (Fig. 2), EO values successfully discriminated study-similar scene pairs (which had relative high levels of configural similarity) from study-new scene pairs (which had relative low levels of configural similarity). Thus, both analyses indicate that EO reflected above-chance viewing of repeated configural information from study to test.

To illustrate the nature of EO and the fixation properties it captures, a heat map demonstrating the density of raw mean fixation overlap for one study-similar scene pair is displayed in Figure 4A. To illustrate the general distribution of fixation overlap within scenes, we created a mean density map of the fixations that overlapped from study to test and therefore contributed to the calculation of EO for all scenes and all subjects (Fig. 4B). These fixations reflected repeated viewing of features that were distributed throughout scenes, with a high density of overlap for features located at and above the scene horizon and surrounding the center. This distribution could have been driven by the high density of important scene features at central locations and possibly due to a bias toward focusing on scene horizons and high-salience objects proximal to the horizon level (e.g., Tatler, 2014). However, this distribution was not likely due to carryover viewing of the scene center from the ISI period, during which participants focused on a fixation cross, as the first two fixations during each scene were removed from analyses (see MATERIALS AND METHODS).

FIGURE 4.

EO association with hippocampal activity. (A) Example heatmap of mean fixations for a study scene, the corresponding similar test scene, and corresponding thresholded map of the fixations that contributing to the raw EO value. (B) Density map of the fixations that contributed to EO values for all scene pairs and all participants. (C) Mean z-scored EO values for similarity hits versus similarity misses. (D) Right hippocampal body activity corresponding to EO. *P < 0.005. Error bars indicate SE. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

To test whether EO was related to configuration memory, we compared EO values for similar test trials with correct yes/no recognition responses (hits) to similar test with incorrect recognition responses (misses). It is important to note that since EO only concerned fixation overlap between similar study-test pairs, it was not possible to compute or compare EO values for similarity false alarms or correct rejections. As reported above, yes/no recognition was not above chance levels on average, and so any differences in EO values for hits versus misses would not reflect accurate explicit memory. EO values were significantly higher for similar hits than for similar misses [Fig. 4C; t(20)=3.40, P<0.005, Cohen’s d=0.80], indicating that EO reflects configural memory. Further, as reported above (Fig. 2B) EO values significantly discriminated study-similar from study-new test pairs, also suggesting that EO values reflected configural memory processing, in that study-similar pairs share higher configural similarity than study-new pairs.

fMRI correlates of EO were identified as activity that varied with trial-by-trial normalized EO values (i.e., parametric analysis), controlling for main effects of the similar condition. Mean normalized EO values collapsed across trials were substantially variable (Minimum EO z=–0.06, Maximum EO z=1.97; Mean SD across subjects=0.52). Fluctuation in EO was positively associated with fMRI activity in the right hippocampal body (Fig. 4D; Table 1). The centroid of this activation area was located in hippocampus proper according to three cytoarchitectonic probabilistic atlases (Eichoff et al. 2005a, 2005b; Desikan et al., 2006; Destrieux et al. 2010).

Similarities and Distinctions Between Explicit Memory and EO Behavior and Neural Correlates

As reported above, EO values differed for similar hits versus similar misses during test, despite the fact that yes/no discrimination was no better than chance. This indicates that EO was associated with correct responding that did not reflect explicit memory. To further test for relationships between EO and explicit memory, we computed trial-by-trial correlations of EO values and familiarity ratings for each subject. These values ranged from r=–0.35 to +0.49, and were not statistically significant at the group level [mean r=0.17; SD=0.22; t(19)=0.70, P=0.46]. Hence, there was no significant relationship between EO and familiarity ratings.

Notably, the hippocampal region associated with EO (Fig. 4D) was spatially distinct from regions identified by the explicit memory analysis (Fig. 3). However, the possibility exists that similar processing within hippocampus is related to both EO and explicit memory, but at sub-threshold levels. We tested this possibility by assessing parameter estimates for the explicit memory contrasts within the region of hippocampal body identified by the EO analysis. Explicit memory parameter estimates did not differ significantly from zero in this EO-related region of hippocampus [t(20)=0.01, P = ns]. We then tested EO parameter estimates in the MTL regions identified by the explicit memory contrast and found no above-zero activity in either hippocampal cluster [left: t(20)=1.28, P = 0.23, right: t(20)=0.54, P = 0.60], or parahippocampal cluster [left: t(20)=0.61, P = 0.55, right: t(20)=0.55, P = 0.61].

Although fMRI correlates of EO and explicit memory were spatially distinct, we tested for possible positive association by subjecting EO and explicit memory parameter estimates within the EO-related MTL region to a bivariate correlational analysis. This yielded a marginal positive relationship [Pearson’s r=0.35 P = 0.06, one-tailed], suggesting weak relationship of greater EO-related activity with greater activity related to explicit memory. Taken together, these findings indicate segregation of fMRI activity related to explicit memory versus EO within MTL, but suggest the potential for concomitant (possibly sub-threshold) contributions by this region of right hippocampal body to both EO and explicit memory.

Exploratory Whole-Brain Analyses of Explicit Memory and EO

Activity related to explicit memory and to EO was identified outside of MTL regions of interest using exploratory whole-brain analyses (see MATERIALS AND METHODS). Explicit memory (similar hits vs. misses) was associated with frontal, temporal, and parietal activity in regions typically associated with explicit memory retrieval (e.g. Buckner et al., 1996; Cabeza et al., 2008) Notably, activations were relatively small compared to most studies of explicit memory retrieval, likely due to the fact that explicit memory was minimal (not significantly above chance) due to the use of configurally similar rather than identical scenes at test, and fMRI activity likely reflected the minimal difference in familiarity between hits and misses. EO was associated with positive activity in regions including superior temporal gyrus, anterior cingulate, caudate, middle frontal gyrus, and inferior parietal cortex (Table 2). These regions are consistent with cortical-hippocampal functional networks that are involved in memory-guided behavior (Ranganath and Ritchey, 2012) and could potentially interact with the EO-related region of hippocampus to support EO (although the current design is not suitable for testing such interaction). EO was also related to reduced activity of ventral visual cortical regions (Table 2) that often show repetition-related activity reduction in priming paradigms (e.g., Squire et al., 1992; Schacter and Buckner, 1998; Ward et al., 2013). This reduced activity potentially reflected repetition-related response reductions resulting from repeated viewing of similar scene elements during EO.

TABLE 2.

Summary of Functional Magnetic Resonance Activations for Primary Conditions of Interest in the Exploratory Whole-Brain Analysis (Excluding Regions Within Medial Temporal Lobe Listed in Table 1)

	MNI centroid coordinates			BA area(s)	Volume (mm3)	Peak t-value
	X (mm)	Y (mm)	Z (mm)
Explicit memory (hits vs. misses)
Medial posterior cingulate	0	−41	3	23	1272	3.76
L. Superior frontal gyrus	−19	67	7	10	746	3.38
L. Posterior cingulate	−1	−47	−27	31	398	3.34
L. Middle temporal gyrus	−59	−27	−12	21	311	4.40
L. Angular gyrus	−34	−55	29	39	199	4.03
L. Medial frontal gyrus	−5	58	40	8	182	3.62
L. Middle occipital gyrus	−34	−73	35	19	152	3.21
L. Middle temporal gyrus	−69	−35	−6	21	148	3.60
R. Anterior cingulate	4	41	−6	24	131	3.82
R. Caudate nucleus	7	9	4	−	128	3.33
L. Precuneus	−2	−57	56	7	124	3.17
R. Middle frontal gyrus	29	59	6	10	118	3.65
L. Thalamus	−19	−22	16	−	104	3.63
L. Superior frontal gyrus	−14	49	36	8	104	5.01
EO (positive effects)
R. Superior temporal gyrus	54	−25	10	41	189	3.38
L. Anterior cingulate cortex	−10	52	3	33	172	3.44
L. Caudate nucleus	−23	−8	−21	−	158	3.60
R. Middle frontal gyrus	23	29	36	46	139	4.43
L. Inferior parietal lobe	−45	−34	41	40	132	3.43
EO (negative effects)
L. Superior occipital gyrus	−11	−94	34	23	142	−3.64
L. Lingual gyrus	−6	−80	5	17	122	−3.37
L. Fusiform gyrus	−42	−59	−6	37	115	−3.65

Effects are positive unless otherwise noted. Peak centroid t-values are listed. All clusters are corrected for multiple comparisons at P<0.05.

BA, Brodmann area; EO, exploration overlap; MNI, Montreal Neurologic Institute.

Accurate Explicit Memory and Memory Awareness for Identical Scenes

Poor explicit memory for similar scenes can be appreciated in contrast to the robust explicit memory that was observed for identical scenes during a final old/new recognition memory test performed after the MRI scanning session (~20 min delay following completion of the in-scanner study-test blocks; see MATERIALS AND METHODS). Hit rates differed significantly from false alarm rates [t(20)=59.57, SE=0.02, P < 0.0001, Cohen’s d=15.75], yielding a mean d′ of 3.68 [t(20)=14.63, P<0.0001 vs. chance discrimination of zero, Cohen’s d=6.54, C=0.06 (a minimal conservative response bias)]. This suggests that even after a post-scanning delay, participants demonstrated robust old-new recognition memory for studied scenes.

Analyses of memory awareness during study also highlighted the contrast between poor explicit memory for similar scenes versus accurate explicit memory for identical scenes in the delayed recognition test. JOEs corresponded positively to recognition performance [G=0.30; P=0.03 computed for 20 participants using the nonparametric gamma (G) correlation (Goodman and Kruskall, 1954; one individual was excluded from this analysis due to a lack of variability in JOE responses). In contrast, there was an unreliable correlation between JOEs and recognition performance for similar scenes during the in-scanner tests [G=–0.13; P=0.34]. A direct paired-samples comparison between mean gamma correlations for each of these two relationships was reliable [t(20)=2.52, P=0.02, Cohen’s d=0.91]. Poor awareness of encoding during study relevant to similar scene recognition thus stood in contrast to excellent awareness of encoding relevant to identical scene recognition, further underscoring the distinction of memory processing relevant to similar scenes from explicit scene memory.

DISCUSSION

We report evidence for hippocampal contribution to an implicit expression of configural scene memory observed in eye-movement measures obtained during direct memory testing. High values of EO reflected viewing of the same regions of scenes at both study and test, even though only configurations of scene elements (not elements themselves) repeated from study to test, thus providing a measure of scene configuration memory. Two lines of evidence suggest that EO was an implicit expression of memory: (1) there was minimal behavioral evidence for explicit memory owing to the use of similar rather than identical scenes at test, and behavioral explicit memory was statistically unrelated to EO and (2) the neural correlate of EO in right hippocampus was distinct from other explicit memory-related hippocampal and MTL activity. Although previous findings have indicated that the hippocampus is involved in relational memory processing for elements of scenes (e.g., Ryan et al., 2000), the present findings are unique in demonstrating implicit scene memory for configural information as expressed via viewing behavior during a relatively unconstrained scene memory task. In contrast, viewing behavior during such memory-testing circumstances is generally thought to reflect explicit processing (e.g., Smith et al., 2006; Holm and Mantyla, 2007; Smith and Squire, 2008). Moreover, we identified activity of cortical regions, including prefrontal cortex, which is also frequently implicated in the deliberate/explicit control of exploration based on memory. Our findings thus underscore hypothesized roles of hippocampus and frontal cortical networks in implicit expressions of memory during ongoing exploratory behavior (Wang et al., 2014).

Some prior evidence is consistent with a specialized role of hippocampus in scene processing (Lee et al., 2005; Graham et al., 2006; Aly, et al., 2013; Maguire and Mullaly, 2013). Although we interpret our EO hippocampal findings as an implicit expression of memory for scene configuration information, it is possible that specific configurations of scenes were particularly salient and that EO therefore reflected specialized perceptual processing of these configurations instead of memory for studied scene configurations per se. However, if EO were to reflect scene processing rather than memory, we would have expected that EO estimates would have been equivalent between similar hits and misses at test. Rather, EO reliably differed between these two categories, thus suggesting an effect of configural memory rather than simply scene perception. Because we did not test EO for other, non-scene stimulus categories, these findings would not speak against the interpretation that scene configuration information is just one of many categories of complex, multi-feature perceptual input that requires relational (i.e., configural) hippocampal processing (Cohen and Eichenbaum, 1991; Cohen and Eichenbaum, 1993; Althoff and Cohen, 1999; Konkel and Cohen, 2009).

The anterior hippocampus is particularly implicated in global reinstatement of complex configural information (e.g., Xu et al., 2010), consistent with the proposal that hippocampus supports pattern completion and retrieval of scene features, configurations, and even distances between scene elements (Morgan et al., 2011; Poppenk et al., 2013). These configural retrieval functions are consistent with our interpretation of hippocampal correlates of EO. Notably, there are growing homologues between these configural memory functions studied in rodent hippocampus and similar forms of configural memory processing in primate and human hippocampus and MTL cortex (i.e., Ekstrom et al., 2003; Doeller et al., 2010; Suthana et al., 2011; Killian et al., 2012). Our findings further support the role of hippocampus in relational/configural memory processing and also indicate that this processing can be expressed implicitly through exploratory eye movements.

Although MTL activity is typically presumed to reflect explicit memory, previous studies have identified relational memory processing indexed by eye movements and reflecting implicit memory (e.g., Ryan et al., 2000; Hannula and Ranganath, 2009). For instance, Hannula and Ranganath (2009) found that eye movements demonstrated relational memory for face-scene pairings related to hippocampal activity, even when explicit recognition failed. Similarly, Greene et al. (2007) found that hippocampal activity differentiated novel from identical visual displays even when overt recognition performance did not. A recent study by Manelis and Reder (2012) showed selective activation of right hippocampus in a region very proximal to our cluster of EO activation for repeated spatial configurations in a contextual cueing task. Using a similar visual search task, Geisbrecht et al. (2013) observed modulation of activity of hippocampus, visual cortex, and parietal cortex without awareness for learned contextual information; a pattern consistent with the present findings. In instances of recognition failure, Ryan et al. (2007) proposed that similarity between previously encoded representations and current information exerts a rapid influence on eye movements.

Critically, the scene stimuli used in the present study are complex and configurally richer than simple stimuli typically used in contextual cueing paradigms (e.g., Chun and Jiang, 1999). The use of complex scenes is an ecological way of studying the properties of relational memory. As opposed to paradigms using sparse object, word, or line drawing stimuli, complex scenes can involve binding of many arbitrarily related details. In particular, structural alignment within scenes may be one form of complex relational information that necessitates binding in memory (e.g., Markman and Gentner, 1993). This type of relational binding is crucial to human episodic memory function, for which MTL function is integral (e.g. Nyberg et al. 1996; Cohen et al., 1997, 1999; Olsen et al., 2012; Monti et al., 2014). The present study is therefore, to our knowledge, the first to capitalize on eye movements as a measure of mapping complex learned scene representations based on configural similarity rather than on repeated/novel visual discrimination, and it is the first study to link this type of complex study-test scene configuration mapping to hippocampal activity that occurs even when explicit yes/no recognition behavior does not reflect this mapping. Notably, in addition to correlations between hippocampal activity and implicit relational/configural memory identified with fMRI, lesion evidence suggests that the hippocampus is necessary for relational and configural memory (Graham et al., 2006; Mundy et al., 2013; Watson et al., 2013), including implicit eye-movement expressions of relational memory (Ryan et al., 2000; Ryan et al., 2007).

Findings from studies of human brain-lesion patients are virtually unanimous in supporting the necessary role of hippocampus in explicit memory (e.g., Scoville and Milner, 1957). However, this evidence has promulgated the view that hippocampus and surrounding MTL cortex support only explicit memory whereas other cortical regions support implicit memory (e.g., Squire and Zola-Morgan, 1991; Squire et al., 2004; Henson, 2003; Reber, 2013). In contrast, other findings indicate that hippocampus and other MTL structures contribute to a variety of implicit memory phenomena including rapid associative learning, incidental learning, and retrieval of long-term memory without awareness (e.g., Schapiro et al., 2012; Wimmer and Shohamy, 2012; Duss et al., 2014; reviewed in Hannula and Greene, 2012; Olsen et al., 2012; Shohamy and Turk-Browne, 2013; Wang et al., 2014), which can be assessed using both direct and indirect memory test formats. The configural and relational qualities of hippocampal memory processing are likely needed in a variety of circumstances, determined by whether specific task demands require relational processing, including tests that are intended to measure implicit as well as explicit memory (Hannula and Greene, 2012).

Previous experiments have also identified evidence for implicit memory during direct memory testing for scenes, particularly by comparing misses to correct rejections (e.g., Blondin and Lepage, 2005; Yi and Chun, 2005; Sayres and Grill-Spector, 2006; Turk-Browne et al., 2006; Xu et al., 2007). We found relatively less parahippocampal activity for similarity misses compared to correct rejections of new scenes, despite no significant discrimination of these categories based on yes/no recognition judgments or familiarity ratings. This finding is consistent with previous reports of parahippocampal repetition reduction effects for scenes such as those above. Indeed, it is possible that this comparison of misses to correct rejections identified neural correlates of implicit memory (e.g., Rugg et al., 1998). However, to the extent that this finding reflected implicit memory, it was a distinct neural signal of implicit memory from that provided by the EO analysis. It is possible that relatively less parahippocampal activity reflected implicit memory for repeated viewing of specific scene elements. In contrast, increased hippocampal activity related to EO reflected implicit retrieval or use of configuration information related to eye-movement behavior. Although it is interesting that distinct neural signals of different implicit memory expressions may have been identified for the same task, future research will be needed to specify the type of implicit memory captured in parahippocampal activity reductions and the relationships between this type of implicit memory and activity related to EO.

Further research is needed to better specify the nature of scene configuration memory supported by hippocampus as well as to identify factors that cause this memory to be expressed implicitly versus explicitly. Our interpretation of hippocampal EO activity reflecting implicit scene configuration memory is compatible with recent evidence for global matching models of hippocampal function (e.g., Aly et al., 2013). That is, one way in which relational configural memory processing by hippocampus could support accurate old/new discrimination is by supporting computation of matching between configuration information present in a current stimulus to long-term memory for previously encountered configurations. Additional work will be needed to determine whether the hippocampal role is in storage or binding of the configurations, cued-retrieval of configurations, and/or in processing the match between current and stored representations. Furthermore, it is possible that the level of global match is a salient factor in memory awareness. In the current study, using similar scenes at test could have allowed global match to occur for configuration but not for item information, thus providing a lower level of match compared to when repetition of identical scenes produced global match for configuration plus item information. Therefore, the observed distinction between implicit memory for similar scenes (during scanning) and explicit memory for identical scenes (post-scanning testing) could have related to different levels of global matching engendered by these testing circumstances. Indeed, in a previous study using similar methods, Ryals et al. (2013) found hippocampal activity associated with accurate explicit memory for similar scenes when item-level global matching could have been encouraged by requiring verbal recall of scene labels at test.

Interestingly, the present findings are also consistent with the “global scanpath” hypothesis of attention and eye movements proposed by Groner and Groner (1989). As opposed to measuring patterns of fixation sequences that repeat from study to test reported by earlier eye-tracking researchers known as “local scanpaths”, (i.e., Buswell, 1935; Yarbus, 1967; Noton and Stark, 1971), global scanpaths reflect the overall distribution of eye movements for the entirety of a viewing event. Importantly, global scanpaths are believed to involve top-down attentional rather than memory processing (Groner and Groner, 1989) Similarly, our novel configural similarity manipulation is also highly consistent with theories of gist-based visual scene recognition involving top-down processing of holistic or global scene properties (i.e., Groner and Groner 1989; Oliva and Torralba, 2006; Torralba et al., 2006; Greene and Oliva, 2009). For instance, Torralba et al. (2006) proposed a dual pathway model where a bottom-up stream of visual processing guides attention based on individual object saliency, and this interacts with a complementary top-down stream based on holistic representations including scene “junctions” and “surfaces”. Our EO measurement may therefore be a novel way to examine global fixation patterns related to attention as well as memory involved in scene exploration.

It will be important to determine whether EO is sensitive to differing degrees of structural versus item-based information similarity in real-world visual scenes and determining how this relates to MTL and frontoparietal activity (e.g., Eckstein et al., 2006; Summerfield et al., 2006; Chun and Turk-Browne, 2008; Võ and Wolfe, 2012, 2013). An important and challenging future direction will also be to quantify and control for semantic content within scenes, which has an unknown relationship with configural properties (e.g., Wu et al., 2014). Future work could also test the roles of similarity versus novelty detection in similar complex scene viewing. Finally, future research will also be needed to tease apart the relationships among global match, configuration memory, and item memory in order to specify why hippocampus can sometimes contribute to implicit expressions of configuration memory (as in the current report) and sometimes to similar expressions of memory occurring with awareness.