Both memory and attention systems contribute to visual search for targets cued by implicitly learned context

Abstract

Environmental context learned without awareness can facilitate visual processing of goal-relevant information. According to one view, the benefit of implicitly learned context relies on the neural systems involved in spatial attention and hippocampus-mediated memory. While this view has received empirical support, it contradicts traditional models of hippocampal function. The purpose of the present work was to clarify the influence of spatial context on visual search performance and on brain structures involved memory and attention. Event-related functional magnetic resonance imaging revealed that activity in the hippocampus as well as in visual and parietal cortex was modulated by learned visual context even though participants’ subjective reports and performance on a post-experiment recognition task indicated no explicit knowledge of the learned context. Moreover, the magnitude of the initial selective hippocampus response predicted the magnitude of the behavioral benefit due to context observed at the end of the experiment. The results suggest that implicit contextual learning is mediated by attention and memory and that these systems interact to support search of our environment.

1. Introduction

Visual search for goal-relevant objects in our complex environment can be facilitated by multiple sources of information that can effectively reduce uncertainty and complexity (Gibson, 1969). Empirical demonstrations of the influence of environmental regularities on search have used natural scenes and objects, as well as the systematic manipulation of synthetic displays. For instance, pre-existing knowledge of environmental regularities in natural scenes, such as scene structure and object-object co-occurrence, can cue the location of behaviorally relevant objects resulting in more efficient search performance as measured by manual responses and eye movements (e.g., Brockmole & Henderson, 2006; Eckstein, Drescher, & Shimozaki, 2006; Hollingworth, 2009; Mack & Eckstein, 2011; Moores, Laiti, & Chelazzi, 2003; Neider & Zelinsky, 2006; Torralba, Oliva, Castelhano, & Henderson, 2006). Similarly, when observers search for target objects in synthetic, cluttered displays of distractors, behavioral performance is often improved and the number of saccades are reduced if the target-distractor spatial configuration is repeated relative to when it is novel (e.g., Chun & Jiang, 1998; Peterson & Kramer, 2001; Tseng & Chiang-Shan, 2004; Zhao et al., 2012). The improvement in search performance that occurs for repeated synthetic configurations is often referred to as the contextual cueing effect (e.g., Chun & Jiang, 1998). Importantly, the context provided by either the naturally occurring or manipulated regularities is thought to provide an associative cue that signals attention to move to the location of the target in much the same way that an arrow cues visual attention in the traditional Posner cueing paradigm (e.g., Chun & Jiang, 1999; Olivers, 2011; Summerfield, Lepsien, Gitelman, Mesulam, & Nobre, 2006).

While studies using natural scenes and objects take advantage of learned associations between target objects and the environment, studies using synthetic displays provide unique insight into the relationship between the mechanisms involved in learning the context and those that support visual search (e.g., visual attention). Three specific findings observed in experiments using synthetic displays are pertinent for the present study. First, studies of contextual cueing have provided neuropsychological and fMRI evidence that this form of learning relies on the hippocampus (Chun & Phelps, 1999; Greene, Gross, Elsinger, & Rao, 2007; Manelis & Reder, in press), a structure typically associated with declarative long-term memory (e.g., Moscovitch, Nadel, Winocur, Gilboa, & Rosenbaum, 2006; Squire, 1992). Second, electrophysiological studies have shown that visual cortical responses that are typically modulated by spatial attention are also enhanced for previously viewed configurations compared novel configurations (Johnson, Woodman, Braun, & Luck, 2007; Olson, Chun, & Allison, 2001). Similarly, fMRI studies have shown that the hemodynamic response in visual cortex peaks faster and is more robust for learned contexts, and that regions of prefrontal cortex show a larger response to learned context (Pollmann & Manginelli, 2009, 2010). Third, the behavioral benefit that arises from context can be observed even when self-reports and performance on post-experiment discrimination tasks indicate that observers do not explicitly recognize the repeated displays (e.g., Chun & Jiang, 1998). Thus, the contextual cueing effect is thought to capture the brain’s sensitivity to environmental regularities that are learned in the absence of awareness (Chun & Jiang, 1998). Together these findings are consistent with the theoretical proposal that the contextual cueing effect is mediated by both the neural systems that support the formation of long-term memories for spatial configurations and attentional orienting systems that support visual search (Chun, 2000; Chun & Jiang, 1999).

The notion that the contextual cueing effect is mediated by structures that are involved in both memory (i.e., hippocampus) and those involved in spatial attention (visual cortex and parietal cortex) is intuitive and is consistent with a number of studies in the literature. Nevertheless, there are aspects of this proposal that have been challenged. Perhaps the most controversial finding is that the hippocampus is involved in a form of implicit memory. Indeed, traditional models of hippocampal function suggest that the medial temporal lobe (MTL) should not be involved in implicit memory tasks (e.g., Moscovitch, et al., 2006; Squire et al., 1992). Consistent with this view, Manns and Squire (2001) reported that patients with focal lesions of the hippocampus can show the contextual cueing effect. More recently, neuroimaging studies have linked hippocampal responses with explicit memory processes rather than implicit contextual cueing by showing that a) when participants have knowledge of the repetitions, BOLD responses in the hippocampus are correlated with the size of the cueing effect (Westerberg, Miller, Reber, Cohen, & Paller, 2011) and b) hippocampus responses aren’t related to the contextual cueing effect, but are instead correlated with explicit recognition of the repeated displays (Preston & Gabrieli, 2008). A second challenge has been the proposal that the contextual cueing effect is mediated largely by response selection processes rather than reflecting the guidance of attention to the target location (Kunar, Flusberg, Horowitz, & Wolfe, 2007). As a result of these challenges, the precise neural mechanisms that mediate the contextual cueing effect and their relationship to behavioral performance remain unclear.

To provide clarity on the issue of whether both attention and memory are key mediators of implicit learning of spatial configurations, we used fMRI to investigate two critical issues. The first issue addresses the role of the MTL memory system. If areas involved in the implicit learning of spatial configurations are involved in the contextual cueing effect, then activity in these regions, particularly the hippocampus, should be modulated by spatial context (e.g. Chun & Phelps, 1999; Greene, et al., 2007). Moreover, the magnitude of this differential response should also be related to the magnitude of the behavioral benefit that arises from contextual learning. The second issue pertains to the role of spatial attention mechanisms in contextual cueing. Specifically, if visual attention is involved, then activity in regions previously implicated in the control of visual attention, including parietal cortex (e.g., Corbetta, Kincade, Ollinger, McAvoy, & Shulman, 2000; Giesbrecht, Woldorff, Song, & Mangun, 2003; Hopfinger, Buonocore, & Mangun, 2000; Kastner, De Weerd, Desimone, & Ungerleider, 1998; Yantis et al., 2002) and activity in visual cortex, where robust attention effects are often observed (e.g., Brefczynski & DeYoe, 1999; Hopfinger, et al., 2000; Kastner, Pinsk, De Weerd, Desimone, & Ungerleider, 1999; Yantis, et al., 2002) should be modulated by repeated spatial context and these modulations should change during the course of learning. Furthermore, if visual attention is involved, then activity in areas of visual cortex that represent the location of the search displays should be modulated by spatial context.

2. Materials and Methods

2.1 Participants

17 right-handed volunteers (mean age=22, 13 female) were paid $20/hour to participate in this study. The data from three participants were excluded due to persistent and severe head motion (>1 voxel). The remaining 14 participants were included in all analyses. All procedures conformed to a protocol approved by the University of California, Santa Barbara Institutional Review Board.

2.2 Stimuli

The stimulus displays consisted of a target (the letter T, 1.5°) presented amongst seven distractors (the letter L, 1.5°). The color of the target and distractors was determined randomly and could be red, green, blue, yellow, or magenta. The target letter was oriented either 90° clockwise or 90° counter-clockwise. The orientation of the distractors was randomly selected from the pool of cardinal orientations (0°, 90°, 180°, 270°). The search displays were lateralized to the left or right visual fields. Within each field, the display was divided into an imaginary 4 × 4 grid, where each square in the grid was 2.4° × 2.4°. The grid was positioned 1.2° from the vertical meridian and centered on the horizontal meridian (i.e., two rows of the grid were in the upper visual field and two rows were in the lower visual field). The locations of the targets and the distractors within the grid were randomly selected under the constraints described below. To prevent collinearity of the contours of adjacent items, the positioning of the search items within each position of the grid was jittered by +/− 0.32°.

A set of eight unique displays was created for each subject in which the location, orientation, and color of the distractors was fixed for a specific target location. These displays, referred to as ‘Old’ displays, were presented to the participant throughout the experiment. An additional set of eight displays was created for each experimental block. In these displays, referred to as ‘New’ displays, the location of the target was never the same as the location of the targets in any of the ‘Old’ displays.

2.3 Design and Procedure

Each trial began with a 250 ms presentation of a search array, followed by a 1750 ms interval within which the participants indicated whether the target was rotated to the left or right. After the response interval, there was a 1000 ms delay within which participants were presented with auditory feedback on their response accuracy (tone duration = 250 ms; high tone [980 Hz] = correct; low tone [680 Hz] = incorrect). An example of the trial sequence is shown in Figure 1a. The brief stimulus presentation time was intended to increase task difficulty and to minimize the likelihood of eye movements. An equal number of trials of the same duration (3000 ms), but in which no stimulus was presented, were randomly interleaved within the sequence of search trials. These ‘no-stimulus’ trials were included to facilitate the event-related analysis.

Figure 1. Experimental task and behavioral results.

Panel A: Sample trial sequence. Each trial began with a 250 ms presentation of a search display, followed by a 1750 ms response interval in which participants indicated whether the target letter T was rotated 90° to the left or 90° to the right. After the response, a feedback tone was presented for 250 ms, followed by a 750 ms inter-trial interval. Panel B: Mean response times (upper two lines) and error rates (lower two lines) as a function of display type and experimental epoch. In this and subsequent figures, error bars for each epoch represent one standard error of the mean difference between Old and New conditions, * denotes differences between conditions significant at p<0.05, one-tailed, and ** denotes differences between conditions significant at p<0.05, two-tailed.

Each session consisted of six experimental blocks of 64 search trials interleaved with 64 no-stimulus trials. To prevent response biases, each Old and New display was presented twice within a block, once with the target rotated 90° clockwise and once with the target rotated 90° counter-clockwise. Within each block, the order of all trial types (visual search and no-stimulus trials) was randomized. At the end of the experimental session, participants were debriefed and then completed a surprise recognition test in which each of the 16 Old displays were presented along with a unique set of 16 New displays. Participants were informed that half of the displays presented in the recognition task had also been presented during the fMRI visual search experiment and the task was to indicate whether they recognized the display as being presented during the main search task or whether the display was novel.

2.4 Imaging Methods

The imaging data were collected using a 3T Philips MRI scanner equipped with an 8-channel phased array head coil. Functional images were acquired using a T2*-weighted gradient-echo, echo-planar imaging sequence with a repetition time (TR) of 3.0 s, an echo time (TE) of 35 ms, and a flip angle (FA) of 90°. Each volume consisted of 53 contiguous slices, with a voxel resolution of 3 mm³ (field of view (FOV) = 24 × 24 cm, matrix = 80 × 80). Anatomical images were acquired using a T1-weighted, spoiled gradient recalled 3D sequence (TR=9.8 ms, TE=4.6 ms, FA = 8°, FOV = 24 × 24 cm, matrix = 256 × 256, yielding a voxel size of .98 × .98 × 1 mm). Image spatial processing was performed in five steps (SPM99, http://www.fil.ion.ucl.ac.uk, Friston et al., 1995). First, the functional images were corrected for differences in slice acquisition order using the first slice acquired during the TR as the reference slice. Second, head motion correction was performed using affine transformations to align the first volume acquired from each block to the very first functional volume acquired during the experimental session. Then the images within each block were aligned to the appropriate block-specific motion-corrected first volume. Third, each participants’ anatomical scan was coregistered with their functional images. Fourth, the anatomical scans were spatially normalized to a common stereotactic space using the MNI template and the resulting parameters were then used to spatially normalize the functional images. Finally, the normalized functional images were spatially smoothed with a 6-mm full-width at half-maximum (FWHM) isotropic Gaussian kernel.

2.5 Data Analysis

2.5.1 Behavior

To increase power, the behavioral performance measures (response time on correct trials and error rate) from the six experimental blocks of the visual search task were collapsed into three experimental epochs (Chun & Jiang, 1998) and analyzed by two-factor repeated-measures ANOVAs. Additionally, and following convention, the key statistical analyses to test for the presence of the contextual cueing effect involved a priori comparisons between the Old and New conditions in the first experimental epoch, where both displays were novel, and the last experimental epoch, where learning of the Old displays should be present (Chun & Jiang, 1998; Jiang & Chun, 2001; Olson & Chun, 2002). Performance in the post-experiment recognition task was evaluated by computing d′ and β using each participants’ proportion hits (i.e., responding ‘old’ to an old display) and false alarms (i.e., responding ‘old’ to a new display). Due to technical difficulties with the button box, one participant’s responses on the post-experiment recognition task were not recorded. During debriefing, however, this participant did not report noticing any repetitions during the experiment.

2.5.2 fMRI

The fMRI data were analyzed using custom routines written in MATLAB® (Natick, MA) in a manner that paralleled the behavioral data, using an implementation of the general linear model that does not assume a specific shape for the hemodynamic response (Ollinger, Corbetta, & Shulman, 2001; Ollinger, Shulman, & Corbetta, 2001). Specifically, for each trial type of interest, the response at six peri-stimulus time points was modeled by a separate parameter, resulting in a total analysis window length of 18 s. Twelve separate trial types were modeled, consisting of the factorial combination of Epoch (1, 2, or 3), Display Type (Old vs. New), and Visual Field (Left vs. Right), restricted to correct trials only. All contrasts compared the estimated response at the third time-point after stimulus presentation. To correct for multiple comparisons across the image volume, 1000 Monte Carlo simulations were performed (Slotnick, Moo, Segal, & Hart, 2003), assuming an individual voxel threshold of p<0.005. Based on these simulations, an extent of 9 contiguous voxels was selected to ensure that all contrasts were corrected for multiple comparisons at p<0.05. Additional analyses were performed using the activations revealed from the general linear model contrasts as functional regions of interest (ROIs). Unless mentioned otherwise, each ROI included all voxels within the activated cluster. In cases where the cluster extended across multiple structures, the functional ROI was partitioned guided by the anatomy. Within each ROI, the average event-related hemodynamic response evoked by the search displays and the no-stim trials was calculated for each participant and each condition. To assess the stimulus-evoked activity, the time courses for each voxel were converted to percent signal change relative to a baseline that included the averaged signal intensity at the onset of the search display and the immediately preceding time-point. Overlap in the hemodynamic responses to the search trials in this fast-rate design was corrected by subtracting the corresponding hemodynamic responses evoked on no-stimulus trials (Burock, Buckner, Woldorff, Rosen, & Dale, 1998; Grent-‘t Jong & Woldorff, 2007; Woldorff et al., 2004). Where appropriate, comparisons between the resulting HRFs were done with paired t-tests or repeated measures ANOVAs at the peak of the response (third TR after trial onset).

2.5.3 Correlations with behavior

In addition to the statistical contrasts of the imaging data, we provided an additional test of the role of MTL structures by correlating individual differences in the contextual cueing effect observed at the end of the experiment (i.e., Epoch 3) with individual differences in the selective response to Old and New displays observed at the beginning of the experiment (i.e., Epoch 1). To ensure that the contrast that defined the ROI was statistically independent of the fMRI data from Epoch 1 being used for the correlation (Vul, Harris, Winkielman, & Pashler, 2009), this analysis was based on all voxels that survived the New > Old contrast over the last two epochs (p<.05, corrected, see section 2.5.2). Because of the relatively small sample size (n=14), before performing the correlation, we performed an outlier analysis to ensure that the resulting correlations were not biased by extreme behavioral or BOLD measurements (>±3 s.d. away from the group mean). Behaviorally, there were no participants that were excluded based on our criterion. Similarly, there no participants excluded based on the difference in percent signal change between the Old and New conditions in the ROI included in the analysis (Fig. 2a). Only after this process were the correlations between behavior and BOLD response computed.

Figure 2. Evidence for modulation of hippocampus responses.

Panel A: Regions of the medial temporal lobe showing a significantly larger response to New displays than Old displays during the last two experimental Epochs (p<0.05, corrected, see Methods). In this and subsequent figures statistical maps are overlaid onto the spatially normalized anatomical image of a single participant. Panel B: Time course of the hemodynamic response at the local maximum (30 −18 −15, see Table 1). Panel C: Scatter plot between individual differences in the magnitude of the differential response to Old and New displays in the first experimental epoch averaged across all voxels shown in the independent statistical contrast shown in Panel A with the magnitude of the contextual cueing effect (ms) in the last experimental epoch.

3. Results

3.1 Behavior

3.1.1 Search Task

Mean response times and error rates are shown as a function of epoch and display type in Figure 1b. Overall, responses times were faster in the last epoch than in the first epoch (F(2, 26)=21.59, p<0.001). There was a marginal epoch x display type interaction, such that during the first epoch, there was no difference in response times for the Old and New display conditions, but during the last epoch participants were faster to discriminate the target in an Old display than in a New display (F(2, 26)=3.22, p<0.06). A planned two-factor repeated measures ANOVA including the first epoch, where no contextual cueing should be present, and the last epoch, where contextual cueing should facilitate performance revealed a significant epoch x display interaction (F(1, 13)=7.23, p<0.02, MSE=169.85). A post-hoc comparison appropriate for within-subjects factorial designs (e.g., Loftus & Masson, 1994) revealed that this interaction was driven by the difference between Old and New displays during the last epoch (t(13)=3.91, p<0.01). The overall mean proportion of errors was 0.09 and did not change as a function of any of the experimental factors (all p-values > 0.15).

3.1.2 Recognition Task

Prior to performing the post-experiment recognition task, participants were debriefed and specifically asked if they noticed that some displays were repeated throughout the course of the experiment. None of the participants reported noticing the repeated displays. These self-reports were validated by the results of the post-experiment recognition task. Specifically, overall mean d′ was 0.12 (SEM=0.11) and it did not change as a function of whether the displays were presented in the left or right visual field (t(12)=0.23, p>0.8). Critically, a one sample t-test revealed that overall mean d′ was not significantly different than zero (t(12)=1.08, p>0.29). Thus, the behavioral contextual cueing effect observed in the search task is unlikely to be due to explicit recognition of the displays, but rather implicit learning of the spatial configuration.

The analysis of the participants’ response bias revealed that the overall mean β was 0.95 (SEM=0.03), which was not significantly different from 1 (i.e., no bias, t(12)=−1.71, p>0.11). As with the d′ analysis, β was not different when the displays were presented in the right or left visual fields (t(12)=1.29, p>0.22).

3.2 fMRI

To investigate the involvement of memory and attention in contextual cueing, we performed three analyses designed to: 1) identify regions that showed an effect of context and to assess whether this effect was related to behavior; 2) identify regions in which the effect of context interacted with learning; and 3) identify regions of retinotopically organized visual cortex that differentially responded to New and Old displays.

3.2.1 Contextual effects and behavioral correlations

The first analysis was aimed at investigating two issues: 1) to determine whether activity in regions of the MTL, particularly the hippocampus, is modulated by spatial context and 2) to determine the extent to which MTL responses early in learning can predict subsequent learning.

To investigate whether regions of the MTL were modulated by spatial context, the responses to New and Old displays in the last two experimental epochs (i.e., later in learning, see Methods) were directly compared. This contrast revealed a single cluster of activity in the right hippocampus, shown in Figure 2a and Table 1. The event-related time-course at the local maximum is shown in Figure 2b and it clearly indicates a differential response at the third time point after the presentation of the search display.

Table 1.

Coordinates of the local maxima revealed by the whole-brain contrasts of Old vs. New displays, the Epoch x Display interaction, and the contrast of lateralized Old vs. New displays in visual areas that differentially responded to displays presented in the left and right visual fields.

Contrast	Region	X	Y	Z	T	Cluster Size
New > Old
	Hippocampus	30	−18	−15	4.57	29
Old > New
	FG/LingG	−15	−69	−15	5.31	13
	IPL	36	−66	39	4.93	12
	PreCun	24	−33	45	3.77	11
	SPL	33	−54	63	3.97	12
	IFG/PreCG	63	18	27	4.03	16
	IFG	−57	21	12	4.18	14
	SFG	−15	54	27	5.46	15
Epoch x Display
	SPL	18	−69	60	8.25	95
	aIPS	−24	−42	57	6.13	46
	IFG/PreCG	54	0	30	6.32	41
L Old > L New
	CalS	12	−90	3	3.46	19
	LingG/FG	9	−81	−15	3.06	13
R Old > R New
	CalS	−18	−90	3	3.31	6
	LingG/FG	−21	−90	−18	2.31	14
	FG	−42	−75	−9	2.79	11

Note: Abbreviations: CalS, calcarine sulcus; FG, fusiform gyrus; IPL, inferior parietal lobe; aIPS, anterior intraparietal sulcus; IFG, inferior frontal gyrus; PreCG, precentral gyrus; PreCun, precuneus; SPL, superior parietal lobe.

To investigate whether MTL responses early in learning can predict subsequent behavioral performance, individual differences in the differential hippocampal response were correlated with individual differences in the behavioral contextual cueing effect. More specifically, the magnitude of each individual’s differential fMRI response (New – Old) in the first Epoch was averaged across all voxels in the cluster that survived the statistical threshold and then correlated with the magnitude of each individual’s behavioral cueing effect observed in the last experimental Epoch. Note that the definition of the ROI (i.e., based on fMRI data from Epochs 2–3) was statistically independent of the fMRI response difference being entered into the correlation analysis (from Epoch 1). The results of this correlation analysis are shown in Figure 2c and revealed that those participants who exhibited larger differential activity in the hippocampus in the first experimental Epoch also showed the largest behavioral contextual cueing effect at the end of the experiment (r(12)=0.65, p<0.02).

An additional correlation analysis was performed to test the extent to which the differential hippocampal response to context was correlated with behavioral performance on the explicit recognition task. This correlational analysis was performed using the differential hippocampal responses from the first epoch (i.e., the same values used in the behavioral correlations outlined above) and it did not reveal a significant correlation with performance on the post-experiment recognition task (r(11)=−0.09, n.s.). The correlation between explicit recognition performance and the differential hippocampal response averaged across all experimental epochs was also not significant (r(11)=−0.13, n.s.).

The hippocampus was the only region that exhibited a larger response to New displays compared to Old displays, but several cortical regions exhibited more robust responses to Old displays than to New displays. The coordinates for these regions are listed in Table 1 and included regions of visual cortex, parietal cortex, and frontal cortex. The visual areas included fusiform and lingual gyri in the left hemisphere. Parietal areas included the right superior parietal lobe and bilateral inferior parietal cortex. Finally, frontal regions included the left superior frontal gyrus and the inferior frontal gyrus bilaterally.

3.2.2 Context x learning interaction

To identify the regions in which the differential response to the New and Old displays changed as a function of learning, a statistical contrast was performed that isolated the regions that were modulated by the interaction between Epoch and Display Type. Two areas that were sensitive to this interaction are shown in Figure 3a. These areas included the parietal cortex bilaterally, including the left anterior intraparietal sulcus and the right superior parietal lobe. An additional cluster of activity in right inferior/precentral gyrus, was revealed by this contrast, but is not shown in Figure 3a. The mean event-related time-course for the right parietal activation is shown in Figure 3b. The time course indicates that in the first experimental epoch these regions exhibited similar BOLD responses to Old and New displays, but by the last experimental epoch, these regions exhibited larger responses to the Old displays than to the New displays (p<0.05).

Figure 3. Evidence for modulation of parietal cortex responses.

Panel A: Regions of parietal cortex sensitive to the Epoch x Display interaction. Panel B: Time course of the hemodynamic response of the SPL local maximum shown in panel a. Abbreviations: SPL, superior parietal lobe; IPS, intraparietal sulcus, IPS; a, anterior; R, right; L, left.

3.2.3 Contextual modulation of visual cortex responses

To test whether the response of visual cortex was modulated by context, we took advantage of the lateralized presentation of the search displays to identify regions of interest that selectively responded to the lateralized displays by directly comparing responses evoked by displays presented in the left visual field, collapsed across display type, with activity evoked on trials evoked by displays presented in the right visual field (p<0.05, corrected; see Methods). Then, within these lateralized regions we contrasted the response to Old and New displays presented in the visual field represented by these regions. For example, the contrast between Old and New displays presented in the left visual field was performed in regions of right visual cortex that differentially responded to left displays more than right displays. To assess the extent to which these regions were modulated by learning, we extracted the event-related time courses evoked by contralateral search displays (e.g., in left visual areas we analyzed the responses to displays presented in the right visual field). The results of the direct comparison of Old and New displays within contralateral visual areas are shown in Figure 4a. Regions of the right calcarine sulcus, fusiform, and lingual gyri exhibited larger BOLD responses to Old displays presented in the left visual field than to New displays presented in the left visual field. The parallel contrast of Old and New displays presented to the right visual field revealed corresponding regions of the calcarine sulcus and lingual gyrus in the left hemisphere. The event-related time courses evoked by the Old and New displays during the first and last experimental epochs, shown in Figure 2b, indicate that during the first few exposures both displays evoked similar responses, but by the end of the experiment Old displays evoked larger BOLD responses than New displays (t(13)=2.72, p<0.02).

Figure 4. Evidence for modulation of visual cortex responses.

Panel A: Regions of visual cortex showing a larger responses to Old displays than to New displays shown in the right visual field (RVF Old > RVF New) and left visual field (LVF Old > LVF New). This contrast was restricted to regions of visual cortex that exhibited a significant differential response to the lateralized displays (p<0.05, corrected, see Method) and was thresholded at p<0.05, uncorrected. Panel B: Time course of the hemodynamic response to the preferred visual stimulus (i.e., right visual cortex response to left visual field stimuli and left visual cortex response to right visual field stimuli) collapsed across hemisphere. Abbreviations: CalS, calcarine sulcus; LingG, lingual gyrus; FG, fusiform gyrus; R, right; L, left.

4. Discussion

Visual context is a powerful cue for reducing the complexity of natural scenes that can facilitate the processing of information that is coupled with the context. Previous neuropsychological, electrophysiological, and neuroimaging studies have suggested that the benefits of context can arise when performing visual search of natural scenes and synthetic displays and that they are due to the combined engagement of structures involved in memory and attention (e.g., Chun & Jiang, 1998; Eckstein, et al., 2006; Hollingworth, 2009; Mack & Eckstein, 2011; Moores, et al., 2003; Neider & Zelinsky, 2006; Pollmann & Manginelli, 2009, 2010; Summerfield, et al., 2006; Torralba, et al., 2006; Westerberg, et al., 2011). Here we investigated the extent to which contributions by the neural systems that support memory and attention can be observed during the early stages of learning and when the presence of context is not explicitly known. The results indicated that the responses in portions of the hippocampus, parietal cortex, and areas of retinotopically organized visual cortex were systematically modulated by visual context, even though participants had no subjective or objective awareness of the context. Moreover, we observed that individual differences in the pattern of selective activation in the hippocampus during the earliest stages of learning were correlated with individual differences in the magnitude of the contextual cueing effect observed at the end of the experiment.

4.1 Hippocampal involvement during contextual cueing

The finding that the hippocampus exhibits a differential response to Old and New displays during implicit contextual cueing is consistent with previous patient and neuroimaging studies of the contextual cueing effect (Chun & Phelps, 1999; Greene, et al., 2007; Manelis & Reder, in press). In contrast, this finding is inconsistent with neuropsychological evidence indicating that patients with focal damage to the hippocampus exhibit a typical contextual cueing effect (Manns & Squire, 2001); neuroimaging evidence that BOLD responses in the hippocampus are associated with explicit recognition of the displays, but not the size of the contextual cueing effect (Preston & Gabrieli, 2008); and with traditional models of hippocampal function (e.g., Moscovitch, et al., 2006; Squire, et al., 1992). There are several methodological and theoretical factors that, when considered together, point to a potential reconciliation between these discrepant findings.

There are three methodological issues to consider. First, previous fMRI studies that have failed to find evidence supporting the role of the hippocampus in implicit contextual cueing (Preston & Gabrieli, 2008; Westerberg, et al., 2011) have typically used BOLD deconvolution approaches that assume a canonically shaped hemodynamic response function (HRF). These HRF functions are usually based on either a single gamma function that captures a peak response that is above baseline (used in AFNI, e.g., Westerberg, et al., 2011) or the combination of two gamma functions that capture both the peak and the post-stimulus undershoot (used in SPM, e.g., Preston & Gabrieli, 2008). Modeling with a specific HRF assumes that the portions of the hippocampus that are important in contextual cueing exhibit this specific response profile (e.g., peak above baseline and post-stimulus undershoot). However, other studies that have not assumed a specific HRF have observed that the hippocampus during the contextual cueing task does not conform to canonical shape (e.g., Greene, et al., 2007). Based on this evidence, we used an analytical approach that did not make a specific assumption about the shape of the hemodynamic response (Ollinger, Corbetta, et al., 2001; Ollinger, Shulman, et al., 2001) to ensure that we were sensitive to atypical profiles. The only assumption we made about the shape of the BOLD response was that the difference between our conditions should be observed at about 6 s after the presentation of the stimulus. Second, previous fMRI studies have shown that the hippocampus is more tightly coupled with explicit recognition of the repeated displays (Preston & Gabrieli, 2008) and may only be recruited when observers are informed that there are repeated contexts in the task (Westerberg, et al., 2011). However, unlike the present study, these previous studies did not have conditions in which there was implicit learning and, as a result, this evidence cannot exclude the possibility that the hippocampus is involved during the implicit learning of spatial context. One possible methodological difference that may have increased our chances of observing implicit learning could be the short display duration. Third, Manns and Squire (2001) reported that the focal hippocampal lesions patients that they tested who showed the contextual cueing effect also had >50% sparing of the hippocampus. In contrast, the patients tested by Chun & Phelps (1999) who did not show contextual cueing had near complete hippocampus lesions (along with surrounding cortex). The difference in lesion size suggests that even a small amount of hippocampal engagement may be sufficient to support contextual cueing in the absence of awareness -- a hypothesis that is also consistent with the correlation between individual differences in hippocampal response and contextual cueing observed in the present study.

At a more theoretical level, the discrepant findings can be reconciled by a reconsideration of the function of the hippocampus. Specifically, there is clear evidence that the hippocampus is involved in explicit memory (Squire, 1992; Squire, et al., 1992) and there is also clear evidence that it is involved in integrating information, including spatial configurations, which may done in the absence of awareness (for recent reviews, see Hannula & Greene, 2012; Henke, 2010). Thus, within the present context, one of the functions of the hippocampus may be to integrate existing representations with new representations. Importantly, the notion that the hippocampus is responsible for integrating contextual information is not predicated on explicit recognition of the information. More broadly, the present results imply that the hippocampus is not solely a declarative memory system, but rather it is critical for the formation of associations – whether explicit or implicit -- that are critical for both configural learning of the sort tested here and the formation of episodic memories (e.g., Cohen & Eichenbaum, 1993; Eichenbaum & Bunsey, 1995; Henke, 2010). We propose that this integrative function is the role of the hippocampus in the contextual cueing task and that the output of this process may serve to guide the control of spatial attention during search (see also Summerfield, et al., 2006; Westerberg, et al., 2011).

The proposal that the hippocampus integrates contextual representations not only explains the overall differential response to Old and New displays (Figure 2a), but it also offers an account of two other results reported here. First, and most importantly, the integrative function of the hippocampus may be critically dependent on the extent to which the hippocampus is engaged during the initial encoding of the repeated contexts (Chun & Turk-Browne, 2007; Turk-Browne, Yi, & Chun, 2006). A strong interpretation of this hypothesis predicts that the magnitude of the hippocampal response to context should be correlated with subsequent behavioral performance. Consistent with this notion, in the present work, we observed that the extent to which the hippocampus differentially responded to Old and New displays was predictive of the subsequent magnitude of the contextual cueing effect. Second, if the reconsideration of hippocampal function described above is correct, then it would be reasonable to assume that the extent to which specific regions of the hippocampus are engaged may be dependent on the nature of the context that is being learned. A strong prediction based on this line of reasoning and on patient and neuroimaging evidence showing that the right hippocampus may be specialized for spatial learning and memory (e.g., Bohbot, Iaria, & Petrides, 2004; Bohbot et al., 1998; Iglói, Doeller, Berthoz, Rondi-Reig, & Burgess, 2010) is that the right hippocampus should be engaged during visual search cued by spatial context. Consistent with this prediction, the right hippocampus showed differential responses to old and new search displays in the present study and in a study recently reported by another lab (Manelis & Reder, in press). While these findings are consistent with the notion that the hippocampus integrates information from the environment with internal representations, further work is needed to test the constraints on this hypothesis and its implications for the role of the hippocampus in visual search.

4.2 Parietal and visual responses during contextual cueing

If spatial attention does play a key role during search of implicitly learned spatial configurations, then one would expect the involvement of attentional control systems that guide the allocation of resources during search for the target. While this hypothesis has intuitive appeal, it has been challenged by recent behavioral work suggesting that the role of attentional systems involved in visual search is limited and is overshadowed by those involved in response selection (Kunar, et al., 2007). In contrast to these recent challenges, here we demonstrated that BOLD responses in regions of parietal cortex, commonly accepted to mediate the control of spatial attention (Corbetta, et al., 2000; Corbetta & Shulman, 2002; Hopfinger, et al., 2000; Yantis, et al., 2002), exhibited an interaction between display type and epoch, such that the selective response to the old displays only emerged late in the experiment. While parietal cortex has also been implicated in response selection in both humans and monkeys (e.g., Astafiev et al., 2003; Colby & Goldberg, 1999; Jiang & Kanwisher, 2003; Rushworth, Nixon, Renowden, Wade, & Passingham, 1997; Rushworth, Paus, & Sipila, 2001; Schumacher & D’Esposito, 2002), we did not observe significant BOLD responses in several key frontal areas that have also been implicated in response selection processes (Dux, Ivanoff, Asplund, & Marois, 2006; Jiang & Kanwisher, 2003; Marois, Larson, Chun, & Shima, 2006; Rowe, Toni, Josephs, Frackowiak, & Passingham, 2000). Based on the evidence reported here, the response selection explanation of contextual cueing cannot be completely ruled out. However, given the observation that the interaction between experimental epoch and context was restricted to posterior parietal cortex, it seems likely that the observed pattern of activation represents the engagement of systems that control attentional orienting.

In addition to the involvement of parietal cortex, we also observed that areas of visual cortex contralateral to the visual field in which the displays were presented exhibited larger BOLD responses to Old displays than to New displays. This finding converges with previous electrophysiogical studies of the contextual cueing phenomenon. For instance, previous intracranial recording studies have shown that both striate and extrastriate areas exhibit larger amplitude responses to displays previously seen by observers relative to novel displays (Olson, et al., 2001). Similarly, more recent work observed that the scalp-recorded N2pc ERP component evoked by Old displays was larger than for New displays within about 200 ms after the presentation of the search displays (Johnson, et al., 2007). Critically, the N2pc component, which has been localized to regions of ventral visual cortex, is generally considered to index the focusing of attention on potential target objects in cluttered arrays of distractors (Eimer, 1996; Hopf et al., 2000; Luck, Girelli, McDermott, & Ford, 1997; Luck & Hillyard, 1994; Woodman & Luck, 1999). In addition, this finding is also consistent with several recent fMRI studies showing that the peak latency of the response in visual cortex occurs earlier for Old displays than for New displays (Pollmann & Manginelli, 2010). Thus, when considered together, the present pattern of selective BOLD responses to Old displays in parietal and visual cortex and the previous electrophysiological and neuroimaging reports converge on the notion that attention plays a key role in the contextual cueing phenomenon.

It should be noted that, although the pattern of activity in parietal and visual cortex is consistent with a role for spatial attention mechanisms because we were unable to track eye position, it is difficult to assess the relative roles of covert and overt attention. First and foremost, our proposal that hippocampal representations may guide attention is not solely aimed at covert attention. Indeed, eye tracking studies have demonstrated that both synthetic spatial context and context in natural scenes play an important role in guiding overt attention (Brockmole & Henderson, 2006; Eckstein, et al., 2006; Hollingworth, 2009; Mack & Eckstein, 2011; Manelis & Reder, in press; Moores, et al., 2003; Neider & Zelinsky, 2006; Torralba, et al., 2006; Tseng & Chiang-Shan, 2004; Zhao, et al., 2012). Moreover, neuropsychological evidence suggests that the hippocampus may mediate memory-guided saccades, particularly their sequential ordering (Müri, Rivaud, Timsit, Corniu, & Pierrot-Deseilligny, 1994). Nevertheless, there are two factors that, in addition to the brief stimulus duration used here, suggest that eye movements alone do not explain our results. First, studies directly comparing covert and overt attention have shown that when eye movements are allowed, there are increases in BOLD responses in both parietal cortex and in the frontal eye fields (FEF, Corbetta, 1998; Corbetta & Shulman, 1999; Gitelman et al., 1999). We did observe modulations in the BOLD response in parietal cortex, but none of the contrasts reported here revealed significant modulations in the FEF BOLD response as would be expected if subjects were moving their eyes towards the lateralized displays. Second, if participants were foveating the visual displays then when comparing the responses to left and right displays, we should have observed a) weak lateralized activity in visual cortex and b) strong lateralized FEF responses, but we did not.

One final point about the patterns of responses in visual and parietal cortex must be addressed. Specifically, it is noteworthy that the effect of repeated context observed in visual and parietal cortex in the last experimental epoch was in the opposite direction of the effect observed in the hippocampus. In visual and parietal cortex the responses to Old displays were larger than the responses to New displays. In contrast, the hippocampal BOLD responses to Old displays were smaller than those evoked by the New displays. We believe that these opposing effects may be explained by the functional roles of the attention and memory systems in this task. Specifically, based on the proposal that the hippocampus is integrating information about contextual representations, we propose that the reduced response in the hippocampus represents a repetition-related reduction in the BOLD response to previously view contexts that may reflect a sharpening of the representation or a facilitation of processing previously viewed contexts relative to novel contexts (for a review of these alternatives, see Grill-Spector, Henson, & Martin, 2006). In contrast, given the role of parietal cortex in the control of spatial attention, it is likely that the increased response to Old displays relative to new displays likely reflects the transient signal responsible for shifting attention to the target location (e.g., Yantis, et al., 2002) or a top-down signal that biases visual cortical responses (e.g., Bressler, Tang, Sylvester, Shulman, & Corbetta, 2008; Kastner, et al., 1999; Szczepanski, Konen, & Kastner, 2010). This interpretation of the differential response in visual cortex is consistent with an explanation based on the notion of attentional enhancement of the target location (e.g., Johnson, et al., 2007; Olson, et al., 2001); however, it is also consistent with studies showing repetition related enhancements with impoverished visual displays (Turk-Browne, Yi, Leber, & Chun, 2007). Further research is required to more completely characterize these functional relationships between attention and memory systems in the contextual cueing task.

4.3 Relationship to other areas involved in contextual processing

A number of studies have implicated retrosplenial cortex (RSC), parahippocampal cortex (PHC), and medial prefrontal cortex (MPFC) as being the core nodes in a contextual processing network (Aminoff, Gronau, & Bar, 2007; Bar & Aminoff, 2003; Kverga et al., 2011). For example, objects presented with strong contextual associates evoke more robust activity and evoke more synchronized activity than objects presented with weak contextual associates (Kverga, et al., 2011). Similarly, these regions also exhibit robust activity when viewing natural scenes (Turk-Browne, et al., 2006). In the present work, we did not observe activity in the nodes of this contextual processing network. However, previous studies that have investigated the contextual processing network have typically done so using natural scenes and objects that draw on well-learned naturalistic associations (Kverga, et al., 2011) or well-learned arbitrary associations (e.g., Aminoff et al., 2007). In contrast, the spatial context in the present experiment was systematically constructed in a synthetic display and it was novel for every participant at the beginning of the experiment. Therefore, one possible interpretation of the lack of activation in the context network is that RSC, PHC, and MPFC are recruited at a later stage of experience with the context.

5. Conclusions

Taken together these results provide strong evidence for the that notion both visual and parietal systems involved in attention and hippocampal systems involved in implicit learning of spatial configurations mediate the contextual cueing phenomenon. When considered together with previous behavioral, patient, and neuroimaging studies of contextual cueing (Chun & Jiang, 1999, 2003; Greene, et al., 2007; Johnson, et al., 2007; Manelis & Reder, in press; Olson, et al., 2001; Pollmann & Manginelli, 2010), along with studies demonstrating both anatomical and functional links between parietal cortex and the medial temporal lobe (Kobayashi & Amaral, 2003; Summerfield, et al., 2006), the empirical evidence converges on the notion that attention and memory systems can operate together to support coherent behavior in a complex environment.

Highlights.

• The hippocampus exhibited differential fMRI responses to spatial contexts during visual search.

• Hippocampus activity correlated with visual search performance, not recognition.

• Parietal cortex activity was modulated by the interaction between context and learning.

• Activity in retinotopically organized visual cortex was modulated by context.