Biasing Perception by Spatial Long-Term Memory

Abstract

Human perception is highly flexible and adaptive. Selective processing is tuned dynamically according to current task goals and expectations to optimize behavior. Arguably, the major source of our expectations about events yet to unfold is our past experience; however, the ability of long-term memories to bias early perceptual analysis has remained untested. We used a noninvasive method with high temporal resolution to record neural activity while human participants detected visual targets that appeared at remembered versus novel locations within naturalistic visual scenes. Upon viewing a familiar scene, spatial memories changed oscillatory brain activity in anticipation of the target location. Memory also enhanced neural activity during early stages of visual analysis of the target and improved behavioral performance. Both measures correlated with subsequent target-detection performance. We therefore demonstrated that memory can directly enhance perceptual functions in the human brain.

Introduction

The link between long-term memory (LTM) and perception is an intuitive one and has been acknowledged for centuries (e.g., Ibn al-Haytham, 1021). According to thinkers like Helmholtz, perception arises from an interaction between incoming stimulation and predictions based on knowledge acquired through experience (von Helmholtz, 1867). However, to date, we still know surprisingly little about the neural mechanisms through which memories come to bias perception in a predictive fashion.

Research in “selective attention” has convincingly demonstrated that high-level representations can exert significant top-down effects on the processing of incoming information. Perception is tuned for events relevant to our current task goal. Neuronal activity is biased toward relevant and predicted events in an anticipatory fashion, and their perception is prioritized from early stages of processing (Desimone and Duncan, 1995; Reynolds and Chelazzi, 2004). Surprisingly, the contribution of memories to these top-down biases has remained relatively ignored.

The role of memory in shaping perception has been more carefully considered in behavioral and eye movement studies using complex scenes. These have indicated that contextual, semantic, and global statistical properties of complex scenes influence scene exploration and object recognition (Mackworth and Morandi, 1967; Yarbus, 1967; Biederman, 1972; Moores et al., 2003; Bar, 2004; Delorme et al., 2004; Malcolm and Henderson, 2010). Theoretical and computational models have been proposed to explain how memory-related variables influence attention and object recognition in complex scenes (Torralba et al., 2006; Peters and Itti, 2007; Zelinsky, 2008). Although these studies provide rich and convincing evidence for the role of memory in perception, the tasks used are not optimal for separating neural signals related to top-down memory biases from those related to perceptual analysis of objects or scenes.

The most direct evidence suggesting that LTM can guide attentional selection during perception comes from the “contextual cueing effect” in visual search tasks (Chun and Jiang, 1998, 2003; Brockmole and Henderson, 2006; Brockmole et al., 2006, 2008). Targets are identified more effectively when they appear within repeated spatial configurations of distractors compared with novel contexts. Electrophysiological recordings during contextual cueing experiments have revealed modulation of target-selection mechanisms (Johnson et al., 2007; Chaumon et al., 2008, 2009; Schankin and Schubö, 2009, 2010; Telling et al., 2010), although findings have not been entirely consistent and studies have not always controlled adequately for eye movements. Furthermore, because the target appears embedded within distractors, it is not possible to test for perceptual modulations unambiguously. Performance improvements could occur because LTM changed the perception of the target within its familiar context, facilitated retrieval of the configuration of distractors, or facilitated decision making or action selection (Kunar et al., 2007; Schankin and Schubö, 2009).

In this study, we used a task that separated presentation of targets from presentation of backgrounds driving spatial contextual LTMs (Summerfield et al., 2006) and measured neural activity during perceptual analysis with high temporal resolution using event-related potentials (ERPs). We tested whether viewing a previously learned complex scene facilitates perception of a task-relevant target by triggering a shift of spatial attention toward its remembered location.

Materials and Methods

The task borrows design elements from contextual cueing in visual search (Chun and Jiang, 1998, 2003) and cued visuospatial orienting tasks (Posner, 1980). In our “memory-orienting” task, participants completed a learning task, in which they explored pictures of naturalistic visual scenes to identify and memorize the location of a predefined item (a small key). On the following day, they completed a LTM-based covert-orienting task, in which they detected the transient appearance of target stimuli within the scenes (Fig. 1c). Some scenes were associated with specific spatial memories for the target location from the learning task (valid trials). Other scenes were equally familiar from the training phase but contained no specific spatial memories (neutral trials). If memory can drive attention and influence perception, then viewing scenes associated with spatial memories should bias attention-related oscillatory brain activity in anticipation of the target stimulus (Worden et al., 2000; Rihs et al., 2007). In addition, targets subsequently appearing at remembered locations should elicit larger electrophysiological potentials that reflect early stages of visual analysis, such as the P1.

Figure 1.

a, Examples of eye-tracking data collected from the learning phase of one representative participant. The scene on the left contained a key, and the scene on the right contained no key. For both scenes, there were extensive eye movements for the first block (row 1). In the final block (row 2), there was an almost direct eye movement to the location of the key on the left scene (see inset). In contrast, there were still widespread eye movements for the scene on the right in which no key had been found. b, Graph shows improved learning during the five blocks of the learning task. Plotted are the mean percentage of keys found in each training block over participants (solid line), as well as the mean RTs for detecting a key within each scene (dashed line). Error bars indicate SEM. c, Schematic of the memory-orienting task, with examples of a valid trial (left column) and neutral trial (right column). Scenes without keys appeared as the “cue.” In valid trials, participants had spatial memory for the location of the key in the learning task (illustrated by the left thought bubble depicting the key in its remembered spatial context), but in neutral trials, they had no specific spatial memory (illustrated by the empty bubble with question mark). After a randomized SOA (500–900 ms), a target key or a foil banana appeared (100 ms) and participants had up to 1000 ms to respond. d, Horizontal electro-oculogram waveform from valid cue trials (left and right cues) and neutral cue trials (for left and right targets) during the orienting task averaged across the 24 participants. e, Eye-tracking data from the orienting task for one representative participant.

Participants.

Twenty-four healthy, right-handed volunteers (mean handedness score: 88%; Oldfield, 1971) participated in the experiment (average age 21.3 years, range 18–27 years; 12 females). The methods and procedures used in the study were noninvasive and had ethical approval from the University of Oxford Central University Research Ethics Committee.

Scene stimuli.

Stimuli used in this experiment were 180 photographic scenes (1000 × 750 pixels, 32-bit color) depicting indoor or outdoor views. Scene stimuli were prepared by the experimenters or obtained, with permission, from photographic images available on the Internet. Three versions of each scene were prepared. In one version, the scene was saved in its original format. In the two remaining versions, a small gold key (12 × 23 pixels) was placed in either the left or the right side of the scene (using PaintShopPro5, Jasc software). The distribution of keys placed in the top and bottom quadrants was balanced (90 keys placed in each quadrant of each visual field). Keys were placed within object arrangements and were usually in plausible places. The assignment of scenes to different experimental conditions (valid or neutral) was counterbalanced across participants. Furthermore, the visual field location of the key for a particular scene was also counterbalanced. The counterbalancing ensured that any effect of validity could not result from differences inherent to specific scenes or to the location of keys within the scenes.

Learning task.

The first part of the experiment was the learning task. Participants viewed 180 scenes repeated in random order over five blocks (Presentation software, Neurobehavioral Systems). Ninety scenes contained a small gold key located somewhere in the left or right side of the scene (22 or 23 in each quadrant). The remaining 90 scenes had no key present. Participants were not aware of which scenes contained a key at first. They explored the scenes overtly and were instructed to find and learn the location of as many keys as possible within the scenes. When they identified a key, they made an initial left-sided mouse click, which revealed a mouse cursor in the center of the screen (small white square, 19 × 19 pixels). They then moved the cursor to the location of the key and made a second left-sided mouse click on the key. Participants were given visual feedback as to whether they had correctly identified the location of the key (“Key Found” for correct responses or “Key Not Found” for incorrect responses). There was a 30 pixel radius of tolerance around the location of the key to allow for inaccuracies in cursor responses. If participants believed that there was no key present in the picture, they could make a right-sided mouse click to move to the next scene. Each scene was viewed until the participants made a response or until a maximum search time was reached. The search time allowed for each scene was reduced over the five blocks to encourage participants to locate the keys more rapidly over each block. The search time was 20 s for the first block, 15 s for the second and third blocks, and 10 s for the final two blocks. Participants received visual feedback at the end of each block regarding the number (N) of keys found (“On this block you found N keys”). Eye movements were recorded using an infrared monitoring system (ISCAN) (Gitelman, 2002) and were inspected online to ensure that the participants were actively searching through the scenes.

The mean percentage of keys found and the mean search time taken to locate the keys were calculated for each of the five learning blocks. To test for the progressive learning of the key locations, we analyzed these percentage and search time measures by a linear contrast over the five blocks using repeated-measures ANOVAs.

Orienting task.

One day after the learning task, participants returned to the laboratory to perform an attentional memory-orienting task, during which the electroencephalogram (EEG) was recorded. The task was performed covertly. Bipolar electro-oculogram (EOG) recordings (Fig. 1d) and eye tracking (Fig. 1e) were used to ensure that participants maintained fixation on a central cross throughout each trial. Participants completed 180 trials in which they had to detect the brief appearance of a bright gold key (80 × 154 pixels) within a complex scene (160 trials) and withhold responses during a minority of catch trials in which a banana (52 × 153 pixels) was flashed instead of a key (20 catch trials). No search was necessary during the memory-orienting task. Instead, participants made a simple speeded detection response upon the appearance of the transient peripheral target stimulus. Participants performed a short practice session with 20 trials using a novel set of scenes before the experiment.

During the main experiment, each trial began with the presentation of a familiar scene. After a random interval of 500–900 ms, either a target (key) or foil (banana) appeared briefly within the scene (100 ms). On valid trials, the location of the key in the learning task predicted with 100% accuracy the location at which the key (80 valid target trials) or banana (10 valid catch trials) would appear. For scenes with no key present in the learning task, participants had no information as to where the key (80 neutral target trials) or banana (10 neutral catch trials) would appear. After the disappearance of the imperative stimulus (key or banana), the scene remained displayed for a further 1000 ms, during which time responses were recorded. This scene was followed by a blank screen for an interval of 2000 ms, during which participants could blink. The fixation cross then returned to the center of the screen for a randomized interval of 500–1500 ms, indicating the start of the next trial.

Behavioral analyses.

The behavioral validity benefits conferred by memory-guided orienting were analyzed using mean response times (RTs) to detect the appearance of the key. Trials corresponding to scenes where the participant had failed to locate the key by the final block of the learning task were excluded from analysis. Only correctly performed target trials were used for the RT analysis.

Further analyses tested for the relationship between performance in the learning task and RT benefits in the orienting task. For each subject and for each key-present scene, we calculated the average search time during the last three blocks of the learning task and the validity effect during the attention-orienting task. The average search time in the learning task provided a rough estimate of the participant's explicit knowledge of the location of the key. The validity effect was computed as the difference between the RT on a given valid trial and the mean RT over neutral trials, divided by the standard deviation of the RTs in neutral trials (z score).

The relationship between knowledge of the key location in the learning task and validity benefits in the memory-orienting task was analyzed by testing whether validity benefits increased linearly as the time to locate the key during the final blocks of the learning task decreased. To simplify the analysis, the trials were separated into five bins (quintiles) according to the search times during the learning task for each participant. The validity scores for each participant were averaged over each quintile. The effect of learning performance on the subsequent attentional validity effects was then probed using a repeated-measures ANOVA testing for linear increases in validity following improved performance (decreasing search times) during the learning task.

Spatial memory recall test.

Explicit memory for the location of the keys within complex scenes was tested by a recall test immediately after the memory-orienting task. Participants were shown a random selection of 30 scenes taken from the learning task in which a key had been present; they were asked to indicate the location where they thought the key had been by positioning and clicking a mouse cursor (as in the learning task). The distance between the correct coordinate of the key location and the recalled location was computed, in number of pixels. Only scenes for which the participants had correctly located the key in the learning task were analyzed. This test provided an approximate measure of explicit long-term memory recall, but it might also have been influenced by viewing the location of the target keys during the orienting task.

EEG recording.

The EEG was recorded continuously with NuAmp amplifiers (Neuroscan) from 40 scalp sites using Ag/AgCl electrodes mounted on an elastic cap (Easy Cap, Falk Minow Services), positioned according to the 10–20 International system (American Electroencephalographic Society, 1991). The montage included six midline sites (FZ, FCZ, CZ, CPZ, PZ, and OZ) and 14 sites over each hemisphere (FP1/FP2, F7/F8, F3/F4, FT7/FT8, FC3/FC4, T7/T8, C3/C4, TP7/TP8, CP3/CP4, P7/P8, P3/P4, PO7/PO8, PO3/PO4, and O1/O2). Additional electrodes were used as ground and reference sites and for recording the EOG. The right mastoid was used as the active reference. Data were then rereferenced offline to the algebraic average of the right and left mastoids. The horizontal and vertical EOGs were recorded bipolarly with electrodes placed around the eyes. The signal was digitized at a sampling rate of 1000 Hz. Data were recorded with a low-pass filter of 200 Hz and with no high-pass filter (DC). Digital codes were sent to the EEG recording computer to mark the presentation of the cue and target stimuli in each trial type.

EEG time-frequency analysis.

Preparatory spatial attention has been shown to induce changes in oscillatory activity within visual areas. A complementary pattern of effects is typically observed across the lower frequency alpha band (8–12 Hz) and the higher frequency gamma band (>30 Hz), with a relative diminution, or desynchronization of alpha and a relative enhancement of gamma over neuronal populations coding the relevant spatial location (Fries et al., 2001). With EEG, it is problematic to measure gamma activity without appropriate controls (Dimigen et al., 2009), but the buildup of a relative of alpha desynchronization at the hemisphere contralateral to the attended location has been well documented (Worden et al., 2000; Rihs et al., 2007). We tested whether the spatial memory for the target location upon the presentation of a valid memory cue would induce similar changes in the level of alpha-band activity in anticipation of the target appearance.

The EEG data used for the induced time-frequency analysis were not filtered, and epochs were prepared starting 750 ms before and ending 750 ms after cue presentation. Epochs containing excessive noise or drift (±100 μV) at any electrode were rejected. Epochs with eye-movement artifacts (blinks or saccades) were also rejected. Blinks and large saccades were identified as large deflections (±50 μV) in the horizontal or vertical EOG electrodes. In addition, all trials were visually inspected for smaller saccades, blinks, and drifts and discarded if necessary. Error and catch trials were also excluded from the analysis. The subsequent analyses were performed using the FieldTrip toolbox software developed at the Donders Institute for Brain, Cognition and Behavior (http://www.ru.nl/fcdonders/fieldtrip) using MATLAB 7 (MathWorks). The evolving power at frequencies between 2 and 30 Hz were estimated at 10 ms intervals using multitaper Hanning wavelets, with a 500 ms temporal smoothing window. Data were transformed into the log-transform of power. Changes in induced alpha (8–12 Hz) were estimated relative to a baseline period comprising 200 ms before cue presentation (−200 to 0 ms). We tested for lateralized differences in the log-power of alpha toward the end of the cue–target interval (400–500 ms) for scenes associated with learned key locations on the left versus right hemifields. Only valid scenes were used in the analysis. The log-power of alpha during this period was compared using a repeated-measures ANOVA with the factor of memory (left, right), hemisphere (contralateral, ipsilateral), and electrode site (O1/2, PO3/4, PO7/8). Because of the relatively short interval between the cues and targets [600–1000 ms stimulus onset asynchrony (SOA)], it was only possible to examine the effects on alpha over a narrow time window. The time period for analysis (400–500 ms) ended 100 ms before the earliest possible presentation of the target, which minimized any contribution of neural activity evoked or induced by target processing on estimates of the alpha-band activity. The window started late to avoid contamination by cue-related ERPs and because previous visual-orienting tasks have typically shown alpha modulations to start ∼300–500 ms after an orienting cue (e.g., Worden et al., 2000; Rihs et al., 2007).

Preprocessing for ERP analysis.

EEG data were filtered offline with a 0.1–40 Hz bandpass filter, epoched to the presentation of the scene cue and the target stimuli. Epochs started 100 ms before and ended 600 ms after stimulus presentation. The time period 50 ms before and after cue or target presentation was used as a baseline. Trials with drifts, ocular artifacts, or errors were removed in the same way as for the time-frequency analysis. The criterion for the minimum number of trials per condition was set at 30. The Greenhouse–Geisser correction factor was used to adjust for possible violations of the sphericity assumption in all ERP analyses (Jennings and Wood, 1976).

To help guide analysis of identifiable potentials, topographical segmentation of the target-related and cue-related ERPs was performed using the Cartool software developed by Denis Brunet (http://brainmapping.unige.ch/Cartool.htm). Topographical analysis was used to test whether memory-based orienting changed the timing or nature of neural processing of spatially predicted versus unpredicted targets, as well as to identify the stable periods of neural functional states (Michel et al., 1992; Pascual-Marqui et al., 1995). The segmentation procedure used a modified hierarchical agglomerative clustering algorithm and was further constrained to identify only clusters lasting at least 20 time points (40 ms) with <97% correlation.

Cue-related ERP analysis.

Waveforms elicited by the scene stimuli serving as valid and neutral memory cues were also analyzed to identify brain activity involved during the control of spatial shifts of attention based on long-term memory. ERPs were constructed separately for cue scenes carrying spatial memories for the target key location on the left and right hemifields and for neutral scenes followed by target appearance on the left and on the right. Electrode sites were recoded as positions on the hemisphere contralateral or ipsilateral to the subsequent target key location. In the case of valid memory cues, this was always equal to the location of the spatial memory for the key location. Initial analyses showed no effects of cue side; therefore, ERPs to cues followed by left and right targets were combined, preserving the relationship between the side of target and the side of electrode position (ipsilateral and contralateral).

We were particularly interested in identifying any spatially specific anticipatory biases that could influence visual excitability upon the presentation of the target or lateralized potentials indicating selection of targets within long-term memory. Based on findings of lateralized potentials signaling identification of target items when searching visual short-term memory (Gratton, 1998; Kuo et al., 2009; Dell'Acqua et al., 2010; Eimer and Kiss, 2010), we asked whether there was any evidence of lateralized potentials signaling the selection of targets when searching long-term memory over lateral posterior electrodes PO7/8 and O1/2. Because the latency of this posterior contralateral negativity (PCN) effect can vary substantially (Töllner et al., 2011), we were not prescriptive about the likely latency of this effect in the context of long-term memory.

We also investigated whether there were any sustained spatial biases toward the end of the cue period, indicative of differential visual excitability, which occur during spatial shifts of attention following visual cues (Harter et al., 1989). We tested for changes in mean voltage over lateral posterior electrodes (PO7/8, O1/2) toward the end of the shortest possible cue–target interval (400–600 ms).

Target-related ERP analysis.

ERPs evoked by valid and neutral targets appearing in the left and right visual field were constructed separately. As initial analyses showed no effects of cue side, waveforms from targets in the left and right visual fields were subsequently combined, preserving the relationship between the side of stimulus and the side of electrode position (ipsilateral and contralateral).

According to the main aim of the experiment, early visual potentials P1 and N1 were compared across valid and neutral trials. The analysis used lateral posterior electrodes where these potentials were most pronounced: O1/2, PO3/4, PO7/8, and P7/8. Mean amplitudes were estimated within a narrow temporal window set around the peak latency of each visual potential in the grand-averaged data. The P1 was analyzed between 110 and 130 ms and N1 was analyzed between 170 and 190 ms. To measure the peak latency of the P1 and N1 visual potentials, the same electrodes were used, but the temporal window was widened to accommodate the interindividual variability in latency times. P1 peak latency was analyzed between 80 and 150 ms and N1 peak latency was measured between 140 and 220 ms. Differences in the mean amplitude or peak latency of the P1 and N1 were analyzed by separate repeated-measures ANOVAs testing the factors of cue (valid, neutral), hemisphere (ipsilateral, contralateral), and electrode location (O1/2, PO3/4, PO7/8, P7/8).

Effects of memory-based orienting on identifiable potentials after P1 and N1 during periods of stable topographies are shown in supplemental Figure 2 (available at www.jneurosci.org as supplemental material).

Results

Learning task

In the learning task, participants searched for the location of a small key embedded within scenes (Fig. 1a). They searched through 180 scenes, 90 of which contained keys, over five training blocks. Their performance improved steadily and was highly accurate by the last block (linear contrast over blocks: F_(1,23) = 111.2, p < 0.001; accuracy in last block 82.6%; range: 65.5-97.7%) (Fig. 1b). Experimental trials in which participants failed to locate the key (range 2–31) were subsequently removed from the behavioral and ERP analyses of the orienting task. From the first to the last training block, mean search times to locate the key decreased from 7.4 s to 1.7 s (linear contrast: F_(1,23) = 708.8, p < 0.001).

Behavioral performance of the memory-orienting task

On the following day, participants returned to perform a memory-orienting task. A target (key) or foil (banana) appeared at a learned or unlearned location within the familiar scenes. The task was performed at a high level of accuracy (98.9% correct). Participants were also successful at withholding responses during the catch trials (75.2% correct), despite their very rare incidence. The ANOVA revealed a main effect of cue (F_(1,23) = 44.35, p < 0.001), showing RTs to be faster for detecting keys at previously learned locations and valid trials (RT = 343 ms, SD = 73.9) than at unlearned locations and neutral trials (RT = 392 ms, SD = 85.3).

Greater validity effects in the orienting task were associated with shorter search times, indexing better memory, during the final blocks of the learning task. A repeated-measures ANOVA testing for linear increases in normalized validity effects in the memory-orienting task following improved performance in the learning task (separated into quintiles) showed a significant linear relationship between the two factors (F_(1,23) = 14.45, p = 0.001).

Spatial-memory recall test

After the orienting task, participants performed a spatial-memory recall test. They were asked to indicate explicitly their memory of the key location based on the learning task by placing the mouse cursor within the scene. Participants were able to indicate the location of keys to within 10% of the screen distance on an average 84% of trials.

Anticipatory spatial biases triggered by long-term memory

To identify spatial biases in brain activity in anticipation of the target stimuli that were triggered by LTM, we analyzed neural activity induced (time-frequency analysis) and evoked (ERPs) by the scene cues.

Time-frequency analysis of alpha-band activity (Fig. 2) provided a clear marker of a memory-triggered spatial bias in visual excitability. There was a significant lateralization in the power of alpha-band neural oscillations in anticipation of the target (hemisphere: F_(1,23) = 6.46, p = 0.02). Alpha-band power was significantly reduced over electrode locations contralateral to the remembered target key location relative to ipsilateral electrodes. There was no difference in the log of alpha power induced by left versus right valid cues overall (F_(1,23) = 0.18, p = 0.67).

Figure 2.

Lateralized alpha desynchronization for scene cues. As there was no effect of the factor of memory, data were combined for scenes cueing target locations on the left and on the right by plotting the EEG at relative contralateral and ipsilateral locations. a, Topographical distribution of log-power of alpha-band activity (8–12 Hz, 400–500 ms) showing greater desynchronization over contralateral than ipsilateral sites. b, The direct subtraction of ipsilateral from contralateral alpha-band activity viewed from a lateral perspective. This subtraction leads to symmetrical effects of opposite polarity over the scalp.

Analysis of ERPs (Fig. 3) showed that scenes with and without long-term memory associations for target locations were processed equivalently through the early visual stages. P1 and N1 potentials elicited by valid and neutral cues showed no effects of cue type (p values > 0.38). However, significant differences between valid and neutral cue scenes were observed during later processing stages. Spatial biases differentiated valid versus neutral scenes over lateral posterior electrodes between 340 and 390 ms (cue × hemisphere: F_(1,23) = 5.3, p = 0.03) (Fig. 3). Subsidiary ANOVAs showed that ERPs elicited by valid memory cues were more negative than those elicited by neutral cues over the contralateral (cue: F_(1,23) = 5.1, p = 0.03) but not ipsilateral (cue: F_(1,23) = 0.63, p = 0.44) hemisphere. This effect is reminiscent of the N2PC effect, also known as the PCN effect, observed during identification of target stimuli when searching visual (Luck and Hillyard, 1994; Eimer, 1996) or visual short-term memory (Kuo et al., 2009) arrays, but occurs later in time (Töllner et al., 2011).

Figure 3.

ERPs for scene cues: grand-averaged ERP waveforms (n = 24) over posterior electrodes (PO7/8) contralateral and ipsilateral to the remembered target location for valid (solid black line) and neutral (dotted black line) trials. The polarity of the waveforms is plotted with positive values upward. Arrow 1 (right panel) indicates the difference in waveforms between valid and neutral scenes between 340 and 390 ms. Arrow 2 (left panel) indicates the difference in waveforms between valid and neutral scenes between 400 and 600 ms.

At the end of the cue–target interval (400–600 ms), there was a trend for a differential spatial bias across cue types over lateral posterior electrodes (cue × hemisphere: F_(1,23) = 4.01, p = 0.06). However, subsidiary analyses did not show any reliable statistical trends of effects when comparing ERPs elicited by valid and neutral cues over contralateral and ipsilateral hemispheres separately (p values > 0.13).

In addition, two nonlateralized ERP effects differentiated valid memory cues from neutral cues (supplemental Fig. 1, available at www.jneurosci.org as supplemental material). These effects may indicate spatially nonspecific retrieval of the target association.

Perceptual modulation of target stimuli

Despite the relatively small size of the target stimulus appearing shortly after the presentation of a complex visual scene, early visual potentials P1 and N1 were clearly visible in the waveforms (Fig. 4) and showed characteristic latencies and distributions. The P1 potential peaked at ∼124 ms at contralateral electrodes (maximal at PO7/8) and peaked later (136 ms) over ipsilateral electrodes. The mean amplitude of the P1 was significantly larger for targets in valid trials compared with neutral trials (F_(1,23) = 5.30 p = 0.031). The latency of the P1 was unaffected (F_(1,23) = 0.30 p = 0.59). Topographical segmentation of the difference waveform isolated the memory-based orienting effect and showed a stable topography between 85 and 149 ms, reflecting the focal enhancement of the P1 amplitude over the lateral posterior electrodes (Fig. 4).

Figure 4.

Modulation of visual potentials in target-related ERPs: early visual potentials P1 and N1 from 0 to 300 ms from grand-averaged waveforms (n = 24) for the lateral posterior electrodes in the contralateral (c) and ipsilateral (i) hemispheres (P7/8, PO7/8, O1/2, PO3/4) for valid (solid black line) and neutral (dotted black line) trials. The polarity of the waveforms is plotted with positive values upward. Normalized topographic segmentation of the difference waveform showed a stable topography between 85 and 149 ms for the P1, reflecting the enhancement of the P1 amplitude over the lateral posterior electrodes, and between 150 and 190 ms for the N1, reflecting N1 attenuation (more positive amplitude) for valid trials compared with neutral trials at posterior contralateral electrodes, but a more negative amplitude for valid trials compared with neutral trials at ipsilateral electrodes. In addition, during the N1 period, positive voltage was also distributed over central midline sites, reflecting an earlier rise of the centrally distributed P2 potential in the valid condition (see supplemental Fig. 2, available at www.jneurosci.org as supplemental material). Legend: ±0.4 μV.

The N1 potential peaked at ∼163 ms at contralateral electrodes (maximal at P7/8). It was smaller and peaked later (167 ms) over ipsilateral electrodes (Fig. 4). Inspection of the N1 potential revealed that amplitude differences between valid and neutral trials differed over ipsilateral and contralateral hemispheres (cue × hemisphere: F_(1,23) = 6.4, p = 0.02; cue × hemisphere × electrode: F_(1.8,41.2) = 3.5, p = 0.04; cue × electrode: F_(2.4,55.0) = 5.1, p = 0.006). At contralateral electrodes, N1 amplitudes were more negative for neutral trials compared with valid trials (cue: F_(1,23) = 5.4, p = 0.03), whereas at ipsilateral electrodes, amplitudes were more negative for valid trials compared with neutral trials (cue × electrode: F_(2.3,53.7) = 6.2, p = 0.002). Memory-based orienting also influenced the latency of the N1 (cue: F_(1,23) = 6.6, p = 0.02). The N1 peaked earlier after valid memory cues than after neutral cues. There was no interaction with the hemisphere factor (cue × hemisphere: F_(1,23) < 0.001, p = 0.99). Topographic segmentation showed a stable topography between 150 and 190 ms. During this time period, the voltage for valid trials compared with neutral trials was more positive at posterior contralateral electrodes and was more negative at ipsilateral electrodes.

To test whether the modulation of the visual perceptual potentials P1 and N1 elicited by the targets was functionally related to the spatial biases in the anticipatory, rhythmic alpha-band activity, we performed correlation analyses. The spatial bias in alpha-band desynchronization (power at contralateral − ipsilateral electrodes) correlated significantly with the spatial bias in the modulation of the P1 potential (r₍₂₂₎ = 0.41, p = 0.05) but not the N1 potential (r₍₂₂₎ = 0.12, p = 0.58). Correlation analyses were also used to test whether the lateralization of the early potentials influenced behavioral measures of target detection in valid trials. Significant negative correlations were obtained between RTs and each of these potentials (P1: r₍₂₂₎ = −0.54, p < 0.05; N1: r₍₂₂₎ = −0.48, p < 0.05). These suggest that relative enhancement of the positive P1 amplitudes contralaterally and relative attenuation of the negative N1 amplitudes (becoming less negative) predicted shorter reaction times.

Discussion

By using a recording method with high temporal resolution, we were able to demonstrate unambiguously that long-term memory for the spatial location of a task-relevant stimulus within a complex scene context changes the neural processing of this target stimulus from early stages of analysis, during visual perceptual processing. Viewing a previously learned complex scene triggered a spatial bias in oscillatory alpha-band neural activity related to anticipation of the target appearing at its remembered location and modulated the earliest identifiable visual potential, P1, elicited by targets. Furthermore, the measures of anticipatory and target-related neural activity correlated with one another, and P1 amplitudes predicted response times. The results build on our previous behavioral findings that spatial LTM can effectively drive attentional orienting (Summerfield et al., 2006), revealing the nature of the anticipatory spatial biases and the stages of target processing that are subsequently affected.

The explicit nature of our learning task and the high levels of accuracy in the spatial recall task suggest that spatial episodic memory traces guided these spatial biases. Such interpretation would be compatible with conclusions drawn based on previous contextual cueing experiments using complex scenes (Brockmole and Henderson, 2006; Brockmole et al., 2006, 2008). However, it is not possible to preclude the additional or alternative contribution of implicit (Chun et al., 1998, 2003) or semantic (e.g., Mackworth and Morandi, 1967) memory traces without further experimentation.

The spatial biases set up by LTM in anticipation of the target appearance were clearly evident in the greater desynchronization of neural activity in the alpha band over the posterior scalp contralateral to the remembered target location. Contralateral increases and ipsilateral decreases in alpha-band desynchronization are triggered by visual spatial attention (Foxe et al., 1998; Worden et al., 2000; Gould et al., 2011) and are proposed to reflect modulation of visual cortical excitability to enhance processing at attended locations (Worden et al., 2000; Kelly et al., 2006; Rihs et al., 2009) and suppress processing at unattended locations (Worden et al., 2000; Sauseng et al., 2005; Thut et al., 2006; Rihs et al., 2007, 2009). The pattern of effects observed in our study parallels that observed in spatial orienting studies using visual cues, strongly suggesting that memory for the target location results in changes in visual excitability in anticipation of the target expected at the remembered location. The correlation between the lateralization of alpha-band activity and subsequent modulation of the P1 potential to the subsequent target supports this interpretation. Because of the short minimum interval between the appearance of the scene cue and the target key in our task (600 ms SOA), it was only possible to estimate changes in alpha-band power over a short time interval. It will be important to replicate this effect with tasks using longer intervals between memory cues and targets.

The lateralized modulation of alpha desynchronization was preceded by a lateralized evoked potential over posterior electrodes that resembled the N2PC potential associated with target identification within visual search arrays (Luck and Hillyard, 1994; Eimer, 1996), although occurring at a later time period (340–390 ms). The potential may provide a marker for activity in spatiotopically organized brain areas, such as extrastriate visual cortex, that is related to LTM retrieval of the target item within its spatial context. Such findings would complement and extend recent observations that retrieval of items within visual short-term memory also engages activity in spatiotopically organized areas (Gratton, 1998; Kuo et al., 2009; Dell'Acqua et al., 2010; Eimer and Kiss, 2010) and suggest a common or similar layout for the organization of perceptual short-term memory and long-term memory visual representations. Future studies should seek to replicate the finding of lateralized potentials linked to spatial retrieval and to test further the spatiotopic specificity of the effect.

Event-related potentials elicited by subsequent targets showed clear effects of spatial LTM on visual potentials P1 and N1, which reflect processing in multiple extrastriate visual areas in ventral and dorsal visual pathways (Clark and Hillyard, 1996; Martínez et al., 1999, 2001; Mangun et al., 2001; Di Russo et al., 2003). Targets appearing at remembered locations within scenes evoked larger visual P1 potentials than targets appearing at memory-neutral locations, an effect that predicted subsequent response times. The effect was similar to the gain modulation of P1 typically observed after endogenous or exogenous cueing of spatial attention by perceptual cues (Mangun et al., 1987; Eimer, 1994; Mangun, 1995; Hillyard and Anllo-Vento, 1998; Luck et al., 2000; Hopfinger and West, 2006). Gain modulation of P1 in visual spatial attention tasks has been proposed to reflect the selective amplification of attended sensory information passing through the visual pathways (Hillyard and Mangun, 1987; Posner and Dehaene, 1994; Luck et al., 2000). Enhancement of the P1 over contralateral electrodes in our task strongly suggests that LTM can also lead to similar gain modulation of early stages of visual processing. The correlation between P1 enhancement and the preceding lateralization of alpha desynchronization suggests that the scene cue led to a spatial bias in visual excitability, which facilitated the initial stages of perceptual analysis of targets appearing at the expected remembered locations.

The following visual potential, N1, was also significantly modulated in our task, again reinforcing the notion that LTM for the location of a target has a strong influence on the neural processing of that event. However, in this case, the pattern of modulation differed from previous reports in studies using perceptual cues to drive spatial attention. The N1 showed an atypical attenuation over the contralateral scalp, followed by enhancement over the ipsilateral scalp (Doherty et al., 2005), an effect that also correlated with subsequent response times. In addition, N1 latency was diminished by valid LTM cues (Di Russo and Spinelli, 1999; Vibell et al., 2007).

Our ERP results may point to important differences in how top-down perceptual biasing mechanisms may differ when generated by memory, as in our task, versus by perceptual cues. Some of the ERP effects found here are similar to those described in previous visual spatial attention experiments, such as lateralization of anticipatory alpha-band activity and amplification of the target P1. A parsimonious explanation for these effects is that viewing the scene cue triggers a shift of visuospatial attention through the dorsal frontoparietal network (Mesulam, 1981; Kastner and Ungerleider, 2001). However, other effects appear qualitatively different, such as the contralateral attenuation and latency shift of the N1. These may indicate that retrieval of spatial memories can trigger additional or alternative mechanisms to modulate perceptual analysis of the subsequent target. The lack of correlation between alpha lateralization and the N1 effect would corroborate this view. For example, brain areas involved in spatial and contextual memory retrieval in medial temporal cortex can influence activity in ventral visual areas directly through feedback connections (Suzuki and Amaral, 1994). One possibility is that attenuation of N1 in memory-guided orienting reflects mechanisms related to perceptual “priming,” whereby the retrieved context would preactivate the target-related representation and/or its location, reducing the perceptual resources required to identify a target in its previously learned context (e.g., Buckner et al., 1995, 1998; Paller and Gross, 1998). Resolving whether the observed N1 modulation is a memory-specific effect will require further investigation. The N1 reflects activation of several visual and possibly multisensory areas (Doherty et al., 2005; Natale et al., 2006), and its modulation can be highly task-sensitive to stimulus parameters and response requirements (Heinze et al., 1990; Luck et al., 1990; Mangun and Hillyard, 1991; Hillyard and Anllo-Vento, 1998).

Direct comparisons between mechanisms of memory-guided and visually guided orienting are not possible without equating several important task parameters, such as the use of rich and naturalistic backgrounds upon which the target stimuli appear. The use of cluttered and naturalistic scenes greatly increases the inherent level of competition among visual objects for driving neuronal responses (Desimone and Duncan, 1995; Reynolds et al., 1999; Rolls and Deco, 2006), which may impact substantially the efficacy of different types of modulatory mechanisms. To understand how top-down attentional mechanisms operate in day-to-day cognition, therefore, it will be important to continue to investigate biases that are generated by memory and that operate within complex stimuli that carry ecological validity.

Our results also validate the use of this task design for investigating the neural mechanisms by which memory signals can bias perception. The design offers greater interpretational power than the contextual cueing manipulations, where context and target remain intrinsically bound. By separating the context from the target during the attention-orienting task, it was possible to dissociate the neural effects triggered by the spatial memory for the target within the scene from the perceptual modulation of target processing. These findings provide empirical support and constraints for theoretical and computations models about the role of memory in object and scene perception (Torralba et al., 2006; Peters and Itti, 2007; Zelinsky, 2008; Friston, 2010). The experimental approach also helps pave the way for understanding the neural mechanisms responsible for the proactive and predictive functions of memory in shaping perception.