Abstract
In everyday situations we often rely on our memories to find what we are looking for in our cluttered environment. Recently, we developed a new experimental paradigm to investigate how long-term memory (LTM) can guide attention, and showed how the pre-exposure to a complex scene in which a target location had been learned facilitated the detection of the transient appearance of the target at the remembered location (Summerfield, Lepsien, Gitelman, Mesulam, & Nobre, 2006; Summerfield, Rao, Garside, & Nobre, 2011). The present study extends these findings by investigating whether and how LTM can enhance perceptual sensitivity to identify targets occurring within their complex scene context. Behavioral measures showed superior perceptual sensitivity (d′) for targets located in remembered spatial contexts. We used the N2pc event-related potential to test whether LTM modulated the process of selecting the target from its scene context. Surprisingly, in contrast to effects of visual spatial cues or implicit contextual cueing, LTM for target locations significantly attenuated the N2pc potential. We propose that the mechanism by which these explicitly available LTMs facilitate perceptual identification of targets may differ from mechanisms triggered by other types of top-down sources of information.
INTRODUCTION
Two large experimental fields have developed side-by-side in cognitive psychology and cognitive neuroscience: one that focuses on attention and one on memory processes. In recent years there has been a surge of interest in understanding how these systems may be related. Most of data and emerging theoretical frameworks focus on how attention can influence what is retrieved from memory (Cabeza, Ciaramelli, Olson, & Moscovitch, 2008; Ciaramelli, Grady, Levine, Ween, & Moscovitch, 2010; Ciaramelli, Grady, & Moscovitch, 2008). Fewer studies have explored the other side of this relationship; that is, how long-term-memories (LTMs) influence the deployment of attention. In a seminal study, Chun and Jiang (1998) showed that implicit LTMs can guide attention during a visual search task. The critical manipulation was that the spatial configuration of distracters could either be repeated from a previous trial, or novel. Implicitly acquired memories in this task resulted in faster reactions times to locate targets in repeated distracter arrays compared to novel ones. This effect has been termed contextual cueing (Chun & Jiang, 1998), and has since been extensively studied and characterized both behaviourally (Chun & Jiang, 1999, 2003; Jiang & Chun, 2001; Jiang & Leung, 2005; Olson & Chun, 2001, 2002) and neurally (Chaumon, Drouet, & Tallon-Baudry, 2008; Chaumon, Hasboun, Baulac, Adam, & Tallon-Baudry, 2009; Chaumon, Schwartz, & Tallon-Baudry, 2009; Chun & Phelps, 1999; Olson, Chun, & Allison, 2001).
Though the contextual cueing paradigm has provided an important foundation for investigating the influence of LTM upon attention, it leaves several important aspects of the putative mechanisms unexplored. Due to the fact that targets appear embedded within repeated distracter configurations, it is difficult to separate context learning and memory retrieval processes from the attentional modulation of target detection. Contextual cueing tasks also typically use arrays of simple stimuli, and so it is important to test whether the effects generalize to more ecologically valid settings in which objects are located within complex, cluttered scenes (for example see Becker & Rasmussen, 2008).
In order to address some of these limitations, and to enable the investigation of the behavioural and neural mechanisms related to orienting attention based on the retrieval of contextual spatial locations of objects within realistic complex scenes, Nobre and colleagues developed a new experimental paradigm (Summerfield, Lepsien, Gitelman, Mesulam, & Nobre, 2006). It builds on and brings together paradigms used to study contextual cueing by repeated configurations of distractors (Chun & Jiang, 1998), orienting of attention by visual cues (Posner, 1978), and search within complex and naturalistic scenes (Biederman, 1972).
In the task used by Summerfield and colleagues (Summerfield et al., 2006; Summerfield, Rao, Garside, & Nobre, 2011), participants first explored complex visual scenes overtly and learned the unique locations of hidden pre-defined targets in the scenes. Twenty-four hours later, participants performed a speeded target detection task, in which the previously learned scenes acted as cues to orient attention based on LTM. The scene was presented briefly (without the target object embedded within it) and the target appeared transiently at the remembered or a non-remembered location. Participants were reliably faster at detecting targets appearing within learned spatial contexts, even at very short intervals between presentation of the scene cue and the target (Summerfield et al., 2006, 2011). Behavioural measures during the learning phase and in subsequent tasks of explicit memory retrieval indicated that participants had conscious access to the learned target locations.
The separation of the learning phase from the orienting task helps to separate the testing of attentional orienting from the development of learning for the target-context association, and ensures that the memories guiding attention are effectively coded in long-term memory. Importantly, separating the presentation of the scene (memory cue) from the target object within the orienting task enables measurement of top-down signals related to memory-guided orienting of attention separately from the modulatory consequences of these biases upon target processing.
One limitation of the task used by Summerfield et al. (2006; 2011) is the rather artificial way in which the target is presented in the orienting task. The target appears transiently, overlaid upon the scene cue, and requires only a simple speeded detection response. Using transient targets that are easy to detect, it is not possible to test whether memory-guided orienting of attention can enhance perceptual sensitivity to perceive target objects, and improve the ability to discriminate their presence within their natural environment of a cluttered complex scene.
In the present study, we modified the orienting task in order to test how LTM modulates behavioural and neural measures of perceptual discrimination of targets within complex scenes. Participants performed a challenging perceptual discrimination task in which they judged whether the target object was present or absent in a previously learned scene. For half of the scenes, participants had learned a consistent location for the key and therefore had a valid memory cue. For the other half, participants had not learned a target location and had no specific spatial contextual association for the target location (neutral memory cue). If long-term memories can bias perceptual discrimination functions, we expect to find an increase in the sensitivity measure (d′) for targets appearing in learned (valid) locations as opposed to unlearned (neutral) locations. Event-related potentials (ERPs) were recorded to reveal modulation of target selection by memory-guided orienting. In particular, we focused on modulation of the N2pc potential. This potential has been identified in studies using simpler visual search arrays. It is characterized by an enhanced negative voltage at posterior electrodes contralateral, as compared to ipsilateral, to the side of the target and is typically observed approximately 200 ms after target presentation. The N2pc component has been associated with the selection of task-relevant target features and/or suppression of distracters (Eimer, 1996; Hickey, Di Lollo, & McDonald, 2009; Kiss, Van Velzen, & Eimer, 2008; Luck & Hillyard, 1994; Mazza, Turatto, & Caramazza, 2009; for an earlier interpretation see Woodman & Luck, 1999, 2003). In our view, the N2pc reflects the spatial layout of a top-down biasing signal prioritizing the features or attributes based on the target identity (see Kuo, Rao, Lepsien, & Nobre, 2009).
Interestingly, the N2pc has been shown to be unaffected by visuo-spatial orienting of attention in visual search (Brignani, Lepsien, & Nobre, 2010; Leblanc, Prime, & Jolicoeur, 2008; Schankin & Schubö, 2010; Seiss, Kiss, & Eimer, 2009), suggesting the relative independence between visual spatial orienting mechanisms and biasing of object features or attributes based on the task requirements (see Brignani et al., 2010). If memory-guided orienting of attention modulates target processing through the same mechanisms as visually guided orienting, there should be no modulation of the N2pc across valid and neutral conditions. However, implicit contextual cueing leads to larger N2pc potentials for targets appearing in repeated displays (Johnson, Woodman, Braun, & Luck, 2007; Schankin, Hagemann, & Schubö, 2011; Schankin & Schubö, 2009; 2010), indicating that different top-down biasing mechanisms may be triggered by LTM cues versus perceptual cues. Here we aim to explore whether and how explicitly acquired spatial LTMs bias the target selection processes indexed by N2pc.
METHODS
Participants
Sixteen healthy right-handed students (7 male, 9 female, mean age 25 yrs, range 19-32 yrs), with normal or corrected-to-normal vision from the University of Oxford consented to participate in this study for monetary compensation. The study was approved by the University of Oxford Central University Research Ethics Committee (CUREC).
Scene stimuli
One hundred and ninety two scenes were used in the experiment. Each scene was prepared in two different formats, one for the learning task and one for the orienting task. For the learning task, a small gold key (15 × 29 pixels, equivalent to 0.3° × 0.7°) was placed in one of the four quadrants of the scene, preferably in a hidden location. Five learning task versions were generated for each scene, with the key placed in one of each of the four quadrants or with the key absent - this was done for counterbalancing purposes. For the orienting task, the scenes with keys were remade to include a larger and brighter key (25 × 49 pixels, equivalent to 0.6° × 1.1°) in the location of the original key. Key presence and location within scenes, and assignment of scenes to different experimental conditions (valid, neutral), were counterbalanced across participants.
All scene stimuli were created from photographs obtained collectively by the lab, and resized to 1000×750 pixels using Matlab (Mathworks, Natick, MA). The stimuli subtended 22° × 17° of visual angle at a viewing distance of 100cm.
All stimuli were presented using Presentation (Neurobehavioral Systems, Albany, CA).
Experimental Procedure
Participants performed three experimental tasks over three days. Over the first two days, they completed a Learning Task, in which they explored visual scenes to learn the location of a target key in each scene (50% of scenes contained a key). By the end of the learning task, participants had formed strong spatial contextual memories of the target location for scenes containing a target, but they had no specific target-context associations for those scenes that did not contain a target (all scenes however, were familiar). On the third day, they completed an attention Orienting Task in which they discriminated the presence or absence of a target key within previously studied scenes. Pre-exposure to the scenes (without any target) in which participants had learned a target location provided valid memory-based cues to orient contextual spatial attention. After the orienting task, participants completed a Spatial Memory Recall Task in which they indicated the location of the target key on each of the previously studied scenes and reported the degree of confidence in their memory recall.
Learning Task
Participants viewed each of the 192 scenes in a random order, repeated over six blocks (the learning task was broken down into three blocks each, over two consecutive days). Half of the scenes contained a small gold key target in one of the four quadrants (24 in each quadrant). The remaining 96 scenes did not contain a target. Participants viewed the scenes and searched for the target overtly. Once located, participants clicked once with the mouse to activate a cursor, after which they clicked on the location of the key with the mouse. After a response, or after the available search time expired, the next scene was presented. The available search time decreased as the blocks progressed, with the maximum duration of each scene randomized within a range (16-24s in block 1, 12-20s in blocks 2 and 3, 10-18s in blocks 4 and 5, 8-16s in block 6). Exposure times for key-absent scenes were yoked to the exposure of key-present scenes. Participants had to find as many keys as possible and memorize their locations. Participants were given visual written feedback as to whether they had correctly identified the location of the key. Feedback was also provided at the end of each learning block notifying them how many keys they had found during that block. Eye movements were recorded using an infrared eye-tracking system (ISCAN).
The encoding of target-context associations over the course of learning, as measured by accuracy and reaction time, was assessed by linear contrasts over the six blocks using repeated-measures analysis of variance (ANOVAs).
Orienting Task
The electroencephalogram (EEG) was recorded while participants performed the memory-guided orienting task on the third day. Participants performed 192 trials. Their task was to discriminate the presence or absence of a gold key within the familiar scenes they had previously studied, using only covert attention (i.e., maintaining visual fixation on the centre of the screen). Each trial began with the presentation of a familiar, studied scene (100ms) for which the participant had or lacked an association with a particular target location. Scenes with associations provided valid memory cues, whereas scenes in which no associations had been learned provided neutral memory cues. After a random inter-stimulus interval (ISI) of 750-1150ms, the probe scene appeared (200ms), with or without an embedded target. There were four conditions of target scenes: valid-present (participants had learned a key location and the target key in the orienting task was in the same location), neutral-present (no key was present in the learning task, but there was a target key present during the orienting task), valid-absent (participants had learned a key location, but the key was absent in the orienting task), and finally neutral-absent (no key was present in the learning task and the key was also absent in the orienting task). Subjects had a 1000-ms time window to respond after the probe scene disappeared, after which feedback was given. The next fixation screen indicated the start of a new trial, and was preceded by a jittered inter-trial interval of 2000-3000ms. Trials were randomly intermixed throughout the task. The orienting task was performed covertly, and eye movements were monitored using an infrared eye-tracking system (ISCAN).
Participants performed a short practice session (12 trials, using a different, novel set of scenes) prior to the orienting task to ensure they understood the task and could refrain from making eye movements.
The effects of memory-based attention on performance were assessed using ANOVAs or paired t-tests. Trials for which participants had failed to learn the location of an available target in the learning task were removed from all analyses (3.05% of the trials were excluded). Analysis of mean response times (RTs) used only correct trials. Trials were also excluded if response times exceeded ± 3 standard deviations (0.56% of the total trials). An ANOVA tested the factors of memory-cue (valid, neutral) and response (present, absent). To quantify perceptual sensitivity, we used a measure from signal detection theory that separates the means of the signal and noise, by giving the relationship between the rate of hits to false alarms within each condition (d′ = z(hits) − z(false alarms)). D-prime was compared between valid and neutral memory-cue conditions using a paired t-test. Accuracy measures (percentage of correct hits, false alarms, correct rejections and misses) were also calculated.
ERP Recording and Analysis
The EEG was recorded continuously during the orienting task, using NuAmp amplifiers (Neuroscan, El Paso, TX) from 40 scalp sites using Ag/AgCl electrodes mounted on an elastic cap, positioned according to the 10-20 international system (AEEGS, 1991). The montage included 7 midline electrode sites (FZ, FCZ, CZ, CPZ, PZ, POZ, and OZ) and 13 sites over each hemisphere (FP1/FP2, F7/F8, F3/F4, FT7/FT8, FC3/FC4, T7/T8, C3/C4, TP7/TP8, CP3/CP4, P7/P8, PO7/PO8, PO3/PO4 and O1/O2). Additional electrodes were used as ground and reference sites and for recording the electrooculogram (EOG). The right mastoid was used as the active reference. Data were then re-referenced off-line to the algebraic average of the right and left mastoids. The horizontal EOG was recorded bipolarly with one electrode on the side of one eye, and the other electrode on the other side of that eye directly next to the nose; this was done to increase the possibility of detecting microsaccades. Vertical EOG was recorded bipolarly with one electrode under the right eye and FP2 used as the other vertical EOG. The signal was digitized at a sampling rate of 1000 Hz. Data were recorded with a low-pass filter of 200 Hz and with no high-pass filter (DC). Data were subsequently filtered offline with a 40 Hz low-pass filter. Digital codes were sent to the EEG recording computer to mark the presentation of the cue and target stimuli in each trial type.
The EEG was epoched offline to the presentation of the cue and of the target stimuli. Epochs for the cue period started 200ms before and ended 850 ms after cue presentation. Epochs for the target period started 1150ms (maximal inter-stimulus interval) before and ended 600ms after the stimulus presentation; this was done to enable removal of any trials with anticipatory saccades. Epochs for both the cue and target were baselined from 50 ms before to 50 ms after stimulus presentation. Epochs containing excessive noise or drift (+/− 100μV) at any electrode were rejected. Trials with blinks or large saccades (+/− 50μV), identified through the horizontal and vertical EOGs, were excluded; additional visual inspection was used to remove trials with any residual artifacts or eye movements. In addition, trials with incorrect responses or corresponding to scenes where participants failed to locate the key by the final block of the learning task were also excluded from the analysis. The criterion for the minimum number of trials per condition was set at 20. Two participants did not reach the criterion of 20 clean ERP trials per condition, and were therefore excluded from all further analyses, thus restricting the analysis to 14 participants.
The main ERP analysis tested how memory-guided attention modulated selection of a target within its cluttered scene context. We compared the N2pc potential elicited by targets embedded within scenes in the valid versus neutral memory-cue conditions. As is common in the visual search literature, the N2pc was analysed over posterior lateral electrodes (O1/2, PO3/4, PO7/8; between 200 and 280 ms). ERPs from targets located on the right and on the left side of scenes were combined by a procedure preserving the relationship between the side of electrode location and the side of target (contralateral and ipsilateral). In order to verify that the N2pc was elicited by selection of the target key within the context, and did not simply result from a spatial shift of attention to a remembered target location, we also tested for the presence of an N2pc in target-absent trials with scenes for which participants had learned a target location (valid-absent trials). Epochs in valid-absent trials were averaged according to the location of the key in the learning task (contralateral and ipsilateral to remembered location). In order to test for modulation of visual processing by contextual spatial memory prior to target selection, mean amplitudes of visual potentials P1 and N1 were measured at contralateral and ipsilateral posterior electrodes (O1/2, PO3/4, PO7/8) during the time windows of 100-120 ms and 140-160 ms, respectively.
Modulation of cue-related activity, in anticipation of the probe-scene was also tested. Results from time-frequency analyses showing significant desynchronization of alpha-band activity over posterior electrodes contralateral to the expected target location have been reported previously (Stokes, Atherton, Patai, & Nobre, 2012; see also Summerfield et al., 2011). Here we tested for cue-related activity by comparing ERPs elicited by spatially informative (valid) and spatially uninformative (neutral) cues. We used the results obtained in a similar study (Summerfield et al., 2011) to guide our analysis. We analysed two components, one over midline electrode sites (C3/Z/4) between 400-470 ms, and another over posterior electrode sites (PO7/8, P7/8) between 500-800 ms. ANOVAs were used to compare mean voltage amplitudes within these time windows across the conditions of interest.
In addition to the ERP analysis, we used a topographical analysis tool (Cartool; developed by Denis Brunet, brainmapping.unige.ch/cartool), which extracts periods of stable topography that are functionally relevant. To reduce to possibility of spurious maps, we merged all clusters correlated above 90% and removed small segments that were present for less than 40 milliseconds.
Spatial Memory Recall Task
Following the orienting task, participants performed a recall task that measured explicit memory for target locations. Participants viewed all 192 scenes, without any target present. For each scene, they were prompted to indicate the location of the target key learned during the learning task. For scenes in which they had a memory for the target location, they used the mouse to click on the remembered target location from the learning phase. If they had no memory, they clicked on the center of the screen. Participants were also asked to rate their confidence in their responses after each scene on a 3-point scale by clicking one of the three mouse buttons to indicate their level confidence (1 = not at all confident; 2 = fairly confident; 3 = very confident).
We used a repeated-measures ANOVA to compare the quality of recollection of the target location, as measured by distance, across the self-reported confidence levels.
RESULTS
Formation of long-term memories for target locations within complex scenes
Over the course of the learning blocks, increasing numbers of targets were located with increasing speed (block 1: mean accuracy ± s.e.m. = 86% ± 1.9, mean search times ± s.e.m. = 4.8 sec ± 0.15; block 6: 96% ± 1.2, 1.4 sec ± 0.13). ANOVAs testing for linear decreases in search times revealed a significant linear contrast over the learning blocks, F(1,12) = 211.55, p<0.001. Similarly, a significant linear increase in accuracy over the learning blocks was revealed, F(1,12)= 28.75, p<0.001 (Figure 1). Eye-tracking data also showed that, as the blocks progressed, participants’ eye movements were directed toward the target location progressively earlier (Figure 1). Subjects had to find a minimum of 90% of targets in order to proceed to the orienting task.
Figure 1. Learning task results.
(A) Eye-movements - from a representative subject - in block one indicate search through whole scene to find the target. (B) Eye-movements in the last block (block 6) show target is located after just a few saccades. (C) As learning blocks progress, participants find more targets and are faster at locating the targets.
The spatial memory recall task performed immediately after the orienting task confirmed that participants retained robust memories for target locations within the learned scenes. In order to obtain an estimate of participants’ explicit memory for the target location, we calculated a 150-pixel (approximately 3.4° visual angle/15% of screen) diameter circle around the target location, and labeled responses as ‘hits’ if the subject’s response was within this window. This calculation was performed only for trials for which the target was absent during the orienting task, in order to avoid any contamination effects from re-exposure to target location during the orienting task. The majority of subjects correctly identified the locations of the learned targets (group mean correct = 75%). In addition, subjects’ confidence ratings varied systematically with the response distance from actual target location, in that they became less confident as the distance from their response to the target location increased (mean distance in pixels ± s.e.m; overall: 55 ± 6; rate 1: 66 ± 10; rate 2: 61 ± 8; rate 3: 38 ± 2, linear contrast: F(1,12) = 9.76, p = 0.009) as shown in Figure 4.
Figure 4. Spatial-memory recall test.
The x-axis shows the various confidence ratings that participants could give after they indicated to where they remembered the key was; the y-axis shows the distance between the actual target location and where participants remembered the target to be. There is a correspondence between participants’ self-rating of confidence and their accuracy at remembering the target location.
Long-term memories increase perceptual sensitivity for targets presented at memory-predicted spatial locations
Figure 2 shows the experimental design for the orienting task and a summary of the pattern of behavioral results. A paired t-test, assuming equal variance, was used to test whether LTM for the target location improved the perceptual sensitivity to detect the target in the orienting array. The d′ was higher for the valid trials compared to neutral trials (valid = 3.49 ± 0.17 s.e.m., neutral = 3.22 ± 0.13, t(1,13) = 2.10, p = 0.05), which indicates that participants were more sensitive at discriminating targets in scenes in which they had learned a target location relative to scenes where no previous locations had been memorized (Figure 2).
Figure 2. Orienting Task: Design & Results.
(A) Each trial consisted of a cue stimulus, which could either be associated with a specific contextual memory for a target location, or not. After a variable inter-stimulus interval, a target scene appeared, with a target (key) embedded in either a learned (‘valid’, top row) or unlearned location (‘neutral’, bottom row) on half the trials; participants had to judge the presence or absence of the target. (B) Mean d′ and reaction times (error bars indicate standard errors) to discriminate the presence versus absence of the key target after a valid or neutral cue scene. Results showed increased perceptual sensitivity and reaction time benefit for targets appearing in learned locations.
ANOVAs were also used to test for the effects of spatial-memory associations during the learning task (valid, neutral) and the presence of the target in the orienting task (present, absent) on accuracy and RT.
Accuracy levels for target-present and target-absent trials in the two conditions are summarised in Table 1. Complementing the d-prime analysis, accuracy measures also showed a significant main effect of memory-cue, F(1,13) = 6.81, p = 0.022, indicating a higher percentage of correct “target-present/target-absent” discriminations on valid trials (0.94 ± 0.01) as compared to neutral trials (0.91 ± 0.02). Accuracy was also higher in absent (0.96 ± 0.01) versus present trials (0.89 ± 0.02), F(1,13) = 16.17, p = 0.001. A significant interaction between the two experimental variables (memory-cue × response: F(1,13) = 6.98, p = 0.020) indicated that the validity effect on accuracy was restricted to target-present trials (valid × neutral in target-present trials: F(1,13) = 10.17, p = 0.007; valid × neutral in target-absent trials: F(1,13) = 0.76, p = 0.40).
Table 1. Accuracy values reported as percentages (mean ± standard error) across all experimental conditions.
CONDITION | hits | false alarms | misses | correct rejections |
---|---|---|---|---|
valid | 92±1.6 | 2±0.6 | 8±1.5 | 95±1.2 |
neutral | 85±2.4 | 2±0.4 | 13±2.3 | 96±0.7 |
Analysis of RT revealed that target discrimination was faster in the valid condition (valid: 641 ± 31 s.e.m., neutral: 667 ± 26; F(1,13) = 11.66, p = 0.005), and in target-present trials (present: 626 ± 30, absent: 682 ± 27; F(1,13) = 15.44, p = 0.002). Moreover, the effects of validity were restricted to target-present trials (memory-cue × response: F(1,13) = 19.26, p = 0.001; target-present trials: F(1,13) = 11.63, p< 0.001; target-absent trials: F(1,13) = 8.15, p = 0.66).
In visual search tasks it is common that target-absent trials show an increase in reaction time, which has commonly been attributed to the fact that participants must search through the whole array in order to determine the absence of a target, whereas during target present trials, the search terminates once the target is located (Treisman & Gelade, 1980). Lower error rates in target-absent trials, alongside increases in RT, have also been shown in other visual search tasks (Yantis & Jonides, 1984).
Long-term memories modulate neural activity related to target selection
The main analysis of interest involved target-present trials with or without memory for target locations. Analysis of the amplitudes of early visual potentials, P1 and N1, revealed no significant differences between valid and neutral conditions (P1: 100-120 ms, F(1,13) = 0.08, p = 0.78; N1: 140-160 ms, F(1,13) = 0.51, p = 0.49).
The topographical segmentation indicated the presence of an N2pc potential in both valid and neutral conditions. Subtraction of waveforms over homologous electrodes over contralateral versus ipsilateral sites resulted in a relative negative voltage over posterior contralateral electrodes covering the period between 200 and 280 ms. The distribution of the contralateral negativity in the valid condition (Figure 3, map 1) extended more anteriorly than in the neutral condition (Figure 3, map 2).
Figure 3. ERP results for the target period.
(A) Grand-average ERP waveforms (collapsed over electrodes O1/2, PO3/4, PO7/8) for the valid and neutral conditions. The N2pc is observed as a greater negative deflection over contralateral sites as opposed to ipsilateral, and is clearly diminished in the valid condition. (B) Difference waveforms (contralateral minus ipsilateral) for both conditions showing a difference in the N2pc but not in the later potential, which is reversed in polarity (here termed PCP: posterior contralateral positivity). On the bottom are the topographical maps of the ERPs for valid and neutral targets. The more frontally distributed pattern on the temporal window corresponding to the N2pc in the valid condition (compared to the neutral condition) suggests a different pattern of neural activity when scenes trigger specific spatial memories and when they do not.
Statistical analysis confirmed the presence of the N2pc potential in target-present scenes at parieto-occipital electrodes (O1/2, PO3/4, PO7/8) (main effect of hemisphere at 200-280 ms: F(1,13) = 24.5, p < 0.001). To isolate the N2pc, the average waveform for the posterior electrodes over ipsilateral electrodes was subtracted from that over contralateral electrodes. T-tests of the subtraction waveforms for each condition against zero, verified the presence of a reliable N2pc in both the valid, t(1,13) = −2.49, p = 0.027 and the neutral, t(1,13) = −5.04, p<0.001, condition.
Of direct relevance to the research question, the amplitude of the N2pc was significantly modulated by memory as revealed by the analysis of the difference waveforms created by subtracting the ipsilateral from the contralateral target-related ERP waveforms. The N2pc was significantly attenuated for valid relative to neutral trials, indicating a strong effect of LTM on the spatially specific processing of targets, t(1,13) = 2.49, p = 0.027. Figure 3 presents the combined difference waveforms for three representative electrodes.
In order to confirm that the N2pc was related to the selection of the target in its scene context and to rule out that it may have been driven simply by the orienting of spatial attention, we also tested for the presence of the N2pc in the valid-absent condition. The subtraction waveform compared electrodes contralateral versus ipsilateral to the attended side, based on memory for the target location in the training phase. No N2pc was evident when no target was present, with the effect being far from significant, t(1,13) = 0.14, p = 0.89.
Visual inspection of the waveforms and topographical segmentation also revealed an unexpected subsequent lateralized effect, following the N2pc, with opposite polarity (see Figure 3). This effect was an enhanced positivity over posterior contralateral (relative to ipsilateral) scalp locations to the target side in the latency window between 320-380 ms post-stimulus (labeled here as PCP: Posterior Contralateral Positivity), which had an equivalent topographical distribution across conditions (Figure 3, map 3). An ANOVA assessed if the amplitude of this potential was differentially modulated by the type of preceding cue. A significant effect of hemisphere, F(1,13) = 14.73, p = 0.002, confirmed the presence of a strong lateralized effect over parieto-occipital electrodes. However, the potential was not affected by memory-guided visual search processes (hemisphere × memory-cue: F(1,13) = 0.002, p = 0.88).
Finally, we tested for differences between ERPs elicited by valid memory cues and neutral cues. The effects were similar to those obtained in Summerfield et al. (2011). There were no lateralized ERPs, but there were two significant non-lateralized effects. ERPs by valid cues elicited a more negative mid-latency potential over midline sites (C3/Z/4, 400-470 ms; F (1,13)= 7.21, p = .019) and a more positive sustained late potential over posterior electrodes (PO7/8, P7/8; 500-800 ms; F (1,13)= 4.91, p = .045).
DISCUSSION
The results of this experiment show that long-term memories (LTMs) enhance attentional guidance during a perceptual discrimination task and influence neural signatures of target selection. Spatial expectations from LTM conferred behavioral benefits, as revealed by increased perceptual sensitivity and decreased reaction times to targets appearing in remembered versus non-remembered locations. The d′ sensitivity index clearly shows that LTMs can influence perceptual analysis of the stimulus, thus confirming that top-down signals from LTM do more than change the response criterion through response biases. These findings replicate and extend previous results by Summerfield et al. (2006, 2011) by showing that predictions based on prior knowledge acquired from experience facilitate perceptual decisions about the presence of relevant objects when they are embedded within their natural scene contexts. The facilitation of RTs suggests that this perceptual benefit does not come at any speed cost. Instead, memory also speeds up responses to identify the target, leading to better perceptual discriminations within shorter latencies.
Target-related ERPs show that LTM can enhance neural processes related to target selection, as reflected by modulations of the N2pc component. This memory-driven modulation of target processing reveals a close and rapid interaction between memory and attention systems in the brain. We were able to identify an enhanced negative voltage over contralateral (versus ipsilateral) posterior electrodes with a similar time-course as the N2pc. Importantly, memory cueing within complex scenes resulted in an interesting and unexpected finding: LTM-based spatial contextual memory cues reduced the magnitude of the N2pc in the valid condition.
Interestingly, the modulation of the N2pc by LTM cues differed qualitatively from what has been observed with spatial cueing of attention in typical visual search tasks. LTM for the target location in our task clearly and strongly attenuated the N2pc. In contrast, visual spatial cues in search tasks, and in other types of perceptual attention tasks, do not influence the N2pc (Brignani et al., 2010; Kiss et al., 2008; Leblanc et al., 2008; Schankin & Schubö, 2010; Seiss et al., 2009). Results from previous visual spatial cueing tasks have been interpreted as suggesting that the N2pc does not reflect the spatial guidance of attention (but see Woodman & Luck, 1999, 2003), or a selection process that is influenced by visual spatial attention. Instead, the N2pc appears to reflect a separate set of mechanisms related to feature-based selection processes guided by the identity of the target (Brignani et al., 2010; Kuo et al., 2009). The fact that we only observed the N2pc when the target stimulus was present in the scene reinforces the notion that the N2pc is linked to target-selection processes.
Furthermore, our results clearly point to possible differences in how memory cues and perceptual cues come to influence target-selection processes. One possible explanation is that LTM for a specific target location within a scene primed the identification of the target attributes, diminishing the amount of visual analysis and suppression of distracting information required for effective target selection and identification. The cue in our task could activate specific memory traces for target/context configurations, facilitating the target selection and thus reducing the amount of resources required for the suppression of distracters. This interpretation is in line with the findings of Luck & Hillyard (1994), who reported that the N2pc is diminished when distracters are irrelevant or removed. Alternatively, these differences may stem from differences in modulations of neural signatures of target selection when targets are embedded in natural complex backgrounds versus simple visual backgrounds. Further experiments directly comparing ERPs produced when memory cues and perceptual cues guide attentional orienting within complex scenes are needed to settle this question.
Intriguingly, the attenuation of the N2pc by LTM also differs from what has been observed in previous experiments using ERPs in the contextual cueing paradigm. These have consistently reported a larger N2pc for targets appearing in repeated as opposed to novel displays (Johnson et al., 2007; Schankin & Schubö, 2009; 2010; Schankin et al., 2011). The discrepancy could result simply from the timings during which the selection processes can start to operate. It may be that, in general, appearance of a target within a learned context enhances the selection processes indexed by the N2pc, but when the context is pre-activated some of the selection processes can proceed ahead of time, in anticipation of the target appearance. In our experiment, the participant is pre-exposed to the scene triggering the contextual memory for the target location. It is possible therefore to process the context-target association, and engage neural processes relating to prioritising the target features and/or inhibiting the irrelevant features in the contextual background. In the contextual cueing paradigm, context and target occur only simultaneously, and all the work for prioritising the target features and suppressing distractor features needs to be carried out online. Evidence that selection has a head start in our memory-orienting task comes from analysis of the lateralised alpha-band activity, which becomes desynchronised over posterior contralateral electrodes in anticipation of the probe scenes (Stokes et al., 2012; see also Summerfield et al., 2011). Notably, we also found two non-lateralized modulations in the ERPs elicited by valid as compared to neutral cues, which may reflect non-spatially specific retrieval of the associations between target and context (see also Summerfield et al., 2011).
Alternatively, the discrepancy between the attenuation of the N2pc found here and the enhancement of the N2pc in previous contextual cueing tasks could be explained by the difference in the types of memory traces involved. In the classical contextual cueing paradigm, it is assumed that the memories guiding attention are implicit in nature (Chun & Jiang, 1998, 2003). In the current experiment, the memories for target-context associations were formed by explicit instruction, and the contexts were rich in visual detail, thus making them more available for explicit recall. When tested explicitly, participants were accurate at retrieving the learned locations of the keys, and reported high confidence levels. This pattern of results also occurred for scenes containing no keys within the orienting task, showing that performance on the memory retrieval was not dependent on re-exposure to the location of keys during the immediately preceding task. There is no way, of course, to rule out the formation and availability of implicit memory traces in our task, but we can be confident that explicit memory traces were also available, and these may have played a role. This difference in the types memory sources available may account for the difference in attentional guidance strategies, target selection and/or distracter suppression processes engaged by the tasks. This interpretation would be in line with proposals by Moscovitch and colleagues, who suggest that explicit, episodic memories may play a unique top-down role in regulating and facilitating a number of cognitive functions, such as priming (Sheldon & Moscovitch, 2010) and problem solving (Sheldon, McAndrews, & Moscovitch, 2011).
Additionally, it has been argued that in arbitrary target-distracter arrays, the local context around the target is sufficient to elicit a contextual cueing effect (Olson & Chun, 2002), whereas in naturalistic scenes, global information is crucial for guiding attention (Brockmole, Castelhano, & Henderson, 2006). It is possible that in the current experiment, the contextual effect is guided by a more holistic scene representation with a target location associated within it, as opposed to spatial configurations with arbitrary target – distracter relationships. Thus, the mechanisms at play in contextual cueing versus in our experiment may differ for multiple different reasons, and these are not mutually exclusive.
We also identified a later, spatially specific effect characterized by a lateralized posterior positivity contralateral to the target location, labeled here as PCP, which was not found to be modulated by memory cues. In reviewing previous ERP studies on PCP-like components, we found a recent description by Hilimire and colleagues (Hilimire, Mounts, Parks, & Corballis, 2010) of a positive posterior contralateral component, called Ptc (approximately 290-340 ms poststimulus), proposed to index additional processing necessary to individuate the target after it is identified under conditions of high competition between stimuli in an array. This finding was not predicted by our initial hypothesis and needs to be interpreted with caution; however, it is plausible to propose that similar later target-related processes are engaged when discrimination of relevant objects within crowded scenes is required.
Earlier visual potentials P1 and/or N1 are also typically modulated by visuospatial attention (Hillyard & Anllo-Vento, 1998). In a previous memory-based orienting task using transiently appearing targets, Summerfield and colleagues recently reported modulation of these early visual potentials. The amplitude of the P1 was enhanced by memory cues, whereas the N1 potential showed a more distinctive pattern of modulation – with contralateral attenuation and ipsilateral enhancement as well as latency reduction (Summerfield et al., 2011). However, the effects on P1 and N1 were not significant in our current experiment. This may simply have reflected the challenging conditions for measuring these potentials under our current task parameters. Typically, visual-spatial tasks use targets that appear transiently onto blank or very simple backgrounds. In our task, however, the target stimuli were intrinsically bound to an associated complex, cluttered scene, which it makes difficult to measure the influence of spatial biases on visual evoked potentials.
The results of the topographical analysis are preliminary, but raise the possibility of different neural sources or functional networks when target selection in our environment is facilitated by long-term memory cues. Further experimentation using alternative methods with higher spatial resolution may help characterise the brain areas involved in guiding target selection during memory-guided attention.
In summary, the present study provides evidence about the role of explicit long-term contextual memories in optimizing visual search and in modulating the ongoing processing of incoming information by biasing neural activity related to target selection. Furthermore, the data imply that the spatial or contextual information from LTM facilitates target selection through a different top-down mechanism than that engaged by attention-directing perceptual cues. Whereas perceptual cues do not influence feature-based selection of targets, memory cues may facilitate identification of target features and substantially diminish the neural resources involved in this process. Furthermore, search for objects in cluttered environments based on explicit memories of specific target-context associations results in a different neural modulation of target identification than that observed when unconsciously memorized contextual relations guide visual search. Further experimentation aimed at comparing the neural mechanisms of top-down biases triggered by memory cues versus perceptual cues will be especially informative.
ACKNOWLEDGEMENTS
This research was supported by a Project Grant to A.C.N. from the Wellcome Trust. S.D. was supported by a Spain’s Ministry of Education and Science/FECYT postdoctoral grant and by a current postdoctoral contract from the Isidro Parga Pondal program (Xunta de Galicia, Spain). In addition, the research was supported by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre based at Oxford University Hospitals Trust Oxford University as part of the Cognitive Health Programme. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.
REFERENCES
- Becker MW, Rasmussen IP. Guidance of attention to objects and locations by long-term memory of natural scenes. Journal of Experimental Psychology. Learning, Memory, and Cognition. 2008;34(6):1325–38. doi: 10.1037/a0013650. [DOI] [PubMed] [Google Scholar]
- Biederman I. Perceiving real-world scenes. Science. 1972;177(4043):77–80. doi: 10.1126/science.177.4043.77. [DOI] [PubMed] [Google Scholar]
- Brignani D, Lepsien J, Nobre AC. Purely endogenous capture of attention by task-defining features proceeds independently from spatial attention. NeuroImage. 2010;51(2):859–66. doi: 10.1016/j.neuroimage.2010.03.029. [DOI] [PubMed] [Google Scholar]
- Brockmole JR, Castelhano MS, Henderson JM. Contextual cueing in naturalistic scenes: Global and local contexts. Journal of Experimental Psychology. Learning, Memory, and Cognition. 2006;32(4):699–706. doi: 10.1037/0278-7393.32.4.699. [DOI] [PubMed] [Google Scholar]
- Cabeza R, Ciaramelli E, Olson IR, Moscovitch M. The parietal cortex and episodic memory: an attentional account. Nature Reviews Neuroscience. 2008;9(8):613–25. doi: 10.1038/nrn2459. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chaumon M, Drouet V, Tallon-Baudry C. Unconscious associative memory affects visual processing before 100 ms. Journal of vision. 2008;8:1–10. doi: 10.1167/8.3.10. [DOI] [PubMed] [Google Scholar]
- Chaumon M, Hasboun D, Baulac M, Adam C, Tallon-Baudry C. Unconscious contextual memory affects early responses in the anterior temporal lobe. Brain research. 2009;1285:77–87. doi: 10.1016/j.brainres.2009.05.087. [DOI] [PubMed] [Google Scholar]
- Chaumon M, Schwartz D, Tallon-Baudry C. Unconscious learning versus visual perception: dissociable roles for gamma oscillations revealed in MEG. Journal of Cognitive Neuroscience. 2009;21(12):2287–99. doi: 10.1162/jocn.2008.21155. [DOI] [PubMed] [Google Scholar]
- Chun MM, Jiang Y. Contextual cueing: implicit learning and memory of visual context guides spatial attention. Cognitive psychology. 1998;36(1):28–71. doi: 10.1006/cogp.1998.0681. [DOI] [PubMed] [Google Scholar]
- Chun MM, Jiang Y. Top-Down Attentional Guidance Based on Implicit Learning of Visual Covariation. Psychological Science. 1999;10(4):360–5. [Google Scholar]
- Chun MM, Jiang Y. Implicit, long-term spatial contextual memory. Journal of Experimental Psychology. Learning, Memory, and Cognition. 2003;29(2):224–34. doi: 10.1037/0278-7393.29.2.224. [DOI] [PubMed] [Google Scholar]
- Chun MM, Phelps EA. Memory deficits for implicit contextual information in amnesic subjects with hippocampal damage. Nature neuroscience. 1999;2(9):844–7. doi: 10.1038/12222. [DOI] [PubMed] [Google Scholar]
- Ciaramelli E, Grady CL, Moscovitch M. Top-down and bottom-up attention to memory: a hypothesis (AtoM) on the role of the posterior parietal cortex in memory retrieval. Neuropsychologia. 2008;46(7):1828–51. doi: 10.1016/j.neuropsychologia.2008.03.022. [DOI] [PubMed] [Google Scholar]
- Ciaramelli E, Grady C, Levine B, Ween J, Moscovitch M. Top-down and bottom-up attention to memory are dissociated in posterior parietal cortex: neuroimaging and neuropsychological evidence. The Journal of Neuroscience. 2010;30(14):4943–56. doi: 10.1523/JNEUROSCI.1209-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Eimer M. The N2pc component as an indicator of attentional selectivity. Electroencephalography and clinical neurophysiology. 1996;99(3):225–34. doi: 10.1016/0013-4694(96)95711-9. [DOI] [PubMed] [Google Scholar]
- Hickey C, Di Lollo V, McDonald JJ. Electrophysiological indices of target and distractor processing in visual search. Journal of Cognitive Neuroscience. 2009;21(4):760–75. doi: 10.1162/jocn.2009.21039. [DOI] [PubMed] [Google Scholar]
- Hilimire MR, Mounts JRW, Parks NA, Corballis PM. Event-Related Potentials Dissociate Effects of Salience and Space in Biased Competition for Visual Representation. PLoS ONE. 2010;5(9):e12677. doi: 10.1371/journal.pone.0012677. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hillyard S, Anllo-Vento L. Event-related brain potentials in the study of visual selective attention. Proceedings of the National Academy of Sciences of the United States of America. 1998;95(3):781–7. doi: 10.1073/pnas.95.3.781. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jiang Y, Chun MM. Selective attention modulates implicit learning. Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology. 2001;54(4):1105–24. doi: 10.1080/713756001. [DOI] [PubMed] [Google Scholar]
- Jiang Y, Leung AW. Implicit learning of ignored visual context. Psychonomic Bulletin and Review. 2005;12(1):100–6. doi: 10.3758/bf03196353. [DOI] [PubMed] [Google Scholar]
- Johnson JS, Woodman GF, Braun E, Luck SJ. Implicit memory influences the allocation of attention in visual cortex. Psychonomic Bulletin and Review. 2007;14(5):834–9. doi: 10.3758/bf03194108. [DOI] [PubMed] [Google Scholar]
- Kiss M, Van Velzen J, Eimer M. The N2pc component and its links to attention shifts and spatially selective visual processing. Psychophysiology. 2008;45(2):240–9. doi: 10.1111/j.1469-8986.2007.00611.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kuo B-C, Rao A, Lepsien J, Nobre AC. Searching for targets within the spatial layout of visual short-term memory. The Journal of Neuroscience. 2009;29(25):8032–8. doi: 10.1523/JNEUROSCI.0952-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leblanc E, Prime DJ, Jolicoeur P. Tracking the location of visuospatial attention in a contingent capture paradigm. Journal of Cognitive Neuroscience. 2008;20(4):657–71. doi: 10.1162/jocn.2008.20051. [DOI] [PubMed] [Google Scholar]
- Luck SJ, Hillyard S. Spatial filtering during visual search: evidence from human electrophysiology. Journal of Experimental Psychology. Human Perception and Performance. 1994;20(5):1000–14. doi: 10.1037//0096-1523.20.5.1000. [DOI] [PubMed] [Google Scholar]
- Mazza V, Turatto M, Caramazza A. Attention selection, distractor suppression and N2pc. Cortex. 2009;45(7):879–90. doi: 10.1016/j.cortex.2008.10.009. [DOI] [PubMed] [Google Scholar]
- Olson IR, Chun MM. Temporal contextual cuing of visual attention. Journal of Experimental Psychology. Learning, Memory, and Cognition. 2001;27(5):1299–1313. doi: 10.1037//0278-7393.27.5.1299. [DOI] [PubMed] [Google Scholar]
- Olson IR, Chun MM. Perceptual constraints on implicit learning of spatial context. Visual Cognition. 2002;9(3):273–302. [Google Scholar]
- Olson IR, Chun MM, Allison T. Contextual guidance of attention: human intracranial event-related potential evidence for feedback modulation in anatomically early temporally late stages of visual processing. Brain. 2001;124:1417–25. doi: 10.1093/brain/124.7.1417. [DOI] [PubMed] [Google Scholar]
- Posner MI. Chronometric explorations of mind. Lawrence Erlbaum; Oxford, England: 1978. p. xiii, 271. [Google Scholar]
- Schankin A, Hagemann D, Schubö A. Is contextual cueing more than the guidance of visual-spatial attention? Biological Psychology. 2011;87(1):58–65. doi: 10.1016/j.biopsycho.2011.02.003. [DOI] [PubMed] [Google Scholar]
- Schankin A, Schubö A. Cognitive processes facilitated by contextual cueing: Evidence from event-related brain potentials. Psychophysiology. 2009;46(3):668–79. doi: 10.1111/j.1469-8986.2009.00807.x. [DOI] [PubMed] [Google Scholar]
- Schankin A, Schubö A. Contextual cueing effects despite spatially cued target locations. Psychophysiology. 2010;47(4):717–27. doi: 10.1111/j.1469-8986.2010.00979.x. [DOI] [PubMed] [Google Scholar]
- Seiss E, Kiss M, Eimer M. Does focused endogenous attention prevent attentional capture in pop-out visual search? Psychophysiology. 2009;46(4):703–17. doi: 10.1111/j.1469-8986.2009.00827.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sheldon S, McAndrews MP, Moscovitch M. Episodic memory processes mediated by the medial temporal lobes contribute to open-ended problem solving. Neuropsychologia. 2011;49(9):2439–47. doi: 10.1016/j.neuropsychologia.2011.04.021. [DOI] [PubMed] [Google Scholar]
- Sheldon S, Moscovitch M. Recollective performance advantages for implicit memory tasks. Memory. 2010;18(7):681–97. doi: 10.1080/09658211.2010.499876. [DOI] [PubMed] [Google Scholar]
- Stokes MG, Atherton K, Patai EZ, Nobre AC. Long-term memory prepares neural activity for perception. Proceedings of the National Academy of Sciences of the United States of America. 2012;109(6):E360–7. doi: 10.1073/pnas.1108555108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Summerfield JJ, Lepsien J, Gitelman DR, Mesulam MM, Nobre AC. Orienting attention based on long-term memory experience. Neuron. 2006;49(6):905–16. doi: 10.1016/j.neuron.2006.01.021. [DOI] [PubMed] [Google Scholar]
- Summerfield JJ, Rao A, Garside N, Nobre AC. Biasing Perception by Spatial Long-Term Memory. The Journal of Neuroscience. 2011;31(42):14952–60. doi: 10.1523/JNEUROSCI.5541-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Treisman A, Gelade G. A Feature-Integration of Attention. Cognitive Psychology. 1980;136:97–136. doi: 10.1016/0010-0285(80)90005-5. [DOI] [PubMed] [Google Scholar]
- Woodman GF, Luck S. Electrophysiological measurement of rapid shifts of attention during visual search. Nature. 1999 Aug;400:867–9. doi: 10.1038/23698. [DOI] [PubMed] [Google Scholar]
- Woodman GF, Luck SJ. Serial deployment of attention during visual search. Journal of Experimental Psychology. Human Perception and Performance. 2003;29(1):121–38. doi: 10.1037//0096-1523.29.1.121. [DOI] [PubMed] [Google Scholar]
- Yantis S, Jonides J. Abrupt visual onsets and selective attention: evidence from visual search. Journal of Experimental Psychology. Human Perception and Performance. 1984;7(5):937–47. doi: 10.1037//0096-1523.10.5.601. [DOI] [PubMed] [Google Scholar]