Complementary attentional components of successful memory encoding

Abstract

Attention during encoding improves later memory, but how this happens is poorly understood. To investigate the role of attention in memory formation, we combined a variant of a spatial attention cuing task with a subsequent memory fMRI design. Scene stimuli were presented in the periphery to either the left or right of fixation, preceded by a central face cue whose gaze oriented attention to the probable location of the scene. We contrasted activity for scenes appearing in cued versus uncued locations to identify: (1) regions where cuing facilitated processing, and (2) regions involved in reorienting. We then tested how activity in these facilitation and reorienting regions of interest predicted subsequent long-term memory for individual scenes. In facilitation regions such as parahippocampal cortex, greater activity during encoding predicted memory success. In reorienting regions such as right temporoparietal junction, greater activity during encoding predicted memory failure. We interpret these results as evidence that memory formation benefits from attentional facilitation of perceptual processing combined with suppression of the ventral attention network to prevent reorienting to distractors.

Introduction

Two of the most fundamental mechanisms in vision are memory and attention (Chun and Turk-Browne, 2007). Memory allows the brain to build on prior experience and improve object recognition (Li and DiCarlo, 2008; Woloszyn and Sheinberg, 2012); without this ability, the visual system would treat every visual feature, object, and scene anew, failing to capitalize on prevalent recurring properties in the environment (statistical regularities). Attention allows the brain to prioritize sensory information that is most relevant to behavior (Kanwisher et al., 2000; Kastner and Ungerleider, 2000; Chun et al., 2011); without such selection, the visual system would be crippled by irrelevant information competing for limited processing resources. Understanding how attention controls memory is thus critical for explaining how the brain can protect existing circuitry while encoding new information (Grossberg, 1980; Seitz et al., 2005).

Although attention and memory have been investigated extensively as separate topics, their interaction has been relatively neglected — despite this issue having been highlighted as one of three major questions for modern neuroscience (Kandel, 2006). This interaction is particularly integral to the theory that memory is an enduring trace of attentional processing (Chun et al., 2011; Johnson and Hirst, 1993). To realize such views in the brain, it is essential to better understand how attentional control networks enable successful memory encoding.

The role of attention in encoding has generally been inferred indirectly. Several brain regions respond differently during the encoding of items that are subsequently remembered versus forgotten (Wagner et al., 1998; Brewer et al., 1998). This variability in memory and neural events is often assumed to reflect elaborative encoding and increased attention to subsequently remembered items (Chun and Johnson, 2011; Turk-Browne et al., 2006; Kim, 2011).

Fewer studies have directly controlled the amount of attention available for encoding. One approach for manipulating attention is to impose a dual-task that divides attentional resources (Cowan and Wood, 1997). For example, subjects might perform a digit-monitoring task while concurrently trying to encode target stimuli such as words. Such divided-attention paradigms have produced varied results, with memory sometimes being impaired and sometimes not (Gopie et al., 2011). This inconsistency suggests that dual-tasks are not always successful in tying up attention, or do not occupy the type of resources required for memory encoding (Uncapher and Rugg, 2005; Kensinger et al., 2003).

A second approach for manipulating attention is to control orienting. Specifically, a target location or object (in the case of overlapping items) is cued to instruct subjects what is likely to be task-relevant. Objects appearing in a cued location are responded to more quickly and accurately (Posner et al., 1980). This attentional facilitation can also affect how well an item is encoded into memory. For example, while being presented with overlapping shapes of two colors, attending to one color and ignoring the other color enhances memory for the attended shape and impairs memory for the unattended shape (Rock and Gutman, 1981; see also MacDonald and MacLeod, 1998). Neural measures of encoding such as repetition attenuation — the lower fMRI response to previously experienced versus novel stimuli (a.k.a. repetition suppression, fMRI-adaptation; Schacter et al., 2007; Henson, 2003; Wiggs and Martin, 1998; Grill-Spector et al., 2006; Turk-Browne et al., 2008) — correlate with long-term memory (Turk-Browne et al., 2006) and are also enhanced by attention. For example, repetition attenuation occurs in scene-selective visual cortex when attention is directed to the scene but not the face component of a repeated composite face-scene image (Yi and Chun, 2005; Yi et al., 2006). Likewise, attending to a location in space produces more repetition attenuation for items appearing at that versus other locations (Eger et al., 2004).

This prior work convincingly shows that attention modulates memory, but how? The control of attention is coordinated by several regions of frontal and parietal cortex (Corbetta et al., 2000; Curtis and D’Esposito, 2003; Hopfinger et al., 2000; Woldorff et al., 2004). These regions can be divided into two interactive and overlapping networks (Corbetta and Shulman, 2002): the dorsal attention network, which includes the superior parietal lobule (SPL), intraparietal sulcus (IPS), and frontal eye fields (FEF); and the ventral attention network, which includes the temporoparietal junction (TPJ), middle frontal gyrus (MFG), and inferior frontal gyrus (IFG). The dorsal network implements goal-directed attention, sending top-down signals into perceptual brain areas to facilitate processing that favors current goals or task rules (Miller and Cohen, 2001; Noudoost et al., 2010). The ventral network contributes to stimulus-driven attention, playing an important role in detecting and shifting attention to unexpected but behaviorally relevant stimuli (Corbetta et al., 2008).

The role of these networks in memory encoding has only recently been investigated (Uncapher et al., 2011). In this study, an arrow cue oriented attention in space, followed by a target object that appeared in either a cued or uncued location. Conjunction analyses revealed that the dorsal network was engaged both by the orienting cues and by cued targets that were subsequently remembered, while the ventral network was engaged both by targets appearing at uncued locations and by cued targets that were subsequently forgotten. These results were interpreted as evidence that both dorsal and ventral attention networks influence memory encoding, but in opposite ways: top-down attention mediated by the dorsal network enhances encoding, while bottom-up attention mediated by the ventral network hinders encoding.

Here we use a complementary approach to further validate and enrich our understanding of how attention enhances memory formation. In particular, we focus on two attentional components of target processing, facilitation and reorienting. Facilitation refers to the improved extraction of information when attention is allocated to the location or features of a stimulus, and is reflected in greater neural response amplitudes in sensory cortical areas specialized for the target (Yantis and Serences, 2003; Reynolds et al., 2004; Maunsell and Treue, 2006). Reorienting refers to the shifting of attention to initially unattended and unexpected stimuli, especially when behaviorally relevant, and is reflected in greater responses in TPJ and the rest of the ventral attention network (Corbetta et al., 2008).

These two components may reflect opposite sides of the same coin during target processing. Targets preceded by an accurate cue should be accompanied by strong facilitation and weak reorienting, since attention is already allocated to the correct location. In contrast, targets preceded by an inaccurate cue should be accompanied by weak facilitation and strong reorienting, since attention must be shifted to the target. Our central proposition is that there is variance over time in the relative balance between facilitation and reorienting even among cued stimuli (see Leber, 2010), and that this variance will be predictive of memory. That is, successful memory encoding is more likely when the brain is in a ‘focused’ attentional state of relatively strong facilitation and weak reorienting, whereas failed encoding is more likely when the brain is in a ‘diffuse’ attentional state that helps monitor the broader environment but results in weaker facilitation of task-relevant items and stronger reorienting to unexpected items. To test this hypothesis, we defined facilitation and reorienting regions based on responses to items appearing in cued versus uncued locations, and asked whether variability in these regions was related to subsequent memory of cued items. We predict that: (1) facilitation regions will respond more strongly during the encoding of subsequently remembered items; and (2) reorienting regions will respond more strongly during the encoding of subsequently forgotten items.

Materials and Methods

Subjects

Thirty-one subjects (14 female; mean age 22.5, range 18–35) participated in this experiment. All subjects were neurologically intact with normal or corrected-to-normal vision. Informed consent was obtained for all subjects, and the study protocols were approved by the Human Investigation Committee of the Yale School of Medicine and the Human Subjects Committee of the Faculty of Arts and Sciences at Yale University.

Apparatus

Stimuli were generated using the Psychtoolbox extension (Brainard, 1997) for Matlab (The Mathworks Inc., Natick, MA). During fMRI scanning, stimuli were displayed with an LCD projector onto a screen mounted in the rear of the scanner bore, which subjects viewed from a distance of 79cm via a mirror attached to the head coil (maximal field of view: 23.5°). Eye position was monitored using a modified ISCAN eye-tracking system (ISCAN Inc., Burlington, MA), in which the camera and infrared source were attached to the head coil above the mirror. Pupil and corneal reflection (CR) were recorded at 60Hz, and gaze angle (pupil-CR) was computed online to confirm accurate fixation.

Procedure

Main task

Each trial began with a fixation dot at the center of the screen, flanked by two placeholder boxes (6.67° × 6.67°), centered 5° to the left and right of fixation (Fig. 1). Subjects were instructed to keep their eyes fixated on the fixation dot at all times. The placeholders indicated the possible stimulus locations, and subjects were instructed that their appearance was a signal that the trial was about to begin. After 1s, a cue appeared at fixation for 200ms. The cue was a modified ‘average’ face (http://www.beautycheck.de), subtending 2.8° × 4° with the eyes looking left or right (orienting cues), or straight ahead (neutral cue). Face cues were used instead of symbolic cues (e.g., an arrow) to minimize overlap with scene processing areas in visual cortex. For instance, the PPA exhibits no response to faces but does respond somewhat to shapes and objects (e.g., Epstein et al., 1999). Moreover, face cues have been used in many studies of spatial attention, and gaze is a very powerful orienting cue (e.g., Friesen and Kingstone, 1998). Subjects were instructed to covertly shift attention to the cued side while remaining fixated.

Figure 1. Task design.

For each trial, subjects fixated a central dot while placeholder boxes appeared in the periphery. After the appearance of a face cue gazing left, right, or straight ahead, subjects were instructed to covertly shift attention to the cued placeholder and maintain attention at this location. A scene image appeared in one placeholder after a variable delay, along with a phase-scrambled image in the opposite placeholder. If the scene appeared on the cued side (Cued), subjects were instructed to remember it for a later test. If the scene appeared on the uncued side (Uncued) or appeared after straight ahead cue (Neutral), they did not need to remember it. Regardless of condition, subjects pressed a button whenever the double-stimulus array appeared. Color-coded feedback was provided on each trial to indicate eye fixation performance.

After a variable delay of 4.5–7.5s, a pair of images appeared in the placeholder boxes for 100ms. One image was a scene (room, building, or landscape), and the other was a phase-scrambled version of a scene. On 80% of orienting trials, the scene appeared on the cued side. For these Cued trials, subjects were instructed to attempt to memorize the scene, and were informed there would be a memory test for these scenes after the scan. For Uncued trials, where the scene appeared on the opposite side, or on neutral trials, where no side was cued, subjects were told they did not need to memorize anything. Regardless, on all trials, subjects were instructed to press a button as soon as the dual images appeared to ensure that they remained vigilant during the task. Responses were made with the index finger of the dominant hand, and no instructions were given about whether to prioritize encoding or button presses.

The purpose of instructing subjects to memorize Cued scenes was to ensure that they remembered enough scenes for a meaningful subsequent memory analysis. Pilot studies revealed that purely incidental encoding led to weak overall memory, possibly due to the peripheral presentation of stimuli, the number of stimuli, and the brief stimulus durations. We decided to instruct subjects not to intentionally encode Uncued scenes so that they would have to use the spatial orienting cues to perform the task. Given the difficulty of the encoding task, they may have disregarded cues entirely if asked to encode all scenes. Although this decision carries some limitations (e.g., we may have encouraged directed forgetting of Uncued scenes), these tradeoffs were mitigated by the fact that we were primarily interested in Cued scenes and that Uncued scenes strongly engaged the ventral attention network as in previous studies.

Trials were presented with jittered onset asynchronies of 9–15s. Cue and scene events were independently jittered, with intervals of 4.5, 6, or 7.5s, skewed towards the shorter intervals (40%, 40%, and 20%, respectively). That is, the face cue in the current trial appeared 4.5–7.5s after the scene in the previous trial, and the scene in the current trial appeared 4.5–7.5s after this cue. Across four runs of the task, there were a total of 96 trials with cued scenes, 24 trials with uncued scenes, and 24 trials with neutral cues (split evenly between left and right presentations). A unique scene was presented on every trial, and the order of scenes was randomized for each subject. The fixation dot remained on the screen between trials. There were also 24 fixation-only trials that were undetectable to subjects (fixation dot was constant).

Subsequent memory test

After the scanning session ended (minimum 10 minutes after final encoding trial), subjects were tested on their memory for the scenes. A total of 192 scenes (144 scenes from the main task and 48 completely new scenes) were presented one at a time in the center of the screen. Subjects made an unspeeded button press response indicating whether they strongly remembered seeing the scene during the main task (“old”), were fairly confident they had not seen the scene before (“new”), or did not feel confident either way (“unsure”). Every scene presented in the main task was probed in the memory test, allowing us to code each fMRI trial based on whether that particular image was subsequently remembered.

As a measure of sensitivity in the memory test, A′ was calculated for each subject and condition, based on the likelihood of responding “old” to scenes from the main task (hits) versus to scenes that were new (false alarms). Subjects with poor or unreliable behavioral performance in the memory test were excluded from all behavioral and fMRI analyses of the main task. The following exclusion criteria were used: A′ for Cued trials below chance (< 0.5), lower A′ for Cued than Uncued scenes (suggesting that the cue was ignored), or runs with no items that were subsequently remembered or no items that were subsequently forgotten.

Data Acquisition

MRI scanning was carried out with a Siemens Trio 3T scanner using an 8-channel receiver array head coil. Functional data were acquired with a T2*-weighted gradient-echo echo-planar imaging sequence (TR = 1500ms, TE = 25ms, flip angle = 90°, matrix = 64 × 64). Twenty-six oblique axial slices (3.5 × 3.5mm in-plane, 5mm thick) were taken oriented parallel to the anterior commissure-posterior commissure line, covering the entire brain. Each functional run contained 338 volumes. Functional data were co-registered to T1-weighted anatomical scans (high-resolution 3D MPRAGE and coplanar FLASH).

Data Analysis

Preprocessing was conducted using Brain Voyager QX (Brain Innovation B. V., Maastricht, Netherlands). The first six volumes of each functional run were discarded, and the remaining data were corrected for slice acquisition time and head motion, spatially smoothed with a 8mm FWHM kernel, temporally high-pass filtered with a 128s period cutoff, normalized into Talairach space (Talairach and Tournoux, 1988), and interpolated into 3mm isotropic voxels.

Data were sorted into five conditions based on cue type and subsequent memory responses. Cued trials were classified for each subject according to whether the subject had subsequently responded “old” (Cued-Remembered), “new” (Cued-Forgotten), or “unsure” (Cued-Unsure). The fMRI data from Uncued and Neutral trials were not analyzed by subsequent memory because there were too few trials of each type. Data were collapsed across left and right scenes.

Regressors were defined by convolving the jittered onsets of face cue and scene events for each of the five conditions and a fixation condition with a canonical hemodynamic response function. Data were combined across runs within-subject using a fixed-effects general linear model (GLM), and across subjects at the group level with a random-effects GLM. Whole-brain contrasts were used to determine brain regions that showed differential responses to scenes as a function of attentional cuing. We focus on responses to the scenes rather than the cues because of our interest in facilitation and reorienting during scene processing, as well as due to a limitation of our design whereby Neutral cues did not provide an appropriate baseline. Specifically, an unforeseen problem with these cues was that they differed from Orienting cues in ways beyond not requiring a shift of attention: Neutral face cues made direct eye contact with the subject, whereas Orienting cues consisted of a face with diverted gaze. Eye contact is a powerful social signal that elicits robust brain activity (Pelphrey et al., 2004), thus confounding the comparison of Neutral and Orienting cues. This confound was apparent in whole-brain analyses of cue processing, reported for completeness in Supplementary Material (Fig. S1).

Regions that showed significantly greater responses to Cued than Uncued trials were classified as ‘facilitation’ regions. The term facilitation is used operationally to refer to processes that were enhanced by the cue, which may include both enhanced perception as a result of attentional modulation and intentional encoding based on the instruction to remember such scenes. Regions that showed significantly greater responses to Uncued than Cued trials were classified as ‘reorienting’ regions. We use the term reorienting broadly to refer to processes engaged when the scene appeared at an uncued location. Although it was not strictly required to attend to the scene on Uncued trials to perform the task (it did not need to be remembered, and the scrambled image was sufficient for the response), it seems likely that subjects nonetheless shifted attention to the scenes (either automatically or intentionally) because they were task-relevant on the vast majority of trials, they were the most interesting and coherent items in the display, and there was no disincentive for attending to them.

Given our interest in comparing Cued-Remembered versus Cued-Forgotten scenes within these regions of interest (ROIs), we contrasted only Cued-Unsure with Uncued when defining the ROIs to avoid non-independence errors. In supplementary material, we report the contrast of all Cued conditions with Uncued to demonstrate that using Cued-Unsure for ROI definition was reasonable (Fig. S2). Moreover, the number of Cued-Unsure and Uncued trials was comparable, t(23) = 1.46, p = 0.16, ensuring roughly equal statistical power in these conditions (there were 3× more Cued than Uncued trials overall). For each of the facilitation and reorienting ROIs, we tested whether they predicted subsequent memory by extracting and comparing the parameter estimates for Cued-Remembered versus Cued-Forgotten scenes using paired t-tests.

Additional whole-brain analyses were performed to explore subsequent memory effects outside of the ROIs. Regions exhibiting greater activity for remembered scenes were identified by the Cued-Remembered > Cued-Forgotten contrast (Brewer et al., 1998; Wagner et al., 1998), and regions exhibiting greater activity for subsequently forgotten scenes were identified by the Cued-Forgotten > Cued-Remembered contrast (Daselaar et al., 2004; Turk-Browne et al., 2006). Whole-brain contrasts were thresholded at p<0.05 cluster corrected, where the minimum voxel extent was determined for each contrast using a cluster-forming threshold of p<0.005 and after accounting for the smoothness of the data (Forman et al., 1995; Goebel et al., 2006). Subsequent memory effects within ROIs were compared using random-effects t-tests.

Eyetracking

Eye position was continuously tracked and recorded for each trial. Accurate fixation was defined as deviations of less than 2° from the central fixation dot. The eyetracker was calibrated at the beginning of the experiment and recalibrated as necessary between runs. Due to technical difficulties illuminating the eye, eyetracking data were unavailable for one subject used in the analyses of the main task. In addition to monitoring eye position in the scanner, all subjects were first trained on the task with an eyetracker outside the scanner (using a different set of scene stimuli). Subjects were also given feedback on their eyetracking performance after each trial during both practice and main tasks: the fixation dot briefly changed color to green (successful fixation for >90% of the trial), yellow (60–90% fixation), or red (<60% fixation).

Results

Behavioral Results

Memory test performance

Scenes from the main task were classified as hits (Remembered) when they received an “old” response and as misses (Forgotten) when they received a “new” response. Novel lures were classified as false alarms when they received an “old” response and as correct rejections when they received a “new” response. Items that received an “unsure” response were excluded from memory analyses because their associated memory strength is ambiguous (Wagner et al., 1998; Turk-Browne et al., 2006). Scenes were further separated based on whether they were Cued, Uncued, or appeared after a Neutral cue.

The test contained fewer new scenes (Novel) than old scenes (Cued, Uncued, and Neutral) in order to minimize interference and reduce the study-test interval (see also Turk-Browne et al., 2006; Preston et al., 2010). However, there was not an overall bias to respond “old”: collapsing across items, the rate of “old” responses (29%) was lower than of “new” responses (42%), t(30) = 2.77, p = 0.01. We assessed whether subjects reliably remembered scenes by comparing the hit rate in each condition against the false alarm rate. For Cued scenes, the mean sensitivity of memory responses (A′ = 0.63) was reliably above chance (0.5), t(30) = 7.85, p < 0.001 (Fig. 2A). For Uncued scenes (A′ = 0.52) and Neutral scenes (A′ = 0.54), reliability was not above chance, t <1 and t(30) = 1.68, p = 0.10, respectively. In addition, sensitivity was reliably higher for Cued scenes than Uncued scenes, t(30) = 6.23, p < 0.001, and Neutral scenes, t(30) = 3.50, p = 0.001.

Figure 2. Behavioral performance.

(A) In the subsequent memory test, subjects were presented with scenes from the main task and novel lure items, and rated each one as “old” (Remembered), “unsure” (Unsure), or “new” (Forgotten). The percentage of items receiving each response is provided for Cued, Uncued, Neutral, and Novel scenes. To measure sensitivity, the hit rate in each conditions (proportion Remembered) was compared against in the false alarm rate for lures using signal detection measure A′. Memory was reliable only for Cued scenes, and was better for these items than for Uncued and Neutral scenes. Error bars reflect ±1 SEM in each condition relative to chance (0.5). (B) In the main task, subjects responded whenever the double-stimulus array appeared. The mean response time (RT) is plotted by memory response for Cued trials and collapsing across memory responses for Uncued and Neutral trials. RTs were faster for the Cued versus Uncued trials. There were no reliable RT differences among Cued memory conditions. Error bars reflect ±1 within-subject SEM. **p<0.01, ***p<0.001.

These results confirm that our cuing and instructional manipulations worked as planned. Because there were many fewer Uncued and Neutral scenes, only Cued scenes were subdivided as Remembered or Forgotten in all subsequent analyses. Despite post-hoc coding, the number of Cued-Remembered and Cued-Forgotten trials did not differ across subjects, t < 1, suggesting that we had equivalent statistical power in each condition. To be conservative when making group-level inferences about whether our procedure elicited reliable memory performance, all subjects were included in these analyses of behavioral data from the memory test. However, in all other analyses of behavioral data and in all analyses of fMRI data, we excluded subjects who did not show reliable memory performance. Namely, we were interested in comparing successful versus unsuccessful encoding, but this distinction was meaningless in subjects with poor overall memory performance. Three exclusion criteria were used: Cued A′ less than chance (4 subjects), Cued A′ less than Uncued A′ (2 additional subjects), or no Cued-Remembered or Cued-Forgotten trials in one or more scanning runs (1 subject). This resulted in a sample of 24 subjects.

Main task performance

Behavioral responses from two subjects were not recorded during the main task due to technical problems. Although not recorded, the experimenter verified that the subjects were responding when necessary by monitoring an LED on the button box control unit. The fMRI data from the main task and the behavioral data from the memory test were recorded properly in these subjects, and thus they are excluded only from the analyses below.

Mean response time (RT) was faster for Cued trials (460 ms) than Uncued trials (503 ms), t(21) = 3.83, p < 0.001, but not Neutral trials (483 ms), t(21) = 1.21, p = 0.24 (Fig. 2B). The lack of difference between Cued and Neutral trials, which would be expected in standard cuing tasks, may result from an unexpected issue with the Neutral cues (Fig. S1), which made eye contact with subjects, and thus may have increased arousal and speeded responses.

Among Cued trials, mean RT for Cued-Remembered (468 ms) did not differ from Cued-Forgotten (465 ms), t < 1. Thus, fMRI differences between these conditions do not reflect time-on-task. Accuracy in the main task was at ceiling (Cued = 99.6%; Uncued = 100.0%; Neutral = 98.9%). Indeed, 13 of 24 subjects achieved perfect performance by responding to every trial.

Eyetracking was used to monitor eye position during each trial and to remind subjects about the importance of fixation. The mean percentage of time that gaze remained within 2° of fixation did not differ for Cued (88.5%), Uncued (88.6%), and Neutral trials (89.0%), ts < 1. Trials lasted several seconds, and thus some fixation breaks, blinks, and calibration errors were to be expected. Most importantly, fixation percentages did not differ across the critical conditions: Cued-Remembered (88.3%) versus Cued-Forgotten (89.0%), t(22) = 1.10, p = 0.28. Thus, fMRI differences between these conditions do not reflect patterns of fixation. We did not exclude individual trials for imperfect fixation behavior. However, we repeated the analyses including only subjects for whom calibration was accurate enough to ensure precise fixation (within 2°) greater than 85% of the time (19 subjects), and obtained the same pattern of fMRI results.

fMRI Results

Regions of interest

To localize regions where scene processing was modulated by the spatial attention cues, we conducted a whole-brain analysis comparing the blood oxygen level-dependent (BOLD) responses evoked on Cued-Unsure and Uncued trials. Facilitation effects were defined by greater responses for Cued-Unsure than Uncued (Fig. 3A), and were observed in five regions (corrected p < 0.05, min extent = 8 voxels; Talairach x/y/z coordinates): bilateral parahippocampal cortex (PHC; right: 25, −35, −6; left: −25, −35, −7), left retrosplenial cortex (RSC; −14, −52, 18), left transverse occipital sulcus (TOS; −35, −78, 22), and midline anterior cingulate cortex (ACC; −6, 39, −3).

Figure 3. Facilitation ROI analysis.

(A) To identify regions in which attention facilitated neural processing, Cued scenes were contrasted against Uncued scenes. Only Cued-Unsure scenes were included in this comparison, to preserve independence from subsequent memory analyses. A group random-effects GLM was used to identify reliable voxel clusters (p<0.05, cluster-size corrected). These clusters were then defined as ROIs in order to examine the role of attentional facilitation in memory encoding. (B) The mean evoked BOLD response for Cued-Remembered and Cued-Forgotten scenes was extracted from each ROI. Right (and to some extent left) PHC exhibited greater responses to Cued-Remembered than Cued-Forgotten. Error bars reflect ±1 within-subject SEM. †p<0.10, *p<0.05.

Reorienting effects were defined by greater responses on Uncued than Cued-Unsure trials (Fig. 4A), and were observed in nine regions (corrected p < 0.05, min extent = 6 voxels): bilateral temporoparietal junction (TPJ; right: 45, −49, 37; left: −48, −47, 41), bilateral superior temporal sulcus and middle temporal gyrus (STS; right: 58, −34, −1; left: −65, -−26, 4), a large cluster covering right middle frontal gyrus and frontal eye fields (MFG/FEF; 31, 11, 41), left MFG (−38, 20, 33), left FEF (−22, −1, 57), right orbitofrontal cortex (OFC; 45, 30, −20), and midline precuneus (PCU; 3, −54, 44).¹

Figure 4. Reorienting ROI analysis.

(A) To identify regions involved in reorienting to unexpected scenes, Uncued scenes were contrasted against Cued scenes. Again, Uncued was only compared against Cued-Unsure to preserve independence. A group random-effects GLM was used to identify reliable voxel clusters (p<0.05, cluster-size corrected). These clusters were then defined as ROIs in order to examine the role of attentional reorienting in memory encoding. (B) The mean evoked BOLD response for Cued-Remembered and Cued-Forgotten scenes was extracted from each ROI. Right TPJ and PCU (and to some extent right MFG/FEF and STS) exhibited greater responses to Cued-Forgotten than Cued-Remembered. Error bars reflect ±1 within-subject SEM. †p<0.10, *p<0.05.

Subsequent memory ROI analysis

To examine the involvement of these facilitation and reorienting regions in memory encoding, we defined them as group ROIs. For each ROI and subject, we extracted the evoked BOLD responses for Cued-Remembered and Cued-Forgotten conditions (neither of which was used to define the ROI) to look at subsequent memory effects.

Among the five facilitation ROIs (Fig. 3B), the right PHC showed a significantly greater response to Cued-Remembered versus Cued-Forgotten, t(23) = 2.57, p = 0.02. The left PHC showed a similar effect that was marginally significant, t(23) = 1.75, p = 0.09. The remaining regions showed numerical differences in the same direction, but they did not reach significance: left RSC, t < 1; left TOS, t(23) = 1.04, p = 0.31; ACC, t(23) = 1.56, p = 0.13.

Among the nine reorienting ROIs (Fig. 4B), two regions showed a significantly greater response to Cued-Forgotten versus Cued-Remembered: right TPJ, t(23) = 2.45, p = 0.02; and precuneus (PCU), t(23) = 2.78, p = 0.01. Two additional regions showed marginally significant effects in the same direction: right MFG/FEF, t(23) = 1.96, p = 0.06; and right STS, t(23) = 1.98, p = 0.06. None of the other ROIs showed a subsequent memory effect, ts < 1.

Weaker responses for Cued-Remembered than Cued-Forgotten could reflect deactivation for Cued-Remembered relative to baseline instead of activation for Cued-Forgotten. Such encoding deactivations have been observed in nearby frontoparietal regions (Daselaar et al., 2004; Turk-Browne et al., 2006). We thus compared Cued-Remembered and Cued-Forgotten to the Fixation condition, which provided an empirical estimate of the baseline BOLD response in our rapid event-related design. In the four reorienting ROIs with at least marginal differences between Cued-Forgotten and Cued-Remembered, there was no reliable deactivation for Cued-Remembered relative to Fixation, one-tailed ps > 0.22. Moreover, there was significant activation for Cued-Forgotten relative to Fixation in three of these ROIs, one-tailed ps < 0.002; just not right STS, t < 1. Thus, the difference between Cued-Forgotten and Cued-Remembered in right TPJ, PCU, and right MFG/FEF is unlikely to reflect encoding deactivation from baseline.

Subsequent memory whole-brain analysis

We focused on subsequent memory effects in the ROIs because of our specific hypotheses about these regions, and to increase statistical power and reduce multiple comparisons. Nevertheless, we also conducted an exploratory analysis of subsequent memory effects throughout the whole brain.

Two regions responded more strongly to Cued-Remembered versus Cued-Forgotten items (corrected p < 0.05, min extent = 7 voxels; Fig. 5A): right PHC (21, −31, −7) and left inferior frontal gyrus (IFG; −46, 24, 12). These results replicate several previous subsequent memory studies, with both scene (Brewer et al., 1998; Turk-Browne et al., 2006) and word stimuli (Wagner et al., 1998). The right PHC region overlapped with the region obtained from the contrast of Cued-Unsure greater than Uncued, suggesting that it reflects attentionally facilitated processing of the scene. The left IFG has been linked to enhanced semantic elaboration of to-be-remembered information (Otten et al., 2001).

Figure 5. Exploratory whole-brain analyses.

To examine subsequent memory effects outside of the facilitation and reorienting ROIs, we contrasted Cued-Remembered versus Cued-Forgotten scenes. A group random-effects GLM was used to identify reliable voxel clusters (p<0.05, cluster-size corrected). (A) Two regions observed in past subsequent memory fMRI studies responded more strongly to Cued-Remembered than Cued-Forgotten. (B) The reverse comparison of Cued-Forgotten greater than Cued-Remembered revealed several regions, including parts of the ventral attention network. The same color scale applies to both panels.

Six regions responded more strongly to Cued-Forgotten versus Cued-Remembered items (corrected p < 0.05, min extent = 5 voxels; Fig. 5B): right TPJ (51, −40, 38), right inferior temporal cortex (IT; 50, −32, −14), posterior cingulate cortex (PCC; 2, −21, 37), PCU (8, −63, 38), and two clusters in right cerebellum (CB; −16, −24, −31; −42, −44, −29). The TPJ and PCU regions overlapped with the regions obtained from the contrast of Uncued greater than Cued-Unsure, suggesting that they reflect increased reorienting during these trials. We thus replicated known subsequent memory effects, in a context where variability in memory may reflect variability in the control of attention, in addition to demonstrating a new pattern of complementary subsequent memory effects in our facilitation and reorienting regions.

Discussion

Prior studies have established that attention is important for behavioral and neural manifestations of memory (Chun and Turk-Browne, 2007). To elucidate the underlying component processes, we combined a spatial attention cuing task with a subsequent memory fMRI design. We tested two hypotheses about how cuing would affect scene processing: (1) that facilitation regions would respond more strongly to subsequently remembered scenes; and (2) that reorienting regions would respond more strongly to subsequently forgotten scenes. These two predictions were supported by several results.

The behavioral data verified that subjects used the spatial orienting cues to shift attention and prepare for the scene: Cued scenes were both detected faster and remembered better than Uncued scenes. Cuing effects in behavior were accompanied by changes in the neural processing of scenes, as measured by the contrast of Cued versus Uncued trials. The network of brain regions obtained from this contrast, including PHC, RSC, and TOS, is similar to the scene-processing network obtained by contrasting scenes versus other object categories (Aguirre et al., 1998; Epstein and Kanwisher, 1998; Epstein et al., 2005; Park and Chun, 2009). Therefore, this replicates prior findings that selective attention (in this case, spatial) to a visual category enhances evoked activity within sensory brain areas specialized for that category (O’Craven et al., 1999; Yi et al., 2006; Norman-Haignere et al., 2012). Because scenes appeared regardless of cue direction, these effects are a pure consequence of top-down attention.

The same conditions also allowed us to identify brain regions where uncued scenes were processed more than cued scenes, including TPJ, MFG, FEF, and PCU. These regions are among the core regions of the frontoparietal attention network, responsible for orienting attention to locations and objects (Woldorff et al., 2004; Corbetta et al., 2000; Hopfinger et al., 2000). In particular, they anchor the ventral attention network in the right hemisphere, which detects behaviorally relevant information at unexpected locations (Corbetta et al., 2008). We observed responses to uncued scenes despite the fact that there was no need to reorient to them: The phase-scrambled distractor image at the cued location was equally relevant for the task of detecting double-image arrays, and subjects were instructed not to attend or remember uncued scenes. Nevertheless, the ventral network and reorienting processes may have been engaged on uncued trials because scenes were task-relevant on other trials, and scenes at uncued locations shared features with expected targets at the cued location (Serences et al., 2005).

After obtaining regions in which cuing affected scene processing, we then examined whether activity in these regions could predict the likelihood of remembering cued scenes. The logic of this analysis was that activity in these regions may index attentional facilitation and reorienting, such that variability in memory that correlates with this activity might be attributable to variability in these attentional processes. For example, in regions where cuing enhanced scene processing in general, the likelihood of remembering a particular cued scene might be predictable from the amount of facilitation on that trial. Indeed, we found greater activity in the PHC for scenes that were later remembered versus forgotten.

While prior subsequent memory studies have been able to predict memory from PHC activity (Brewer et al., 1998; Wagner et al., 1998; Turk-Browne et al., 2006; Yoo et al., 2012), the reasons why have remained mysterious. For example, item analyses in prior studies have ruled out the possibility that predictability is a consequence of inherent memorability. Our data suggest that variability in memory arises from variability in the orienting and/or focus of spatial attention, which in turn introduces variable amounts of attentional modulation in PHC. Indeed, this interpretation is supported by findings that the dorsal attention network — possibly the source of modulation in PHC and other visual areas (Noudoost et al., 2010) — is more involved during successful memory formation (Uncapher et al., 2011). Given that spatial attention can influence baseline activity in visual cortex (Kastner et al., 1999), this explanation also accounts for why PHC activity even before an item is even delivered can predict successful encoding (Turk-Browne et al., 2006; Yoo et al., 2012).

Similar to the logic above, in regions where accurate cuing suppressed processing in general, the likelihood of remembering a particular cued scene might be predictable from the amount of suppression on that trial. This is precisely what we found in right TPJ, PCU, and to a weaker extent, in right MFG/FEF and STS, where responses were greater when scenes were later forgotten versus remembered. These results reflect a full crossover interaction with the PHC: the same remembered items were associated with higher activity in facilitation regions and lower activity in reorienting regions. We interpret these results as evidence that reduced engagement of the ventral attention network improves memory encoding. Indeed, while several regions showed up bilaterally in response to uncued scenes, the effects of subsequent memory were limited to the right hemisphere, where the ventral attention network is dominant. Moreover, this result provides further support for the negative relationship between the ventral attention network and memory encoding (Uncapher et al., 2011), using a different paradigm, a complementary analysis approach, and tighter eye movement controls.

Why was the ventral attention network more engaged during unsuccessful memory formation? One possibility is that this activity reflects reorienting to the scene when it appeared. That is, forgotten scenes may not have been cued as well as remembered scenes, with attention initially allocated to the wrong location or, due to a long cue-target interval, initially allocated to the cued location but shifted away because of mind-wandering or a lapse of vigilance. Different regions of the ventral network could implement one or more specific components of reorienting to forgotten scenes, such as detecting the violation of expectation when a scene appeared at an unattended location, or shifting the locus of attention to this new location (Shulman et al., 2009).

Alternatively, greater ventral network activity could reflect distraction during scene encoding. That is, even if attention was initially allocated to the cued location of forgotten scenes, distractors may have captured attention after the scene appeared. Indeed, lower TPJ activity (as for our remembered scenes) is related to reduced distractor interference (Todd et al., 2005; Shulman et al., 2007; Anticevic et al., 2010). This role of the TPJ in resisting distraction and protecting target representations in working memory may further enhance encoding into long-term memory. This interpretation is consistent with theoretical and meta-analytic approaches in the study of episodic memory (Wagner and Davachi, 2001; Cabeza et al., 2008; Uncapher and Wagner, 2009; Kim, 2011). Relatedly, perceptual attention to external distractors may trade off with reflective attention to internal thoughts (Chun et al., 2011), with suppression of the ventral network resulting in increased reflective attention and enhanced encoding of target information (Chun and Johnson, 2011; Todd et al., 2005). By conceptualizing memory as a consequence of attentional processing (Chun and Johnson, 2011), frontoparietal attention networks may ultimately be no less important for successful memory formation than dedicated memory systems.

Highlights.

• How does attention improve the encoding of items into long-term memory?

• Facilitation and reorienting are two attentional processes that operate at encoding

• Remembered items elicit more activity in areas where attention facilitates processing

• Forgotten items elicit more activity in areas that help reorient spatial attention

• Memory formation benefits from suppression of the ventral attention network