Influence of early attentional modulation on working memory

Abstract

It is now established that attention influences working memory (WM) at multiple processing stages. This liaison between attention and WM poses several interesting empirical questions. Notably, does attention impact WM via its influences on early perceptual processing? If so, what are the critical factors at play in this attention-perception-WM interaction. I review recent data from our laboratory utilizing a variety of techniques (electroencephalography (EEG), functional MRI (fMRI) and transcranial magnetic stimulation (TMS)), stimuli (features and complex objects), novel experimental paradigms, and research populations (younger and older adults), which converge to support the conclusion that top-down modulation of visual cortical activity at early perceptual processing stages (100–200 ms after stimulus onset) impacts subsequent WM performance. Factors that affect attentional control at this stage include cognitive load, task practice, perceptual training, and aging. These developments highlight the complex and dynamic relationships among perception, attention, and memory.

1. Introduction

“The true art of memory is the art of attention.”

- Samuel Johnson (1759)

As appreciated by British author Samuel Johnson over 250 years ago, memory and attention are intricately interwoven. Today, it is known that the relationship between memory and attention is bi-directional and multifaceted. The goal of this review is to focus only on that which is elegantly depicted by Johnson—the influence that attention has on memory, specifically, on working memory (WM) performance. WM is a theoretical construct that defines our ability to maintain and manipulate information in mind for brief periods of time to guide subsequent behavior (Baddeley, 2003). It has been shown that selective attention, the ability to focus our cognitive resources on information that is relevant to our goals, influences WM at multiple stages of processing. This includes the preparatory period before a memory task (Bollinger, Rubens, Zanto, & Gazzaley, 2010 ; Schmidt, Vogel, Woodman, & Luck, 2002), the selection and encoding of stimuli when encountered (Vogel & Machizawa, 2004; Vogel, McCollough, & Machizawa, 2005), the maintenance of relevant information in mind (Awh, Jonides, & Reuter-Lorenz, 1998; Jha, 2002; Postle, Awh, Jonides, Smith, & D’Esposito, 2004) and memory retrieval (Theeuwes, Kramer, & Irwin, 2010).

To better understand the relationship between attention and WM, it is necessary to consider the timing of attentional influences on stimulus processing and determine how this impacts subsequent memory. This is reminiscent of the long-standing debate that questioned if attentional selection targeted early perceptual stages of sensory processing (within 200 milliseconds after stimulus onset) or later post-perceptual stages (Broadbent, 1958; Duncan, 1980; Treisman, 1969). An extensive literature has now revealed that attention modulates the processing of visual stimuli at both early (i.e. time periods involved in basic stimulus identification) (Hillyard, Vogel, & Luck, 1998) and later phases of sensory processing (Eimer, 1996). In human studies, early influences were first documented by showing spatial attention modulated neural activity as early as 100 ms after stimulus presentation (P1 component of the event-related potential (ERP)) (Van Voorhis & Hillyard, 1977). The modulation of activity based on task goals, known as top-down modulation, has now also been shown to occur as early as 100–200ms after stimulus onset for attention to stimulus features (Schoenfeld, et al., 2003; Zanto & Gazzaley, 2009) and whole objects (Martinez, Teder-Salejarvi, & Hillyard, 2007; Rutman, Clapp, Chadick, & Gazzaley, 2010), and has been localized to visual association cortices (Di Russo, Martínez, Sereno, Pitzalis, & Hillyard, 2002).

As described above, there is now physiological evidence that attention targets both perceptual and post-perceptual stages of sensory processing. But the question remained as to which of these stage(s) of attentional modulation translates to subsequent memory of those stimuli. Studies using the attentional blink paradigm have supported a role for late processing by showing that the P1 (~100 ms) is not influenced by attentional blink interference that disrupts WM, while the later occurring P3 component (~300 ms) is suppressed (Vogel, Luck, & Shapiro, 1998). This was described in a recent article on the interaction of attention and WM: “…attention can operate at a relatively late stage of processing, to determine which stimuli will gain access to working memory after the completion of early perceptual and semantic processing” (Awh, Vogel, & Oh, 2006). Of note, these data reveal that attentional influences on later processing stages can influence WM, but do not preclude an influence by earlier states of processing.

To explore the role of selective attention on WM performance, our laboratory has been studying the modulation of visual cortical activity while research participants are engaged in novel selective attention, delayed-recognition tasks. In these paradigms, both relevant and irrelevant visual stimuli are presented (selective attention), with the relevant information being maintained over a several-second delay period (WM). We address temporal and anatomical aspects of object- and feature- based attentional modulation by recording correlates of neural activity using electroencephalography (EEG) and functional MRI (fMRI). Both ERPs, obtained by time-locking and then averaging EEG signals, and the Blood Oxygen Level Dependent (BOLD) signal recorded with fMRI, reflect cortical activity driven by local processing and the summation of postsynaptic potentials on synchronously active, large ensembles of neurons (Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001; Silva, 1991). While changes in fMRI signals can be localized to cortical regions separated by millimeters, EEG can resolve changes in neural activity on the level of milliseconds. Thus, the temporal and spatial resolutions of these techniques offer complementary information about top-down modulation.

In this review, I will present a synthesis of recent contributions from our laboratory that elucidates the relationship between early activity modulation in sensory cortices driven by attentional goals and subsequent WM performance. In the process, I briefly entertain the question of why successful cognitive operations demand such rapid modulation. This review includes a convergence of results documenting: 1) The influence of early processing of interfering stimuli on WM, 2) The role of cognitive load, interference, task practice and perceptual training in modifying early processing to alter performance, 3) Alterations at this early processing stage negatively impacting WM performance in older adults, and 4) The role of prefrontal control regions in causally controlling early activity modulation to influence subsequent memory.

2. Basic mechanisms of top-down modulation

The first experimental paradigm we utilized to explore this question consisted of three tasks in which participants viewed four sequentially presented stimuli, two faces and two natural scenes, in a randomized order (Figure 1) (Gazzaley, Cooney, McEvoy, Knight, & D'Esposito, 2005). The tasks differed in their instructions informing the participants how to process the stimuli on each trial: 1) Remember Faces and Ignore Scenes, 2) Remember Scenes and Ignore Faces, and 3) Passively View Faces and Scenes without attempting to remember or evaluate them. All trials were followed by a brief delay period and, in the memory trials, a response period requiring participants to report whether the probe stimulus matched one of the previously presented, relevant cued stimuli. fMRI measures associated with stimuli presented during the “cue” period, revealed that modulation of visual cortical activity in stimulus-selective regions was driven by object-based attention, as documented by other studies (O'Craven, Downing, & Kanwisher, 1999; Vuilleumier, Armony, Driver, & Dolan, 2001; Wojciulik, Kanwisher, & Driver, 1998). In our studies, the inclusion of the passive view condition enables us to record both increases (enhancement) and decreases (suppression) of activity measures relative to that associated with the passive view baseline task (Gazzaley, Cooney, McEvoy, et al., 2005). Our results revealed that top-down modulation involves both enhancement of activity associated with stimuli that are relevant to task goals, and suppression of activity associated with stimuli that are irrelevant (this occurred only in a scene-selective cortical region; a face-selective region exhibited enhancement only). Using the same basic paradigm, we tested the effects of motivation on top-down modulation and visual WM by manipulating the monetary reward participants received on different task blocks (Krawczyk, Gazzaley, & D'Esposito, 2007). Reward significantly reduced response time on correct trials and amplified both enhancement and suppression of activity in the scene-selective region. This suggests that reward motivation can play a pivotal role in driving WM performance via top-down modulation during stimulus encoding.

Figure 1.

Basic experimental paradigm with the following timing parameters. 1) Cue stimuli: 800 ms, time between cue stimuli: 200 ms, Delay period: 9 sec, Probe stimuli: 1000 ms (Gazzaley, et al., 2008; Gazzaley, Cooney, McEvoy, et al., 2005; Gazzaley, et al., 2007). 2) Cue stimuli: 800 ms, time between cue stimuli: 200 ms, Delay period: 7 sec, Probe stimuli: 1000 ms (Rissman, et al., 2009). 3) Cue stimuli: 400 ms, time between cue stimuli: 600 ms, Delay period: (jittered) 8,10,12 sec, Probe stimuli: 2 sec (Krawczyk, et al., 2007).

Most germane to this review, we capitalized on the high temporal resolution of EEG to study the timing of top-down modulation on visual processing using this selective attention, delayed-recognition task. Analysis revealed early modulation of activity based on attention to face stimuli, as indicated by modulation of the amplitude of the P1 component (100 ms) and the latency of the N170 component (170 ms) in posterior electrodes (Gazzaley, et al., 2008; Gazzaley, Cooney, McEvoy, et al., 2005). Comparable to the fMRI data, this early activity modulation exhibited both enhancement and suppression relative to the passive view baseline task. Modulation of activity as early as 100ms had been observed largely for spatial attention (Hillyard, et al., 1998), and more recently for feature-based attention (Schoenfeld, et al., 2007). Object-based studies using illusory surface paradigms have also documented significant modulation of the P1 component (Valdes-Sosa, Bobes, Rodriguez, & Pinilla, 1998), but modulation at such an early perceptual phase for complex real-world objects, such as faces, had not been documented. Moreover, while modulation of the magnitude of neural activity has served as the foundation for our understanding of top-down modulation of cortical activity, less has been documented about top-down influences on the speed of activation. An attention-mediated shift in N170 latency reflects top-down modulation of the speed of neural processing, likely mediated by a shorter time to reach maximal synchronized neural activity in visual cortex. An ERP latency shift reflects the sum contribution of large ensembles of neurons, a population measure of processing speed that may not be readily observable at the single neuron level (Silva, 1991). This may account for the lack of observed latency differences in attention studies utilizing single-cell recording techniques to assess spike activity. It has been proposed that amplification of activity magnitude improves signal-to-noise ratio, allowing more information to be extracted from relevant stimuli (Hillyard, et al., 1998). Likewise, faster processing speed may reflect an augmentation in the efficiency of neural processing, further facilitating information extraction (Titchener, 1908). We assert that by modulating both activity magnitude and processing speed, top-down signals bias the likelihood of successful representation of relevant information in a competitive system.

Using this selective attention, delayed-recognition paradigm, we extend our understanding of the mechanistic basis of top-down modulation by revealing: 1) Activity modulation driven by attention to complex, real world objects occurs as early as 100 ms, 2) Both activity magnitude (i.e., amplitude increases) and speed of processing (i.e., the latency shifts) at early perceptual stages are modulated by top-down influences, 3) Modulation at these time points occurs both above (i.e., enhancement) and below (i.e., suppression) the perceptual baseline level of activity, and 4) Manipulating reward levels result in greater enhancement and suppression of relevant and irrelevant stimuli and improved WM performance. Although of mechanistic value, these studies did not establish a direct connection between early top-down modulation of activity and WM performance. The absence of neural-behavioral relationships was presumably the result of the WM tasks not being demanding enough to reveal a range of performance measures. However, they did launch both a paradigmatic and analytical approach to studying top-down modulation and WM in our laboratory, and placed an emphasis on early measures of stimulus processing, which can serve as functional biomarkers to explore the influence of attention on WM in a variety of experimental manipulations.

3. Influence of interfering stimuli

The experimental paradigm illustrated in Figure 1 was modified to increase task difficulty and facilitate a direct assessment of the role that early attentional modulation has on WM performance (Rutman, et al., 2010). In a revised paradigm, the task goals and overall design were preserved, but participants were presented with overlapping, “double-exposure” images of faces and scenes with instructions to either remember the faces or the scenes while simultaneously ignoring the other stimuli (Figure 2). This is comparable to the approach used in other studies of object-based attention (Furey, et al., 2006; O'Craven, et al., 1999; Serences, Schwarzbach, Courtney, Golay, & Yantis, 2004; Yi & Chun, 2005) and increased the difficulty of the WM tasks. Consistent with our previous finding of activity modulation within 100 ms for attention to sequentially presented faces, this study also revealed significant modulation of the P1 amplitude based solely on the goal of remembering or ignoring the faces (i.e., bottom-up, perceptual information was equivalent). Importantly, in this study we were able to capitalize on individual performance differences to reveal that the degree to which participants modulated the P1 amplitude during stimulus encoding predicted their subsequent WM recognition accuracy (r = .45, p < .05) (Figure 3). This finding represents the first correlation between early neural measures of selective activity modulation in visual cortices during encoding and subsequent WM performance. Our interpretation is that early modulation of cortical activity reduces interference and biases perceptual processing toward higher fidelity sensory representations of relevant stimuli that then confers an advantage in maintaining that information in mind.

Figure 2.

Basic experimental paradigm with the following timing parameters. 1) Cue stimuli: 800 ms, time between cue stimuli: 200 ms, Delay period: 4 sec, Probe stimuli: 500 ms (Rutman, et al., 2010). 2) Cue stimuli: 800 ms, time between cue stimuli: 400 ms, Delay period: 8 sec, Probe stimuli: 1000 ms (Chadick & Gazzaley, 2008; Chadick & Gazzaley, Submitted).

Figure 3.

Top-down modulation of the P1 component and its relationship with WM performance. A, Grand Average waveform of P1 (n=19); B, P1 peak amplitudes (n=19); C, Neural-behavioral correlation. All peak amplitudes of memory tasks show significant differences across tasks (PV-O is not significantly different than FM-O or SM-O). Error bars represent standard error of the mean. Asterisks denote significant difference (single - p<0.05, double, - p<0.01, triple, p<0.0001. C, Measures of attentional modulation (P1 Modulation Index) correlate significantly with working memory recognition (Accuracy Index). Participants with greater attentional modulation of P1 amplitude (~100ms post-stimulus presentation) show greater subsequent memory of encoded stimuli (R=0.45, p<0.05). Face memory-overlap (FM-O), Scene memory-overlap (SM-O), Passive view-overlap (PV-O), Face memory (FM), Scene memory (SM). Modified from (Rutman, et al., 2010).

We extended this evaluation to feature-based attention via another paradigm manipulation (Zanto & Gazzaley, 2009). The overall design was identical to that illustrated in Figure 1, except instead of faces and scenes, participants were presented with four sequential apertures of dots: two were differently colored and stationary, and two were gray and moving coherently in different directions (Figure 4). Attention was selectively modulated by instructing participants to remember color hues (ignore motion) or motion direction (ignore color). In addition to allowing us to characterize of top-down modulation based on features, this design offered several advantages: the WM task was more difficult than the face/scene sequential presentation task, perceptual thresholding of the stimuli allowed us to normalize WM task difficulty based on individual perceptual differences, and in comparison to the face/scene overlap task, the temporal separation of relevant and irrelevant stimuli allowed us to dissociate the impact of attending and ignoring stimuli on WM performance. ERPs time-locked to relevant and irrelevant stimuli revealed significant early top-down modulation for color and motion stimuli, within 200 ms of stimulus presentation onset (as observed by others: Beer and Roder, 2004; (Schoenfeld, et al., 2007)). To evaluate the relationship of early feature-based modulation and WM performance, trials from each participant were split into fast (high WM performance) and slow (low WM performance) responses based on the median response time (RT) (of note, accuracy was also reduced in the slower trials). This approach revealed that significant top-down modulation was not present for either stimulus type in low-performance trials, but was present in the high-performance trials (Figure 5). Further analysis showed that the lack of significant modulation in the low-performance trials was due to misdirected allocation of attention to irrelevant stimuli, as revealed by excessive activity at the early time points (Figure 6). On these trials, participants were allocating as much attention to irrelevant stimuli as they were when these stimuli were relevant. This pattern of results was replicated for both stimulus features. Moreover, excessive attention directed to irrelevant stimuli was reflected in the WM maintenance period as an increased memory load (i.e., irrelevant stimuli were not just attended to more, but were maintained in WM, similar to the findings of Vogel and colleagues (Vogel, et al., 2005). Notably, there were no significant differences in ERP modulation for relevant stimuli across performance subtypes. These results suggest that neural enhancement of encoded information is not the primary determinant of high-level WM performance, but rather, optimal performance is dependent upon effectively filtering irrelevant information at early processing stages to prevent overloading a limited WM capacity.

Figure 4.

Basic experimental paradigm with the following timing parameters. 1) Cue stimuli: 800 ms, time between cue stimuli: 200 ms, Delay period: 4 sec, Probe stimuli: 800 ms (Zanto & Gazzaley, 2009, Submitted; Zanto, Rubens, et al., 2010; Zanto, Toy, et al., 2010). 2) Cue stimuli: 800 ms, time between cue stimuli: 1200 ms, Delay period: 8 sec, Probe stimuli: 800 ms (Zanto & Gazzaley, Submitted; Zanto, Rubens, et al., 2010). White arrows indicate motion and were not present during the experiment, except in the probe for the Passive view.

Figure 5.

Attentional modulation and WM performance. A, ERP waveform for attended (solid line) and ignored (dashed line) motion stimuli. Attentional modulation is observed at the P1. B, Comparison of the P1 modulation index (difference between attended and ignored stimuli) across different trial groupings: all trials, low- and high-performance trials. C, ERP waveforms for attended (solid line) and ignored (dashed line) colored stimuli. D, Comparison of the N1 modulation index. Asterisks indicate a significant difference between attend and ignore, whereas the bracket indicates a significant difference between indices (Zanto & Gazzaley, 2009).

Figure 6.

ERP comparisons between low- and high- WM performance trials. Low-performance (dark gray line) and high-performance (light gray line) trials for attended and ignored stimuli. Inset bar graphs compare designated peak ERP measures between low- and high-performance trials. A, No differences observed between low- and high- performance at the P1 peak when attending to motion, or B, the N1 peak when attending to color. B, When participants are instructed to ignore color, an enhanced N1 is observed during low-performance trials, which is similar in magnitude to the N1 when attending to color (dashed line - waveform from C). D, When instructed to ignore motion, an enhanced P1 is observed during low-performance trials that is similar in magnitude to the P1 when attending to motion (dashed line - waveform from A) (Zanto & Gazzaley, 2009).

The impact of early processing of irrelevant information on subsequent WM performance was further evaluated using EEG and a face delayed-recognition paradigm with a single distracting, irrelevant face presented in the middle of the maintenance period (Distractor task) (Figure 7) (Clapp, Rubens, & Gazzaley, 2010). This differed from our previous paradigms in that the distractor onset was delayed several seconds after the to-be-remembered face stimuli was presented, and thus the neural signature of distractor processing could be analyzed independent of influences by temporally adjacent relevant processing demands. An across-participant regression analysis revealed that N170 latency suppression indices for the distracting stimuli significantly predicted subsequent WM accuracy, such that those participants who processed the distractors less at this early time point, exhibited better WM performance (Figure 8). These findings were replicated in an fMRI study, which showed that the magnitude of BOLD signal modulation to face distractors in a face-selective visual region negatively correlated with WM accuracy across participants. Moreover, consistent with the results of the motion/color EEG experiment, early EEG measures of processing relevant, encoded stimuli did not correlate with WM performance. This confirms using different types of stimuli and task design that the influence of early distractor processing is the primary mediator of successful WM performance.

Figure 7.

Basic experimental paradigm with the following timing parameters. 1) Cue stimuli: 800 ms, Delay 1 period: 2.8–3.2 sec, Interference Stimuli: 800 ms, Delay 2 period: 2.8–3.2 sec, Probe stimuli: 1000 ms (Clapp & Gazzaley, 2010; Clapp, et al., 2010). 2) Cue stimuli: 800 ms, Delay 1 period: 7.2 sec, Interference Stimuli: 800 ms, Delay 2 period: 7.2 sec, Probe stimuli: 1000 ms (Clapp, et al., 2010; Clapp, et al., Submitted).

Figure 8.

Modulation of Occipito-Temporal Electrode of Interest ERPs: A & C) ERPs to interruptors (IS), passively-viewed stimuli (PV) and distractors (DS). A) P100 amplitude reveals significant enhancement. B) The amount that participants allocate attention towards an interruptor (IS, enhancement) negatively correlates with their WM performance (R=−0.7, p < .001). Likewise, the amount of attention allocated away from a distractor (DS, suppression) positively correlates with WM (R= 0.5, p < .05). C) N170 results showing significant enhancement of the N170 Latency. D) The same significant correlations were obtained as for the P100, such that the amount of attention allocated towards the interruptor and away from distractors predicts WM performance (R=−0.76, p < .0001, R=.64, p < .005 respectively) (Clapp, et al., 2010)

To further assess the impact that early processing based on attentional goals has on WM performance, a face delayed-recognition task that included a different category of interfering stimulus was evaluated in the experiment (Clapp, et al., 2010). In this task, a face was presented in the middle of the maintenance period that required attention as a component of a secondary task (gender and age decision) (Interruptor task) (Figure 7). We categorize this type of interference as an “interruptor”, distinguishing it from a "distractor" that should be entirely ignored (Clapp, et al., 2010). Analysis of ERPs time-locked to the interruptor revealed that early processing of these stimuli, as assessed by N170 latency modulation indices, also negatively correlated with subsequent WM accuracy, such that participants who processed the interrupting stimuli more, performed worse on the WM task (Figure 8). This significant correlation was also replicated in an fMRI experiment (Clapp, et al., 2010). Moreover, in another experiment that used a motion version of this task (Figure 9), early processing of both distractors and interruptors, as reflected by P1 and N1 modulation indices, again predicted subsequent WM performance measures (Berry, Zanto, Rutman, Clapp, & Gazzaley, 2009). These experiments generalize the impact on memory via early processing of interfering stimuli to include both distractors and interruptors.

Figure 9.

Basic experimental paradigm with the following timing parameters. Cue stimuli: 800 ms, Delay 1 period: 2.8–3 sec, Interference Stimuli: 800 ms, Delay 2 period: 2.8–3 sec, Probe stimuli: 800 ms (Berry, et al., 2010; Berry, et al., 2009). White arrows indicate motion and were not present during the experiment, except in the probe for the Passive view.

In summary, EEG data obtained using different stimulus types (i.e., complex objects and stimulus features) and experimental paradigms converge to reveal that attentionally-driven modulation of visual cortical activity early in the sensory processing stream influences subsequent WM performance. This influence was mediated by excessive processing of both distracting irrelevant information, and interrupting information that was attended to as a secondary task.

4. Influence of cognitive load

Cognitive operations exhibit capacity limitations, such that increasing demands result in diminished performance (Kahneman, 1973). Several studies in our laboratory have evaluated the influence of WM load on attentional modulation and how this influences WM performance. In a simple instructional manipulation of the selective, delayed recognition paradigm using sequential face/scene stimuli (Figure 1), both sets of faces and scenes were relevant (i.e., all 4 items). This resulted in diminished WM accuracy relative to performance on tasks when only two items were relevant. Of note, this diminished performance occurred with a constant sensory load (the other two stimuli were irrelevant) and varied only with the mnemonic demands. EEG and fMRI analysis revealed significant decrements in enhancement indices for the four items, using both fMRI measures of modulation and EEG N170 latency measures, relative to levels of modulation obtained when only two stimuli were relevant (Gazzaley, Cooney, McEvoy, et al., 2005). Our interpretation of this finding is that in the presence of increased mnemonic demands, top-down attentional resources were exceeded, resulting in both diminished early activity modulation and WM performance. Building upon this finding, we repeated the original EEG experiment (remembering 2 items and ignoring 2 items) with more trials, so that there was sufficient statistical power to evaluate the two relevant face stimuli independently. Consistent with our hypothesis, the second relevant stimulus revealed diminished N170 amplitude relative to the first relevant stimulus (Bollinger, Masangkay, Zanto, & Gazzaley, 2009). Thus, even a WM load of a single face stimulus seems to reduce the degree of early attentional modulation attainable for a second stimulus. These results reveal the complexity of the interaction between these processes: WM load results in less attentional modulation of subsequent stimuli, which results in an overall diminished WM performance.

In another experiment, we utilized fMRI and a modification of the paradigm illustrated in Figure 1 to explore the influence of increasing WM load from distinct information on top-down modulation (Rissman, Gazzaley, & D’Esposito, 2009). In a dual task condition, each trial began with the auditory presentation of six random (high load) or sequentially ordered (low load) digits prior to the selective attention, delayed-recognition task. Thus, in this manipulation, the same number of faces and scenes were to be maintained (2 faces or 2 scenes), but a series of digits was also maintained. When taxed by a high cognitive load, participants exhibited diminished WM performance on the task, as well as a selective failure to suppress the neural processing of task-irrelevant scene stimuli. The over-processing of distractor scenes under high load was indexed by elevated activity relative to low load and passive viewing levels. In contrast, the load manipulation did not affect the participants’ ability to enhance activity associated with the relevant information. These results highlight the critical role of domain-general WM resources in goal-directed regulation of distractor processing, and is consistent with cognitive load theory as described by Lavie and colleagues (Lavie, 2005).

In summary, 1) Attentional enhancement of early visual cortical activity exhibits capacity limitations that can be exceeded by the WM load of previously presented relevant information within the same domain, and 2) When maintaining relevant information, increasing the general cognitive load on WM resources (concurrent digit load task) results in preserved enhancement, but with a deficit in suppression (i.e., the over-processing of irrelevant stimuli).

5. Influence of task practice

As previously described, successful suppression of interfering stimuli, both distractors (stimuli to be ignored) and interruptors (stimuli requiring attention), confers a behavioral benefit on WM performance (Clapp, et al., 2010; Zanto & Gazzaley, 2009). Indeed, optimal WM performance seems to rely upon successful suppression of irrelevant stimuli within 200 ms of stimulus onset, as revealed using P1 and N1 markers of attentional modulation. Behavioral studies have revealed that more efficient suppression of interfering visual stimuli occurs with practice, as measured via diminution of the motion aftereffect (Vidnyanzky and Sohn 2005) and decreased dominance of interfering stimuli in a binocular rivalry paradigm (Paffen et al. 2008). However, our understanding of the neural basis of changes in interference processing with practice, and its influence on WM performance, is limited. We hypothesized that early visual processing of external interference exhibits plasticity, such that it is modifiable with practice. Accordingly, we predicted that the negative impact of interfering information on WM performance would be abated by practice-related changes in the modulation of neural activity associated with early stages of perceptual processing.

To address this, we conducted an experiment to assess practice-mediated changes in processing interfering stimuli and its impact on WM performance, and evaluated neural mechanisms of these effects using EEG (Berry, et al., 2009). We explored practice effects in three delayed-recognition tasks by comparing performance across two blocks within a single experimental session. The paradigm was a modification of the tasks illustrated in Figure 7. In this experiment, participants were instructed to maintain in mind the direction of motion of a field of dots across a delay period (Figure 9). One of the tasks involved no interference, while the other two tasks included motion interference that consisted of a field of rotating dots presented in the middle of the delay period. These intervening stimuli were either to-be-ignored distractors or interruptors, which required a simple perceptual discrimination. A passive viewing task utilizing the same temporal design served as a baseline from which to measure processing enhancement of encoded and interruptor stimuli and suppression of distractor stimuli. Neural measures from posterior electrodes were evaluated to examine practice-related changes in early visual cortical activity.

We showed that WM performance was disrupted by both types of interference, but that interference-induced disruption abated across a single experimental session (Berry, et al., 2009). Interestingly, WM improved for tasks that included interfering stimuli, but not for the same task without interference. Morevoer, WM accuracy and response time improved in a manner that was correlated with changes in early neural measures of interference processing in visual cortex (i.e. P1 suppression and N1 enhancement). This was not true for neural indices associated with the cue and probe stimuli of any task. Overall, these findings confirm conclusions already presented in this review that the degree to which interfering stimuli are processed early in the time course exerts a major influence on WM performance, and further reveal that susceptibility to interference is modified with practice over the course of hours via an influence on attentional modulation at an early stage of processing.

6. Changes with aging

Cognitive impairment associated with normal aging impacts multiple domains, including attention and WM (Craik & Salthouse, 2000). However, the search for an underlying mechanistic theory of cognitive aging has been impeded by a tendency to study attention and memory in isolation. We hypothesized that age-related WM deficits may be the consequence of impaired attentional processes, specifically an alteration in top-down modulation (Gazzaley & D'Esposito, 2007a). The selective attention, delayed-recognition task with sequentially presented faces and scenes (Figure 1) was first used to study a healthy cohort of healthy older adults (60–77 years of age), with neural data recorded using fMRI. The fMRI data revealed that older adults exhibit a deficit in the suppression of irrelevant information relative to younger adults, with no significant differences in the enhancement of relevant information (using the scene-selective modulation indices) (Gazzaley, Cooney, Rissman, & D'Esposito, 2005). We recently repeated this fMRI experiment in older adults using the face/scene overlap version of the task (Figure 2), which replicated the finding of an age-related, selective-suppression deficit (Chadick & Gazzaley, 2008). In addition to the suppression deficit, older participants were behaviorally impaired on the WM tasks, exhibiting both reduced accuracy and a slower reaction time compared to younger participants. To directly evaluate the relationship between top-down modulation during encoding and subsequent WM recognition performance in the older subjects, we performed a regression analysis between the scene suppression index and face WM accuracy (Gazzaley, Cooney, Rissman, et al., 2005). This analysis revealed that the suppression index significantly correlated with WM performance, such that the degree of top-down suppression predicted WM recognition accuracy (true for both the sequential and overlapping presentation versions of this experiment). These findings support the link between attention and WM impairments in normal aging with an underlying deficit in top-down suppression, consistent with the inhibitory deficit hypothesis of aging (Hasher, Zacks, & May, 1999; Hasher & Zacks, 1988). Although not cause, these data suggest that older individuals are able to focus on pertinent information, but are overwhelmed by interference related to a failure to ignore distracting information, resulting in memory impairment for the relevant information.

To explore the time-course of this age-related relationship between attentional modulation and WM performance, we performed the same selective attention, delayed-recognition tasks in older adults, but utilized EEG recordings (Gazzaley, et al., 2008). To temporally dissect modulation effects, five posterior EEG measures associated with visual processing were evaluated that have been previously shown to be modulated by selective attention throughout the stimulus-present time-frame: P1 (50–150ms) (Gomez Gonzalez, Clark, Fan, Luck, & Hillyard, 1994), N1 (120–220ms) (Gazzaley, Cooney, McEvoy, et al., 2005; Gomez Gonzalez, et al., 1994), gamma synchronization (200–300ms) (Gruber, Muller, Keil, & Elbert, 1999), P300 (300–500ms) (Picton, 1992) and alpha desynchronization (500–650ms) (Muller & Keil, 2004). Analysis revealed that younger participants exhibited both significant enhancement and suppression of all of these measures relative to passive view levels. However, while older participants also exhibited enhancement at all time points, they only showed suppression at the latest measure, alpha desynchronization. Across age-group comparisons of modulation indices revealed a significant age-related suppression deficit only in the earliest of these measures, P1 amplitude and N1 latency, in the setting of preserved enhancement (Gazzaley, et al., 2008) (Figure 10). This study thus provided electrophysiological evidence of a selective deficit in top-down suppression with aging, which manifests as early as 100ms after stimulus onset. The presence of intact suppression later in the time-course demonstrates that suppression abilities are not abolished with normal aging, but delayed to a later processing stage, revealing an interaction between deficits in suppression and processing speed in older adults. Interestingly, an impairment in WM performance existed despite successful later suppression, implying that interference by irrelevant information overwhelms a limited WM capacity very rapidly, and is unable to be successfully compensated by later processing. When the older population was divided into two subgroups based on scene WM accuracy, N1 latency analysis revealed that lower-performing older adults exhibited a reduced suppression index compared to younger adults. However, higher-performing older adults did not show a significant suppression deficit relative to the younger cohort. Independent of the important aging aspects of this finding, these results once again highlight the impact of early distractor processing on subsequent WM performance.

Figure 10.

EEG data revealing an age-related deficit in top-down suppression in the earliest measures: P1 amplitude and N1 latency. All within-group t-tests are designated as significant by brackets (P < .05). The asterisk denotes that only P1 amplitude and N1 latency revealed a significant age × task interaction plus a significant across-group suppression deficit. Error bars indicate standard error of the mean (Gazzaley, et al., 2008).

These studies revealed that older adults exhibit a deficit in suppressing task-irrelevant information that is associated with diminished WM performance. However, it remained unclear if the suppression deficit was specific to mechanisms involved in attention-driven suppression or if it was a reflection of an inability of older adults to rapidly identify stimuli and assess their relevance/irrelevance upon presentation. To address this, we recorded EEG in another cohort of healthy older participants (aged 60–80 years) while they performed two different versions of the face/scene delayed-recognition task (Figure 1), both with and without prior knowledge of when relevant and irrelevant stimuli would appear. Each trial contained two faces and two scenes presented sequentially followed by a nine second delay and a probe stimulus. In separate blocks, participants were given the following instructions: remember faces, remember scenes, remember the xth and yth stimuli (x and y could be 1^st, 2^nd, 3^rd or 4^th. They did not know in advance whether the stimuli would be faces or scenes), or passively view all stimuli. The remember faces/rememeber scenes tasks were the same as those used in previous experiments, i.e., participants had to first identify stimuli upon presentation and then either remember or ignore them as instructed. While in the sequence tasks (remember the xth and yth stimuli), participants could anticipate the appearance of the distractors prior to presentation. WM performance was the same regardless of task instructions. Neural analysis revealed that for both task types, enhanced activity occurred at posterior electrodes to attended stimuli, but the suppression of activity for irrelevant stimuli did not occur. The lack of significant suppression at early stages of visual processing was again revealed by P1 amplitude and N1 latency modulation indices. These results mirror previous findings of (Gazzaley, et al., 2008), and extend them by revealing that prior knowledge of stimulus relevance/irrelevance does not influence early neural processing in older adults, nor does it translate to improved WM performance. These results suggest that the inability to suppress irrelevant information early in the visual processing stream by older adults is related to mechanisms specific to top-down suppression, possibly reflective of a more general inability of older adults to use anticipatory cues to guide optimal behavior (Bollinger, Rubens, Masangkay, Kalkstein, & Gazzaley, Submitted). Of note, data from our lab also revealed that age-related changes in later, postperceptual processing (e.g., the selection negativity: ~300–350ms in older adults) detrimentally influences WM performance in older adults (Zanto, Toy, & Gazzaley, 2010). The impact of later perceptual processing on WM has not been systematically evaluated in all of the studies presented in this review, so it is unclear if there is a differential influence on memory performance by early or late processing.

A deficit by older adults in suppressing early processing of distrators, even in the setting of anticipating when they will appear, was also replicated using the paradigm illustrated in Figure 7. As was found in younger adults (Clapp, et al., 2010), there was a significant correlation in older adults between the degree of early suppression (using the N170 latency index) and subsequent WM performance (Clapp & Gazzaley, 2010). This same relationship was also replicated in older adults using fMRI (Clapp, Rubens, Sabharwal, & Gazzaley, Submitted). Moreover, the same significant across-participant correlation in younger adults for the early processing of interruptors and diminished WM performance was also observed in older adults (Clapp & Gazzaley, 2010).

As revealed in younger adults (Berry, et al., 2009), even limited practice on a WM task with interference (i.e., block one vs. block two) resulted in WM performance improvements in older adults. This was again associated with attentional influences on the early stages of processing distracting stimuli (Clapp & Gazzaley, 2010). After observing this rapid visual processing plasticity in older adults with only brief practice, we entertained the possibility that extensive training of perceptual discrimination abilities may transfer to improvements in WM performance in older adults. To examine the neural effects of perceptual discrimination training, as well the consequences on WM, two cohorts of healthy older adults (ages 60–89 years) were evaluated with EEG while they preformed delayed recognition tasks before and after either ten hours of visual discrimination training over a three to five week period (training group), versus a control group without training (Berry, et al., 2010). Stimuli used in the training program were Gabor patterns (sine-waves windowed by a 2D Gaussian), which expanded or contracted two successive times per trial. Participants pressed one of two buttons for each movement to indicate whether they perceived the stimuli expanding or contracting. Training was adaptive such that the speed of expansion/contraction and the duration of the inter-stimulus interval scaled with improvements in response accuracy, so as to continue to challenge the trainee.

The results of this study revealed that training on a perceptual discrimination task resulted in improved WM by impacting early visual processing of the encoded stimuli. Specifically, training resulted in decreased N1 amplitude for the encoded stimuli in four different WM conditions (Figure 11). The reported decrease in N1 amplitude is consistent with the results of perceptual learning studies in experimental animals that show tuning curves become narrower for the trained population of visual cortical neurons (Yang & Maunsell, 2004), and this is likely associated with smaller responsive neural populations as reflected by decreased EEG (Alain & Snyder, 2008; Ding, Song, Fan, Qu, & Chen, 2003) and fMRI signals (Mukai, Kim, Fukunaga, Japee, & Marrett, 2007). Notably, this did not occur for the same stimuli presented in a passive view condition, suggesting that discrimination training influenced the impact attentional modulation had on early perceptual processing of the encoded stimuli. In one of the delayed-recognition tasks (the task at the original perceptual threshold, with no interference present), WM performance significantly improved in the training group alone. An evaluation of individual differences revealed that WM improvement positively correlated with the magnitude of the N1 amplitude decreases (Figure 11). We propose that the N1 amplitude decrease reflects a change in the influence attention has on perception as mediated by training (i.e. fewer resources are required to perceive the stimuli, presumably at a higher fidelity level), which then engenders improved WM accuracy under the appropriate conditions. This study highlights the complex interaction between perception, attention and memory, and again confirms the influence of early visual processing on WM.

Figure 11.

ERPs during stimulus encoding and relationship with WM performance. Posterior occipital N1 amplitude (120–220 ms) significantly decreased at T2 for the A) training, but not B) control group. Statistics are based on electrode of interest (EOI) clusters selected for each participant. Scalp topographies of T2-T1 at the latency of mean N1 peak +/− 1sd illustrate the location of the training related functional plasticity. C, Across participants, decreased N1 amplitude during encoding correlated with WM performance improvements (r = 0.82, p < 0.001) (Berry, et al., 2010).

7. Prefrontal cortex control

The prevailing view is that top-down modulation of visual processing is not an intrinsic property of visual cortices, but rather is mediated by a distributed network of interacting brain regions, or neural networks. Extensive reciprocal connections between the prefrontal cortex (PFC) and virtually all cortical and subcortical structures situate the PFC in a unique neuroanatomical position to monitor and manipulate diverse cognitive processes (Barbas, 2000; Goldman-Rakic & Friedman, 1991). Tract-tracing studies in experimental animals have revealed long-range reciprocal connections between the PFC and the parietal, temporal, cingulate and insular cortices, the limbic system, as well as extensive subcortical connections (Cavada & Goldman-Rakic, 1989; Morecraft, Geula, & Mesulam, 1992; Ongur, An, & Price, 1998; Petrides & Pandya, 1999, 2002; Ungerleider, Gaffan, & Pelak, 1989; Webster, Bachevalier, & Ungerleider, 1994). Several of these well-defined pathways have also been described in humans with post-mortem dissection (Heimer, 1983) and more recently with in vivo diffusion tensor magnetic resonance imaging (Makris, et al., 2004).

The majority of evidence supporting a PFC role in meditating top-down modulation is based on data from physiology and neuroimaging studies that report simultaneous engagement of PFC and sensory regions on tasks when cognitive control is demanded and sensory cortical activity is modulated (Corbetta & Shulman, 2002; Hopfinger, Buonocore, & Mangun, 2000; Rainer, Asaad, & Miller, 1998; Ungerleider, Courtney, & Haxby, 1998). However, these data are indirect evidence that the PFC actually mediates top-down modulation. Functional connectivity analysis of fMRI data offers stronger support for a PFC role as a source of sensory activity modulation. Using this approach to analyze data obtained from younger adults engaged in the paradigm described in Figure 1, we revealed that a region in the PFC (left middle frontal gyrus) was more functionally connected with a scene-selective visual region when scenes were remembered and less so when scenes were ignored (Gazzaley, et al., 2007). Note that functional connectivity reflects activity correlations between regions using trial-by-trial variability as the dependent measure (Gazzaley, Rissman, & Desposito, 2004; Rissman, Gazzaley, & D'Esposito, 2004). Moreover, the strength of functional connectivity between these regions correlated with the magnitude of attentional enhancement for relevant stimuli and suppression of visual processing associated with irrelevant stimuli. Although this analysis is correlational, and does not permit an assessment of causality or directionality, these findings suggest that PFC modulates activity levels in visual cortices via biasing the strength of functional coupling in accordance with stimulus relevance.

A recent study using this same analytical approach and data obtained using the paradigm described in Figure 2, where there was direct competition between relevant and irrelevant information, revealed that visual cortical areas associated with processing relevant information were functionally connected with a frontal-parietal network (i.e., middle frontal gyrus, bilateral inferior frontal junction, and intraparietal sulcus), while visual cortical areas that process irrelevant stimuli were simultaneously coupled with a “default network” (i.e., medial prefrontal cortex and posterior cingulate cortex) (Chadick & Gazzaley, Submitted). Importantly, we also demonstrate that the strength of visual cortex/default network functional connectivity was predictive of a participant’s ability to suppress task-irrelevant information. This provides the first evidence that sensory cortical regions are functionally and dynamically coupled with distinct neural networks based on an individual’s goals, and suggests differential involvement of these large-scale networks in top-down enhancement and suppression.

Despite the substantial contribution made by correlational studies, the ideal experimental design to causally assess the role of the PFC in top-down modulation involves disruption of function in the PFC coupled with physiological recordings of distant brain regions while participants are engaged in a task. There have been several studies that have implemented such a design in experimental animals (Fuster, Bauer, & Jervey, 1985), and in humans (reviewed; (Gazzaley & D'Esposito, 2007b). These studies support the conclusion that top-down modulation, utilizing both enhancement and suppression mechanisms, is a source of PFC control over diverse mental processes. Lesion studies in humans (Barcelo, Suwazono, & Knight, 2000), and more recently transcranial magnetic stimulation (TMS) used to perturb function in frontal and parietal regions, provide causal evidence that these areas are a source of top-down activity modulation in visual cortex (Capotosto, Babiloni, Romani, & Corbetta, 2009; Ruff, Blankenburg, Bjoertomt, & Bestmann, 2006; Silvanto, Muggleton, Lavie, & Walsh, 2009; Taylor, Nobre, & Rushworth, 2007). However, the stage of visual processing impacted by PFC control and its influence on WM performance had not been evaluated from a causal perspective.

We recently identified a region within the PFC as a potential source of top-down modulation underlying attention to visual features, the inferior frontal junction (IFJ), using the experimental paradigm described in Figure 5 and fMRI functional connectivity analysis, (i.e. color and motion; Zanto, Rubens, Bollinger, & Gazzaley, 2010). Furthermore, EEG results suggested that the IFJ may exert an influence on visual processing as early as 100 ms post-stimulus onset. Activity at this time point had previously been shown to be modulated by attention to color and motion stimuli (Zanto, Rubens, et al., 2010; Zhang & Luck, 2009), as well as related to subsequent WM performance (Zanto & Gazzaley, 2009). To explore the causal role of PFC-mediated top-down modulation, as driven by selective attention on subsequent WM performance, we perturbed function within the IFJ via repetitive TMS (rTMS) prior to participants performing a selective attention, delayed-recognition task and recorded the consequences with EEG (Zanto & Gazzaley, Submitted). The paradigm utilized in this two-session experiment required participants to selectively attend to relevant visual features of sequentially presented stimuli (motion or color), ignore the irrelevant stimuli and maintain the attended features until the information was probed after a brief delay (Figure 5). The first session used fMRI to identify neural networks associated with top-down modulation. Based on each participant's functional connectivity data, during the second session, the right IFJ was targeted with 1 Hz rTMS to disrupt the network subserving top-down modulation. Immediately following 15 minutes of rTMS, participants performed the same experimental task as during the first session while EEG and WM performance data were acquired. We assessed the impact of rTMS on ERP measures of modulation during stimulus presentation, as well as the impact on WM accuracy. Our goals were to determine the extent to which top-down modulation was driven by the IFJ, the timing of this influence, and how this impacted WM performance.

Results indicated that top-down modulation during early visual processing stages of the memoranda was causally related to subsequent WM performance (Zanto & Gazzaley, Submitted). There are four findings supporting this conclusion. First, color processing showed declines in both P1 modulation and WM accuracy after rTMS treatment. Second, both P1 modulation and WM performance recovered with time as the effect of TMS effect abated (i.e., in the second half of the block). Third, motion processing, which exhibited bilateral IFJ connectivity in the fMRI analysis, did not show an effect of right IFJ rTMS (i.e. P1 modulation and WM accuracy were not different than sham). Finally, in an across-participant regression analysis, the rTMS-induced effect on the P1 modulation during color processing predicted changes in WM accuracy. Critically, the data revealed the IFJ to be a PFC control region that mediates the causal connection between early top-down activity modulation in the service of attentional goals and WM. Thus, we conclude that top-down modulation of early stimulus processing imposed by attentional demands is directly related to subsequent WM performance.

8. Conclusions

A richer perspective on the dynamic interaction between attention and WM requires an understanding of the stage of sensory processing at which attentional influences impact subsequent memory of those stimuli. It is conceivable, and has been proposed that the transfer of information to WM storage occurs after basic perceptual processing has been completed (e.g., the WM deficit elicited in the attentional blink paradigm reflects an influence after stimulus identification has been completed (>200ms) (Vogel 1998)). Here, a convergence of evidence from a series of studies performed in our laboratory utilizing different techniques, stimuli, paradigms and populations, reveals that top-down modulation of visual cortical activity at early perceptual processing stages (100–200 ms after stimulus presentation) influences subsequent WM performance.

To summarize the data presented in this review that directly informs this conclusion: Capitalizing on individual performance differences in younger adults, a correlation between early processing of competing object-based information and WM accuracy was revealed (Rutman, et al., 2010). This relationship was further shown to be selective for attention to interfering stimuli, both to-be-ignored distractors, and to-be-attended interruptors (Berry, et al., 2009; Clapp, et al., 2010). Using performance variability across trials to probe neural-behavioral relationships, revealed that optimal WM performance is dependent upon effectively filtering irrelevant information at early processing stages (Zanto & Gazzaley, 2009). Complementary evidence is found in older adults who exhibit the same relationship between early attentional modulation of interfering stimuli and subsequent WM performance (Clapp & Gazzaley, 2010; Gazzaley, et al., 2008; Zanto, Hennigan, Ostberg, Clapp, & Gazzaley, 2010). Early visual processing of relevant information exhibits capacity limitations that are exceeded when WM load of previously presented relevant information increases in the same domain (Chadick & Gazzaley, 2008), and this is associated with diminished WM accuracy (Gazzaley, Cooney, McEvoy, et al., 2005). Limited practice on delayed-recognition tasks with interfering stimuli results in diminished early processing of these interfering stimuli and this correlates with improvements in WM performance in both younger (Berry, et al., 2009) and older adults (Clapp & Gazzaley, 2010). Training on a perceptual discrimination task by older adults alters early processing of relevant information in an attentionally dependent manner, which predicts improvements in WM performance (Berry, et al., 2010). Lastly, prefrontal cortex (e.g., IFJ) control of early attentional modulation of visual cortical activity causally influences subsequent WM performance (Zanto & Gazzaley, Submitted).

As described throughout this review, and as is evident from an extensive literature (Desimone & Duncan, 1995), top-down attentional control is driven by competitive demands to perceive a complex environment. Top-down control is often in opposition to automatic, bottom-up, and pre-potent responses. There is a consensus that unintentional perception and action (e.g., automatic or bottom-up processes) tend to transpire more rapidly than intentional processes, such as those involved in top-down, attentional modulation (Cohen, Dunbar, & McClelland, 1990; Curtis & D'Esposito, 2003; Gray, 2004; Grossberg, 1999; Libet, 2004; Lieberman, 2007; Logan, 2008; Strack & Deutsch, 2004). It could be argued that top-down modulation must ‘get a head start’ and be initiated as early as possible in order to direct processing (Morsella, Zarolia, & Gazzaley, 2010). As reviewed above, if top-down control does not occur rapidly enough, then WM performance in the setting of interference is diminished. With this conclusion in mind, it is clear why intentional/voluntary processing is often characterized as highly predictive, anticipatory, and concerned with overcoming undesired processing (Bethoz, 2002; Bollinger, et al., 2010 ; Buzsáki, 2006; Hawkins & Blakeslee, 2005; Norman & Shallice, 1980; Summerfield, Egner, Greene, & Koechlin, 2006).

In addition to establishing evidence of the relationship between early attentional modulation and WM, the reviewed findings illustrate a broad range of influences on early perceptual processing. Of note, although these findings offer converging evidence that early processing of relevant and irrelevant stimuli influences subsequent WM, it does not preclude a role of top-down modulation during other task stages (e.g., post-perceptual phases of encoding and the WM maintenance period) in WM performance. Further studies utilizing novel approaches will be needed to continue to explore the intricacies of the functional network dynamics that mediate interactions between perception, attention and memory.