Working Memory as Internal Attention: Toward an Integrative Account of Internal and External Selection Processes

Abstract

Working memory (WM) and attention have been studied as separate cognitive constructs, although it has long been acknowledged that attention plays an important role in controlling the activation, maintenance, and manipulation of representations in WM. WM has, conversely, been thought of as a means of maintaining representations to voluntarily guide perceptual selective attention. It has more recently been observed, however, that the contents of WM can capture visual attention, even when such internally maintained representations are irrelevant, and often disruptive, to the immediate external task. Thus the precise relationship between WM and attention remains unclear, but it appears that they may bi-directionally impact one another, whether or not internal representations are consistent with external perceptual goals. This reciprocal relationship seems, further, to be constrained by limited cognitive resources to handle demands in either maintenance or selection. We propose here that the close relationship between WM and attention may be best described as a give-and-take interdependence between attention directed toward actively maintained internal representations (traditionally considered WM) versus external perceptual stimuli (traditionally considered selective attention), underpinned by their shared reliance on a common cognitive resource. Put simply, we argue that WM and attention should no longer be considered as separate systems or concepts, but as competing and impacting one another because they rely on the same limited resource. This framework can offer an explanation for the capture of visual attention by irrelevant WM contents, as well as a straightforward account of the underspecified relationship between WM and attention.

Background

All of us are taxed with juggling our inner mental lives and immediate external task demands. The environment comprises an immeasurable array of stimuli from which we must somehow filter out a limited few upon which to act. Meanwhile, our minds are continually undulating with goals, reminders, and other thoughts that may or may not be related to the immediate task at hand. This theoretical review will examine how these demands—selecting and maintaining internal representations versus attending to external stimuli—bear on one another, in a way that may optimize or obstruct performance in either domain.

Working memory (WM) is broadly considered the temporary maintenance and manipulation of information that is no longer available to the senses. While sensory afterimages and recently activated representations can also persist for a short time in the absence of sensory input (cf. Cowan, 1998; Massaro, 1972; Turvey, 1973), we use WM only to refer to that information that is actively, intentionally internally maintained. In one of the earliest formulations of the concept of WM, Baddeley and Hitch (1974) described a system in which attention serves as a filter for the information that is to become maintained internally. WM was defined as a dedicated system for the short-term maintenance and processing of information, subserved by domain-specific stores and controlled by a central executive attention system. Other influential theories of WM (Cowan, 1988; Oberauer, 2009) have challenged the idea of a separate system dedicated to short-term storage, but instead contend that attention controls the activation of long term memory representations for their use in the short-term. In this conceptualization, a number of long-term representations can be activated in WM, but only one receives the focus of attention, which is the subject of current processing. Taking an individual differences approach to understanding WM, others (Engle, Kane, & Tuholski, 1999; Kane, Bleckley, Conway, & Engle, 2001) have proposed that attentional control is the critical faculty that determines WM capacity. More recently, furthermore, Postle (2006) asserted that WM is not a specialized system, but arises as a result of the recruitment of multiple brain systems by attention (see also Theeuwes, Belopolsky, & Olivers, 2009).

All of these major theories of WM thus share a common acknowledgment of the important role of attention in controlling the activation, maintenance, and manipulation of short-term internal representations. A growing body of research, however, supports the idea that WM can conversely influence what gets attended. Prominent theories of visual attention describe the manner in which internal representations can guide or bias visual selection. In the guided search framework (Wolfe, 1994), top-down commands—or internal goals which would presumably be held in WM—can ascribe higher weights to certain features to impact where attention is deployed. The influential biased competition model (Desimone & Duncan, 1995) likewise describes how the active maintenance of an item in WM will result in biasing of visual processing in favor of matching items over other, competing objects. These models explain how the maintenance of a representation of an intended search target—or a search template—will promote the selection of that target. This is an adaptive mechanism for narrowing attention in a cluttered scene to find a target object, but also suggests that stimuli matching WM representations might guide attention even when they are not immediately task relevant.

In fact, it has recently been repeatedly observed that the contents of WM can affect the allocation of—or capture—visual attention, even when such internally maintained representations are irrelevant, and often disruptive, to the current external task (e.g., Downing, 2000; Olivers, Meijer, & Theeuwes, 2006; Soto, Heinke, Humphreys, & Blanco, 2005). Thus it may not simply be that attention is the controller of WM (in whatever way WM is conceptualized), but that the representations maintained in WM can conversely exert a powerful influence on attentional selection. A number of reviews have examined the interplay and similarities between the broad categories of attention and WM (e.g., Awh & Jonides, 2001; Awh, Vogel, & Oh, 2006; Chun, 2011; Chun & Johnson, 2011; Gazzaley & Nobre, 2012; Theeuwes, Belopolsky, & Olivers, 2009) and all ultimately conclude that the two are closely linked—although the precise mechanism of this linkage remains unclear. In the present article, we propose to take the conceptual association between WM and attention one step further, arguing that they should no longer be considered as separate constructs but as reliant on the same mechanism of prioritization of processing (traditionally called attention) that can be geared either at internal (traditionally considered WM) or external (traditionally considered selective attention) representations.

Although a recent taxonomy of attention (Chun, Golomb, & Turk-Browne, 2011) draws a fundamental distinction between the content and operation of internally versus externally targeted attention (but see Chun & Johnson, 2011), the present review will describe and synthesize a compelling collection of evidence for their close interdependence. If WM representations can be considered internally attended information (cf. Nobre et al., 2004), this literature converges on the notion that such internal attention guides the selection of stimuli that are currently available for perception. As part of the same broader prioritization scheme, WM (internal attention) and perceptual selection (external attention) closely interact and impact one another. This close relationship would be a natural byproduct of their mutual reliance on a common cognitive resource (supramodal attention), resulting in a reciprocal, push-pull exchange of attentional allocation to either the internal or external domain.

A number of recent studies have explicitly investigated the degree to which, and conditions under which, the contents of WM will automatically guide the distribution of attention during visual search. Importantly, this phenomenon is observed beyond the purposeful holding of an attentional template to guide selection, but when the internal attentional content is irrelevant—and often harmful—to the external perceptual (search) task. The burgeoning literature that probes this particular circumstance will be the focus of this review, for two main reasons. First, the capture of attention as observed in these studies is presently without a complete explanation, which may be offered by the framework suggested here. Second, although there are a multitude of investigations of WM and attention, we believe that this particular slice of the literature speaks most directly to the manner in which WM contents might act like other attention biases. If WM and visual attention demands mutually impact one another, and WM operates like other established forms of attentional cuing, that would support the possibility that WM can be considered internally oriented attention. The goal of this review will be to describe a tradeoff between internal and external selection, underpinned by their reliance on a shared resource, and thus a reconceptualization of the relationship between WM and attention. To this end, we will first describe canonical findings and boundary conditions of the guidance of external selection processes by internally maintained information, and subsequently present evidence for the thesis that the relationship between internal maintenance and external selection is fully reciprocal.

Internal Maintenance Guides External Selection

Features Matching Internally Attended Information Capture Visual Attention

Imagine that you are sitting in your car at a stoplight, rehearsing your to-do list, continually reminding yourself to pick up your yellow shirt at the dry-cleaner (think “yellow shirt, yellow shirt, yellow shirt”). You can envision that if a person walks through the crosswalk wearing a yellow shirt, you will notice him immediately, while not being as seized by others in blue suits or green dresses. This happens without the objective to look for a yellow shirt, and possibly at the expense of the more pressing task of monitoring the traffic. This sort of occurrence may also be beneficial, however, if the active memory content overlaps with currently relevant goals—for instance, if your immediate task were to locate and pick up a yellow shirt-clad friend curbside at the airport. In daily life, and in the laboratory, it has been repeatedly demonstrated that actively maintained information (which we here argue to be equivalent to internally attended information) can guide the selection of external stimuli as recipients of attention (with and without intention).

In an early demonstration of the relationship between mental and perceptual images, Farah (1985) found that imagining a letter yielded better detection for a faint matching letter in a visual display. Pashler and Shiu (1999) further, observed an attentional blink for a target preceded by an image matching a visualized item—suggesting that attention was deployed to the visualized image—even when subjects were specifically instructed to ignore the imagined item. Later studies explicitly instructed participants to maintain certain items in WM—or to actively attend to them internally. In a seminal study of this kind, Downing (2000) showed participants a to-be-remembered face, and later gave them a match/non-match face probe for the memory item. During the delay, the memory face and a novel face appeared simultaneously on the screen, followed by a bracket at the location of either one of the two faces—the orientation of which was to be discriminated by the participants. Discriminations occurred reliably faster at the location of faces that matched the memory item, suggesting that maintaining a face in WM biases attention towards matching faces. This would ordinarily serve as an adaptive mechanism, as we often internally maintain task goals to guide our allocation of attention.

Consistent with Pashler and Shiu (1999), later studies confirmed that such capture (as observed by Downing, 2000) can happen involuntarily—even when attending to a memory-match is detrimental to a more immediate task. For instance, using a multiple-item visual search task, Soto and colleagues (2005) found that items held in WM could confer both costs and benefits on visual search. Participants saw a colored shape and were asked to keep it in memory; subsequently, they were presented with a visual search array consisting of a number of colored shapes with lines embedded within them. The search task required locating the single target slanted line (inside one of the shapes) and discriminating the direction of its tilt. The memory item could be either present or absent from the search array, and when present could contain either the target or a distracter line. On “valid” trials the memory item reappeared containing the target, while on “invalid” trials it reappeared containing a distracter. On neutral trials, the memory item never reappeared in the search display. On memory probe trials, a colored shape was displayed after the participant responded to the search target, and the task was to indicate whether that item was identical (in shape and color) to the memory item. Performance was faster in the valid, and slower in the invalid, than in the neutral condition, but items did not influence search if they were merely primed (displayed prior to the search but in the absence of a memory requirement). The costs of invalidity were observed even under conditions where the memory items were never valid predictors of the target location, suggesting that internally attended information involuntarily guided external selection.

Olivers and colleagues (2006) also found slowing of search when distracter items matched items held in WM. Participants were asked to remember a colored disk, then to perform a visual search for a diamond among circles, and discriminate the item inside the diamond. The search array either contained targets and distracters of all the same color, or a colored singleton distracter that could either match the memory color or be of an unrelated color. Participants then performed a memory test for the colored disk, choosing the matching item from a display of three colored disks. Distracters related to the internally maintained item were consistently found to slow search RTs, although this occurred only when a memory task was performed after the search task—not with mere exposure to a stimulus or when performing the memory task prior to the search (Olivers et al., 2006). Distracters matching the WM content attracted attention above and beyond the capture of an unrelated color singleton, suggesting that task-irrelevant WM content can operate similarly to an internally attended search target template.

The observation of attentional capture by WM contents is not limited to measurements of response time. In the paradigms described above, eye movement analyses (Soto et al., 2005; Soto et al., 2006) also determined that the percentage of first fixations landing at the target location was greater on valid vs. invalid trials (Soto et al., 2005), and more fixations overall landed at the locations of objects matching the features of the memory item—internally maintained content seemed to exert its influence early on, affecting the first saccade of the search (Olivers et al., 2006; Soto et al., 2005). Search stimuli matching an irrelevant actively maintained item have also been found to impact the direction of gaze correction after an errant saccade (Hollingworth & Luck, 2009). Performing a unique form of visual search—wherein the target was identified on each trial as the one stimulus in the array that expanded and contracted in size—participants were to make a saccade to that target as quickly as possible. During the saccade, the circular array was rotated so that the fixation would land, not on the target, but halfway between it and a distracter. Participants would thus need to correct their gaze to fixate on the proper target. This gaze correction was slower and less accurate when the adjacent distracter matched a WM stimulus. Thus, just as attentional biasing increases responsiveness to attended target stimuli, multiple measurements of response time and eye movements indicate that attention is drawn to external stimuli matching actively maintained internal representations, whether or not they are consistent with the goals of the relevant visual task.

Internally maintained representations are beneficial when they overlap with—but detrimental when inapplicable to—the current externally geared endeavor. WM (internally maintaining information) and attention (selecting external information) thus seem to share processes and content, indicating that WM may best be thought of as attention to internal representations. Mechanistically, we assume that this interaction between internal and external selection processes exists because WM entails endogenously activating perceptual long-term memory representations in sensory cortex (e.g., visual neurons representing particular color or shape features), which in turn primes these neurons to be more easily activated by external stimuli matching their preferred features (cf. Desimone & Duncan, 1995; Postle, 2006).

Properties of the Guidance of Visual Attentional Selection by Internal Representations

While there have been many observations of external attentional capture by internally maintained material, it is of great interest how and at what stage of processing the internal content has its influence. It could, for instance, alter early perception of external stimuli, or create a later decision bias towards particular items—an understanding of which would clarify the mechanism linking internal and external attention. If, for instance, irrelevant memory content has a behavioral impact similar to that of cueing attention, it would support the notion that WM representations are akin to internally attended items. There is some behavioral evidence that an object maintained in WM can act “early” after the onset of a visual search array and may enhance perceptual selection among competitors. First, the initial saccade of search is influenced by the relationship between internally and externally attended information (Olivers et al., 2006; Soto et al., 2005), as are the fastest 5% of RTs (Soto et al., 2005). Second, using a signal detection analysis, Soto and colleagues (2010) determined—just as perceptual sensitivity is enhanced at selectively attended spatial locations (Bashinski & Bacharach, 1980)—that items held in WM actually impact perceptual processing (but see Cosman & Vecera, 2011). In accord with earlier studies indicating an attentional boost for items matching the contents of WM, A’—a nonparametric index of perceptual sensitivity—was increased for targets that coincided with memory items (relative to when the memory item occurred with a distracter), even with highly salient targets and small search arrays, and even when the memory-matching item was more likely to be a distracter than a target.

Finally, internally attended information can override or modulate the influence of bottom-up attention-capturing features. Perceptually prominent—or “pop-out”—targets are one means of producing bottom-up, or externally generated salience. While it might be expected that bottom-up salience would supersede the impact of irrelevant WM content, Soto, Humphreys, and Heinke (2006) found WM content to exert an early and involuntary influence in the search for a pop-out target, just as top-down attention strategies can countermand capture by bottom-up salience (e.g., Folk, Remington, & Johnston, 1992; Leber & Egeth, 2006). Even in efficient search, when targets were easily detectable based on color or shape differences from distracters, internally maintained WM contents conferred costs and benefits—depending on whether memory items were related to targets or distracters—further supporting the notion that the effects of WM contents on external selection processes are comparable to those of attention. Just as when attention is endogenously guided by spatial cueing (e.g., Posner, 1980) or the holding of a search template (e.g., Wolfe, 1994), internally maintained representations can have effects early on in the search for an unrelated target, and enhance perceptual sensitivity to matching items in the environment.

While internally attended representations may be able to influence early perceptual processing based on visual feature similarity, however, they can also impact external attention via more abstract stimulus dimensions. Memory-matching items have been shown to impact visual search whether the WM cue was presented visually (e.g., an image of a red circle) or verbally (e.g., the words “RED CIRCLE”). The efficiency of visual search, moreover, is influenced by the semantic relatedness of WM items, attention targets, and distracters (Balani, Soto, & Humphreys, 2010), similar to the manner in which visual search may be impacted by knowledge from context (e.g., Chun & Jiang, 1998) or long-term memory (e.g., Henderson, Malcolm, & Schandl, 2009). Even when WM and search stimuli do not share any features (physical or semantic), WM task instructions can influence an intervening attention task (Zhe Chen & Tsou, 2011; Pan, Xu, & Soto, 2009). Thus, like attentional biasing, actively maintained items can impinge on both early and late stages of processing in favor of matching items in the environment. Whether by way of shared physical features, semantic qualities, or task instructions, items held in WM influence what gets attended in the external world.

Boundaries of the Impact of Internal Representations on External Attention Allocation

Effective Prioritization of the External Search Target

As described above, what gets selected in the environment is interrelated with what is attended internally. Those guiding internal representations could be items for active maintenance in the absence of sensory input (WM) or search templates for an external task. While there is abundant evidence that WM contents will drive attentional selection, several studies have found null or contradictory results, revealing conditions under which search targets can be effectively shielded from, or prioritized over, the memory contents. If WM is akin to limited attention resources directed at internal items for active maintenance, we would expect that such attention would be unable to be sustained toward internal and external domains simultaneously. Consequently, in some circumstances WM representations would gain attention and in others a search template for an external task would win. According to a recent theory of the relationship between representations in visual WM and attention, items can be assigned different activation statuses (Olivers, Peters, Houtkamp, & Roelfsema, 2011). Only one item receives the prioritized focus of attention, but others are able to occupy an “accessory state”—meaning still active and accessible, but not the center of attention (see also Oberauer, 2009). At the neural level, this could correspond to a graded endogenous activation distributed over multiple neuronal ensembles, where at any given time only one ensemble (e.g., visual neurons responsive to yellow triangles) is primed to a degree that is sufficient to substantially lower its threshold for externally evoked activation. Multiple representations may alternate in the focus of attention, however, and each may be able to guide external attention depending on their current status (cf. Beck, Hollingworth, & Luck, 2012). This framework offers a possible means by which internally attended content may impact visual attention allocation, and explains why in some cases the search target may be protected from or prioritized over the memory content.

In keeping with the activation model of when internal representations will impact selection, it seems that those representations must be actively maintained in order to involuntarily capture external attention; merely priming a stimulus is not enough. When participants are just shown a colored shape, for instance, but not instructed to remember it, the item has no impact on reaction time or eye movements in a subsequent visual search among colored shapes (Hollingworth & Luck, 2009; Soto et al., 2005, 2010). Likewise, if a memory is probed prior to a visual search—and the information thus no longer needs to remain active—it also exerts no influence on the allocation of visual attention (Olivers et al., 2006; Pan et al., 2009). Thus, when attention resources are not directed at internal representations for active maintenance—and can be focused on the external task—those representations do not seem to impinge on the allocation of visual attention.

WM representations also appear to have a diminished impact on external selection when memory and search templates are perceptually and/or semantically similar (Downing & Dodds, 2004; Houtkamp & Roelfsema, 2006; Peters, Goebel, & Roelfsema, 2009; Woodman & Luck, 2007). In one example (Woodman & Luck, 2007), participants were asked to maintain a colored square memory item over a delay, with an intervening visual search discrimination task for the location of a gap on a colored square. When memory items could only reappear as distracters—and it was therefore never beneficial to attend to the memory-matching item in the array—there was no significant effect of the reappearance of a memory-match, though search RTs were slightly (insignificantly) faster when this was the case. When the memory item matched the target on a small percentage of trials—and was therefore sometimes beneficial—visual search was fastest when the memory item matched the target, and slowest when it matched a distracter. WM content, therefore, does not automatically capture external attention. Rather, internal attention is allocated differently depending on task conditions and goals, and subsequently varies in its impact on external attention deployment.

Woodman, Luck, & Schall (2007) speculated that a search target template is only stored in WM when it varies from trial to trial. When a search target changes from trial to trial, likewise, the guidance of external selection by unrelated WM representations is typically diminished or abolished (Downing & Dodds, 2004; Houtkamp & Roelfsema, 2006; Olivers, 2009; Peters et al., 2009). In other words, the search template seems only to retain the priority status in the focus of attention when it is variable and more demanding to maintain (i.e., requires internal attention), and thus wins the competition for selection. When the search template is stable from trial to trial—and can presumably be shifted to an accessory state—the actively maintained WM content is internally attended and liable to influence selection. In one such demonstration (Downing & Dodds, 2004), participants were simultaneously shown a search target and a memory target—both selected from a set of abstract shapes and varying from trial to trial—and subsequently performed a search task or a memory task. Participants were faster to detect the search target when the non-matching memory item was present, suggesting that the internal maintenance of the search target conferred a competitive advantage for selection of that target—and deprioritization of the irrelevant memory content—over distracters during visual search. A later study used line-drawings of real-life objects—varying from trial to trial—as both search and memory stimuli (Houtkamp & Roelfsema, 2006). Participants were shown two items that served as targets for two back-to-back search tasks. The second search target, thus, needed to be maintained in WM during performance of the first search. When the second target appeared in the first array it was considered a memory item, and its presence influenced neither search accuracy nor response times on target-present trials. In this task context—one of several to use a changing search target and yield similar findings (e.g., Downing & Dodds, 2004; Olivers, 2009; Peters et al., 2009)—it appears as though the search template maintained priority (internal attention focus) status and was able to guide attention more so than other memory contents, which were presumably conferred accessory status. In addition to the changing search target—which likely necessitated internal attention—the above studies used complex objects and shapes as stimuli (cf. Zhang et al., 2010), which were likely demanding to maintain, and may have provoked their retention in the focus of attention.

Even in some of these studies (above) that call into question the involuntary guidance of attentional selection from internally maintained information, however, such guidance is often observed when the visual search target is absent (Balani et al., 2010; Downing & Dodds, 2004; Houtkamp & Roelfsema, 2006). In these experimental paradigms—with variable search targets from trial to trial and complex, categorically similar memory and search stimuli—the search target template appears to be effectively prioritized and may consequently win the competition for selection when it is present. When the search target does not appear in the display, on the other hand, search-irrelevant memory-matching items may gain a relative priority status and exert an influence on search beyond that of other distracters. Thus, actively maintained internal representations guide perceptual selection, and the extent to which this is favorable or detrimental is determined by the internal content to which attentional resources are deployed and its relevance to the external task.

Modulation of the Impact of Internally Attended Information

As elaborated upon above, an actively maintained search template, stimulus complexity, and similarity between memory and search targets appear to be able to reduce or eliminate the impact on selection of irrelevant internally maintained information (i.e., undermine its priority status); variations in WM and cognitive load have been shown to likewise modulate the extent of attentional guidance by internal representations. When a single item is maintained in WM, search performance is commonly impaired when the memory item appears as a distracter (e.g., Olivers et al., 2006; Soto et al., 2005; Soto & Humphreys, 2008). When, on the other hand, subjects have had to maintain an additional memory item and engage in an articulatory suppression task (Soto & Humphreys, 2008), the reappearance of the memory item in the search display made very little difference in search performance (although neither increased memory load nor articulatory suppression alone hampered the typically-observed capture effect). Several studies have employed articulatory suppression as a means of encouraging the WM content to be coded visually rather than verbally (e.g. Downing & Dodds, 2004; Woodman & Luck, 2007). When the memory content has subsequently had no impact, it has been taken as evidence that irrelevant internal representations only win the competition for selection when they are verbally rehearsed. Another possibility, though, is that activation of the internally attended content is attenuated when attentional resources must be diverted to the articulation task. Even without the use of concurrent articulatory suppression, nonetheless, increasing the memory load to four items has been demonstrated to diminish the ability of the internal memory content to capture external selection (Zhang, Zhang, Huang, Kong, & Wang, 2011). In certain task contexts, neither a moderately increased load nor a concurrent task seem to be sufficiently attention-demanding to prevent the WM content from influencing visual search; when load is more substantially amplified or combined with concurrent articulation, and attention resources must be dispersed across several items and tasks, however, any given memory item may be less likely to occupy the influential focus of attention, and thus guide selection.

Top-down control over biases of selection

As indicated by the variations in task (e.g., variable search template) and stimulus qualities (e.g., complexity, similarity between memory and search targets) that can modify attentional guidance by WM contents, there are limits to unintentional biases of selection by internally attended items. Involuntary shifts of attention have been found to depend on the attentional control settings invoked by task demands (Folk, Remington, & Johnston, 1992), and goal-directed selection of specific features can override salience-driven capture of attention (Bacon & Egeth, 1994), hinting at the potential for strategic control to alter the prioritization (or attention allocation) of internal representations and their subsequent impact on external selection.

When the location of an upcoming search target is spatially pre-cued, for instance, the influence of a memory-matching item in the search display can be eliminated (Pan & Soto, 2010). Some researchers further suggest that when processing is extended in time—and, presumably, cognitive control processes are given time to come online and intervene—memory-matching distracters will be less disruptive to visual search (Han & Kim, 2009). The influence of memory-matching search items has likewise been observed to be diminished at protracted search durations and intervals between stimuli (Dombrowe, Olivers, & Donk, 2010, Expt 1; Han & Kim, 2009). In other studies, short and long duration search displays have been equally susceptible to influence from internal memory representations when they occured in random order, but the magnitude of this impact was substantially decreased for short duration displays when they were blocked and predictable (Dalvit & Eimer, 2011). Hence, it appears that the search task (external allocation of attention) can be strategically prioritized over the memory task (internal allocation of attention) when allowed or required by the temporal characteristics of the search.

The degree of external attentional capture by irrelevant internally attended representations can also be modulated by the probability that an actively maintained memory item will coincide with the search target. When the memory content is sometimes (as opposed to never) a valid predictor of the search target location—and may thus be worth attending—preferential eye movements towards memory-matching search stimuli are greater (Soto et al., 2005), as is slowing of search by a memory matching distracter (Woodman & Luck, 2007). When memory-matching to targets versus distracters is varied—following the logic of classic attention studies (Posner, 1980)—increased probabilities of valid trials (memory item matching search target) amplify the benefits of matching targets, while increased probabilities of invalid trials (memory item matching search distracter) diminish the costs of non-matching targets (Carlisle & Woodman, 2011a). Thus it appears that participants can strategically attend to memory-matching search stimuli when they are reliably helpful (i.e., predictably correspond to targets), and can likewise willfully avoid memory-matching search stimuli when they are predictably harmful (i.e., correspond only with distractors). Even when WM items are 100% predictably invalid (always coinciding with a distracter), however, they can still slow search for an unrelated target (Kiyonaga, Egner, & Soto, 2012)—indicating that external attentional capture by internally attended representations is partly, but not completely, modifiable by top-down control.

In summary, in some task contexts the irrelevant actively maintained information dependably interferes with visual search, while in others the search target is able to consistently override the influence of stimuli matching those unrelated representations. This prioritization of one locus of attention over another can also be modulated within an experimental paradigm by the application of strategic, top-down control. Just as top-down control of attention can be applied to give precedence to certain stimuli in a visual scene or auditory environment, internal attention can be strategically controlled to modify its impact on external selection. The observation that preference can be given to either the internal memoranda or a relevant external search target suggests that limited attentional resources can be applied to either domain, but there is a trade-off between the two.

Neural Mechanisms of Internal Biases of External Selection

While a number of studies have used neural data to illuminate the relationship between WM and attention, it is not yet well-understood how the WM capture of visual attention, specifically, is reflected in the brain. A few studies have begun to shed some light on the timing and localization of the neural mechanisms at play, which can further inform models of the broader relationship between internally and externally geared attention. As described above, when the memory and search targets are from completely different categories (e.g. remember a colored shape, search for a slanted line) visual attention is typically captured by stimuli matching the internally maintained representations. This capture is also reflected in electrophysiological measurements from participants performing this sort of task. One such study found the N2pc ERP component to a search target to be enhanced when it also coincided with an internally maintained memory item (Mazza, Dallabona, Chelazzi, & Turatto, 2011). The N2pc—seen over visual cortex contralateral to an attended stimulus among distracters about 200 msec after stimulus onset—is used to isolate brain responses reflecting the focusing of attention (Luck & Hillyard, 1994), and is thought to originate in the extrastriate visual cortex (Hopf et al., 2000). In another study using a variant of the task from Soto et al. (2005), the N2pc had an earlier onset and greater amplitude when a memory-matching distracter fell on the same side of the display as the target, suggesting that there is facilitated selection of the side of space containing the target when a memory-matching stimulus is also present (Kumar, Soto, & Humphreys, 2009). Thus, just as the N2pc is sensitive to task set and endogenous cueing (e.g., Kiss, Jolicoeur, Dell’Acqua, & Eimer, 2008; Kiss, Van Velzen, & Eimer, 2008), it can be altered by irrelevant actively maintained information. Hence, in some task contexts, the behavioral impact of the reappearance of WM items may be explained by earlier and enhanced attentional focusing, reflected in the N2pc ERP component. In other cases—when the memory item resembles the search target and distracters perceptually and semantically—search targets appear somehow to be able to maintain their priority, and there is consequently an absence of any observed difference in the amplitude of the N2pc component in response to memory items versus targets in the search display (Carlisle & Woodman, 2011b; Peters et al., 2009).

A task paradigm nearly identical to that used by Soto et al. (2005) was also combined with functional magnetic resonance imaging (fMRI) to identify the brain regions involved in such attentional guidance (Soto, Humphreys, & Rotshtein, 2007). A network of superior frontal gyrus, midtemporal and occipital areas—which have previously been shown to be sensitive to stimulus repetition—was enhanced to the appearance of a stimulus in the search array matching actively maintained content (but decreased for the appearance of a merely primed stimulus). This enhanced response to the reappearance of internally maintained items (whether or not they are valid) corresponds to the behavioral capture of external attention by actively maintained memory-matching content (but not by merely primed cues). Transcranial magnetic stimulation (TMS) applied over the primary visual cortex also boosted the impact of an actively maintained memory-matching visual search target, but had an opposite effect when the cue was simply primed (Soto, Llewelyn, & Silvanto, 2012).

While several brain regions have been identified as being sensitive to the reappearance of a memory-matching item, regardless of its role in the search display, a fronto-thalamic network (bilateral anterior PFC peaking in BA 10 and bilateral thalamic nuclei including the pulvinar) activated most robustly to a match between the WM item and the search target, and the least for a match between the WM item and a distracter. These regions appeared to be sensitive to the behavioral relevance of the stimulus. Using a different experimental paradigm, involving emotional words and faces, the same set of regions were also identified as being sensitive to both the reappearance of memory-matching information, and its behavioral significance (Grecucci, Soto, Rumiati, Humphreys, & Rotshtein, 2010). These studies suggest that the anterior PFC compares internal and external representations and prioritizes them according to task goals, while the thalamus (in particular the pulvinar) may be involved in guiding visual attention based on the goal-relevant information held in the prefrontal areas. Parietal patients, furthermore, exhibit reduced extinction of contralesional search stimuli when they match an internally maintained item (Soto & Humphreys, 2006)—suggesting that perhaps the recruitment of the frontal, temporal, and occipital regions that are responsive to the appearance of memorymatching stimuli can partially overcome the attentional impairment due to parietal injury.

Imaging data have also been used to pursue a more specific explanation of the observed reduced visual search capture by WM items under load (Soto, Greene, Chaudhary, & Rotshtein, 2011). This study examined the functional connectivity—using a psychophysiological interaction analysis—between the prefrontal cortex and posterior occipital regions during visual search, while holding either one (low load) or three (high load) items in WM. Functional coupling was greater when a memory item reappeared matching a search target than a distracter, but was amplified in the low compared to the high load condition. The robust coupling between the prefrontal cortex and posterior visual regions during memory-matching visual search with low WM load suggests that attentional capture by memory items occurs via top-down modulation of sensory areas—just as attentional orienting and pre-target biasing of sensory cortices are thought to originate in frontal cortex (e.g. Kastner, Pinsk, De Weerd, Desimone, & Ungerleider, 1999). The disruption of this functional coupling under high WM load—along with greater frontal activation in this condition—implies that high internal processing loads may occupy limited neural resources and prevent the top-down modulation of posterior occipital visual areas that guides selection. The capture of visual attention by internally maintained items, thus, may only occur when cognitive and neural resources are available to bias sensory representations.

Finally, a recent study specifically investigated the neural substrates of top-down control over WM biases on visual selection, by contrasting fMRI data between conditions where WM cue validity for a subsequent visual search was either fully predictable (100% valid or invalid cues) or unpredictable (50% valid/invalid cues) (Soto, Greene, Kiyonaga, Rosenthal, & Egner, In Press). Accompanying significant behavioral modulations of cue validity effects by predictability, anticipatory biasing signals emerged in posterior parietal cortex (including intraparietal sulcus) when cue validity was predictable, followed by modulation of activity in posterior cingulate and medial temporal lobe (MTL) structures, including the hippocampus, during visual search. These observations suggest that parietal cortex interacts with the MTL to strategically enhance or attenuate the impact (or activation status) of the internal memorandum on the external selection process.

Reciprocity between Internal Maintenance and External Selection

As reviewed above, WM and attention have been shown to be closely linked. While it has long been contended that attention can influence the contents of WM (i.e. Baddeley & Hitch, 1974; Cowan, 1988), it conversely seems to be the case that the contents of WM influence what gets attended, both voluntarily and involuntarily. Not only, however, may there be a bi-directional relationship between attention to internal and external material, but this close linkage may be underpinned by their shared dependence on a single resource, resulting in a give-and-take competition. Although it is not directly addressed in the WM biasing of attention literature, this possibility is supported by a number of behavioral and neural observations that internal and external attention task demands bear on one another. Increased WM load can interfere with a simple non-verbal choice reaction time task (Chen & Cowan, 2009), increase susceptibility to visual distraction (de Fockert, 2001) and impair visual search (Han & Kim, 2004). Items can, likewise, be displaced from WM when a visual search must be performed (Woodman et al., 2007; Woodman & Luck, 2010; Woodman & Vecera, 2011), and events in the outside world can alter internal spatial maps (Theeuwes et al., 2009). Theeuwes and colleagues (2009), in fact, have described the close linkage between visual working memory, attention, and the preparation to perform an action, and suggested they all be unified into a single concept. Top-down (internal) and bottom-up (external) attention have been proposed to closely interact and influence each other in long-term memory retrieval (Cabeza, Ciaramelli, Olson, & Moscovitch, 2008), and a competitive interaction has, furthermore, been observed between brain regions responsive to long-term memory (internal) versus perceptual (external) information (Sestieri, Shulman, & Corbetta, 2010).

Spatial attention has been demonstrated to be oriented toward mental representations (Nobre et al., 2004), and such attention promotes WM retrieval (Lepsien, Griffin, Devlin, & Nobre, 2005). Identifying a target in both an externally displayed perceptual search array, and an internally maintained array that is no longer displayed, can elicit an N2pc ERP component of equivalent time course and scalp distribution, suggesting that these internal and external search processes occur by the same mechanism (Kuo, Rao, Lepsien, & Nobre, 2009). The magnitude of interference from distracting external stimuli—as measured by the neural response in visual cortex—has been shown to predict WM accuracy (Clapp, Rubens, & Gazzaley, 2010). Modulation of a prefrontal control region with TMS has, further, been shown to impact perceptual processing and WM accuracy (Zanto, Rubens, Thangavel, & Gazzaley, 2011), and training in perceptual discrimination has been shown to transfer to improvements in WM (Berry et al., 2010). WM and visuospatial attention tasks correspondingly engage overlapping fronto-parietal brain regions (LaBar, Gitelman, Parrish, & Mesulam, 1999; Mayer et al., 2007), and some of these regions have been identified as the seat of a unified attentional bottleneck for diverse attention-demanding operations, like perceptual encoding and decision-making (Tombu et al., 2011). One particular brain region, the intraparietal sulcus (IPS), has emerged as a likely candidate for the seat of a domain-general limited attention resource. This area has been shown to reflect the coordination of multisensory attention (Anderson, Ferguson, Lopez-Larson, & Yurgelun-Todd, 2010; Palva, Monto, Kulashekhar, & Palva, 2010), spatial and feature-based attention (Egner et al., 2008), and capacity limitations in working memory across modalities (Chein, Moore, & Conway, 2011; Cowan et al., 2011; Todd & Marois, 2004; Xu & Chun, 2006), as well as modulation of other brain regions based on distinct task demands (Majerus et al., 2006).

This attentional bottleneck has, furthermore, revealed itself behaviorally in the form of the attentional blink (wherein the processing of one stimulus can block awareness of another, the extent of which is modulated by processing load; Jolicoeur, Dell’Acqua, & Crebolder, 2001) and the psychological refractory period (manifested in an increase in reaction time to a second stimulus as the stimulus onset asynchrony decreases; Tombu & Jolicoeur, 2003), reflecting the limitation on attentional resources in time. Given such observed limitations, if internal maintenance requires the deployment of attention to mental representations, we would expect that WM demands might tie up resources and prevent attention to external stimuli. Using this same logic, we would conversely expect, when attention is allocated to external stimuli, that attention-demanding rehearsing or refreshing of internal representations would suffer.

A well-supported model of the relationship between storage and processing demands in WM, in fact, proposes that the amount of information that can be maintained will depend upon intervening attention-demanding processes, as storage and processing share a common resource that alternates between task demands (Barrouillet, Bernardin, & Camos, 2004). This time-based resource-sharing model does not explicitly address the relationship between internal and external attention, but has generally been investigated using some kind of external processing task during WM maintenance. In theory, however, any type of attention-demanding process (which could also be internal) should make use of the same common resource and have the same impact on storage capacity. Time-based resource-sharing advocates that (WM) storage capacity is constrained by the time consumption of concurrent (attention) processing (see Barrouillet, Portrat, & Camos, 2011 for review). For instance, color-naming of an incongruent color-word stimulus (e.g., the word “red” written in green ink) would be more disruptive to WM storage than a congruent stimulus (e.g., the word “red” written in red ink) because processing of the incongruent stimulus takes longer (Stroop, 1935), thus leaving attention less time or opportunity to refresh the memorandum. As the above-reviewed literature shows that irrelevant internally attended information can impair intervening external attention processing, the time-based resource-sharing literature demonstrates that unrelated (and in this case external) attention processing can impair concurrent internal maintenance.

Time-based resource-sharing has typically been studied using a WM span task that requires participants to remember strings of letters or words that are interspersed with some sort of attention-demanding processing task (e.g., simple equations or a Stroop task). This processing task can be made more or less time-consuming by varying the number of operations to be completed (i.e. more or fewer math equations) or by invoking lengthier executive processing (i.e. high or low Stroop conflict). The degree of this cognitive load is proposed to determine the extent of its impact on WM span. This is thought to occur when processing demands occupy the assumed common attentional resource, because there is no opportunity to rehearse or refresh the WM representations, and the memoranda therefore cannot be recovered (Barrouillet et al., 2004). Across many experiments (Barrouillet et al., 2004; Camos, Lagner, & Barrouillet, 2009; Camos, Mora, & Oberauer, 2010; Vergauwe, Barrouillet, & Camos, 2009, 2010) it has been demonstrated that WM span suffers when greater processing demands are packed into shorter periods of time. Thus it appears that the external, attention-demanding task impacts the ability to maintain internally attended information (though any reciprocal impact of the WM storage component on the effectiveness of the processing component has not been examined).

A Bi-Directional Tradeoff between Attention to Internal and External Information

While most prominent WM theories posit a critical role for attention in controlling access to and manipulation in active internal maintenance, the observed influence of the internal memory information on external attention allocation is also striking. What has been conspicuously absent from this literature is an examination of the impact that intermittent task demands (i.e., external visual search) might have on the maintenance of internal memoranda. Given the evidence for a domain-general, limited attention resource—as well as specific demonstrations of reciprocity between WM and attention—it seems only reasonable to assume that there may actually be a bi-directional, give-and-take relationship between attention to internal and external information. Internal memoranda can impact performance on an unrelated external task, but that unrelated attention task may in turn result in modulation of the memoranda. For instance, it has been found that processing efficiency is increased at internally maintained locations, but memory for those locations is impaired when attention to them is blocked (Awh, Jonides, & Reuter-Lorenz, 1998). Might it be the case that internally maintained representations are susceptible to alteration depending on the role that they play (and attention allocated to them) in the external environment? This would provide further evidence for the reciprocity between WM and selective attention, and the conceptualization of WM as internally-targeted attention.

Only a handful of the studies reviewed here examine performance on the WM probe as a function of the relationship between the memory content and search target. Most studies simply report the accuracy on this task component, to confirm that participants followed the task instruction of maintaining the item in memory. Several observe no significant variation in memory performance as a function of visual search task condition, but these studies are predominantly those that use only invalid (memory item matches a distracter) and neutral (memory item does not reappear), but never valid (memory item matches the target) cues. While we might expect that the reappearance of an invalid memory item as a distractor should produce worse memory performance than a neutral distractor (because of some attempt to inhibit or ignore it during the search), this difference is confounded with the fact that during invalid trials (as opposed to neutral) participants have the opportunity to resample or refresh their memory representation during the search display for the later memory test. Thus, it is critical to have a valid cue comparison condition, when the memory cue also reappears in the search display, but is helpful, rather than harmful, to search. From the few studies that meet this criterion and examine memory performance, a picture is emerging that the reappearance of memory-matching items in a visual search context can also impact the memory representation, depending on whether it is harmful or helpful to search.

When Woodman & Luck (2007) included some proportion of valid (WM item corresponds to search target) trials, they found not only that memory matching visual search targets produced the fastest responses, but that memory probe responses were more accurate following these valid search arrays. This suggests that the role of the internally attended content in an intervening task can also influence its representation. This finding cannot be explained by the simple reappearance of the memory item as an opportunity to refresh its representation, because invalid search arrays also featured memory-matching items, but did not produce a memory benefit. Carlisle and Woodman (2011a) also examined memory performance and found, in several different conditions, that accuracy for the memory item was best following valid trials and worst following invalid (WM item corresponds to a distracter) trials.

Grecucci et al. (2010) investigated whether emotional words kept in WM would steer attention towards faces expressing those emotions. Participants were asked to remember an emotional word for later recognition, and were then shown two faces among which they had to search for a face of a particular gender and indicate its position in the display. The faces expressed different emotional states, which were totally irrelevant to the search task, but could match the emotional memory word. Memory responses were faster when the emotional descriptor matched the target, and slower when it matched a distracter. Increased functional coupling between the pulvinar and temporal parietal junction when an internally maintained item matched a visual search distracter was associated with greater slowing of visual search (thus presumably more attention allocated to the memory-matching distracter) but better memory accuracy, further supporting the concept of a trade-off between the two (Rotshtein, Soto, Grecucci, Geng, & Humphreys, 2011).

Finally, Kiyonaga et al. (2012) explicitly examined performance on memory probes that only occurred instead of—never following—the search array. Memory performance was modulated by the context of validity within an entire block (which was either 100% valid, 100% invalid, or 50% valid/50% invalid). In this way, memory performance was not contaminated by any incentive to attend to the memory item in the search array for the purposes of the memory test, but only determined by the manner in which the WM content might be modulated in preparation to either enhance (when valid) or inhibit (when invalid) its impact on the search task. Indeed, WM recognition performance was slowest on blocks when the WM content was consistently associated with distracters—participants took longer to retrieve and report memory items when they were presumably strategically attempting to inhibit their impact on visual search. Thus, while few studies to date have explicitly tested for full reciprocity between internal maintenance and external selection processes, the extant data are very compatible with a bi-directional trade-off due to a shared resource.

Conclusions

As reviewed above, while one literature (on time-based resource sharing) has examined the impact of intervening external attention demands on WM span, a separate literature (on WM–visual search interactions) has investigated the impact of internally maintained memory items on external attention. Though they take different perspectives, both point to the notion that attention to internal representations—homologous with WM—can influence external attention allocation, and conversely that the conditions of external attention can have consequences for the internal representations. These literatures converge on the prospect that attention to internal representations and external stimuli occur via a shared resource, resulting in a competitive interaction between them. Within this framework, internally maintained items often win the competition for external attentional selection. While the external selection of items matching internal attention will bolster the representation of that item, the processing of other (potentially relevant) external stimuli will suffer. External stimuli, on the other hand, can also capture bottom-up attention and subsequently gain active internal maintenance. When that occurs, however, the representation of other internally maintained representations lose their priority status (i.e., focus of attention) and their processing will suffer.

At first blush, there are a number of seemingly diverging findings and theories regarding the relationship between attention and WM; we argue, however, that the current proposal may in fact be compatible with these observations. Specifically, some authors suggest that, counter to what would be predicted by a model in which the two share resources and compete, WM capacity is not limited by attention (Duncan, Schramm, Thompson, & Dumontheil, 2012; Oberauer, Lewandowsky, Farrell, Jarrold, & Greaves, 2012). Oberauer and colleagues (e.g., Lewandowsky, Oberauer, & Brown, 2009; Oberauer & Lewandowsky, 2008; Oberauer et al., 2012), for instance, assert that WM storage is not limited by the prevention of attentional refreshing (as predicted by the TBRS model) but mainly by interference from distractors, which require time to be removed from the limited capacity store. In the framework of the current proposal, we would suggest that the same limited capacity attention resource would be directed toward removing interfering representations from storage just as would be employed to refresh relevant representations. In this way, regardless of the specific hindrance of internal maintenance, its resolution requires some manner of attention.

In a number of different contexts, additionally, WM maintenance has been shown to be unimpaired by a concurrent task, demonstrating both the independence of WM and visual search operations (e.g., Woodman, Vogel, & Luck, 2001), and the separability of different domains within WM (e.g., Baddeley & others, 2003; Cocchini, Logie, Sala, MacPherson, & Baddeley, 2002). While these appear to undermine the current proposal, again, they are not truly in opposition. For one, we would only expect the competition between internal and external attention to be evident when the two are sufficiently taxed. Given enough time for attention to alternate between demands (whether due to lenient task timing or a sub-capacity load in either domain) this limitation would not reveal itself. Secondly, the current proposal makes no strong assertion about domain-generality or specificity within WM. Although the model is broadly consistent with a domain-general attention resource, this internal/external tradeoff could also occur within modalities or domains. A reconciliation between the current proposal and these seemingly contradictory findings would be that different neural regions and mechanisms may handle information in distinct domains (e.g., verbal vs. spatial), and information can be maintained in those regions for some amount of time (perhaps due to a sluggish, non-instantaneous decay of neural activity) without focused attention from the shared resource. When either are maximally loaded, however, the attention resource that is responsible for a host of actions (like processing, manipulating, refreshing, inhibiting, etc.) becomes strained, and performance suffers in one or both domains.

Our understanding of the give-and-take relationship proposed here is in its relative infancy, and is ripe for future study. Because WM and attention have generally been considered separate constructs, the possibility that they reflect a shared cognitive resource directed at prioritizing internal and external information, respectively, has rarely been explicitly investigated. The neural mechanisms of this interaction are, additionally, little understood, and should be a fruitful avenue for further clarifying the determinants and means of attention distribution. There is little question that WM and attention are related, and there is now abundant evidence that attention-demanding processes can impinge on one another in their use of a common resource. A parsimonious explanation of the seemingly complicated relationship between these cognitive demands may therefore be a unified, capacity-limited cognitive resource that is alternatingly allocated toward internal and external information. The extent to which this resource is dedicated to one domain over the other can be influenced by task goals and other internal representations, which will determine the external stimuli selected for processing. Conversely, because these attention domains influence each other bi-directionally, externally selected stimuli can also alter internally maintained representations, including task goals. In this way, we can understand why our visual attention might be captured by an irrelevant object that coincides with our internal thoughts, but also why we might update our immediate priorities when external stimuli demand our attention. This account thus offers an explanation for the observed interactions between WM and attention, as well as a new framework within which to study limitations on, and effective prioritization of, attention allocation to our internal thoughts and goals versus pressing external demands. In many circumstances, our internal goals apply to an external task. It is those instances when the two compete, however, that will inform the comprehension of these critical cognitive capacities on which we rely during every waking moment.