Memory: Enduring Traces of Perceptual and Reflective Attention

Abstract

Attention and memory are typically studied as separate topics, but they are highly intertwined. Here we discuss the relation between memory and two fundamental types of attention: perceptual and reflective. Memory is the persisting consequence of cognitive activities initiated by and/or focused on external information from the environment (perceptual attention) and initiated by and/or focused on internal mental representations (reflective attention). We consider three key questions for advancing a cognitive neuroscience of attention and memory: To what extent do perception and reflection share representational areas? To what extent are the control processes that select, maintain, and manipulate perceptual and reflective information subserved by common areas and networks? During perception and reflection, to what extent are common areas responsible for binding features together to create complex, episodic memories and for reviving them later? Considering similarities and differences in perceptual and reflective attention helps integrate a broad range of findings and raises important unresolved issues.

Different research traditions tend to emphasize either attentional phenomena (Posner et al., 2007) or memory phenomena (Eichenbaum et al., 2007; Squire and Wixted, 2011). Although this “divide and conquer” approach has been extremely useful for advancing knowledge about cognition, there is increasing recognition that fully understanding each may entail understanding the other. Here we focus on the similarities and differences across these domains and an emerging picture of how they interact. We build, in particular, on two previous theoretical frameworks: the external/internal taxonomy of attention (Chun et al., 2011) and the Multiple-Entry-Modular (MEM) model of cognition, which views memory as traces of perceptual and reflective processing (Johnson and Hirst, 1993). Drawing on these ideas, and empirical findings that they incorporate, we propose an integrative Perceptual/Reflective Attention/Memory (PRAM) framework, which serves to organize current findings and theoretical ideas regarding the relation between attention and memory, and which highlights key unresolved questions.

Cognition can be broadly divided into perceptual processes initiated by and/or directed at external sensory information from the environment, and reflective processes initiated by and/or directed at internal mental representations. Perceptual processes operate on “incoming”, external stimuli (e.g., reading text, listening to a song). Reflective processes are directed at internal representations, such as thoughts, memories, imagery, decision options, problem solving, and self-directed processes. That is, reflective processes can operate on representations in the absence of corresponding external stimuli or independent of current external input (e.g., thinking about what to have for dinner, remembering a friend’s remark).

At any given moment, not all features, objects and events in the environment or in the mind can be processed equally (Marois and Ivanoff, 2005). Both perception and reflection are inherently selective, requiring mechanisms of attention – modulating, sustaining, and manipulating the information that is most relevant for current and/or future behavior (Chun et al., 2011). The byproducts of these perceptual and reflective attentional processes are registered as changes or records in the cognitive system, changes that we call “memory”.

Although the border between perceiving and reflecting can be fuzzy, there are meaningful differences. Logically, perceiving and thinking are unlikely to engage exactly the same neural hardware or have exactly the same memorial consequences. That would produce an epistemological quagmire in which we could not tell fact from fantasy in perceiving, thinking or remembering (Johnson, 2006). On the other hand, if there were no interaction between perception and reflection, we would not be able to constructively and creatively cumulate knowledge across experiences of perceiving and thinking. To what extent do perception and reflection activate the same representational and processing regions? To what extent do they have similar and different memorial consequences? Under what conditions do they operate independently and by what mechanisms do they interact?

Our review and PRAM framework lead us to several hypotheses that invite further testing. (1) Perception and reflection engage some of the same areas (e.g., posterior sensory areas) for representing information (e.g., concrete items such as objects, faces and scenes). However, the extent to which they engage the same or different representations within these areas is an open question. The degree of overlap should predict the extent to which perception and reflection influence each other and how likely they are to be confused, for example, in source memory tasks. (2) Perception and reflection both involve frontal and parietal regions that control the direction/focus of attention. However, whether they engage the same regions for similar cognitive functions is an open question, and should dictate when perception and reflection interfere with or facilitate each other (3) It is well accepted that the hippocampus/medial temporal lobe region associates attended information with other existing representations throughout the brain (Davachi, 2006; Ranganath, 2010). The resulting configural representations bind multiple features (e.g., perceptual, spatial, temporal, semantic, emotional details) together and give representations an episodic quality (i.e., they have source or contextual information). Are there differences in configural processing active during both perception and reflection?

Perceptual attention

Sensory information (e.g., sights, sounds, smells) arrives from different locations in space and points in time. Perceptual attention selects and modulates this information according to current task goals. In the PRAM framework, such processing yields persisting records (traces or memories). Because most research has used visual stimuli, our review will focus on the visual modality.

Limited processing capacity prevents equal attention to all items, but when cues direct selective attention to specific locations, perceptual performance is enhanced for cued items (Posner et al., 1980), as is memory for these attended items (Eger et al., 2004; Uncapher et al., 2011). Perceptual attention can also select on the basis of features. In a classic study, Rock and Gutman (1981) showed participants two abstract shapes that spatially overlapped on each trial. One shape was red and one was green, and each participant was told to attend to shapes in only one of the colors. Shapes in the other color were poorly remembered later, even though they spatially overlapped with attended shapes that were remembered. Even with only brief exposures, we appear to store a great deal of detailed perceptual information about selected information (Hollingworth and Henderson, 2002; Potter, 1976). For example, in one study (Brady et al., 2008), participants saw 2,500 pictures of objects, each for 3 sec, with instructions to try to remember them. On a later forced-choice recognition test, participants selected the correct previously seen item 92% of the time; even more remarkable, performance was still 87% when participants were required to discriminate between an original picture and the same object in a different state or orientation. In priming studies using even briefer presentations, and no instructions to remember, participants show memory for quite specific representations, although participants do not consciously recognize the items (repetition priming; e.g., Tulving and Schacter, 1990; Wiggs and Martin, 1998; Henson and Rugg, 2003). For example, if participants saw an item for 1 sec, they were subsequently better able to identify it under degraded stimulus conditions, even when they did not remember having seen the item before (e.g., Jacoby and Dallas, 1981). A neural measure of such behavioral priming measures of implicit memory is provided by repetition suppression or attenuation during fMRI: Compared to the first exposure to a stimulus, repeating a stimulus results in less activity in representational areas that are active when stimuli of that class are perceived (Grill-Spector et al., 1999; Schacter and Buckner, 1998). For example, object repetition attenuates activity in the lateral occipital complex (LOC, Grill-Spector et al., 1999; Buckner et al., 1998), while face repetition and scene repetition attenuation effects are found in the fusiform face area (FFA, Jiang et al., 2006) and the parahippocampal place area (PPA, Epstein et al, 2000), respectively.

Perceptual attention enhances stimulus-specific representations, as measured with fMRI repetition attenuation. An object appearing in a cued location shows more repetition attenuation than an object appearing in an uncued location (Eger et al., 2004; Chee et al., 2007). In one study, participants were presented on each trial with a face and scene that overlapped spatially and were cued to attend either to the face or the scene. Repetition attenuation was observed in PPA when scenes were repeated on a subsequent trial only when participants were instructed to attend to the scene on both the first and second presentation (Yi and Chun, 2005). Thus attention gates the learning that occurs incidentally during perception, and the expression of learning on implicit tests.

Reflective attention

Component processes of reflection (Johnson and Hirst, 1993) are the cognitive elements of what is often referred to as controlled/executive processing or working memory (Baddeley, 1992; Smith and Jonides, 1999). Refreshing is the act of briefly thinking of, and thereby foregrounding, a percept or thought that was activated moments earlier. Rehearsing maintains information, typically several items, in a phonological loop, over longer intervals of several seconds (Baddeley, 1992). (Ranganath et al., 2005 make a similar distinction between early and late maintenance processes.) Selectively refreshing an activated representation of a perceptual stimulus that has just disappeared, a thought that just became active, or an item that is currently in an active rehearsal set, boosts the strength of that item relative to other active items, making it the focus of attention (Cowan, 2001) and giving it a competitive advantage for additional processing. Thus, refreshing and rehearsing, individually and together, constitute reflective attention that selects, maintains, and manipulates the contents of working memory.

Reviving representations that are not currently active involves the component processes of reactivating or retrieving. Once revived, these longer-term memory representations can be further extended briefly by refreshing and/or rehearsing them. Other reflective processes include noting and/or discovering relations among active representations (processes underlying, for example, elaboration and organization of information), and initiating or shifting between representations, features, tasks or goals (processes needed, for example, to control or manipulate the sequence of mental events).

Thus, reflective attention selects, maintains, and manipulates information from working memory and long-term memory, and promotes long-lasting memories (Craik and Lockhart, 1972; Roediger and Karpicke, 2006; Tulving, 1962). For example, in one study comparing refreshing to perceptual repetition, participants viewed and read aloud words as they appeared one at a time. Some words appeared and were read aloud only once, some words appeared and were read aloud twice in succession (repeated – perceptual processing), and other words were read once and followed by a cue that signaled participants to think of (refresh) the immediately preceding word and say it out loud. A surprise test at the end of the experiment revealed greater recognition memory for words that had been refreshed than words that had been read once or read twice (Johnson et al., 2002). Even greater effects on long-term memory are yielded when information is reactivated and retrieved on different occasions over time (Roediger & Karpicke, 2004). If accurate source features are revived, reflectively reviving events can protect against memory distortion (Henkel, 2004).

To What Extent do Perception and Reflection Share Representations?

Do representations that are the outcome of perceptual attention also serve as targets for reflective attention? Reflection modulates activity in many of the same representational areas as perceptual attention. For example, both refreshing and rehearsing modulate activity in posterior areas involved in perception (Curtis and D’Esposito, 2003; Harrison & Tong, 2009; Johnson et al., 2009; Ranganath et al., 2005). Johnson et al. (2009) directly compared selective perceptual and reflective attention and found similar effects on sensory representations (Figure 1). Participants were shown a scene and a face on each trial and either cued in advance to attend perceptually to the scene or face, or cued after the stimulus was removed to refresh the scene or the face. Both perception (attend) and reflection (refresh) showed comparable enhancement and suppression effects relative to a passive viewing condition.

Figure 1. Comparing reflective (refresh) and perceptual (orient) attention.

Task design. (A) Refresh condition: Participants first saw a fixation cross, then a face and a scene picture, followed by an arrow cueing them to briefly think back to, or visualize, either the face or the scene, as indicated by the direction of the arrow. (B) Orient condition: Participants first saw an arrow, cueing them to look only at the picture on the left or right side of the screen. This was followed by a face and a scene picture, and then a fixation cross. (C) Act (control) condition: Participants first saw a fixation cross, then a face and a scene picture, followed by a gray square cueing them simply to press a button (and not think about either picture). (D–J) Activation for each of seven scene-selective regions of interest across the five conditions of the task. (K) Locations of these regions overlaid on the MNI single-subject template brain (Adapted from Johnson and Johnson, 2009, Journal of Cognitive Neuroscience).

Although perceptual representations and refreshed representations in working memory may engage the same brain areas, long-term memory could reside in areas different from those of the processes that gave rise to them (Barsalou et al., 2008). However, fMRI evidence suggests that long-term memory often involves reactivation of the same areas engaged during encoding. Retrieving visual events during long-term memory tasks activates visual cortex, while retrieving auditory events from memory activates auditory cortex (Wheeler et al., 2000). Importantly, the extent to which encoding activity is reinstated during long-term remembering, depends in part on what reflective agenda is engaged during remembering (McDuff et al., 2009).

Further evidence that perception and reflection may each later re-engage the same representations comes from a study in which Turk-Browne et al. (2006) examined how repetition attenuation in scene-selective regions was related to measures of explicit subsequent memory. Each scene was repeated somewhere in the study sequence and the experimenters sorted the data according to whether or not each scene was later recognized. Repetition attenuation in PPA (and behavioral priming) on the second presentation were only significant for repeated items that were later remembered (see also Gonsalves et al., 2005; Chee et al., 2007), consistent with the idea that repetition attenuation (a perceptual effect) draws on the same level of representation (PPA) as does the phenomenal experience of remembering. Other evidence that reflection and perception can operate on the same representations comes from an fMRI study that measured repetition suppression to assess representational strength of previously viewed and previously refreshed scenes. There were similar levels of repetition suppression in PPA for items seen and refreshed once, as for items seen twice (Yi et al., 2008). The impact on long-term memory from viewing an item once and refreshing it was equivalent to having seen the item twice. This provides strong evidence that refreshing active representations of perceptual events engages the same representation (not simply the same representational area), and that the consequences last beyond a few seconds. These findings also support the idea that perception and reflection interact to influence memory through the engagement of common representations. Other evidence that perception and reflection can share common representations is that a reflective representation may serve as a template for perceptual attention (Olivers et al., 2011). Additional research is needed to clarify to what extent individual memories can be decoded from brain activity at test. Currently, decoding category-specific activity within ventral cortex during recall, using multi-voxel pattern analysis (MVPA, Polyn et al., 2005), can signal the class of an item one is likely remembering (e.g., scene, face, object). Also, the ability to discriminate more specifically what a person is remembering is starting to show promise. In a face recognition task, MVPA reliably decoded whether or not participants said they had seen faces but not whether they had actually seen them (Rissman et al., 2010). This is consistent with behavioral and fMRI evidence that true and false memories are attributions about mental experiences based on their qualitative characteristics (Johnson, 2006; Mitchell et al., 2009). Mental imagery of specific visual orientations can be decoded above chance from low-level visual cortex (Kamitani and Tong, 2005), and mental imagery of a small set of well-learned, scenes can be decoded above chance in scene-sensitive cortex (Johnson, 2011) MVPA of the hippocampus can differentiate episodic memories of three film clips of everyday actions (Chadwick et al., 2010). Furthermore, perceptual representations of more diverse natural images can be reconstructed using quantitative receptive-field models that characterize the relationship between visual input and fMRI activity in early visual cortex (Naselaris et al., 2009; Nishimoto et al., 2011). Collectively, these promising findings (see Danker and Anderson, 2010), suggest that decoding of more specific memory representations, at least of visual images, may be possible within the next few years.

There is utility in having both perceptual representations that are more specific, and reflective representations that are more abstract and global. PRAM posits that classifiers should transfer across perceptual and reflective tasks more successfully for more abstract, global representations. Different brain regions represent different types of information in perception (Bar, 2004; Epstein et al., 2007; Park et al., 2007; Park & Chun, 2009) and we would expect people to be differentially successful in representing such information during reflection. For example, PPA represents scene details whereas retrosplenial cortex (RSC) represents less viewpoint specific, more global information that relates a scene to the larger environment (Epstein and Higgins, 2007; Bar, 2004; Park et al., 2007). In a direct comparison of perceiving and refreshing stimuli across several areas of visual cortex (perception showed greater activity in middle occipital gyrus and PPA than did refreshing, but there was little difference between perception and refreshing in RSC and precuneus Johnson et al., 2007),. At least for the hierarchy of visual processing, PRAM predicts that perceptual and reflective representations should be more confusable in high-level areas than in mid-level or low-level visual areas. Indeed, in subsequent memory tasks, precuneus activity during imagery is associated with later false memory for the imagined items (Gonsalves et al., 2004). Thus, understanding similarities and differences in how different brain regions represent perceptual and reflective information may help explain cases where the distinction between perception and reflection breaks down, such as in schizophrenia (Simons et al., 2006) or false memories for childhood events (Loftus, 2003).

Because even a simple stimulus such as a face or scene is not represented only in one area, the relative contribution of different regions to perceptual and reflective representations is a potential way we discriminate between them. Cross-validation of classification on brain activity engaged during perception and reflection would be interesting not only for explicit memory tasks, but also for implicit memory tasks. At what level of specificity can we decode representations, even those not giving rise to the subjective experience of recollection or familiarity? And using combined information from multiple brain areas, can a classifier do better than participants in discriminating between real and false memories?

Control of Perceptual and Reflective Attention

For both perception and reflection, control mechanisms select, and sustain processing of target information, and combat interference from perceptual or mnemonic distraction.

Attention to perceptual events recruits frontal and parietal areas that modulate and maintain activity in other brain areas. For example, changes in activity in posterior representation areas as a function of attention are accompanied by increased activation in frontal eye fields (FEF) and dorsal (SPL, IPS) and ventral (IPL, SMG, TPJ, AG] parietal cortex (Corbetta et al., 2000; Hopfinger et al., 2000; Kastner et al., 1999). Such activity supports perceptual awareness (e.g., Asplund et al., 2010; Dehaene et al., 2006).

Reflective processes also depend heavily on frontal and parietal mechanisms. Refreshing typically activates left dorsolateral prefrontal cortex and left parietal regions (SMG and PCu) (Raye et al., 2002). Refreshing one among several active representations (Johnson et al., 2005) also recruits anterior cingulate cortex (ACC, an area associated with competition, Carter et al., 1998) and left ventrolateral PFC (BA 45, an area associated with resolving interference, D’Esposito et al., 1999; Thompson-Schill et al., 1997). Initiating refreshing, or shifting between refreshing and another task agenda, recruits left rostrolateral PFC (BA 10, Raye et al., 2007), an area associated with task switching, engaging subgoals, and attending to internal representations (Braver and Bongiolatti, 2002; Burgess et al., 2007; Henseler et al., 2011). In contrast, rehearsing information tends to recruit left ventrolateral PFC (BA 44), premotor, pre-SMA and parietal cortex (SMG) (Chein and Fiez, 2010; D’Esposito et al., 1999; Raye et al., 2007; Smith and Jonides, 1999;). Tasks requiring both maintenance and manipulation typically show both VLPFC and DLPFC activity (Cohen et al., 1997).

The frontal and parietal areas active during refreshing and rehearsing are typically found in more complex tasks requiring executive function (Duncan and Owen, 2000; Smith and Jonides, 1999). That is, the foregrounding (refreshing) of task-relevant information within working memory is important for most executive tasks that involve selective attention, task maintenance, task switching, or manipulation of information (Miller and Cohen, 2001; Duncan and Owen, 2000; Smith and Jonides, 1999). Furthermore, encoding activity in regions associated with component processes of reflective attention predicts long-term memory. Greater activity in DLPFC during refreshing at encoding is associated with better subsequent long-term recognition memory (Raye et al., 2002). Rote (phonological) rehearsal is associated with activity in left ventrolateral PFC, as well as supplementary motor area (SMA) (Jonides et al., 1998). Amount of activation in these regions when participants are instructed to rehearse predicts subsequent recognition memory (Davachi et al., 2001). Relational processing at encoding is associated with increased activity and subsequent memory effects in ventrolateral PFC and, especially, dorsolateral PFC (e.g., Blumenfeld and Ranganath, 2007; Staresina and Davachi, 2006).

Neural activity that occurs during remembering has also been vigorously investigated. Many studies show activity in DLPFC and VLPFC during recognition and recall in long-term memory tasks, and there are increasing efforts to differentially associate different PFC areas with subprocesses involved in reviving and/or evaluating information (e.g., Mitchell et al., 2009). For example, there is evidence that rostrolateral PFC maintains memory-relevant goals or specific agendas to look for a particular type of information (e.g., Dobbins and Han, 2006).

Dissociable Control Mechanisms of Perceptual and Reflective Attention

The review above indicates that frontal and parietal regions are engaged during both perceptual and reflective attention. This similarity likely reflects the fact that they are serving related functions. However, according to PRAM, perceptual and reflective attention should be dissociable at the neural level. A growing body of work makes distinctions similar to that between perceptual and reflective attention: stimulus-oriented vs. stimulus independent attending (Burgess et al., 2007), selective attention vs. memorial selection (Nee and Jonides, 2009), attentional orienting in the perceptual domain vs. the working memory domain (Lepsien and Nobre, 2006), and attentional modulation of sensory information and information in working memory (Awh et al., 2006). Although the literature directly comparing perception and reflection is still quite small, recent studies are beginning to advance our understanding of the relation between perception and reflection and their consequences for memory.

In one direct comparison of perceptual attention and reflective attention to word stimuli (Roth et al., 2009), regions more active for perceptual attention included right frontal cortex and bilateral posterior visual cortex. Activity more specific to refreshing was recorded in left dorsolateral frontal cortex, left temporal cortex, and bilateral inferior frontal cortex. Another comparison between perceptual selection and reflective selection found that the superior parietal lobule and frontal eye fields were more specific to perceptual selection, while left ventrolateral prefrontal cortex was more specific to reflective selection (see Figure 2, Nee and Jonides, 2009). Attention to locations within mental representations revealed stronger activations in frontal cortex compared to when attending to locations in perceptual arrays (Nobre et al., 2004). Furthermore, rostromedial PFC was more active during perceptual attention, while rostrolateral PFC was more active during reflective attention (Henseler et al., 2011). Burgess et al. (2007) note that the region of rostral medial PFC that was more active for perceptual and reflective cognitive tasks is anterior to the medial PFC area that is typically found when participants are not engaged in a task (“rest”) or when they are specifically instructed to engage in self-referential thinking (see below).

Figure 2. Comparing perceptual attention and reflective attention.

(A) Regions active for high versus low selection in a perceptual selection task (top), memory selection task (middle), and the conjunction of both tasks (bottom). (B) Regions unique to perceptual selection (left) and memorial selection (right). (C) Differential activation of the rostromedial and rostrolateral PFC during attentional orientation to external and internal information, respectively. Center: Brain activation map showing significantly stronger activation of the anterior rostromedial PFC during the orientation of attention to external as compared to internal information (blue), and significantly stronger activation of the rostrolateral PFC during the orientation of attention to internal as compared to external information (red). The scale below shows the color-coding of the displayed T-values. Periphery: Parameter estimates extracted from the rostromedial PFC (left side) and rostrolateral PFC (right side) color coded for the different tasks (blue: Externally Oriented position task; purple: Externally Oriented target task; red: Internally Oriented task). [(A) and (B) from Nee and Jonides (2009, Neuroimage); (C) from Henseler et al. (2011, Neuroimage).

Sestieri, Shulman & Corbetta (2010) compared a perceptual attention task in which participants looked for specific targets in a video (e.g., Can you detect a man standing on the street wearing red pants?) with a reflective attention task in which they answered recognition questions about videos seen previously (“Richard mentioned his problem with alcohol before his intimacy problem.”). For the perceptual task they found activity in SPL and posterior IPS, regions commonly found in perceptual attention tasks. For the memory task, they found areas of AG, SMG, lateral IPS, and medial areas (PCu, PCC, RSC). These findings suggest a dissociation of regions engaged during perceptual and reflective attention. However, this study did not equate items across perceptual and reflective conditions. Furthermore, as noted by Sestieri et al., the memory retrieval task likely involved an “ensemble of processes” (p. 8453) and thus was not designed to contrast specific component processes of perceptual and/or reflective attention.

Functional connectivity analyses help to segregate functionally different networks (Fox et al., 2006; Corbetta et al., 2008; Chadick & Gazzaley, 2011). PRAM predicts different patterns of connectivity between representational areas and frontal and/or parietal cortex for perceptual vs. reflective tasks. Also, the timing of activity between frontal and parietal control mechanisms may yield differences between perceptual and reflective attention. For example, frontal activity occurs before parietal activity during top-down perceptual attention, while parietal activity precedes frontal activity during bottom up perceptual attention (e.g., Buschman and Miller, 2007). It would be useful to see if such findings extend to reflective attention tasks.

Dissociations between patterns of enhancement and suppression also show differences between perceptual and reflective attention. During encoding of multiple items presented in a sequence, Older adults showed intact enhancement but disrupted suppression effects relative to young adults, suggesting that enhancement and suppression are dissociable processes (Gazzaley et al., 2005). Although it provided evidence regarding overall enhancement and suppression effects during encoding, the design of the Gazzaley et al. study did not separately assess effects of perceptual and reflective attention. Evidence that perceptual and reflective attention are also dissociable comes from a study finding that older adults showed disrupted suppression during refreshing, but not during perceptual attention, while enhancement effects in both perceptual and reflective attention were preserved (Mitchell et al., 2009). Large-scale brain networks for enhancement and suppression appear dissociable Functional connectivity analysis during a perceptual attention task revealed that visual cortical areas that process target information coupled with right MFG and bilateral IFJ during enhancement, and with mPFC and PCC during suppression (Chadick & Gazzaley, 2011). The differences between enhancement and suppression in connectivity suggest that on-task perceptual attention contributed to enhancement effects and that off-task, self-referential attention (activating the “default network” [see below]) contributed to suppression effects. Additional studies are needed to compare such network effects for perception and reflection.

Bottom-up and Top-down Orienting in Perception or Reflection

Perceptual attention is controlled by two orienting systems (Corbetta and Shulman, 2002; Corbetta et al., 2008). A dorsal system includes the frontal eye fields (FEF) and intraparietal cortex (IPS, SPL) and is involved in goal-directed, top-down attention to stimuli. A ventral network includes inferior frontal cortex and IPL (TPJ and is specialized for bottom-up detection of salient or unexpected events. The ventral network has a right hemisphere bias and mediates the ability to “reorient” quickly to salient events that are potentially rewarding or dangerous to an observer. Reorienting involves interruption and resetting of ongoing activity in the dorsal network, which otherwise suppresses the ventral network during focused and sustained attention to an ongoing task. One would expect that perceptual attention would be important for encoding events for LTM and indeed, this is the case. For example, Uncapher et al. (2011) found that cuing top-down perceptual attention to an upcoming target location engaged the IPS and was associated with better subsequent memory, while cuing participants to an invalid non-target location engaged TPJ and was associated with poorer subsequent memory. Presumably, activity in TPJ reflected perceptual capture and/or reorienting necessary when the cued location did not contain a target. These findings provide important evidence of the role during encoding of top-down and bottom-up perceptual attention, but the study did not compare perceptual and reflective attention.

Whether a simple dorsal/ventral distinction applies to remembering is a subject of current debate. Lateral parietal activity is commonly found to be associated with correct recognition memory for old items (Vilberg and Rugg, 2008). It has been proposed that the dorsal/ventral distinction in perceptual attention may generalize to the kind of reflective attention processes engaged during remembering. Cabeza et al (2008) and Ciaramelli et al. (2008) suggested that superior parietal cortex supports retrieval search, monitoring and verification, similar to its role in the top-down, voluntary control of perceptual attention, and that inferior parietal cortex is active when there is clear and more detailed recollection, similar to the exogenous capture of attention by salient, bottom-up perceptual events. However, a comparison by Hutchinson et al. (2009) of their meta-analysis of regions involved in top-down and bottom up attention, with previously published analyses of top-down and bottom up effects in episodic remembering (Ciaramelli et al., 2008; Vilberg and Rugg, 2008), did not support the idea of overlap between perceptual attention and memory processes, especially for ventral parietal cortex. And, as noted above, Sestieri et al. (2010) found different parietal areas associated with their perceptual and memory search tasks.

Nevertheless, as Wagner et al., 2005) suggested, parietal activity is associated with a number of factors important for memory judgments, including a subjective sense that the relevant information is old or new (independent of the memory’s veracity, Johnson, 2006), level of detail that the memory supports, and retrieval orientation – the type of detail that participants are asked to retrieve about target memories. That is, parietal mechanisms may be involved in attending to internal, mnemonic representations, act as a buffer to integrate details that have been activated, reflect the overall strength of memories, and/or play a role in the evaluation of the task relevance of what is remembered (Wagner et al., 2005; Cabeza et al., 2008; Vilberg and Rugg, 2008; Shimamura, 2011).

Importantly, the PRAM framework assumes that the distinction between perceptual and reflective attention is orthogonal to the distinction between top-down and bottom-up attention (Chun et al., 2011; Corbetta and Shulman, 2002). Thus, efforts to compare control mechanisms for perceptual and reflective information should attempt to equate whether attention is directed to the task stimuli in a top-down or bottom-up manner. Studies to date typically relied on top-down manipulations (Nee and Jonides, 2009; Henseler et al., 2011; Roth et al., 2009). It would be helpful to introduce stimuli that capture attention in a bottom-up manner to assess the extent to which a common ventral network is engaged in both perceptual and reflective tasks. That is, it would be useful to directly compare four conditions: top-down and bottom-up attentional conditions in both perceptual and reflective tasks.

Resolving Interference within Perceptual and Reflective Attention

Perception and reflection both need selective mechanisms to resolve interference. Perception requires focusing on task-relevant information from among perceptually present task-irrelevant information. Perceptual competition makes it more difficult to find a T among Ls than among Os in visual search, and can even produce quite dramatic examples of blindness to unattended information (Simons and Chabris, 1999; reviewed in Marois and Ivanoff, 2005). Resolution of competition (successful selection) occurs when goals bias activation in favor of goal relevant features (Desimone and Duncan, 1995).

During perceptual identification, the strength of sensory evidence for a target can be measured by the strength of activity within a cortical region for the target category. Competing perceptual hypotheses, as under conditions of perceptual noise and degradation, arise from similar levels of activity within different cortical representational regions. For example, when trying to discriminate a face from a house under near-threshold conditions of degradation, successful recognition is dependent on FFA activity being greater than baseline and greater than PPA activity. Frontal and parietal mechanisms presumably evaluate the sensory evidence, and to suppress competing information. In particular, the findings shown in Figure 3 are consistent with the idea that DLPFC (left superior frontal sulcus) evaluates sensory evidence from posterior sensory areas to make perceptual discrimination decisions for target stimuli degraded with noise; stronger activity was associated with better psychophysical performance (Heekeren et al., 2004). Other regions including IPS and frontal eye field were more active during perceptually difficult conditions.

Figure 3. Perceptual and reflective resolution of competition.

(A) Activity in posterior DLPFC (left SFS) during perceptual discrimination of faces vs. houses showing a higher response to suprathreshold images of faces and houses relative to perithreshold images, and a correlation with the difference in activity in face vs. house selective areas, suggesting that this region integrates evidence from sensory processing areas to make perceptual decisions. (B) Signal changes in the posterior portion of the DLPFC predicted task performance. Points represent average BOLD change and performance for each condition (suprathreshold face, perithreshold face, perithreshold house and suprathreshold house) and participant. (C) Frontoparietal regions that were more active during memory retrieval for trials on which a classifier showed less activity for a target category under conditions of A–B, A–C interference (B and C were faces and scenes). (D) Responses in several regions of interest was characterized by marked activation for AC trials associated with low-fidelity reactivation. (Panels A and B adapted from Heekeren et al., 2004, Nature; C and D adapted from Kuhl et al., 2011, PNAS).

Reflection also involves the resolution of interference—competition from representations cued because they share features (e.g., spatial or temporal context, objects or people, perceptual or emotional qualities) with a target event. Examples of retrieval competition include proactive interference, when older memories interfere with the ability to retrieve more recent information (e.g., where did I park my car today?), or retroactive interference, when newer memories interfere with older ones (e.g., What was my former address? Wixted 2004). Resolution of such competition or interference is an act of selective reflective attention analogous to perceptual attention (Kuhl et al., 2011; Heekeren et al., 2004).

Patients with frontal lobe damage show greater deficits under conditions of increased mnemonic competition (Shimamura et al., 1995; Smith et al., 1995). Functional neuroimaging helps to specify the relative roles of different areas of PFC. For example, left ventrolateral PFC is more active in the face of interference, both during encoding (Dolan et al., 1997; Fletcher et al., 2000) and retrieval (Badre et al., 2005; Thompson-Schill et al., 1997). After learning person-location pairs (e.g., “The hippie is in the park”), subsequent retrieval of any association (hippie-park, yes/no?) is slower when more facts are associated with the same concept, producing slower response times and greater activity in left VLPFC (BA 44/45) (Sohn et al., 2003). As noted above, left VLPFC is also more active as interference accumulates over trials during working memory tasks, and may play a role in overcoming proactive interference (Jonides and Nee, 2006).

Retrieval competition in LTM is often studied with an A–B/A–C paradigm. A list of A–B pairs is studied, with the goal of being able later to recall B (target) when A (cue) is presented. Later in the experiment, some of the A cues are paired with new targets (A–C) for new learning. Relative to other completely new D–E cue-target pairs, subsequent memory for A–C pairs is impaired (proactive interference from the earlier A–B pairs), while memory for A–B pairs is impaired (retroactive interference from the more recent A–C pairs). For example, Kuhl et al. (2011) asked participants to associate cue words with faces or scenes, and a given cue was associated with both a face and a scene. Since faces and scenes have distinguishable representations in ventral-occipito-temporal cortex (including FFA and PPA), Kuhl et al. used MVPA to decode the relative strength of face and scene activation during memory retrieval to investigate how recall for an A–C pairing was affected by the earlier A–B pairing. Competition between associates B and C (from opposing face-scene categories) was assessed by the degree to which the classifier favored either face or scene activity. Compared to control items without competition, classifier performance was poorer for items with face/scene competition, suggesting that target and competing memories were being simultaneously reactivated. Furthermore when the classifier indicated more conflict, frontal and parietal areas were more strongly engaged, suggesting a role for these areas in resolving mnemonic conflict between target and competing memories (see Figure 3). Active regions included dorsolateral prefrontal cortex, medial prefrontal cortex, and lateral and medial parietal cortex. Overall, the results support a model in which multiple representations are reactivated in sensory areas, and control mechanisms in frontal and parietal lobes serve to resolve the interference and select a representation.

What is the fate of competing memories that are not selected during remembering? When goal-relevant memories are consistently and repeatedly retrieved, competing memories are often forgotten. That is, retrieval competition appears to at least sometimes be resolved through inhibition of competing memories, mediated by PFC mechanisms (Anderson et al., 2004). Furthermore, over time, forgetting is accompanied by reduced involvement of cognitive control mechanisms required for detecting (anterior cingulate cortex) and resolving (dorsolateral and ventrolateral prefrontal cortex) mnemonic competition (Kuhl et al., 2007). Thus forgetting has the adaptive benefit of reducing the burden on cognitive control mechanisms (Anderson, 2003).

Fluctuations in Perceptual and Reflective Attention

As a series of items appears at the focus of perceptual attention, an observer may try to sustain attention equally to every item but, typically, some items are encoded and retrieved better than others. Considering variations in perceptual and reflective attention can help explain this variability. Emotional significance or perceptual salience can draw more attention to some items (“attentional capture”), enhancing memory (Mather, 2007; Phelps, 2006). People may be more successful in noting associations or using elaborative strategies that facilitate encoding for some items than others (Craik and Lockhart, 1972). Distraction or mind wandering will reduce memory (Weissman et al., 2006). Both perceptual and reflective attention have limited capacity, and the fact that they often trade-off suggests that they are not entirely independent.

Researchers have identified some of the neural changes associated with trial to trial fluctuations in attention and their consequences for memory, and PRAM attempts to explain subsequent memory according to variation in either perceptual or reflective attention to different items. As described above, in studies of LTM using simple materials (e.g., words, pictures) various lateral PFC, lateral parietal and MTL regions show greater activity for subsequently remembered than forgotten items (Blumenfeld and Ranganath, 2007; Diana et al., 2007; Kim, 2011; Uncapher and Wagner, 2009). In contrast, greater activity for subsequently forgotten than remembered items is often found in medial PFC and medial parietal cortex (posterior cingulate, precuneus, TPJ; Kim et al., 2010; Otten & Rugg, 2001; Park et al., 2008; Turk-Browne et al., 2006; Uncapher et al., 2011). Interestingly, these same regions are active when participants are cued to engage in reflective, self-referential, internally-oriented processing (e.g., Gusnard et al., 2001; Ochsner et al., 2005), such as thinking about their aspirations or obligations (Johnson et al., 2006). This suggests that forgotten items in non-self-referential cognitive tasks were ones for which the participant’s attention was momentarily diverted from the task and focused on more personal concerns. In fact, these same anterior and posterior medial regions are spontaneously more active during “rest”, when no task at all is specified, than during many cognitive tasks (and are part of what is known as the “default network”, Raichle et al., 2001; Buckner et al., 2008). It should be noted that when self-referential processing is relevant to a later memory task, activation in medial PFC is positively related to later memory (e.g., Macrae et al., 2004; Kim and Johnson, 2010).

Limited attention is not necessarily allocated in an all-or-none manner but may be distributed between primary (target) and other (non-target or task irrelevant) information (Pashler, 1998). For example, according to load theory (Lavie, 2005), the amount of processing that nominally “unattended” stimuli receive depends on how much processing is devoted to the primary, attended target task. Difficult primary tasks (high load) consume attention; easy primary tasks (low load) do not fully consume attention which thus spills over to process “unattended” stimuli. Consistent with load theory, increasing the perceptual difficulty of a target task can eliminate negative priming from unattended distractors (Lavie and Fox, 2000). In contextual learning tasks, increasing the difficulty of the search through attended arrays decreases implicit learning of unattended arrays (Jiang and Chun, 2001). In repetition attenuation studies, increasing the perceptual difficulty of a primary task reduced repetition attenuation for repeated cycles of task-irrelevant background images (Yi et al., 2004). Interestingly, there is some evidence that in reflective tasks (e.g., recall), the negative impact of a concurrent reflective task (e.g., recognition) depends at encoding on whether the two tasks engage similar processes and, at retrieval, depends on whether the two tasks engaged similar representations (Fernandes and Moscovitch, 2000).

Our distinction between perceptual and reflective attention relates to how Lavie (2005) distinguished between perceptual and central (e.g., working memory, executive control) difficulty. This helps predict when perceptual and reflective attention trade off with each other or when they are independent. In some situations, reflection and perception clearly interfere with each other. For example, carrying on a conversation on a cell phone dramatically reduces perception and memory for stimuli encountered in a driving task (Strayer, Dreews & Johnston, 2003). Perceptual distraction (visual or auditory) disrupts reflective memory for visual details of pictures (Wais & Gazzaley, 2011; Wais, Rubens, Boccanfuso & Gazzaley, 2010). In other situations, there is little or no evidence of interference between perception and reflection. For example, in one study, Yi et al. (2004) manipulated working memory load (central/reflective processing), and found no impact on processing or implicit memory for an unattended, repeating background. Importantly, perceptual load manipulations of comparable difficulty did affect background processing. Another case where reflection did not disrupt perceptual learning comes from a study by Watanabe et al. (2001). Participants were given a primary task that required them to detect and be able to report target stimuli in a series of rapidly changing visual stimuli (a rapid serial visual presentation or RSVP task). In RSVP tasks, rapidly presented stimuli are perceptually processed to a level at which they are identified, but memory for the target depends on more than perceptual processing—it depends on central (reflective) processes that encode the target into working memory (Chun & Potter, 1995). One possibility is that this is accomplished via briefly refreshing the target. In the Watanabe et al. study, the RSVP task occurred against a background display of coherently moving dot stimuli embedded in enough random noise that their trajectory could not be consciously perceived or guessed above chance levels. Perceptual learning occurred for this unconscious but coherent background motion in spite of the perceptual and reflective demands of the primary RSVP task.

These examples highlight the question of what kinds of perceptual processes are and are not affected by reflective demands and vice/versa. At least part of the answer should depend on whether perceptual and reflective attention similarly or differentially engage the same or different brain areas and networks. Furthermore, the answer to this should in turn depend on exactly which perceptual and which reflective processes are being compared. Insofar as different reflective processes (e.g., refreshing, rehearsing, retrieving) differentially engage specific frontal and parietal regions, we would expect them to differentially interact with specific perceptual attention tasks. For example, some types of perceptual learning show effects in a very early visual processing area (V1) but not in other visual areas (V2, V3) nor in parietal or frontal cortex (Yotsumoto, Watanabe & Sasaki, 2008). V1 is an area relatively unlikely to be activated during reflection.

Thus, more work is needed to clarify the relations between perceptual load and reflective load (or central load, Lavie, 2005). These two types of load have been dissociated in some prior studies, but not all. For example, active manipulation of information in working memory (e.g., counting backwords), which involves reflective processing, impairs concurrent visual search efficiency (Han and Kim, 2004). Perceptual secondary tasks frequently disrupt reflective processing as well. For example, when making categorical decisions about visual stimuli, participants can be asked to concurrently perform an easy or difficult auditory monitoring task. Dividing attention with a difficult secondary task engaged DLPFC and superior parietal regions, impairing both visual task performance and subsequent memory for the stimuli (Uncapher & Rugg, 2005).

While both inferior frontal gyrus (IFG) and hippocampus show subsequent memory effects, these areas are affected by divided attention (dual task) in some experiments (Kensinger et al., 2003) but not others (Uncapher & Rugg, 2005). Differences in experimental outcomes may be explained by how well participants can share processing across dual tasks, that is, whether or not the two overlapping tasks recruit the same type of attentional processing (Uncapher & Rugg, 2008) or involved the same general representational areas (Fernandes and Moscovitch, 2000). In addition, different encoding conditions yield qualitatively different types of memory experiences. For example, Kensinger et al. (2003) found that words encoded under difficult divided attention conditions yielded a sense of familiarity, while words encoded with easier concurrent tasks yielded a more detailed (“recollective”) experience. Overall, whether two tasks interfere with each other should depend on whether common processes are important for the task and the type of representations involved. In sum, because perceptual load and reflective (central) load interfere differently with perceptual tasks (Lavie and De Fockert, 2005; Yi et al., 2004), they will likely have different effects on reflective tasks. Thus, dual-task studies sensitive to the distinction between perceptual and reflective attention will be important for conclusions about the presence or absence of divided attention costs. Furthermore, task switching studies would produce revealing findings regarding the important question of the mechanisms by which the cognitive system switches between perceptual and reflective attention (e.g., Burgess et al., 2007).

The brain is active even when at rest, and investigators have begun to explore the functional connectivity between areas when participants are not given an explicit task (Fox and Raichle, 2007). Early interest focused on the relation between a general “task positive network” including regions often found in cognitive tasks and a “task negative network” including regions that often deactivate during cognitive tasks and activate during rest (Fox and Raichle, 2007). These networks are also evident during sleep and anesthesia, consistent with the idea that they originate from intrinsic connectivity rather than uncontrolled, spontaneous cognition. Investigators are beginning to identify other “resting state networks” (RSNs) that are similar to networks found during explicit task manipulations (Smith et al., 2009). Thus, a potential direction for future research is whether dissociable intrinsic networks can be identified that are associated with differences in perceptual vs. reflective attention (when the content is held constant).

The Role of Attention in Binding Features and in Consolidation

It was once thought that the hippocampus was the memory region and that frontal and parietal cortex served other functions (cognition, attention). However, as noted above, the specific roles of frontal and parietal cortex in both attention and memory are under active investigation. It is also now recognized that other structures in the MTL (entorhinal cortex, perirhinal cortex and parahippocampal cortex) are important for memory (Ranganath, 2010). Although some maintain that evidence that various MTL structures have different functions in memory is weak (Squire et al., 2004), others have concluded they play differential roles in either item vs. relational memory, the types of features they process (e.g., object vs. spatial), or the level of representation at which binding occurs (Davachi, 2006; Eichenbaum et al., 2007; Shimamura, 2010). Nevertheless there is common agreement that the hippocampus (and perhaps other MTL structures, Shimamura, 2010) mediates binding among features (e.g., location, color, time) and of features with prior knowledge (e.g., schemas, Tse et al., 2007).

The importance of the hippocampus for long-term episodic memory is beyond debate based on patient and lesion data (Squire and Wixted, 2011; Eichenbaum et al., 2007). Consistent with patient data are neuroimaging findings of hippocampal activity during long-term memory tests, especially during source memory tasks (Weis et al., 2004) and correlations between hippocampal activity and the subjective experience of remembered details (Addis et al., 2004). Neuroimaging data from studies of long-term memory have also made it clear that the hippocampus is engaged not only during remembering, but also during encoding. For example, hippocampal activity during encoding predicts better source memory on a later test (Davachi, Mitchell & Wagner., 2003) further suggesting a major role for the hippocampus in initial feature binding.

Although most research on the MTL has focused on its role in long-term memory, it is increasingly evident that the hippocampus plays a much broader role in perception and reflection.. With respect to short-term memory, MTL damage impairs working memory for visual objects across delays as short as 4 s (Olson et al., 2006). Furthermore, object-location conjunction information can be impaired across delays as short as 8 s with MTL damage (Hannula et al., 2006; Olson et al., 2006). During perception, contextual representations mediated by the hippocampus/MTL can facilitate object recognition (Bar, 2004), guide the focus of attention (Chun and Phelps, 1999; Summerfield et al., 2006) and generate perceptual anticipation (Turk-Browne et al., 2010). Differences in eye movement patterns when processing a particular stimulus that has been seen before vs. one that has not provide an implicit measure of memory, and hippocampal activity and its connectivity with lateral PFC predicts eye movement measures of memory for relational information (Hannula and Ranganath, 2009). Furthermore, MTL damage can also impair perceptual tasks requiring difficult object discriminations (Baxter, 2009; but see Suzuki, 2009) or visual associations (Degonda et al., 2005; Chun and Phelps, 1999). These findings of hippocampal involvement in long-term memory, working memory, and perception make clear that the hippocampus is engaged in an on-going fashion during cognition. Is there a general function being served in these various situations? One possibility is that the hippocampus helps bridge temporal and spatial gaps between features of experience so that information that is not strictly contiguous can be bound together (Johnson & Chalfonte, 1994; Staresina & Davachi, 2009). Of course, the hippocampus may bind whatever features are contiguous (perceptually or reflectively) and other regions (e.g., frontal and parietal) may actually do the bridging, for example, via refreshing (Park et al., 2009).

From the PRAM perspective, a critical issue is how perceptual and reflective attention affect MTL function. Assuming that attention modulates MTL regions, are different frontal, parietal, and/or MTL regions engaged during perceptual and reflective attention? Do attentional networks that include MTL depend on the type of perception (e.g., focal, peripheral) or the type of reflection (e.g., refreshing, reactivating) or the type of target (scenes vs. objects vs. faces)?

Intriguing recent work demonstrates that hippocampal-cortical interactions occur not only during encoding, but also during retention intervals during which participants have no explicit task (“rest”). For example, functional connectivity between the hippocampus and the lateral occipital complex (LOC) during a rest period predicts subsequent memory for face-object pairs, presumably reflecting a consolidation process during the retention interval (Tambini et al., 2010). Tambini et al. concluded that evidence of hippocampal-LOC connectivity during resting was unlikely to reflect active rehearsal of the target information during rest.

Can patterns of incidental functional connectivity after an experience be distinguished according to whether they were initiated by perceptual encoding or by reflective encoding? Does reflection during encoding contribute to memory not only via, for example, elaborated encoding, but also by jumpstarting critical consolidation processes during rest? Another important question is whether incidental functional connectivity during rest can be distinguished from spontaneous reflection (reactivating, retrieving) during rest. Can we discriminate reactivations that have functional significance but (a) do not yield a subjective experience of target information coming to mind; (b) yield the phenomenal experience of a target coming to mind, but not attributed to memory; (c) yield the phenomenal experience of remembering; (c) are the result of active attempts to remember?

Another function that both perception and reflection share is binding multiple features into coherent objects or event representations. Failure to bind correctly can result in illusory conjunctions during perception (Treisman and Schmidt, 1982), or source memory errors during recall (Johnson, 2006). The intraparietal sulcus serves to bind different features together, important for feature integration in perceptual tasks (Corbetta et al., 1995; Friedman-Hill et al., 1995), and episodic memory tasks (Uncapher et al., 2006; Mitchell et al., 2009). That is, greater activity in IPS is correlated with successful encoding and recall of combinations of features.

What are the respective roles of parietal cortex and hippocampus in binding? Two potential differences require further investigation. The first difference concerns the type of information that undergoes binding (reviewed in Mitchell et al., 2009). The hippocampal system can bind and associate objects and events with each other and across domains (e.g., associating two different faces, or a face and a place, Mayes et al., 2007), while the parietal cortex might be more focused on binding multiple features into coherent objects or episodes (e.g., the screen location and font color of words, Uncapher et al., 2006).

Related to this difference in the level of binding, the second potential difference may be one of time scale. The function of the hippocampal system for long-term, durable binding seems uncontroversial. Parietal cortex was originally considered to be primarily involved in binding features during perception.. However, Shimamura (2011) recently highlighted how episodic retrieval (reflective attention in PRAM) contributes to feature binding, and proposes that ventral parietal cortex is a convergence zone for integrating episodic information (the cortical binding of relational activity [CoBRA] model).

Necessity of brain regions for perception and reflection

Although there is increasing evidence that different brain areas cooperate and interact during both perceptual attention and reflective attention, whether such activity is necessary for a particular function is indeterminate from brain imaging data alone. To simplify greatly, damage to the parietal cortex impairs spatial attention, but memory less so. In contrast, damage to the hippocampus and other medial temporal lobe regions impair explicit memory, but perception less so. However, newer work challenges this simplification, as parietal damage can result in memory impairments in specific situations such as free recall, but not recognition (Berryhill et al., 2007), and produces deficits in perceptual binding (Friedman-Hill et al., 1995), but not associative learning (Simons et al., 2008). Conversely, hippocampus/MTL damage can impair perceptual/attentional tasks (Murray et al., 2007; Chun and Phelps, 1999). Thus, more neuropsychological work is needed to investigate to what extent parietal mechanisms are necessary for reflective processes, and to what extent hippocampus and medial temporal lobe structures are necessary for perception. For disrupting both frontal and parietal function in humans, transcranial magnetic stimulation studies are promising (Miller et al., 2008; Zanto et al., 2011; Morishima et al., 2009).

Conclusion

The fields of attention and memory are beneficiaries of an increasingly vast amount of research in cognitive neuroscience, each complex and rich in its own right. The goal of a framework is to synthesize available evidence and suggest new directions for systematic analysis (Johnson, 2007). The PRAM framework and related empirical findings suggest that considering the similarities and differences between perception and reflection can help clarify and integrate the study of attention and memory to advance understanding of each in a symbiotic way, and points to potentially fruitful areas of additional research.