Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Nov 11.
Published in final edited form as: J Cogn Neurosci. 2008 Aug;20(8):1371–1380. doi: 10.1162/jocn.2008.20094

When a Thought Equals a Look: Refreshing Enhances Perceptual Memory

Do-Joon Yi 1, Nicholas B Turk-Browne 2, Marvin M Chun 2, Marcia K Johnson 2
PMCID: PMC3213859  NIHMSID: NIHMS220858  PMID: 18303973

Abstract

Cognition constantly involves retrieving and maintaining information that is not perceptually available in the current environment. Studies on visual imagery and working memory suggest that such high-level cognition might, in part, be mediated by the revival of perceptual representations in the inferior temporal cortex. Here, we provide new support for this hypothesis, showing that reflectively accessed information can have similar consequences for subsequent perception as actual perceptual input. Participants were presented with pairs of frames in which a scene could appear, and were required to make a category judgment on the second frame. In the critical condition, a scene was presented in the first frame, but the second frame was blank. Thus, it was necessary to refresh the scene from the first frame in order to make the category judgment. Scenes were then repeated in subsequent trials to measure the effect of refreshing on functional magnetic resonance imaging repetition attenuation—a neural index of memory—in a scene-selective region of the visual cortex. Surprisingly, the refreshed scenes produced equal attenuation as scenes that had been presented twice during encoding, and more attenuation than scenes that had been presented once during encoding, but that were not refreshed. Thus, the top–down revival of a percept had a similar effect on memory as actually seeing the stimulus again. These findings indicate that high-level cognition can activate stimulus-specific representations in the ventral visual cortex, and that such top–down activation, like that from sensory stimulation, produces memorial changes that affect perceptual processing during a later encounter with the stimulus.

Introduction

To adapt to a constantly changing environment, the brain needs to distinguish what to learn from what to ignore. Accordingly, learning from perceptual experience is modulated by high-level cognitive control. In behavioral experiments, attended information produces better subsequent implicit and explicit memory than ignored information (e.g., Turk-Browne, Yi, & Chun, 2006; Turk-Browne, Jungé, & Scholl, 2005; Mulligan, 2003; Jiang & Chun, 2001; Bentin, Moscovitch, & Nirhod, 1998). In functional neuroimaging, neural signatures of encoding and memory in the ventral visual cortex are enhanced when stimuli are task-relevant (Henson & Mouchlianitis, 2007; Yi & Chun, 2005). Yet, cognitive control is not limited to selecting among incoming perceptual input. In fact, much of cognition is specialized for maintaining or reviving information that is not perceptually available in the current environment. Little is known about how such higher-order, reflective operations are related to learning in perceptual systems.

There is evidence that reflecting on perceptual information is supported by activity in the inferior temporal cortex. For example, rehearsing visual objects in short-term memory or retrieving object representations from long-term memory is frequently associated with increased functional magnetic resonance imaging (fMRI) signals in category-selective cortical areas (M. R. Johnson, Mitchell, Raye, D'Esposito, & Johnson, 2007; Polyn, Natu, Cohen, & Norman, 2005; Mechelli, Price, Friston, & Ishai, 2004; Ranganath, DeGutis, & D'Esposito, 2004; Ishai, Haxby, & Ungerleider, 2002; Ishai, Ungerleider, & Haxby, 2000). These findings suggest that common neural substrates may represent the visual information manipulated by high-level cognition whether perceptual input is currently available or not (Kosslyn, Ganis, & Thompson, 2001). To what extent do the inferior temporal representations that are activated when a stimulus is not externally present act like those activated with perceptual input? Imagery of a visual object typically activates only a subset of the cortical regions that respond to the perception of the object (Mechelli et al., 2004; Ishai et al., 2002; Ishai, Ungerleider, & Haxby, 2000). Nevertheless, such sparse representations might still result in long-term changes that can be detected by indirect, perceptual memory tests. Imagery tasks often involve retrieving information from long-term memory and/or combining elements and discovering novel relations. Hence, imagery may involve a number of component cognitive processes. The consequences for later perception of the various component processes recruited during complex imagery tasks remain to be clarified.

One basic reflective process that may subserve a simple form of imagery is refreshing (e.g., Raye, Johnson, Mitchell, Greene, & M. R. Johnson, 2007; Raye, Johnson, Mitchell, Reeder, & Greene, 2002). Refreshing is a mechanism by which an active representation is briefly sustained or foregrounded—as in a brief thought directed at a just vanished image on the TV or word on the radio. Refreshing is an elementary reflective process that has been proposed to support more complex cognitive activities, such as those involved in working memory, long-term memory, and imagery (Johnson et al., 2005; M. R. Johnson et al., 2007; Johnson & Hirst, 1993). A number of studies have shown that refreshing involves areas in the dorsolateral prefrontal cortex (Johnson et al., 2005; Johnson, Mitchell, Raye, & Greene, 2004; Raye et al., 2002), and more recent evidence indicates that refreshing can activate category-specific brain regions (M. R. Johnson et al., 2007).

The current study tested whether reflectively refreshing a picture can produce changes that affect neural activity associated with later perception of the same picture. While performing a simple categorization task for pictures of scenes during an fMRI session, participants were required to think of a just-seen picture. Then we asked, what are the consequences of this brief reflective operation of refreshing for subsequent perceptual processing? Does refreshed information, like perceived information, leave memory records that affect future perception?

To answer this question about the possible impact of reflection on perception, refreshed scenes were perceptually presented again in subsequent trials, allowing us to measure fMRI repetition attenuation in scene-selective visual cortex. Repetition attenuation, also known as fMR adaptation or repetition suppression, refers to the reduced fMRI signal observed in the ventral visual cortex for repeated compared to novel stimuli. Importantly, repetition attenuation is stimulus-specific, and is believed to reflect sharpening of perceptual representations with experience or reduced visual processing necessary for object identification (Grill-Spector, Henson, & Martin, 2006; Wiggs & Martin, 1998; Desimone, 1996). Moreover, the degree of repetition attenuation predicts the likelihood of subsequent recognition of a perceptual stimulus (Turk-Browne et al., 2006).

To explore the consequences of refreshing on repetition attenuation, the evoked fMRI responses in the scene-selective visual cortex to previously refreshed scenes (refresh) were compared to scenes that were either previously presented twice (repeated), previously presented once without refreshing (skipped), or never presented before (novel). If refreshing has memorial effects similar to perception, attenuated fMRI responses should be observed for refreshed scenes compared to novel or skipped scenes. Moreover, it is interesting to compare the fMRI responses between refreshed and repeated scenes in order to assess the relative effects of top–down revival versus bottom–up stimulation on subsequent perception. In addition to the indirect test of memory provided by repetition attenuation, memory was also assessed during a surprise recognition test after scanning in order to measure the behavioral effect of refreshing on long-term, explicit memory.

Methods

Participants

Nine participants volunteered for monetary compensation (4 women, mean = 21 ± 3 years old).1 Informed consent was obtained from all participants. The study protocol was approved by the Human Investigation Committee at Yale University.

fMRI Acquisition

A Siemens Trio 3-T scanner with a standard birdcage head coil was used to acquire functional data with a gradient echo-planar imaging sequence. Each functional volume (2000 msec repetition time; 25 msec echo time; 90° flip angle; 5 mm thickness with no gap) comprised 27 axial slices parallel to the anterior commissure–posterior commissure line, covering the entire brain. Five functional scan runs were conducted, each acquiring 190 volumes. Visual stimuli were projected on a rear LCD projection screen and seen through an angled mirror attached to the head coil. A magnet-compatible button box was used to collect responses.

Task and Procedure

In the first functional run, participants performed a one-back repetition detection task in order to localize scene-selective visual cortex, the parahippocampal place area (PPA) (Epstein, Harris, Stanley, & Kanwisher, 1999; Epstein & Kanwisher, 1998; Aguirre, Detre, Alsop, & D'Esposito, 1996; Maguire, Frackowiak, & Frith, 1996). There were sixteen 14-sec stimulation blocks interleaved with 8-sec baseline periods. Half of the blocks presented faces and the other half presented scenes. During these blocks, face or scene images (true color, 15.5° × 15.5°) were sequentially presented every 700 msec (200 msec interstimulus interval) against a dark gray background. A white-outlined black square (1.6° × 1.6°) was superimposed in the center of the screen throughout the run. Two or three images per block were immediately repeated to which participants made unspeeded responses with the right index finger to indicate a repetition was detected. The order of face and scene blocks was counterbalanced between participants.

In the remaining four functional runs, participants performed an indoor–outdoor scene categorization task. As shown in Figure 1, each trial consisted of two 500-msec frames (15.5° × 15.5°) presented in succession with a 500-msec fixation blank between them. Each frame contained either a true color scene or an empty square outlined in light gray. The same fixation mark as in the PPA localizer was presented throughout the experiment. Participants were instructed to make no overt judgment in the first frame and to make a speeded keypress response to the content in the second frame. To prompt an indoor or outdoor response, a white dot (0.6° diameter) was added to the center of the fixation in the second frame. Responses slower than 2 sec were considered as missed trials.

Figure 1.

Figure 1

Open in a new tab

Schematic design and procedure. Eight types of trials were presented in an intermixed order. A black square was shown as fixation throughout the experiment. A trial contained two 500-msec frames with a 500-msec blank interval between them; a white dot in the second frame prompted participants to make a speeded scene categorization response. In the first presentation trials (left), the repeat condition (first row) consisted of the same scene in the two frames, the refresh condition (second row) consisted of a scene only in the first frame, and the skip and novel conditions (third and fourth rows) consisted of two different scenes. In the second presentation trials (right), the repeat, refresh, and skip conditions repeated scenes from the first frame of the first presentation trials.

As shown in the left side of Figure 1, there were three critical conditions depending on scene presentation in the second frame of a “first presentation” trial. (1) In the repeat condition, both frames contained the same scene. Participants thus responded to the scene repeated within the trial. (2) In the refresh condition, the first frame contained a scene, whereas the second frame was empty. In order to make a response, participants were required to think back to the scene in the first frame.2 (3) In the skip condition, the two frames contained different scenes. Thus, participants made an indoor–outdoor response to the second scene while skipping the first scene. This condition and our instructions helped ensure that participants were not preparing their response during the first frame, and instead waited for the second frame. The trials of these three conditions formed “first presentation” priming trials, which are distinguished from “second presentation” test trials described below.

Scenes in the first frame of the first presentation priming trials were presented again in the second frame of the “second presentation” test trials (right side of Figure 1). The first frame of the second presentation trials was always empty. In fact, these trials all looked identical and their conditions (repeat, refresh, and skip) were distinguishable only by how the scenes were presented during the first presentation trials. Repeat trials contained scenes that had initially appeared as repeat scenes in the first presentation phase, refresh trials contained scenes that had initially appeared as refresh scenes, and skip trials contained scenes that had initially appeared as the first scene in skip trials in the first presentation phase. In addition to these three trial types, two novel conditions were also included: One condition presented two different scenes and the other presented a scene in the second frame only. The novel trials containing two scenes were identical to the skip condition during the first presentation, but the first scene in skip trials was later repeated, whereas none of the novel scenes were ever repeated. In other words, the novel scenes provided the PPA signal baselines for the first and second presentation trials, respectively. In short, signaled by a white dot, participants made a keypress response on the second frame of every trial, indicating whether the currently presented picture, or the picture that they were to refresh, was an indoor or outdoor scene.

Each run consisted of 64 trials, with eight trials per condition. There were an equal number of indoor and outdoor responses in each condition. All trials were pseudorandomly intermixed, the order of which was predetermined to minimize the event correlation between any two conditions after being modeled by canonical hemodynamic response functions (HRFs; rs < .25). The intertrial interval was 4, 8, or 12 sec (mean = 5.5 sec). In addition, we confirmed that the repetition lag was not different among conditions in which scenes were repeated between trials. The number of intervening trials was 2.34 for the refresh, 2.26 for the repeat, and 2.32 for the skip conditions (ps > .4). The elapsed time was 17.94 sec for the refresh, 17.56 sec for the repeat, and 17.97 sec for the skip condition (ps > .4).

After the scanning session, participants moved to an adjacent behavioral testing room to perform a surprise scene recognition test. The lag between the scanning session and the recognition test was about 10 min. The recognition test contained all 128 scenes that had been presented in the first frames during the indoor–outdoor categorization task (4 conditions × 32 scenes). These old scenes were randomly intermixed with 32 new scenes. Participants viewed each scene one by one and made an unspeeded response by pressing one of three buttons: “old” if they recognized the scene, “new” if they did not recognize the scene, and “unsure” if they were not confident about whether they had seen the scene or not.

fMRI Analyses

Preprocessing and statistical analyses were conducted using statistical parametric mapping (SPM2, Wellcome Department of Imaging Neuroscience, University College, London, UK). The first four volumes of each run were discarded due to magnetization equilibration. The remaining volumes were then corrected for slice timing, realigned, normalized (resampling voxel size, 3 × 3 × 3 mm), and smoothed (Gaussian kernel, 8 × 8 × 8 mm). A high-pass frequency filter (cutoff: 128 sec period) and autocorrelation correction were applied to the time series.

The PPA regions of interest (ROIs) were localized for individual participants from the first functional run. Blocks of scenes and faces were separately modeled by canonical HRFs, with six movement parameters as covariates of no interest. A statistical parametric map of the t statistic was generated from a linear contrast between scene and face blocks. For each individual observer, the maximally scene-selective voxel was identified bilaterally in the cortical area straddling the parahippocampal gyrus and the collateral sulcus. Example loci of individual PPA ROIs are shown in Figure 2A. The coordinates of the maximum voxel were then used as the center of a 4-mm-radius spherical ROI (mean Talairach coordinates: x = −30, y = −44, z = −8; x = 30, y = −42, z = −8).

Figure 2.

Figure 2

Open in a new tab

Behavioral and fMRI results. (A) Examples of bilateral PPA ROIs superimposed on an axial slice of the MNI template. The PPA ROIs from different participants are shown around the collateral sulcus. (B) Response times. (C, D) Mean fMRI signal changes in the left and right PPA ROIs, respectively. Error bars indicate standard errors of the means for the first and second presentations.

In the ROI analysis, the mean time courses were first extracted from each ROI using the MarsBar toolbox (http://marsbar.sourceforge.net). Parameter estimates of event-related activity were obtained using the general linear model of the eight critical conditions and a dummy condition. Trials from the eight conditions containing an incorrect or no response were treated together as dummy events. Each condition was modeled by an HRF and a temporal derivative to account for variable hemodynamic delays. Six movement parameters were also included as covariates of no interest. The peaks of fitted HRFs for the four conditions were entered into statistical analyses as percent signal change.

For the whole-brain analyses, the same general linear model was applied to each voxel across the entire brain. A two-stage random-effect analysis was then performed using a one-sample t test on contrast images, which were obtained for individual participants for each comparison of interest. Resulting SPMs of the t statistic were thresholded at p < .001 (uncorrected, cluster threshold k = 5 voxels).

Results

Behavior in the Scanner

One participant was excluded from the behavioral and neuroimaging analyses due to exceptionally slow response times (RTs; slower than the mean ± 3 SD of the remaining participants). Participants categorized scenes very accurately, committing errors only in 0.9% of both first and second presentation trials. The differences between error rates among conditions were not statistically significant (ps > .1). Participants infrequently (3.5% of all trials) responded to scenes in the first frame or made anticipation errors in the second frame (RTs < 200 msec). The trials with outlier RTs (> 3 SD in each condition) were also excluded (4.2% of all trials). The remaining 91.4% of trials were then considered for RT, fMRI, and recognition analyses (mean 29.2 trials per condition). The mean RTs are shown in Figure 2B.

In the first presentations, participants showed a within-trial priming effect when the same scene was repeated in a trial. The repeat condition produced faster RTs than the other conditions (ps < .01). In contrast, it took longer for participants to think back to a just-seen picture, indicating effortful reflective processing in the refresh condition, although the numerical difference with the skip and novel conditions did not reach significance (ps > .2). The fact that refresh responses were slower than repeat responses confirms that participants were not preemptively preparing their response during the first frame. The skip and novel conditions did not differ from each other (p > .8).

In the second presentation trials, participants showed between-trial priming effects, categorizing the previously seen pictures more quickly than novel pictures. Relative to the novel condition, the repeat condition produced decreased RTs [t(7) = 2.42, p < .05] and the skip condition showed a trend [t(7) = 1.95, p = .09]. Importantly, the RTs in the refresh condition were shorter than those in the novel condition [t(7) = 2.78, p < .05] and comparable to those in the repeat condition (p > .3). The difference between the refresh and skip conditions was not significant (p > .1). Note that the RTs in the repeat condition were faster on the first than on the second presentation trials because the repeat condition of the first presentation trials presented two identical scenes, whereas that of the second presentation trials presented a scene only in the second frame.

PPA ROI Analysis

The fMRI signal changes from the left and right PPA ROIs are plotted in Figure 2C and D, respectively. The first and second presentation trials were analyzed separately with repeated-measure analyses of variance (ANOVAs) with two factors: hemisphere (left and right) versus condition (repeat, refresh, skip, and novel).

The first presentation trials showed a main effect of condition [F(3, 21) = 29.17, p < .01], a marginal main effect of hemisphere [Right > Left, F(1, 7) = 3.98, p = .09], and an interaction between hemisphere and condition [F(3, 21) = 4.21, p < .05]. Presenting two identical scenes produced less PPA activation than two different scenes (Turk-Browne, Yi, Leber, & Chun, 2007; Epstein, Graham, & Downing, 2003; Kourtzi & Kanwisher, 2001). This within-trial repetition attenuation effect was stronger in the right hemisphere. Relative to the skip and novel conditions, the repeat condition produced a significantly lower response in the right PPA (ps < .005) and marginally in the left PPA (ps < .1). The refresh condition produced the lowest PPA response in both hemispheres, presumably due to greater stimulus-driven, bottom–up activation in the other conditions which involved seeing two successive pictures (ps < .05). The skip and novel conditions produced comparable PPA signals, as would be expected, and in line with the RT results.

The second presentation trials showed only a main effect of condition [F(3, 21) = 8.63, p < .001; other effects, ps > .1]. The previously repeated scenes produced responses that were significantly lower than the novel scenes in bilateral PPA, and significantly lower than the previously skipped scenes in the right PPA (ps < .05), revealing between-trial repetition attenuation (Turk-Browne et al., 2006). More importantly, the previously refreshed scenes also attenuated bilateral PPA signals relative to both the novel scenes and the previously skipped scenes (ps < .05). This attenuation effect for previously refreshed scenes was comparable to that for the previously repeated scenes. Interestingly, the skip condition did not produce significant attenuation relative to the novel scenes in either hemisphere (ps > .1).

Refresh-related Activity

To examine refresh-related activity in additional brain regions, we contrasted the refresh condition with the repeat condition in the first presentation trials. As shown in Table 1 and Figure 3, refreshing was accompanied by increased activation in regions including middle frontal, middle temporal, and parietal cortices. The frontal activation near or at the dorsolateral prefrontal cortex replicated prior studies on refreshing (Johnson et al., 2004, 2005; M. R. Johnson et al., 2007; Raye et al., 2002).

Table 1.

Talairach Coordinates of Cluster Centers Revealed in Exploratory Whole-brain Analyses, Organized by Analysis Contrast

Brain Region L/R kE BA x y z t
First Presentation Trials
Refresh > Repeat (p < .001, k =5)
 Middle frontal gyrus L 12 6 −39 −3 53 16.17
 Middle frontal gyrus L 6 6 −50 5 33 6.15
 Middle temporal gyrus L 24 21 −59 −49 8 11.02
 Middle temporal gyrus L 6 21 −59 −41 0 8.35
 Lateral occipital gyrus L 7 37 −56 −62 −7 7.94
 Supramarginal gyrus L 14 40 −62 −51 25 7.08
 Middle frontal gyrus R 6 6 30 8 44 6.79
 Medial frontal gyrus R 37 6 6 17 46 7.88
 Middle frontal gyrus R 10 10/46 42 42 20 9.09
 Middle frontal gyrus R 5 10/46 36 50 17 5.70
 Intraparietal sulcus R 11 40 39 −44 44 9.44
[Novel + Skip] > Repeat (p < .001, k = 5)
 Middle occipital gyrus L 5 19 −27 −92 24 6.44
 Cuneus R 5 18 18 −95 13 7.04
 Middle occipital gyrus R 32 19 33 −89 18 10.89
 Fusiform gyrus R 49 19/37 33 −48 −18 9.04
Second Presentation Trials
Conjunction of [Novel > Repeat] and [Novel > Refresh] (p < .005, k = 5)
 Superior occipital gyrus L 11 19 −39 −77 29 3.83
 Retrosplenial cortex R 24 23/30 24 −52 14 3.99
 Superior occipital gyrus R 5 19 42 −83 24 4.36
 Parahippocampal gyrus R 8 36 24 −33 −14 3.35
 Fusiform gyrus R 7 37 30 −50 −13 3.70
Open in a new tab

kE = number of contiguous voxels; BA = Brodmann's area.

Figure 3.

Figure 3

Open in a new tab

Refresh-related activity from the whole-brain Refresh > Repeat contrast (p < .001, k = 5). IPS = intraparietal sulcus; LOG = lateral occipital gyrus; MFG = middle frontal gyrus; MdFG = medial frontal gyrus; MTG = middle temporal gyrus; SMG = supramarginal gyrus.

Repetition Attenuation Effects beyond the PPA ROI

Additional contrasts revealed the voxels that showed attenuation for scene repetition within and between trials (Table 1). When two scenes in a trial were identical (first presentation repeat), voxels in the right fusiform gyrus, right cuneus, and bilateral middle occipital gyrus showed decreased activation, relative to when two different scenes were presented (first presentation novel and skip). Additional within-trial repetition attenuation effects were observed in the bilateral collateral sulcus and parahippocampal gyrus with a less stringent threshold of p < .005 (k = 5).

Between-trial repetition effects were examined with a conjunction of two contrasts: greater activation for novel scenes than for previously refreshed scenes and for previously repeated scenes (novel, refresh, and repeat in the second presentation trials, respectively). The voxels that showed attenuation for both refresh and repeat conditions were localized in the right fusiform gyrus, right parahippocampal gyrus, right retrosplenial cortex, and bilateral superior occipital gyrus. These regions are part of a distributed network of regions involved in scene processing and navigation (Maguire, 2001; Ishai, Ungerleider, Martin, & Haxby, 2000).

Subsequent Scene Recognition

The analysis of explicit scene recognition performance focused on “old” responses although similar results were found with “new” responses (because they are roughly complementary). The scenes that had been repeated or refreshed during initial presentations produced better long-term, explicit memory relative to novel scenes. As shown in Figure 4 (bars on the left), scene recognition was better in the repeat and refresh conditions than in the novel condition (ps < .05). The difference between the repeat and refresh conditions was not statistically significant (p > .3). The repeat condition produced better recognition than the skip condition [t(7) = 2.393, p < .05] whereas the refresh condition did not (p > .1). The skip and novel conditions did not differ (p > .1), and both produced scene recognition well above the rate at which participants responded “old” to the scenes that were newly introduced during the recognition test (i.e., false alarms).

Figure 4.

Figure 4

Open in a new tab

Scene recognition performance. Error bars indicate standard errors of the mean.

Discussion

The current study demonstrates a situation in which briefly thinking about a recent perceptual stimulus is comparable to seeing it again. Refreshing a single presentation resulted in fMRI repetition attenuation comparable to what was observed after two presentations. Previous neuroimaging studies have suggested that top–down control mechanisms, such as those operating during working memory and visual imagery, revive inferior temporal object representations (Mechelli et al., 2004; Ranganath et al., 2004; Ishai et al., 2002; Ishai, Ungerleider, & Haxby, 2000; O'Craven & Kanwisher, 2000). Our results extend these findings by showing that such reflective representations are encoded into memory in a way that influences subsequent perception. One possibility is that the same population of inferior temporal neurons may be responsible for both bottom–up and top–down object representations, given the assumption that repetition attenuation reflects reduced responsiveness of a subpopulation of selective neurons (Grill-Spector et al., 2006). Alternatively, refreshing a scene, like seeing it again, may facilitate extracting information during subsequent perception relevant to the specific response required (in this case, an indoor–outdoor judgment; cf. Dobbins, Schnyer, Verfaellie, & Schacter, 2004).

Besides the main findings, other aspects of our results are worth considering. First, the skip condition did not produce reliable repetition effects relative to the novel condition (Figure 2B, C, and D), even though single presentations are often sufficient to produce repetition attenuation. With jittered intertrial intervals and no pretrial cue, participants should attend to first frame scenes equally well across conditions. Moreover, given the relatively long presentation and interstimulus interval (each 500 msec), it does not seem likely that the scene in the second frame interfered with the perceptual processing of the scene in the first frame. On the other hand, if there is such interference, the skip condition represents an analog of common situations in which the visual environment is constantly changing and generating such interference/inhibition. Perceptually or reflectively attending to a stimulus immediately again may help overcome such interference in natural viewing situations. Alternatively, participants might actively inhibit the memory of skipped scenes, which would, in turn, reduce repetition effects during a later encounter (cf., Anderson & Green, 2001). Although the effect of active inhibition on the later expression of implicit memory deserves more investigation, related evidence suggests that several acts of inhibition for a particular stimulus might be required to suppress its memory (e.g., more than five; Depue, Banich, & Curran, 2006). The lack of repetition attenuation for skipped scenes may also reflect the lack of a match between task demands during the initial perception and the later expression of implicit memory. That is, participants did not make responses to skipped scenes until the second presentation trials, and thus, responses were not repeated along with the scene repetitions; such changes in stimulus–response context have been shown to reduce repetition attenuation effects (Dobbins et al., 2004; Schacter, Dobbins, & Schnyer, 2004).

Second, refreshed scenes were not recognized better than repeated scenes during a subsequent long-term memory test, which is not consistent with the expectation that reflective operations such as refreshing induce better encoding. Long-term memory is not, however, simply a result of being mentally active during encoding. Rather, the activity during encoding has to be useful for the subsequent memory test. One explanation for the lack of difference between the repeat and refresh conditions in recognition is that we have largely equated the processing that affects recognition memory in this situation—that is, participants are considering the same type of information during encoding in both conditions as they determine whether the scene depicts an indoor or outdoor place. Moreover, note that although refreshing sometimes produces better recognition memory than repeating (words, pictures of simple objects), it does not always. For example, it has been reported that repeating abstract, random irregular patterns produced better memory than refreshing them, which may be related to the difficulty participants would have in refreshing such patterns (Johnson, Raye, Mitchell, Greene, & Anderson, 2003).

How does refreshing influence perceptual memory, as measured in our case by repetition attenuation? The multiple entry, modular (MEM) memory model provides a psychological framework to understand how perceptual representations are mediated by top–down cognitive processes (Johnson & Hirst, 1993). The model assumes a cognitive architecture consisting of two systems: perceptual and reflective. Perceptual and reflective systems interface with each other through agendas or goals that recruit processes that produce and act on representations. Refreshing is a component process in the reflective system and may, in the present context, constitute attention to decaying perceptual representations in the category-specific visual cortex (M. R. Johnson et al., 2007; Ranganath & D'Esposito, 2005). Reviving such representations would result in changes in memory, later expressed as repetition effects assessed during perception. Repetition attenuation could reflect changes in a common representation or representational format affected by perception and reflection (e.g., “tuning” the “same” representation by different operations). Alternatively, attenuation could reflect summation over common or similar elements of different perceptually derived and reflectively generated representations when cued by a memory probe. Another possibility is that attenuation could reflect changes as a consequence of reflection in the ease with which elements of a perceptual stimulus cue its identity or other task-relevant responses (e.g., category membership or associative relations). Note that such memory effects are the consequences of an executive, self-generated mental operation which is postperceptual. In MEM, such reflective attention to active internal representations is distinguished from perceptual attention to externally present stimuli (e.g., Johnson, Mitchell, Raye, McGuire, & Sanislow, 2006). The findings from the current study replicate the effects of such perceptual attention on neural memory (Yi, Kelley, Marois, & Chun, 2006; Yi & Chun, 2005) and, in addition, provide strong evidence that postperceptual, reflective operations can be as effective as perceptual operations in modulating memory as indexed by the degree of later activation of representations in the inferior temporal cortex during perception.

Acknowledgments

This research was supported by NIH grant AG15793 (M. K. J.), NIH grant EY014193 (M. M. C.), a foreign Natural Sciences and Engineering Research Council of Canada Post-Graduate Scholarship (N. B. T.-B.), and Ministry of Science and Technology of the Republic of Korea 21st Century Frontier Research Program Brain Research Center grant (D.-J. Y.). We thank Jenika Beck for recruiting participants.

Footnotes

1

Previous findings indicating that a single refresh can produce top–down modulation in the PPA (M. R. Johnson et al., 2007), along with our a priori focus on the impact of refreshing on repetition attenuation in the PPA, suggested that we would have sufficient statistical power to observe such a refresh-induced repetition attenuation effect with even a small sample size.

2

There is previous evidence of activity in visual processing areas, including the PPA, during the refreshing of scenes (M. R. Johnson et al., 2007). In addition, in unpublished responses to postexperimental questions (Johnson et al., 2005, Experiment 1; M. R. Johnson et al., 2007), participants indicated on a scale provided that, on average, they were able to “mentally revisualize” 56% of scenes during refreshing, and on an open-ended question, they often reported refreshing visual properties of scenes (e.g., “I recalled the overall picture and not the specific details”; “Thinking lines and blocks of light and dark”; “I thought of a specific part of the picture”). Thus, we expected participants in the present study similarly to think of visual properties of scenes during refreshing.

References

  1. Aguirre GK, Detre JA, Alsop DC, D'Esposito M. The parahippocampus subserves topographical learning in man. Cerebral Cortex. 1996;6:823–829. doi: 10.1093/cercor/6.6.823. [DOI] [PubMed] [Google Scholar]
  2. Anderson MC, Green C. Suppressing unwanted memories by executive control. Nature. 2001;410:366–369. doi: 10.1038/35066572. [DOI] [PubMed] [Google Scholar]
  3. Bentin S, Moscovitch M, Nirhod O. Levels of processing and selective attention effects on encoding in memory. Acta Psychologica. 1998;98:311–341. doi: 10.1016/s0001-6918(97)00048-6. [DOI] [PubMed] [Google Scholar]
  4. Depue BE, Banich MT, Curran T. Suppression of emotional and nonemotional content in memory: Effects of repetition on cognitive control. Psychological Science. 2006;17:441–447. doi: 10.1111/j.1467-9280.2006.01725.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Desimone R. Neural mechanisms for visual memory and their role in attention. Proceedings of the National Academy of Sciences, U S A. 1996;93:13494–13499. doi: 10.1073/pnas.93.24.13494. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Dobbins IG, Schnyer DM, Verfaellie M, Schacter DL. Cortical activity reductions during repetition priming can result from rapid response learning. Nature. 2004;428:316–319. doi: 10.1038/nature02400. [DOI] [PubMed] [Google Scholar]
  7. Epstein R, Graham KS, Downing PE. Viewpoint-specific scene representations in human parahippocampal cortex. Neuron. 2003;37:865–876. doi: 10.1016/s0896-6273(03)00117-x. [DOI] [PubMed] [Google Scholar]
  8. Epstein R, Harris A, Stanley D, Kanwisher N. The parahippocampal place area: Recognition, navigation, or encoding? Neuron. 1999;23:115–125. doi: 10.1016/s0896-6273(00)80758-8. [DOI] [PubMed] [Google Scholar]
  9. Epstein R, Kanwisher N. A cortical representation of the local visual environment. Nature. 1998;392:598–601. doi: 10.1038/33402. [DOI] [PubMed] [Google Scholar]
  10. Grill-Spector K, Henson R, Martin A. Repetition and the brain: Neural models of stimulus-specific effects. Trends in Cognitive Sciences. 2006;10:14–23. doi: 10.1016/j.tics.2005.11.006. [DOI] [PubMed] [Google Scholar]
  11. Henson RN, Mouchlianitis E. Effect of spatial attention on stimulus-specific haemodynamic repetition effects. Neuroimage. 2007;35:1317–1329. doi: 10.1016/j.neuroimage.2007.01.019. [DOI] [PubMed] [Google Scholar]
  12. Ishai A, Haxby JV, Ungerleider LG. Visual imagery of famous faces: Effects of memory and attention revealed by fMRI. Neuroimage. 2002;17:1729–1741. doi: 10.1006/nimg.2002.1330. [DOI] [PubMed] [Google Scholar]
  13. Ishai A, Ungerleider LG, Haxby JV. Distributed neural systems for the generation of visual images. Neuron. 2000;28:979–990. doi: 10.1016/s0896-6273(00)00168-9. [DOI] [PubMed] [Google Scholar]
  14. Ishai A, Ungerleider LG, Martin A, Haxby JV. The representation of objects in the human occipital and temporal cortex. Journal of Cognitive Neuroscience. 2000;12(Suppl. 2):35–51. doi: 10.1162/089892900564055. [DOI] [PubMed] [Google Scholar]
  15. Jiang Y, Chun MM. Selective attention modulates implicit learning. (A).Quarterly Journal of Experimental Psychology. 2001;54A:1105–1124. doi: 10.1080/713756001. [DOI] [PubMed] [Google Scholar]
  16. Johnson MK, Hirst W. MEM: Memory subsystems as processes. In: Collins AF, Gathercole SE, Conway MA, Morris PE, editors. Theories of memory. East Sussex, England: Erlbaum; 1993. pp. 241–286. [Google Scholar]
  17. Johnson MK, Mitchell KJ, Raye CL, Greene EJ. An age-related deficit in prefrontal cortical function associated with refreshing information. Psychological Science. 2004;15:127–132. doi: 10.1111/j.0963-7214.2004.01502009.x. [DOI] [PubMed] [Google Scholar]
  18. Johnson MK, Mitchell KJ, Raye CL, McGuire JT, Sanislow CA. Mental rubbernecking to negative information depends on task context. Psychonomic Bulletin & Review. 2006;13:614–618. doi: 10.3758/bf03193971. [DOI] [PubMed] [Google Scholar]
  19. Johnson MK, Raye CL, Mitchell KJ, Greene EJ, Anderson AW. fMRI evidence for an organization of prefrontal cortex by both type of process and type of information. Cerebral Cortex. 2003;13:265–273. doi: 10.1093/cercor/13.3.265. [DOI] [PubMed] [Google Scholar]
  20. Johnson MK, Raye CL, Mitchell KJ, Greene EJ, Cunningham WA, Sanislow CA. Using fMRI to investigate a component process of reflection: Prefrontal correlates of refreshing a just-activated representation. Cognitive, Affective, & Behavioral Neuroscience. 2005;5:339–361. doi: 10.3758/cabn.5.3.339. [DOI] [PubMed] [Google Scholar]
  21. Johnson MR, Mitchell KJ, Raye CL, D'Esposito M, Johnson MK. A brief thought can modulate activity in extrastriate visual areas: Top–down effects of refreshing just-seen visual stimuli. Neuroimage. 2007;37:290–299. doi: 10.1016/j.neuroimage.2007.05.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kosslyn SM, Ganis G, Thompson WL. Neural foundations of imagery. Nature Reviews Neuroscience. 2001;2:635–642. doi: 10.1038/35090055. [DOI] [PubMed] [Google Scholar]
  23. Kourtzi Z, Kanwisher N. Representation of perceived object shape by the human lateral occipital complex. Science. 2001;293:1506–1509. doi: 10.1126/science.1061133. [DOI] [PubMed] [Google Scholar]
  24. Maguire EA. The retrosplenial contribution to human navigation: A review of lesion and neuroimaging findings. Scandinavian Journal of Psychology. 2001;42:225–238. doi: 10.1111/1467-9450.00233. [DOI] [PubMed] [Google Scholar]
  25. Maguire EA, Frackowiak RS, Frith CD. Learning to find your way: A role for the human hippocampal formation. Proceedings: Biological Sciences. 1996;263:1745–1750. doi: 10.1098/rspb.1996.0255. [DOI] [PubMed] [Google Scholar]
  26. Mechelli A, Price CJ, Friston KJ, Ishai A. Where bottom–up meets top–down: Neuronal interactions during perception and imagery. Cerebral Cortex. 2004;14:1256–1265. doi: 10.1093/cercor/bhh087. [DOI] [PubMed] [Google Scholar]
  27. Mulligan NW. Effects of cross-modal and intramodal division of attention on perceptual implicit memory. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2003;29:262–276. doi: 10.1037/0278-7393.29.2.262. [DOI] [PubMed] [Google Scholar]
  28. O'Craven KM, Kanwisher N. Mental imagery of faces and places activates corresponding stimulus-specific brain regions. Journal of Cognitive Neuroscience. 2000;12:1013–1023. doi: 10.1162/08989290051137549. [DOI] [PubMed] [Google Scholar]
  29. Polyn SM, Natu VS, Cohen JD, Norman KA. Category-specific cortical activity precedes retrieval during memory search. Science. 2005;310:1963–1966. doi: 10.1126/science.1117645. [DOI] [PubMed] [Google Scholar]
  30. Ranganath C, D'Esposito M. Directing the mind's eye: Prefrontal, inferior and medial temporal mechanisms for visual working memory. Current Opinion in Neurobiology. 2005;15:175–182. doi: 10.1016/j.conb.2005.03.017. [DOI] [PubMed] [Google Scholar]
  31. Ranganath C, DeGutis J, D'Esposito M. Category-specific modulation of inferior temporal activity during working memory encoding and maintenance. Cognitive Brain Research. 2004;20:37–45. doi: 10.1016/j.cogbrainres.2003.11.017. [DOI] [PubMed] [Google Scholar]
  32. Raye CL, Johnson MK, Mitchell KJ, Greene EJ, Johnson MR. Refreshing: A minimal executive function. Cortex. 2007;43:135–145. doi: 10.1016/s0010-9452(08)70451-9. [DOI] [PubMed] [Google Scholar]
  33. Raye CL, Johnson MK, Mitchell KJ, Reeder JA, Greene EJ. Neuroimaging a single thought: Dorsolateral PFC activity associated with refreshing just-activated information. Neuroimage. 2002;15:447–453. doi: 10.1006/nimg.2001.0983. [DOI] [PubMed] [Google Scholar]
  34. Schacter DL, Dobbins IG, Schnyer DM. Specificity of priming: A cognitive neuroscience perspective. Nature Reviews Neuroscience. 2004;5:853–862. doi: 10.1038/nrn1534. [DOI] [PubMed] [Google Scholar]
  35. Turk-Browne NB, Jungé J, Scholl BJ. The automaticity of visual statistical learning. Journal of Experimental Psychology: General. 2005;134:552–564. doi: 10.1037/0096-3445.134.4.552. [DOI] [PubMed] [Google Scholar]
  36. Turk-Browne NB, Yi DJ, Chun MM. Linking implicit and explicit memory: Common encoding factors and shared representations. Neuron. 2006;49:917–927. doi: 10.1016/j.neuron.2006.01.030. [DOI] [PubMed] [Google Scholar]
  37. Turk-Browne NB, Yi DJ, Leber AB, Chun MM. Visual quality determines the direction of neural repetition effects. Cerebral Cortex. 2007;17:425–433. doi: 10.1093/cercor/bhj159. [DOI] [PubMed] [Google Scholar]
  38. Wiggs CL, Martin A. Properties and mechanisms of perceptual priming. Current Opinion in Neurobiology. 1998;8:227–233. doi: 10.1016/s0959-4388(98)80144-x. [DOI] [PubMed] [Google Scholar]
  39. Yi DJ, Chun MM. Attentional modulation of learning-related repetition attenuation effects in human parahippocampal cortex. Journal of Neuroscience. 2005;25:3593–3600. doi: 10.1523/JNEUROSCI.4677-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Yi DJ, Kelley TA, Marois R, Chun MM. Attentional modulation of repetition attenuation is anatomically dissociable for scenes and faces. Brain Research. 2006;1080:53–62. doi: 10.1016/j.brainres.2006.01.090. [DOI] [PubMed] [Google Scholar]

RESOURCES