Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2008 Aug 1.
Published in final edited form as: Neuroimage. 2007 May 21;37(1):290–299. doi: 10.1016/j.neuroimage.2007.05.017

A Brief Thought Can Modulate Activity in Extrastriate Visual Areas: Top-Down Effects of Refreshing Just-Seen Visual Stimuli

Matthew R Johnson 1, Karen J Mitchell 2, Carol L Raye 2, Mark D’Esposito 3, Marcia K Johnson 1,2
PMCID: PMC2093956  NIHMSID: NIHMS27953  PMID: 17574442

Abstract

Current models of executive function hold that the internal representations of stimuli used during reflective thought are maintained in the same posterior cortical regions initially activated during perception, and that activity in such regions is modulated by top-down signals originating in prefrontal cortex. In an event-related functional magnetic resonance imaging study, we presented participants with two pictures simultaneously, a face and a scene, immediately followed either by a repetition of one of the pictures (perception) or by a cue to think briefly of one of the just-seen, but no longer present, pictures (refreshing, a reflective act). Refreshing faces and scenes modulated activity in the fusiform face area (FFA) and parahippocampal place area (PPA), respectively, as well as other regions exhibiting relative perceptual selectivity for either faces or scenes. Four scene-selective regions (lateral precuneus, retrosplenial cortex, PPA, and middle occipital gyrus) showed an anatomical gradient of responsiveness to top-down reflective influences versus bottom-up perceptual influences. These results demonstrate that a brief reflective act can modulate posterior cortical activity in a stimulus-specific manner, suggesting that such modulatory mechanisms are engaged even during transient ongoing thought. Our findings are consistent with the hypothesis that refreshing is a component of more complex modulatory operations such as working memory and mental imagery, and that refresh-related activity may thus contribute to the common activation patterns seen across different cognitive tasks.

Keywords: executive function, fMRI, refreshing, top-down control, working memory

Introduction

Contemporary theories of cognitive control propose that a primary role of the prefrontal cortex (PFC) is to produce top-down signals that influence levels of neural activity, and hence the flow of information processing, in other brain regions (Miller and Cohen, 2001). Such top-down modulation is likely a key mechanism underlying working memory (WM; Baddeley, 1992; Baddeley and Hitch, 1974) and reflective processes more generally (e.g., the maintenance, manipulation, encoding, and revival of representations of external stimuli, ideas, beliefs, and goals; Johnson, 1992; Johnson and Hirst, 1993). For example, one current influential hypothesis about WM is that representations of recently perceived external stimuli that are no longer present are maintained by the top-down activation of posterior regions that are initially active during the perception of such stimuli (Curtis and D’Esposito, 2003; Petrides, 1994; Ranganath and D’Esposito, 2005; Ruchkin et al., 2003).

Evidence for this view has been obtained by examining activity in regions of inferior temporal cortex that are differentially responsive to different classes of stimuli. For example, the fusiform face area (FFA) and the parahippocampal place area (PPA) are known to activate differentially to faces and scenes, respectively, during perception (Epstein and Kanwisher, 1998; Kanwisher et al., 1997; Maguire et al., 2001; McCarthy et al., 1997). Several studies have demonstrated that these areas also exhibit selective delay-period activity during WM maintenance of the appropriate class of stimuli (e.g., Druzgal and D’Esposito, 2003; Postle et al., 2003; Ranganath et al., 2004). These findings are consistent with the idea that selective reflection produces top-down modulation of activity in posterior perceptual cortical regions.

Presumably, such top-down modulation occurs not only when individuals attempt to hold a target class of stimuli in mind over several seconds, but also during the more transient processes commonly engaged during ongoing thought. Often we do not know in advance which information will be relevant later and do not have the time, or a reason, to actively rehearse it for several seconds. In addition to extended maintenance that may be driven in a prospective top-down fashion, we often have to make rapid selections from activated representations based on information that becomes available only after the stimulus has disappeared. Furthermore, we may foreground or sustain such retroactively selected representations only very briefly as part of a continually changing stream of mental representations. The current study examined whether posterior areas show selective activation when a top-down signal is retroactive and relatively brief.

We obtained evidence relevant to this question by examining posterior activity associated with the cognitive process of refreshing. Refreshing is thinking briefly of an already active representation of a thought or percept (Johnson, 1992; Johnson and Hirst, 1993). Like perceptual repetition, refreshing often benefits memory for the targeted information (e.g., Johnson et al., 2005). Johnson and colleagues (Johnson et al., 2005; Raye et al., 2007) have proposed that refreshing serves both maintenance and executive functions, depending on task circumstances. They have identified PFC regions associated with refreshing, including refreshing selectively from among several active representations of the same type (e.g., words; Johnson et al., 2005; Raye et al., under review). Previous studies of refreshing have reported refresh-related activity in posterior areas as well as PFC (supramarginal gyrus, precuneus; Raye et al., 2002, 2007), and refreshing is assumed to involve interactions between PFC and posterior areas (including top-down modulation). Those studies were not designed, however, to demonstrate selective top-down modulation of activity in content-specific posterior areas. The present study thus asked whether a brief instance of refreshing would be sufficient to induce stimulus-specific changes in activity in extrastriate visual regions such as those sensitive to face and place information.

We presented participants with an initial display consisting of two stimuli, a face and a scene, followed by a cue to refresh (briefly think of) one of the stimuli. We examined the differential effects of refreshing a face or scene on activity in regions that included FFA, PPA, and other regions differentially sensitive to faces and scenes, as identified in a separate functional localizer task. This design allowed us to demonstrate that a transient reflective thought could indeed produce changes in posterior cortical activity that were related to which specific active representation was refreshed. We also compared the effect of refreshing a stimulus to seeing it again and identified scene-selective posterior areas where top-down (i.e., reflectively driven) and bottom-up (i.e., perceptually driven) processing produced similar or different levels of activity.

Materials and Methods

Participants

Fifteen young, right-handed, self-reported healthy adults with normal or corrected-to-normal vision participated in the study (6 males, mean age=21.3 years ± 2.7). One additional participant was excluded due to head movements during scanning. Participants were screened for MRI compatibility, gave written informed consent, and were compensated. The procedure was approved by the Yale University School of Medicine Human Investigation Committee.

Localizer Task

In the scanner, all participants performed a standard task (e.g., Wojciulik et al., 1998; Yi and Chun, 2005) commonly used to identify the location of each individual’s FFA, PPA, and other areas responding preferentially to one stimulus class over the other. Pictures of either faces or scenes were presented sequentially and participants were instructed to press a button with their right index finger when they saw the same picture twice in a row. Each participant performed two runs of the Localizer task; each run consisted of 8 task blocks (4 blocks of all faces, 4 blocks of all scenes) distributed pseudo-randomly with blocks of rest (fixation cross displayed onscreen) separating all task blocks. Face, scene, and rest blocks were all 16s long, with face and scene blocks consisting of 20 sequential stimulus presentations (500ms onscreen, 300ms fixation in-between stimuli).

Experimental Task

Participants saw one or two pictures (faces, scenes) presented on the initial slide of each trial; then participants either viewed one of the pictures again or refreshed one of the pictures they had just seen. Although all combinations of these factors produced 12 conditions, in the present report we focus on the 4 conditions that consisted of first presenting a face and a scene, followed by either an instance of reflection (Refresh) or a second instance of perception (Repeat) (see Fig. 1). Importantly, the two Refresh conditions were identical in terms of visual presentation, as both consisted of an initial slide of one face and one scene picture, followed by a second slide with just a dot cue; however, the refresh cue in one condition instructed participants to think of the face, and the cue in the other instructed participants to think of the scene. Thus, any activation differences between these two refresh conditions are attributable to the top-down influence of briefly thinking back to a specific stimulus, after the picture has disappeared from view. The 4 conditions of interest constituted a 2 (Refresh, Repeat) × 2 (critical stimulus: Face, Scene) design, where “critical stimulus” denotes the picture to be refreshed or repeated. These 4 conditions are hereafter labeled according to the refreshed or repeated content of slide 2: (1) Ref_F: Participants saw a dot and refreshed (thought of) the face picture; (2) Ref_S: Participants saw a dot and refreshed the scene picture; (3) Rep_F: Participants saw the face picture repeat (the scene did not); (4) Rep_S: Participants saw the scene picture repeat (the face did not).

Figure 1. Task designs.

Figure 1

Open in a new tab

(A) Experimental task, Repeat condition. Participants saw an initial slide with one face and one scene picture, followed by a second slide in which one of the just-presented stimuli was repeated. (B) Experimental task, Refresh condition. Participants saw an initial slide with one face and one scene picture, followed by a cue (a dot) to think back to one of the stimuli. In both conditions, to decrease uncontrolled mental activity between trials, participants performed a task during the inter-trial interval in which they pressed buttons to indicate which direction each of a series of arrows was pointing.

Participants completed 6 runs of 36 trials each, totaling 18 trials of each condition across the experiment. Each trial lasted 12s and can be conceptualized as 6 slides, each lasting 2s (see Fig. 1). In the first slide, two pictures (one face and one scene) were shown simultaneously side-by-side near the center of the screen (onscreen for 1500ms with a 500ms inter-stimulus interval [ISI]). During the second slide, on Repeat trials (Fig. 1A), only the critical stimulus was redisplayed (1500ms onscreen, 500ms ISI) in the same location it had just appeared on slide 1; on Refresh trials (Fig. 1B), a dot (·) was presented (1500ms onscreen, 500ms ISI) in the location where the critical stimulus had just appeared, cuing participants to think of the picture that had occupied that location on the previous slide. To reduce uncontrolled mental activity between trials, slides 3, 4, and 5 of each trial comprised an inter-trial interval task wherein a series of 3 upward- or downward-pointing arrows were displayed (each 1400ms onscreen, 600ms ISI) and participants pressed a button for each arrow to indicate the direction it was pointing. Slide 6 of each trial was a blank screen (2000ms). Although no overt responses were required for the refresh or repeat tasks, we were confident that participants would engage in the expected processing based on several previous studies (e.g., Johnson et al., 2005; Raye et al., 2002, 2007), also using covert responses, that found reliable refresh-related activity, as well as subsequent effects on memory similar to those found when participants responded overtly (e.g., Johnson et al., 2002).

Face and Scene Stimuli

Equal numbers of face and scene stimuli were used. All stimuli were grayscale images, 300 pixels by 300 pixels. Faces were forward-facing complete head shots of young to middle-aged individuals of various ethnicities, with neutral or pleasant expressions. The stimulus set contained both male and female faces, at a ratio of about 3:1. Scene stimuli were pictures of outdoor landscapes (e.g., beaches, forests, mountains).

Across participants, Experimental task stimuli were counterbalanced with regard to the condition and run in which they appeared. Face and scene stimuli were also balanced so that each appeared equally often on the left and right sides of the screen and critical items came equally often from each position in each condition. No stimulus in the Experimental task was seen in more than one trial. There were 6 practice trials, using stimuli not used in the Experimental task, to familiarize participants with the procedure prior to entering the scanner. In the Localizer task, 28 faces and 28 scenes (not used in the Experimental task, but similar in appearance) were presented several times each.

fMRI Acquisition & Analysis

Imaging data were acquired on a 1.5T Siemens Sonata scanner at the Yale University Magnetic Resonance Research Center. The imaging session totaled approximately 1.25 hours. Medium-resolution T1 anatomical images were followed by 6 functional runs of the Experimental task (226 volumes, 7:32 per run) and 2 functional runs of the Localizer task (132 volumes, 4:24 per run). Six volumes were discarded from the beginning of each run to allow tissue to reach steady-state magnetization, and each run was also “padded” with several additional volumes at the end to allow us to fully model the tail end of the hemodynamic response from the last trial. Functional echoplanar images were whole-brain volumes with the following parameters: 24 axial slices, interleaved acquisition, TR=2000ms, TE=35ms, flip=80°, 3.75mm × 3.75mm × 3.8mm voxels with 0mm skip.

fMRI data analysis was performed using SPM2 (Wellcome Department of Imaging Neuroscience, University College London, UK). Pre-processing included slice timing correction, motion correction using INRIAlign (Freire and Mangin, 2001; Freire et al., 2002), spatial normalization to the echoplanar image template provided with SPM (resampling images to 3mm isotropic voxels during normalization), and spatial smoothing (8mm FWHM Gaussian kernel). Single-subject statistics were modeled using the canonical hemodynamic response function with its temporal derivative. The Localizer task was modeled as a block design with separate regressors for blocks of face images and blocks of scene images, and contrasts were evaluated for each participant comparing face-block activity to scene-block activity (Face>Scene), and vice versa (Scene>Face). The Experimental task was modeled as an event-related design with separate regressors for each condition; slides 1 and 2 of each trial were collapsed and modeled as a single event, while the inter-trial interval (arrows task) was not explicitly modeled.

Individual coordinates for each participant’s FFA and PPA were located by examining the Face>Scene and Scene>Face contrasts, respectively, from the Localizer task and selecting the maximum of each FFA/PPA cluster. Bilateral FFA and PPA were located for all participants (P thresholds at which FFA and PPA emerged ranged from .001 to .05, uncorrected). Other face- and scene-selective regions of interest (ROIs) were determined at the group level using random-effects analyses of the Face>Scene and Scene>Face contrasts from the Localizer task; these results were examined at an a priori statistical threshold of P<.001, uncorrected, and an extent threshold of 4 voxels. We considered this an appropriate threshold for initially defining ROIs whose role in face and scene perception would subsequently be verified in separate analyses of the Experimental task data. To determine prefrontal areas of refresh-related activity, we examined a group random-effects analysis of the Refresh>Repeat contrast from the Experimental task1 at an uncorrected threshold of P<.01, as we had a priori hypotheses for these areas’ locations based on prior refresh studies.

Thus, coordinates for FFA and PPA were determined separately for each participant; coordinates for refresh-related PFC ROIs as well as other face- or scene-selective ROIs were determined using cluster maximum coordinates from the group random-effects analyses. In all ROI analyses, voxel values from each participant’s SPM contrast images were extracted from a 6mm sphere around the appropriate coordinate of interest and averaged to produce a single value for the region. MNI coordinates were converted to Talairach space using the Matlab script mni2tal (http://imaging.mrc-cbu.cam.ac.uk/imaging/MniTalairach).

Results

Top-down Effects of Refreshing in FFA/PPA

We first examined activity in the primary posterior areas of interest, FFA and PPA, during the two Refresh conditions: Ref_F and Ref_S. We predicted that we would find greater activity in PPA for refreshing scenes than faces, and greater activity in FFA for refreshing faces than scenes.

Example PPA and FFA locations for a representative participant are shown in Figure 2A, and parameter estimates of activation for the two Refresh conditions are presented in Figure 2B. A clear top-down effect was seen in bilateral PPA, with greater activity when refreshing scenes than when refreshing faces (left: P<.005, right: P=.01, both one-tailed paired t-tests). There was also a strong trend for a top-down effect in right FFA, with greater activity when refreshing faces than when refreshing scenes (P=.06, one-tailed). No top-down effect was observed in the left FFA; however, this is not particularly surprising as the right FFA is generally found to be more responsive than the left, with some previous studies failing to locate the left FFA reliably in all participants (Gazzaley et al., 2005; Kanwisher et al., 1997).

Figure 2. Effects of reflection and perception in parahippocampal place area (PPA) and fusiform face area (FFA).

Figure 2

Open in a new tab

(A) Example locations of the PPA and FFA are shown for a representative participant. Bilateral PPA and FFA were located for all participants and used as regions of interest for later analyses. (B) For bilateral PPA and FFA, activation estimates are plotted for the two Refresh conditions only, to show top-down effects of refreshing. After identical perceptual (bottom-up) stimulation, activity in bilateral PPA was greater for refreshing a scene than for refreshing a face. Activity in right FFA was greater for refreshing a face than for refreshing a scene. (C) For bilateral PPA and FFA, activation estimates are plotted for all four conditions of the Experimental task. In both regions, activity was greater for perception than for reflection, as expected. Error bars represent standard error of the mean. Conditions: Ref_F = refresh face, Ref_S = refresh scene, Rep_F = repeat face, Rep_S = repeat scene. See text for further details.

Perception vs. Reflection in FFA/PPA

Figure 2C shows parameter estimates of activation for all four conditions in FFA/PPA. In all cases, activity in perceptual (Repeat) conditions was numerically greater than the corresponding reflective (Refresh) condition. A 2 (Left vs. Right) × 2 (Refresh vs. Repeat) × 2 (Face vs. Scene) within-subjects ANOVA for each ROI showed a significant main effect of task (Repeat>Refresh; PPA: [F(1,14)=12.47, P<.005], FFA: [F(1,14)=7.23, P<.05]). This would be expected, assuming that the level of activity in such regions provides one of the cues that allows us to discriminate perception from reflection (reality monitoring; Johnson and Raye, 1981)2. Other effects included a main effect of Face/Scene (PPA: [F(1,14)=54.70, P<.001], FFA: [F(1,14)=4.54, P=.051]), with each area responding more to its preferred stimulus class; a Left/Right × Refresh/Repeat interaction [F(1,14)=4.71, P<.05] in FFA only, with right FFA showing a larger difference than left FFA between Repeat and Refresh trials; a trend for a Left/Right × Face/Scene interaction [F(1,14)=3.31, P=.09] in FFA only, with right FFA showing somewhat greater specificity for faces (versus scenes) than left FFA; and a Refresh/Repeat × Face/Scene interaction [F(1,14)=7.41, P<.05] in PPA only, due to a particularly strong response in the Rep_S condition.

Effects in Other Scene-Selective ROIs

In addition to FFA and PPA, we examined several other posterior ROIs that exhibited relative selectivity for one stimulus class (i.e., areas identified in the group Localizer analysis with the Scene>Face and Face>Scene contrasts). Note that we do not claim that these ROIs are strongly selective for face or scene stimuli (i.e., activating more for one stimulus class than all other stimulus classes) in the same manner that FFA and PPA have traditionally been thought to be (Downing et al., 2006; Epstein and Kanwisher, 1998; Kanwisher et al., 1997; but see Grill-Spector et al., 2006). Rather, in the present study, we considered any differential response between faces and scenes sufficient for studying the effects of perceiving and reflecting upon stimuli from these two categories. Our hypothesis was that, to the extent that a brain area exhibits a relative preference for either face or scene stimuli during perception, it may exhibit the same preference during reflection if it is involved in maintaining stimulus representations.

Besides PPA, the only posterior areas identified as responding more to scenes than faces (shown in Fig. 3A) were bilateral retrosplenial cortex (RSC), bilateral precuneus (PCu), and a bilateral region of middle occipital gyrus (MOG). Significant modulatory refresh effects (greater activity for refreshing scenes than refreshing faces) were observed in left RSC (P<.05) and left MOG (P=.01), and there were trends in right RSC (P=.053) and left PCu (P=.07; all one-tailed paired t-tests).

Figure 3. Effects of reflection and perception in scene-selective regions of interest (ROIs).

Figure 3

Open in a new tab

A number of other posterior regions displayed relative selectivity for scenes (Scene>Face contrast) during the Localizer task. (A) Bilateral retrosplenial cortex (RSC; Talairach coordinates [−15, −60, 14] and [21, −55, 17]), precuneus (PCu; [−18, −71, 45] and [24, −70, 48]), and middle occipital gyrus (MOG; [−30, −86, 21] and [33, −84, 15]) all showed greater activity for scenes than for faces. Significant or trend-level top-down modulation effects (Ref_S>Ref_F) were seen in bilateral RSC, left PCu, and left MOG. (B) When we compared activity in the four scene-selective ROIs (lateral PCu, RSC, PPA, and MOG), we observed a perceptual/reflective gradient whereby anterior-superior regions had nearly the same activity levels for both perception and reflection, but more posterior-inferior regions had greater activation for perception. Absolute levels of reflective (Refresh) and perceptual (Repeat) activity are plotted for all four ROIs. (C) The same analysis as in (B), presented in terms of the difference in activation between perception and reflection (Repeat-Refresh). The gradient effect is clear; there is almost no difference in lateral PCu but a large difference at the other end of the gradient, in MOG. Error bars represent standard error of the mean. Conditions: Ref_F = refresh face, Ref_S = refresh scene, Rep_F = repeat face, Rep_S = repeat scene. See text for further details.

We also conducted separate 2 (Left vs. Right) × 2 (Refresh vs. Repeat) × 2 (Face vs. Scene) within-subjects ANOVAs (similar to those performed for the FFA and PPA ROIs) for each of these additional scene-selective ROIs (see Fig. 3A for activation plots). In RSC, the only significant effect was a main effect of Face/Scene [F(1,14)=5.74, P<.05], with greater activity for scenes than for faces. In lateral PCu, there were strong trends for a main effect of Face/Scene [F(1,14)=4.27, P=.058], with greater activity for scenes than faces, and a crossover interaction of Left/Right × Refresh/Repeat [F(1,14)=4.30, P=.057], with greater activity for Refresh than Repeat trials in left PCu but greater activity for Repeat than Refresh trials in right PCu. In MOG, there were several notable effects, including a significant main effect of Refresh/Repeat [F(1,14)=19.15, P<.001], with greater activity for Repeat than Refresh trials; a significant main effect of Face/Scene [F(1,14)=38.68, P<.001], with greater activity for scenes than faces; a significant interaction of Left/Right × Face/Scene [F(1,14)=7.46, P<.05], with a greater difference between face- and scene-related activity in left MOG than in right MOG; and a significant interaction of Refresh/Repeat × Face/Scene [F(1,14)=17.33, P<.001], with a greater difference between Rep_F and Rep_S activity than between Ref_F and Ref_S activity.

Perceptual/Reflective Gradient for Scene-Selective ROIs

We collapsed activity in the four scene-selective ROIs (from anterior-superior to posterior-inferior: lateral PCu, RSC, PPA, MOG) across hemisphere and stimulus class, in order to consider these regions only in terms of their responses to refreshed and repeated stimuli. An intriguing pattern emerged, as shown in Figures 3B and 3C. PCu showed essentially no difference between perceptual and reflective activation, but there was increasingly greater activity for perception than reflection through RSC, PPA, and finally MOG, which exhibited the greatest difference between perception- and reflection-related activity. A one-way repeated-measures ANOVA of the (Repeat-Refresh) activation difference (see Fig. 3C) for these four regions showed a significant effect of region [F(3,42)=17.69, P<.001] with a significant linear trend [F(1,14)=67.61, P<.001].

Effects in Face-Selective ROIs

In addition to FFA, posterior areas responding more to faces than scenes included the right inferior occipital gyrus (IOG; Fig. 4). There was a trend towards a modulatory refresh effect (greater activity for refreshing faces than refreshing scenes; P=.08). We also performed a 2 (Refresh vs. Repeat) × 2 (Face vs. Scene) within-subjects ANOVA for activity in right IOG (see Fig. 4 for activation plots). There was a significant main effect of Refresh/Repeat [F(1,14)=10.23, P<.01], with greater activity for Repeat than Refresh trials, and a main effect of Face/Scene [F(1,14)=10.94, P<.01), with greater activity for faces than for scenes.

Figure 4. Effects of reflection and perception in a face-selective region of interest (ROI).

Figure 4

Open in a new tab

We considered one posterior region, right inferior occipital gyrus (IOG; Talairach coordinates [42, −82, −3]), that displayed relative selectivity for faces (Face>Scene contrast) during the Localizer task. There was a trend-level top-down modulation effect (Ref_F>Ref_S) in right IOG. Error bars represent standard error of the mean. Conditions: Ref_F = refresh face, Ref_S = refresh scene, Rep_F = repeat face, Rep_S = repeat scene. See text for further details.

Data from a face-selective area of right superior temporal gyrus (STG) are not shown, as this region failed to show any significant effects or trends in the Experimental task. Clusters identified from the Face>Scene localizer contrast that were either very small or a result of task-induced deactivation are not discussed.

PFC Activity: Replication of Prior Refresh Studies

A pattern of refresh-related activity in PFC (Fig. 5) was seen that replicated previous findings (Johnson et al., 2005; Raye et al., 2007)3. We were particularly interested in two PFC regions that have previously been associated with refreshing across multiple studies (Johnson et al., 2005). One of these regions, located in DLPFC (Fig. 5A) has been associated with the refresh process specifically, and the other, in anterior PFC (Fig. 5B), is thought to subserve initiation of various non-automatic processes (Raye et al., 2007). In each of these PFC areas, separate 2 (Left vs. Right) × 2 (Refresh vs. Repeat) × 2 (Face vs. Scene) within-subjects ANOVAs confirmed a main effect of refreshing, significantly in DLPFC [F(1,14)=6.54, P<.05] and as a strong trend in anterior PFC [F(1,14)=4.17, P=.06]. There was a weak tendency [F(1,14)=2.98, P=.11] in DLPFC for a Left/Right × Refresh/Repeat interaction, due to a somewhat stronger refresh effect on the left than on the right. This is consistent with prior studies that found refresh-related activity either more strongly or exclusively on the left, depending on the particulars of the task and the type of material being refreshed (Johnson et al., 2005).

Figure 5. Effects of refreshing in prefrontal cortex.

Figure 5

Open in a new tab

Activation maps are presented at a threshold of P<.01, extent threshold 4 voxels. Two areas corresponding to regions observed in prior refresh studies were considered. (A) A region of dorsolateral prefrontal cortex (DLPFC; Talairach coordinates: [−50, 10, 30] and [48, 8, 41]) previously shown to be associated with foregrounding a representation which was recently active (Raye et al., 2002, 2007). Estimates of activation are plotted for left and right DLPFC for the four conditions of the Experimental task. In all cases, there was greater activity for refreshing stimuli than seeing them repeated, with little effect of stimulus type. (B) A region of anterior prefrontal cortex (PFC; Talairach coordinates: [−36, 45, 23] and [42, 48, 20]) previously shown to be associated with initiating non-automatic processes, including but not limited to the refresh process (Raye et al., 2007). Estimates of activation are plotted for left and right anterior PFC for the four conditions of the Experimental task. In all cases, there was greater activity for refreshing stimuli than the relatively automatic act of watching them repeated, with little effect of stimulus type. Error bars represent standard error of the mean. Conditions: Ref_F = refresh face, Ref_S = refresh scene, Rep_F = repeat face, Rep_S = repeat scene.

Discussion

The present study demonstrates that (1) refreshing, a relatively simple cognitive process that reliably shows activation in PFC, is capable of modulating activity in FFA and PPA; (2) refreshing can also modulate activity in several other, less-well-studied areas exhibiting relative specificity for faces or scenes; and (3) among scene-selective regions, there was an anatomical gradient whereby perception evoked more activity than reflection in more posterior-inferior areas, but perception and reflection evoked roughly equal activity in more anterior-superior areas. These results show that even one of the simplest acts of reflection – a relatively transient thought of something that was observed just a moment ago – can induce stimulus-specific modulation of multiple extrastriate visual areas, and such top-down modulation of internal representations of external stimuli appears to rely, to varying degrees, on a distributed network of regions initially used to perceive the stimuli. When considered in the context of existing literature, our results suggest that refreshing may be an important component process of more complex operations that have previously been shown to exhibit top-down effects, such as visual mental imagery and visual working memory maintenance over extended delays.

Modulatory Refresh Effects in Posterior Cortical Regions

As hypothesized, we observed refresh-related top-down modulation in FFA and PPA. The PPA effect was significant and bilateral whereas the FFA effect was only at a trend level on the right. FFA may have exhibited weaker task-related modulation due to ceiling effects; anecdotally, most participants reported paying more attention to faces than scenes during initial presentation. Also, whereas PPA showed relatively little response to faces, FFA exhibited a positive response to scenes. PPA may constitute a better marker for top-down modulation effects than FFA, due to its lesser responsiveness to non-preferred stimuli (Gazzaley et al., 2005).

Top-down modulation of posterior cortex has been reported during mental imagery. For example, O’Craven and Kanwisher (2000) found top-down effects in FFA and PPA during mental imagery of faces and scenes, respectively, but to our knowledge, no imagery study has demonstrated face- or scene-specific modulation effects in any of the other stimulus-specific regions we report (RSC, PCu, MOG, and IOG). Other mental imagery studies (reviews: Kosslyn et al., 2001, Mellet et al., 1998a) have reported activations throughout the visual processing stream, extending in some cases to primary visual cortex, that may overlap some of the regions reported here. However, without directly contrasting activity related to different imagined stimuli, it cannot be concluded from such activations alone that the activated regions represent information about a specific stimulus or class of stimuli.

Gradient of Responsiveness to Reflection/Perception

When we compared our four scene-selective ROIs (lateral PCu, RSC, PPA, MOG) in terms of responsiveness to reflection and perception (Fig. 3B,C), we observed a gradient: Posterior-inferior areas showed greater activity for perception than reflection, and the difference decreased anterior-superiorly up to lateral PCu, where the effects of perception and reflection did not differ. (Note, however, that even perceptually biased regions showed at least trends toward top-down effects.)

Our gradient finding, while novel to our knowledge, is consistent with prior literature in several important ways. The fact that perception- and reflection-related activity in PCu did not differ as in other posterior areas is consistent with the finding that PCu activity during imagination is associated with reality monitoring failures (Gonsalves et al., 2004) and that the PCu may be playing a more general role such as relaying top-down signals from PFC to other posterior regions (Mechelli et al., 2004). The idea of a posterior reflective-perceptual gradient is also consistent with lesion studies of imagery deficits (review: Bartolomeo, 2002), which have demonstrated intact imagery with impaired perception and vice versa, suggesting that perceptual regions are not all equally necessary for both functions. Our finding of an anatomical gradient for the relative strengths of reflection- and perception-related activation are generally consistent with Bartolomeo’s claims that occipital damage is not necessary or sufficient to produce imagery deficits, that temporal lobe damage often does accompany various kinds of imagery deficits, and that “cortical areas which are related to vision, but at a higher level of integration than previously proposed, might be crucial for visual mental imagery abilities” (p. 373).

Recently, Shomstein and Behrmann (2006) reported what could be an analogous result in the domain of visual attention. The authors examined attentional modulation in retinotopic visual areas V1, V2, V3, and V4 to colored squares flashed on a screen, and found gradually decreasing degrees of modulation from area V4 to V1. Although there are many differences between that study of perceptual attention to colored squares and our study of reflective attention via refreshing to complex face and scene stimuli, the convergent finding of a gradient of sensitivity to top-down effects may reflect a general principle of how top-down mechanisms operate.

The gradient we observed could be interpreted in a number of ways. One possibility consistent with the lesion literature is that it is indicative of these areas’ decreasing role in reflective processes and increasing role in perceptual processes along the gradient. Mechanistically, it is unclear whether such gradients result from a single control region (e.g., PFC) exerting differential amounts of direct influence on different regions within the gradient, or whether the control region primarily exerts its influence on one region (e.g., PCu) that sequentially “trickles down” via feedback connections to the other areas in the gradient, with a decrease in top-down influence occurring at each step. Other mechanisms are possible as well; future studies will need to investigate further the functional properties of top-down gradients and the mechanisms that give rise to them.

Refreshing as a Minimal Executive Function

The assumption that prefrontal executive processes produce reflective, stimulus-specific, top-down activation of posterior areas used during perception is a core idea in models of visual WM (Curtis and D’Esposito, 2003; Petrides, 1994; Postle et al., 1999, 2003; Ranganath and D’Esposito, 2005; Ruchkin et al., 2003; Rypma and D’Esposito, 1999) and mental imagery (Farah, 1984; Kosslyn, 1980, 1994). Our findings support such models by demonstrating that the relatively simple and transient cognitive process of foregrounding a representation via refreshing may be one mechanism responsible for PFC activity and associated posterior modulation observed in more complex tasks. For example, imagery studies often involve either long-term memory retrieval (e.g., Ganis et al., 2004; Ishai et al., 2000, 2002; Kosslyn et al., 2005; Mazard et al., 2005; Mechelli et al., 2004; Mellet et al., 1998b) or the construction of a complex image from visually or verbally described components (e.g., Mellet et al., 1996; Yomogida et al., 2004). Hence, the observed neural activity in such studies could be a consequence of retrieval of information from long-term memory, manipulation involved in image construction, refreshing activated representations, or some combination of these and/or other processes. Determining the functions and neural correlates of component cognitive processes such as refreshing can thus be useful for deconstructing more complex operations and associated neural activity into constituent parts, in turn allowing for more precise descriptions of the functional specificity of brain regions. Furthermore, identifying the neural substrates of such component processes may help account for the surprisingly similar patterns of activity observed across quite different tasks (e.g., Duncan and Owen, 2000; M.R. Johnson and M.K. Johnson, in preparation).

In short, the MEM cognitive framework (Johnson, 1983; Johnson and Hirst, 1991) postulates refreshing to be a basic component process contributing to reflective thought. Refreshing is proposed to be a minimal executive function (Raye et al., 2007) that is a fundamental component of WM maintenance, selection, and manipulation (Johnson et al., 2005). The present findings strengthen the claim that refreshing plays an important role in executive functions by demonstrating that it not only engages the PFC, as shown before (Johnson et al., 2005; Raye et al., 2002, 2007), but is also capable of biasing information processing in posterior areas of cortex in a content-specific manner.

Acknowledgments

This research was supported by NIH AG15793. We thank the staff of the Yale Magnetic Resonance Research Center for their help with fMRI data acquisition.

Footnotes

1

This contrast involved all 6 Refresh and 6 Repeat conditions in the session (including the 2 Refresh and 2 Repeat conditions reported here in more detail).

2

The finding of numerically greater perception-related activity held even for each ROI’s non-preferred stimulus class (i.e., Rep_S>Ref_S in FFA, Rep_F>Ref_F in PPA), although it was only significant in right FFA (P<.05, two-tailed paired t-test). This was not entirely unexpected, as bilateral FFA showed some degree of reactivity to scene stimuli in the Localizer task (compared to a fixation-cross baseline; data not shown). The greater reactivity of the FFA to scene stimuli (versus the PPA’s reactivity to face stimuli) may also account for the fact that FFA tended to exhibit greater activity than PPA across the conditions of the Experimental task.

3

As noted in the Materials and Methods, refresh-related activity was assessed using a Refresh>Repeat contrast. For the opposite contrast, Repeat>Refresh, we found greater activation for repeating than refreshing a face or scene in areas activated by face or scene perception in the Localizer task. This is consistent with previous reports of greater activity in visual areas in repeat than refresh conditions for words, and pictures of people or scenes (Raye et al., 2002, Figure 1; Johnson et al., 2005, Figure 1.3).

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Baddeley AD. Working memory. Science. 1992;255:556–559. doi: 10.1126/science.1736359. [DOI] [PubMed] [Google Scholar]
  2. Baddeley AD, Hitch G. Working memory. In: Bower GH, editor. The Psychology of Learning and Motivation. Vol. 8. Academic Press; San Diego: 1974. pp. 47–90. [Google Scholar]
  3. Bartolomeo P. The relationship between visual perception and visual mental imagery: a reappraisal of the neuropsychological evidence. Cortex. 2002;38:357–378. doi: 10.1016/s0010-9452(08)70665-8. [DOI] [PubMed] [Google Scholar]
  4. Curtis CE, D’Esposito M. Persistent activity in the prefrontal cortex during working memory. Trends Cogn Sci. 2003;7:415–423. doi: 10.1016/s1364-6613(03)00197-9. [DOI] [PubMed] [Google Scholar]
  5. Downing PE, Chan AW, Peelen MV, Dodds CM, Kanwisher N. Domain specificity in visual cortex. Cereb Cortex. 2006;16:1453–1461. doi: 10.1093/cercor/bhj086. [DOI] [PubMed] [Google Scholar]
  6. Druzgal TJ, D’Esposito M. Dissecting contributions of prefrontal cortex and fusiform face area to face working memory. J Cogn Neurosci. 2003;15:771–784. doi: 10.1162/089892903322370708. [DOI] [PubMed] [Google Scholar]
  7. Duncan J, Owen AM. Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends Neurosci. 2000;23:475–483. doi: 10.1016/s0166-2236(00)01633-7. [DOI] [PubMed] [Google Scholar]
  8. Epstein R, Kanwisher N. A cortical representation of the local visual environment. Nature. 1998;392:598–601. doi: 10.1038/33402. [DOI] [PubMed] [Google Scholar]
  9. Farah MJ. The neurological basis of mental imagery: a componential analysis. Cognition. 1984;18:245–272. doi: 10.1016/0010-0277(84)90026-x. [DOI] [PubMed] [Google Scholar]
  10. Freire L, Mangin JF. Motion correction algorithms may create spurious brain activations in the absence of subject motion. NeuroImage. 2001;14:709–722. doi: 10.1006/nimg.2001.0869. [DOI] [PubMed] [Google Scholar]
  11. Freire L, Roche A, Mangin JF. What is the best similarity measure for motion correction in fMRI time series? IEEE Trans Med Imaging. 2002;21:470–484. doi: 10.1109/TMI.2002.1009383. [DOI] [PubMed] [Google Scholar]
  12. Ganis G, Thompson WL, Kosslyn SM. Brain areas underlying visual mental imagery and visual perception: an fMRI study. Brain Res Cogn Brain Res. 2004;20:226–241. doi: 10.1016/j.cogbrainres.2004.02.012. [DOI] [PubMed] [Google Scholar]
  13. Gazzaley A, Cooney JW, McEvoy K, Knight RT, D’Esposito M. Top-down enhancement and suppression of the magnitude and speed of neural activity. J Cogn Neurosci. 2005;17:507–517. doi: 10.1162/0898929053279522. [DOI] [PubMed] [Google Scholar]
  14. Gonsalves B, Reber PJ, Gitelman DR, Parrish TB, Mesulam MM, Paller KA. Neural evidence that vivid imagining can lead to false remembering. Psychol Sci. 2004;15:655–660. doi: 10.1111/j.0956-7976.2004.00736.x. [DOI] [PubMed] [Google Scholar]
  15. Grill-Spector K, Sayres R, Ress D. High-resolution imaging reveals highly selective nonface clusters in the fusiform face area. Nat Neurosci. 2006;9:1177–1185. doi: 10.1038/nn1745. [DOI] [PubMed] [Google Scholar]
  16. Ishai A, Ungerleider LG, Haxby JV. Distributed neural systems for the generation of visual images. Neuron. 2000;28:979–990. doi: 10.1016/s0896-6273(00)00168-9. [DOI] [PubMed] [Google Scholar]
  17. Ishai A, Haxby JV, Ungerleider LG. Visual imagery of famous faces: Effects of memory and attention revealed by fMRI. NeuroImage. 2002;17:1729–1741. doi: 10.1006/nimg.2002.1330. [DOI] [PubMed] [Google Scholar]
  18. Johnson MK, Raye CL. Reality monitoring. Psychol Rev. 1981;88:67–85. [Google Scholar]
  19. Johnson MK. A multiple-entry, modular memory system. In: Bower GH, editor. The Psychology of Learning and Motivation: Advances in Research and Theory. Vol. 17. Academic Press; New York: 1983. pp. 81–123. [Google Scholar]
  20. Johnson MK, Hirst W. Processing subsystems of memory. In: Lister RG, Weingartner HJ, editors. Perspectives on cognitive neuroscience. Oxford University Press; New York: 1991. pp. 197–217. [Google Scholar]
  21. Johnson MK. MEM: Mechanisms of recollection. J Cogn Neurosci. 1992;4:268–280. doi: 10.1162/jocn.1992.4.3.268. [DOI] [PubMed] [Google Scholar]
  22. Johnson MK, Hirst W. MEM: Memory subsystems as processes. In: Collins AF, Gathercole SE, Conway MA, Morris PE, editors. Theories of memory. Lawrence Erlbaum Associates, East Sussex; England: 1993. pp. 241–286. [Google Scholar]
  23. Johnson MK, Reeder JA, Raye CL, Mitchell KJ. Second thoughts versus second looks: An age-related deficit in reflectively refreshing just-activated information. Psychol Sci. 2002;13:64–67. doi: 10.1111/1467-9280.00411. [DOI] [PubMed] [Google Scholar]
  24. Johnson MK, Raye CL, Mitchell KJ, Greene EJ, Cunningham WA, Sanislow CA. Using fMRI to investigate a component process of reflection: prefrontal correlates of refreshing a just-activated representation. Cogn Affect Behav Neurosci. 2005;5:339–361. doi: 10.3758/cabn.5.3.339. [DOI] [PubMed] [Google Scholar]
  25. Kanwisher N, McDermott J, Chun MM. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci. 1997;17:4302–4311. doi: 10.1523/JNEUROSCI.17-11-04302.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Kosslyn SM. Image and mind. Harvard University Press; Cambridge, MA: 1980. [Google Scholar]
  27. Kosslyn SM. Image and brain: the resolution of the imagery debate. The MIT Press; Cambridge, MA: 1994. [Google Scholar]
  28. Kosslyn SM, Ganis G, Thompson WL. Neural foundations of imagery. Nat Rev Neurosci. 2001;2:635–642. doi: 10.1038/35090055. [DOI] [PubMed] [Google Scholar]
  29. Kosslyn SM, Thompson WL, Sukel KE, Alpert NM. Two types of image generation: Evidence from PET. Cogn Affect Behav Neurosci. 2005;5:41–53. doi: 10.3758/cabn.5.1.41. [DOI] [PubMed] [Google Scholar]
  30. Maguire EA, Frith CD, Cipolotti L. Distinct neural systems for the encoding and recognition of topography and faces. NeuroImage. 2001;13:743–750. doi: 10.1006/nimg.2000.0712. [DOI] [PubMed] [Google Scholar]
  31. Mazard A, Laou L, Joliot M, Mellet E. Neural impact of the semantic content of visual mental images and visual percepts. Brain Res Cogn Brain Res. 2005;24:423–435. doi: 10.1016/j.cogbrainres.2005.02.018. [DOI] [PubMed] [Google Scholar]
  32. McCarthy G, Puce A, Gore JC, Allison T. Face-specific processing in the human fusiform gyrus. J Cogn Neurosci. 1997;9:605–610. doi: 10.1162/jocn.1997.9.5.605. [DOI] [PubMed] [Google Scholar]
  33. Mechelli A, Price CJ, Friston KJ, Ishai A. Where bottom-up meets top-down: neuronal interactions during perception and imagery. Cereb Cortex. 2004;14:1256–1265. doi: 10.1093/cercor/bhh087. [DOI] [PubMed] [Google Scholar]
  34. Mellet E, Tzourio N, Crivello F, Joliot M, Denis M, Mazoyer B. Functional anatomy of spatial mental imagery generated from verbal instructions. J Neurosci. 1996;16:6504–6512. doi: 10.1523/JNEUROSCI.16-20-06504.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Mellet E, Petit L, Mazoyer B, Denis M, Tzourio N. Reopening the mental imagery debate: lessons from functional anatomy. NeuroImage. 1998a;8:129–139. doi: 10.1006/nimg.1998.0355. [DOI] [PubMed] [Google Scholar]
  36. Mellet E, Tzourio N, Denis M, Mazoyer B. Cortical anatomy of mental imagery of concrete nouns based on their dictionary definition. Neuroreport. 1998b;9:803–808. doi: 10.1097/00001756-199803300-00007. [DOI] [PubMed] [Google Scholar]
  37. Miller EK, Cohen JD. An integrative theory of prefrontal cortex function. Annu Rev Neurosci. 2001;24:167–202. doi: 10.1146/annurev.neuro.24.1.167. [DOI] [PubMed] [Google Scholar]
  38. O’Craven KM, Kanwisher N. Mental imagery of faces and places activates corresponding stimulus-specific brain regions. J Cogn Neurosci. 2000;12:1013–1023. doi: 10.1162/08989290051137549. [DOI] [PubMed] [Google Scholar]
  39. Petrides M. Frontal lobes and working memory: evidence from investigations of the effects of cortical excisions in nonhuman primates. In: Boller F, Grafman J, editors. Handbook of Neuropsychology. Vol. 9. Elsevier Science; Amsterdam: 1994. pp. 59–82. [Google Scholar]
  40. Postle BR, Berger JS, D’Esposito M. Functional neuroanatomical double dissociation of mnemonic and executive control processes contributing to working memory performance. Proc Natl Acad Sci USA. 1999;96:12959–12964. doi: 10.1073/pnas.96.22.12959. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Postle BR, Druzgal TJ, D’Esposito M. Seeking the neural substrates of visual working memory storage. Cortex. 2003;39:927–946. doi: 10.1016/s0010-9452(08)70871-2. [DOI] [PubMed] [Google Scholar]
  42. Ranganath C, DeGutis J, D’Esposito M. Category-specific modulation of inferior temporal activity during working memory encoding and maintenance. Brain Res Cogn Brain Res. 2004;20:37–45. doi: 10.1016/j.cogbrainres.2003.11.017. [DOI] [PubMed] [Google Scholar]
  43. Ranganath C, D’Esposito M. Directing the mind’s eye: prefrontal, inferior and medial temporal mechanisms for visual working memory. Curr Opin Neurobiol. 2005;15:175–182. doi: 10.1016/j.conb.2005.03.017. [DOI] [PubMed] [Google Scholar]
  44. Raye CL, Johnson MK, Mitchell KJ, Reeder JA, Greene EJ. Neuroimaging a single thought: dorsolateral PFC activity associated with refreshing just-activated information. NeuroImage. 2002;15:447–453. doi: 10.1006/nimg.2001.0983. [DOI] [PubMed] [Google Scholar]
  45. Raye CL, Johnson MK, Mitchell KJ, Greene EJ, Johnson MR. Refreshing: A minimal executive function. Cortex. 2007;43:135–145. doi: 10.1016/s0010-9452(08)70451-9. [DOI] [PubMed] [Google Scholar]
  46. Ruchkin DS, Grafman J, Cameron K, Berndt RS. Working memory retention systems: a state of activated long-term memory. Behav Brain Sci. 2003;26:709–728. doi: 10.1017/s0140525x03000165. [DOI] [PubMed] [Google Scholar]
  47. Rypma B, D’Esposito M. The roles of prefrontal brain regions in components of working memory: effects of memory load and individual differences. Proc Natl Acad Sci USA. 1999;96:6558–6563. doi: 10.1073/pnas.96.11.6558. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Shomstein S, Behrmann M. Cortical systems mediating visual attention to both objects and spatial locations. Proc Natl Acad Sci USA. 2006;103:11387–11392. doi: 10.1073/pnas.0601813103. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Wojciulik E, Kanwisher N, Driver J. Covert visual attention modulates face-specific activity in the human fusiform gyrus: fMRI study. J Neurophysiol. 1998;79:1574–1578. doi: 10.1152/jn.1998.79.3.1574. [DOI] [PubMed] [Google Scholar]
  50. Yi DJ, Chun MM. Attentional modulation of learning-related repetition attenuation effects in human parahippocampal cortex. J Neurosci. 2005;25:3593–3600. doi: 10.1523/JNEUROSCI.4677-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Yomogida Y, Sugiura M, Watanabe J, Akitsuki Y, Sassa Y, Sato T, Matsue Y, Kawashima R. Mental visual synthesis is originated in the fronto-temporal network of the left hemisphere. Cereb Cortex. 2004;14:1376–1383. doi: 10.1093/cercor/bhh098. [DOI] [PubMed] [Google Scholar]

RESOURCES