Dissociable Neural Mechanisms for Goal-Directed Versus Incidental Memory Reactivation

Abstract

Remembering a past event involves reactivation of distributed patterns of neural activity that represent the features of that event—a process that depends on associative mechanisms supported by medial temporal lobe structures. Although efficient use of memory requires prioritizing those features of a memory that are relevant to current behavioral goals (target features) over features that may be goal-irrelevant (incidental features), there remains ambiguity concerning how this is achieved. We tested the hypothesis that although medial temporal lobe structures may support reactivation of both target and incidental event features, frontoparietal cortex preferentially reactivates those features that match current goals. Here, human participants were cued to remember either the category (face/scene) to which a picture belonged (category trials) or the location (left/right) in which a picture appeared (location trials). Multivoxel pattern analysis of fMRI data were used to measure reactivation of category information as a function of its behavioral relevance (target vs incidental reactivation). In ventral/medial temporal lobe (VMTL) structures, incidental reactivation was as robust as target reactivation. In contrast, frontoparietal cortex exhibited stronger target than incidental reactivation; that is, goal-modulated reactivation. Reactivation was also associated with later memory. Frontoparietal biases toward target reactivation predicted subsequent memory for target features, whereas incidental reactivation in VMTL predicted subsequent memory for nontested features. These findings reveal a striking dissociation between goal-modulated reactivation in frontoparietal cortex and incidental reactivation in VMTL.

Introduction

Our experiences consist of combinations of features (e.g., perceptual, contextual, semantic, and affective details) that are collectively stored as events in memory (Johnson et al., 1993). A hallmark of the medial temporal lobe system, and the hippocampus in particular, is that it supports the binding of event features into an integrated memory (Squire, 1992; Johnson and Chalfonte, 1994; Davachi and Wagner, 2002; Shimamura, 2010) and allows individual cues to reactivate sets of features (Treves and Rolls, 1994). Indeed, reactivation mediated by the hippocampus may occur automatically (Moscovitch, 1992) and even implicitly (Turk-Browne et al., 2010). However, despite the potential richness of memories, and the automaticity with which features may be reactivated, remembering tends to be goal directed; that is, we typically seek to retrieve information from memory selectively. For example, returning from a wedding, one may face specific questions about the reception. Where was she sitting? What was he wearing? Frontoparietal cortex is thought to be involved in biasing mnemonic processing in favor of goal-relevant representations (Miller, 2000; Buckner and Wheeler, 2001; Badre and Wagner, 2007; Cabeza et al., 2008), though the nature of these frontoparietal control mechanisms remains under-specified.

It has been argued that the cognitive control mechanisms that guide memory retrieval are distinct from the mechanisms that support reactivation per se, in that control regions do not actively represent mnemonic content (Buckner and Wheeler, 2001). However, in studies of visual perceptual attention and working memory in nonhuman primates, a hallmark of neurons in frontoparietal cortex is that they actively and preferentially represent goal-relevant features of a stimulus (Rainer et al., 1998; Duncan, 2001; Assad, 2003). For example, neurons in monkey lateral parietal cortex that typically do not code for stimulus color, will adaptively code for color if it determines the appropriate behavioral response (Toth and Assad, 2002). It is unclear, however, whether frontoparietal cortex contributes to memory retrieval in a similar manner; that is, by preferentially representing goal-relevant features of a memory.

Here, we report a human fMRI study of goal-directed memory retrieval that probed mnemonic content within frontoparietal and temporal lobe regions. Pictures (faces or scenes) were studied in one of two locations (left or right). During a memory test, participants reported either the visual category of the picture (category trials) or the picture's location (location trials). Multivoxel pattern analysis (MVPA) of fMRI data were used to measure reactivation of visual category information during test trials (Polyn et al., 2005; McDuff et al., 2009; Kuhl et al., 2011, 2012a). Category reactivation was considered as a function of its behavioral relevance; that is, on category trials (target reactivation) versus location trials (incidental reactivation). We separately considered target/incidental reactivation in frontoparietal versus ventral/medial temporal (VMTL) regions. Our primary hypothesis was that goal-directed remembering involves preferential reactivation of target features of a memory within frontoparietal cortex. In contrast, we predicted that incidental reactivation would be relatively greater in VMTL, due to binding of event features.

Additionally, we assessed whether reactivation during initial retrieval would predict subsequent memory. A tenet of episodic memory theories is that reactivation allows for re-encoding of an experience, thereby promoting memory strengthening (Johnson, 1992; Treves and Rolls, 1994; Eichenbaum, 2004). To the extent that frontoparietal cortex preferentially reactivates goal-relevant features, this bias should correspond to selective strengthening of target features. In contrast, if incidental reactivation of nontarget event features is more likely to occur in VMTL, this may represent an opportunity for goal-irrelevant features of an event to be strengthened.

Materials and Methods

Participants.

Twenty-six paid individuals (eight female) from the Yale University community participated. Participants were between the ages of 18 and 35 (M = 21.9); all were right-handed, native English speakers. Two additional participants (both male) were excluded due to excessive head motion. All gave informed written consent in accordance with procedures approved by the Yale University Institutional Review Board.

Materials.

Stimuli consisted of words (e.g., candle, hose) paired with pictures of well known people (e.g., Elton John, Angelina Jolie) or locations (e.g., Big Ben, Grand Canyon). Words (144 total) were nouns from the Medical Research Council psycholinguistic database and were selected based on word length (3–9 letters, M = 5.5). Pictures were drawn from online sources and consisted of grayscale photographs (72 faces, 72 scenes).

Experimental task.

The experiment consisted of three phases: study, test, and post-test. Participants first studied words paired with pictures of either faces or scenes (study phase), with each face/scene appearing on the left- or right-hand side of a computer screen (Fig. 1A). Participants' memories were then selectively tested (test phase): for each word, they were either cued to recall the corresponding picture's category (face vs scene; a category trial), cued to recall the location at which a picture was presented (left vs right; location trial), or not tested at all (baseline condition). The study and test phases were conducted during fMRI scanning. Participants completed 12 functional scans (blocks), each lasting 4 min and 26 s. Each scan began with 12 study trials and was followed by eight test trials. After fMRI scanning, participants completed a surprise memory post-test during which they were shown each word again and asked to recall both the category and location of the corresponding picture (Fig. 1B). Each phase is described in more detail below.

Figure 1.

Experimental paradigm. A, During fMRI scanning, participants completed alternating study/test cycles. During study, participants encoded words paired with pictures of faces or scenes appearing on the left- or right-hand side of the display. Each study phase was followed by a test phase that tested source memory for some of the words. For half of the test phases, participants were asked to recall the category of the corresponding picture (F = face, S = scene, DK = don't know; category trials); for the other half, participants were asked to recall the spatial location of the corresponding picture (L = left, r = right, DK = don't know; location trials). Critically, participants did not know which feature they would be tested on until after each study phase. fMRI data from the study phases were used to train pattern classifiers to discriminate between face versus scene trials; classifiers were then “tested” on data from the test phases to determine whether patterns of encoding activity were reactivated during retrieval. B, After exiting the scanner, participants completed a post-test that probed memory for both the category and location corresponding to all previously studied words. Category memory was tested first (1 = definitely face, 2 = probably face, 3 = probably scene, 4 = definitely scene) and was immediately followed by a test of location memory for that item (1 = definitely left, 2 = probably left, 3 = probably right, 4 = definitely right). C, Post-test feature memory was considered accurate if participants selected the relevant feature with high confidence. Accuracy was considered as a function of prior testing: category previously tested, location previously tested, or no prior test (baseline). Testing a feature improved later memory for that feature, relative to baseline, but also improved later memory for the corresponding nontested (incidental) feature. The testing benefit was, however, stronger for the feature that had been tested (target) than the nontested (incidental) feature. Error bars reflect within-subject SE of the interaction.

During the study, words were horizontally centered on the screen and appeared above the pictures. Pictures were presented either on the left- or right-hand side of the screen (and were thus not horizontally aligned with the words). Opposite to the picture was a rectangle with a white border and black interior (an empty box). The empty box was the same size as the picture and was included to more strongly emphasize the picture location (e.g., if the picture was in the left position, it was clear that the right position was “empty”). Participants were instructed to encode both the picture category and the picture location that was associated with each word. Each word-picture pair was presented for 4 s and was followed by an 8 s baseline period during which participants completed an arithmetic task. The baseline period began with the presentation of a fixation cross (1 s), followed by a two-digit number (1 s), an addition sign (1 s), a second two-digit number (1 s), an equal sign (1 s), and then a third number along with a question mark (2 s) that was either the sum of the first two numbers (50% likelihood) or ±1, 2,10, or 11 units (each equally likely for a total probability of an incorrect total = 50%). Participants indicated whether the sum was correct or incorrect by pressing keys corresponding to their index or middle fingers, respectively, on a button box. The baseline arithmetic task was intended to prevent participants from covertly rehearsing study items in between trials.

After the 12 study trials within each block, participants were informed whether the upcoming test phase would test their memory for the picture category (face/scene) or the picture location (left/right) that corresponded to various words. Half (6) of all test phases tested category memory; half tested location memory (two pseudo-random orders were alternated across participants). Eight of the 12 items from each study phase were tested during each test phase, with the untested items functioning as a no-test baseline condition. Each test trial consisted of the presentation of a word, in the same screen position as during the study phase, along with a reminder of the test task and relevant response keys (i.e., “F–S–DK” for the category blocks, to indicate “face,” “scene,” or “don't know” as response options; “L–R–DK” for the location blocks to indicate “left,” “right,” or “don't know”). Participants were instructed to use the index, middle, and ring fingers of their right hand to make responses. Each test trial lasted 4 s and was followed by an 8 s baseline period identical to the study phase (i.e., with the same arithmetic task). Participants' responses were recorded for an additional 1 s after the trial ended (i.e., during the 1 s fixation cross) to reduce the number of trials with no response.

Before entering the scanner, participants completed two short practice study/test rounds to orient them to the task. Upon entering the scanner, participants completed two additional practice rounds to further develop comfort with the task and to gain familiarity with the button box used for responding.

After exiting the scanner, participants completed a surprise post-test that probed their memory for the picture category (face/scene) and location (left/right) that corresponded to each word presented during the study phases. Two-thirds of these words had been tested during the test rounds; one-third had not been tested. Each post-test trial first tested participants' memory for the picture category. They made one of four responses: “definitely face,” “probably face,” “probably scene,” or “definitely scene.” Immediately after making their response, participants were asked to indicate the picture location: “definitely left,” “probably left,” “probably right,” or “definitely right.” The post-test was completely self-paced with no emphasis placed on responding quickly. The post-test typically lasted 10–15 min.

fMRI data acquisition.

Imaging data were collected on a 3T Siemens Trio scanner at the Anlyan Center at Yale University using a 12-channel head coil. Before the functional imaging, two T1-weighted anatomical scans were collected (in-plane and high-resolution 3D). Functional data were collected using a T2*-weighted gradient EPI sequence; TR = 2000 ms, TE = 25 ms, flip angle = 90°, 34 axial-oblique slices, 224 mm FOV (3.5 × 3.5 × 4 mm). A total of 12 functional scans were collected. Each scan consisted of 133 volumes; the first five volumes were discarded to allow for T1 equilibration.

fMRI data preprocessing.

fMRI data preprocessing was conducted using SPM8 (Wellcome Department of Cognitive Neurology, London, UK). Images were first corrected for slice timing and head motion. High-resolution anatomical images were coregistered to the functional images and segmented into gray matter, white matter, and CSF. Segmented gray matter images were “skull-stripped” and normalized to a gray matter Montreal Neurological Institute template. Resulting parameters were used for normalization of functional images. Functional images were resampled to 3-mm³ voxels and smoothed with a Gaussian kernel (8 mm FWHM). Functional data were then high-pass filtered (0.01 Hz), detrended, and Z-scored within scan. After relevant trials and corresponding volumes had been selected, data were Z-scored again. First, Z-scoring was performed across all voxels within each volume at each point in time (i.e., mean response for each volume at each time point = 0), which had the effect of expressing the activity of a given voxel at a given point in time relative to activity in other voxels at the same point in time. Second, Z-scoring was performed, for each voxel, across all volumes corresponding to the study phase and, separately, across all volumes corresponding to the test phase (mean response for each voxel within each phase = 0); this had the effect of expressing the activity of a given voxel on a given trial relative to the response of that voxel on other trials from the same phase. Thus, main effects of phase (e.g., study trials being associated with larger responses) would be eliminated.

Pattern classification analyses.

Pattern classification analyses were performed using a penalized (L2) logistic regression classifier and implemented via the Princeton MVPA toolbox and custom MATLAB code. Classification analyses were first performed using three broad regions of interest (ROIs): prefrontal cortex (PFC), lateral parietal cortex (LPC), and VMTL. Classification analyses were also performed using ROIs representing subregions of each of the broad ROIs. ROIs were created using the Anatomical Automatic Labeling (AAL) atlas (Tzourio-Mazoyer et al., 2002). The PFC ROI was comprised of subregions representing inferior, middle, superior, orbitofrontal, and medial frontal gyri (including anterior cingulate cortex). The LPC ROI, which was modified slightly from the AAL atlas, was comprised of subregions representing superior parietal lobule, angular gyrus, supramarginal gyrus, and temporoparietal junction (the most inferior aspect of supramarginal gyrus). The VMTL ROI was comprised of subregions representing parahippocampal and fusiform gyri as well as hippocampus. All subregions were constructed such that they contained zero overlapping voxels. For classification analyses performed using the broad ROIs, analyses were restricted to the 500 voxels from each region that were most sensitive to category differences during encoding (face vs scene), as determined by a subject-specific univariate contrast (i.e., a feature selection step). Equating the number of voxels across ROIs facilitated comparison across regions. Similarly, for subregion analyses, the 50 most category-sensitive voxels were selected from each subregion.

Each classifier was trained to discriminate face versus scene trials based on all of the scanned study trials; the classifier was then tested on the scanned test trials. Classifier performance was assessed in two ways. Classification accuracy represented a binary coding of whether or not a given test trial was correctly classified according to the category of the corresponding (studied) picture. Classifier output represented a continuous measure of the probability (range = 0–1) that the classifier assigned to the relevant category for each trial. For general assessment of classifier performance, classification accuracy was used as the measure of interest. Classifier output was used as the measure of interest when considering relationships with response speed and subsequent memory (where gradations in reactivation strength might be related to differences in memory) and for assessing region-to-region correlations.

Classification analyses were performed both on a trial-by-trial basis and on a TR-by-TR basis. For trial-by-trial classification analyses, each trial was reduced to a single image/volume by averaging volumes across TRs. For study trials, TRs 3–5 (representing 4–10 s post-trial onset) were equally weighted; for test trials, TRs 3–6 (4–12 s post-trial onset) were equally weighted. A slightly wider window was used for test trials because reactivation at test (retrieval) is likely to be delayed relative to activation at encoding. For TR-by-TR classification analyses, study trials (training data) were again averaged using equal weighting across TRs 3–5, but the classifier was separately tested at each TR (1–6) of each test trial.

Statistical analyses.

Comparison of behavioral and classification performance as a function of ROI and experimental condition was assessed via ANOVA. Comparison of classifier evidence to chance performance was assessed via one-sample t tests using a contrast value of 0.5. Planned comparisons of classifier performance across trial types (category vs location) used paired-sample t tests. All analyses used a threshold of p < 0.05 to reject the null hypothesis and all t tests were two-tailed.

Results

Behavioral performance

During the scanned test rounds, participants' responses were considered accurate if they indicated the correct picture category (face/scene) or location (left/right), as appropriate. Overall, participants were generally successful at remembering the category (M = 84.5%, SD = 12.5%) or location (M = 80.5%, SD = 13.2%) (Table 1). Performance in the category blocks was significantly higher than performance in the location blocks (t₍₂₅₎ = 2.07, p < 0.05).

Table 1.

Accuracy and response time (RT) for category and location trials in the immediate test rounds

	Proportion of responses				RT Correct
	Correct	Don't know	Error	No response
Category
Mean	84.5%	2.2%	12.2%	1.2%	2470
SD	12.5%	5.1%	9.5%	3.4%	364
Location
Mean	80.5%	4.7%	14.0%	0.7%	2475
SD	13.2%	9.5%	9.5%	2.2%	422

Memory at post-test (high-confidence correct responses; Fig. 1C, Table 2) was assessed using ANOVA with two levels of feature type (category, location) and three levels of experimental condition reflecting prior testing (baseline, prior category trial, prior location trial). Overall, participants recalled category information (face/scene) with high confidence more often (M = 54.4%) than location information (left/right; M = 43.7%), as revealed by a main effect of feature (F_(1,25) = 12.18, p < 0.005). Importantly, post-test memory was influenced by the testing condition, as revealed by a main effect of condition (F_(2,50) = 28.60, p < 0.001). Planned subsequent comparisons indicated that category memory at post-test was greater in the category-tested condition (M = 59.9%) and location-tested condition (M = 56.7%) than the baseline (no test) condition (M = 46.6%; all t > 4.6, p < 0.001). Thus, category memory at post-test was more accurate if either category or location memory was previously tested, relative to no test. Likewise, location information was more frequently recalled with high confidence in both the location-tested condition (M = 49.9%) and the category-tested condition (M = 46.8%) than in the no test condition (M = 34.4%; all t > 5.2, p < 0.001). Notably, although testing either feature (category or location) benefitted the other feature, the testing benefit was greater for the tested feature, as reflected by a significant interaction between tested feature (category vs location) and feature memory at post-test (category vs location; F_(1,25) = 8.89, p < 0.01).

Table 2.

Mean confidence rating at post-test for each feature as a function of test phase condition

	No test	Category tested	Location tested
Category
Mean	1.89	1.71	1.78
SD	0.37	0.40	0.38
Location
Mean	2.10	1.92	1.87
SD	0.29	0.33	0.35

Confidence ratings are rescored such that 1 = high confidence correct, 4 = high confidence incorrect.

Reactivation during initial memory test

To measure neural reactivation during test trials, we considered performance of a pattern classifier that was trained on patterns of activity elicited during encoding (study trials) and tested on activity elicited during test trials. In particular we focused on reactivation of category information (face/scene) because this is a form of information that is robustly reactivated during remembering (Polyn et al., 2005; Kuhl et al., 2011, 2012a). To generate a trial-level measure of classifier performance, fMRI activity was averaged across several time points corresponding to a given trial and this averaged response was then fed to the classifier. Each test trial was coded according to whether the pattern classifier correctly identified the category to which it corresponded. Because the pattern classifier could only succeed in classifying test trials to the extent that patterns of encoding activity were reinstated during test trials, successful classification of test trials was considered evidence for category reactivation. Reactivation could then be separated according to participants' goals on a given trial; that is, whether category or location information was goal-relevant. Separate classifiers were trained and tested for three broad ROIs: PFC, LPC, and VMTL (Fig. 2A; see Materials and Methods).

Figure 2.

Goal-modulated reactivation. A, Anatomical ROIs (PFC, LPC, VMTL) displayed on a standard-space template brain. B, Trial-level evidence for category reactivation (i.e., classification accuracy) during category (orange) and location (blue) test trials. Category reactivation was well above chance (50%) both during category trials (target reactivation) and location trials (incidental reactivation). LPC was associated with robust goal-modulation: greater category reactivation during category relative to location trials. Error bars represent SEM; ***p < 0.005. C, Classification accuracy on a TR-by-TR basis (shown in seconds) for each region of interest, separately for category (orange) and location (blue) trials. LPC was associated with sustained target reactivation. In contrast, VMTL target reactivation peaked earlier and subsided more quickly. Shaded area of the time course represents mean ± SEM.

On trials where category information was goal relevant (category trials), mean classification accuracy for the previously studied category was significantly greater than chance (0.5) for each ROI (t test, all p < 0.0001; Fig. 2B). Thus, distributed patterns of neural activity elicited during event encoding were reactivated during retrieval in PFC, LPC, and VMTL.

Critically, we next asked whether category information was incidentally reactivated during location trials; that is, whether category information was reactivated when location information was goal relevant. Indeed, during Location trials, classification accuracy for the category (which was goal irrelevant) was well above chance in each of the three ROIs (t test, all p < 0.005; Fig. 2B). An ANOVA with three levels of region (PFC, LPC, VMTL) and two levels of condition (category vs location trials) indicated no main effect of condition (F < 1). Importantly, however, the relative strength of target versus incidental reactivation differed across regions, as revealed by a significant interaction between region and test condition (F_(2,50) = 5.90, p = 0.005). Planned comparisons indicated that the goal modulation effect (target-incidental reactivation) in LPC was significant when considered on its own (p = 0.003) and was greater than the effects in VMTL (p = 0.003) and PFC (p = 0.03). The effect in PFC did not reach significance (p = 0.31) and was not significantly greater than the effect in VMTL (p = 0.11).

To further characterize the differences in target versus incidental reactivation across regions, we compared the temporal profile of reactivation (Fig. 2C). That is, we compared classification accuracy at each of the six TRs (2 s intervals) of a test trial (see Materials and Methods). Comparing target reactivation in LPC versus VMTL, we observed a robust time × region interaction (F_(5,125) = 3.49, p = 0.006): early in the trial (4–6 s), target reactivation was more robust in VMTL than LPC (t₍₂₅₎ = 2.48, p < 0.05), whereas by the last time point (10–12 s), this relationship had reversed (t₍₂₅₎ = −2.31, p < 0.01). Target reactivation also remained well above chance in LPC at the last time point (p = 0.005) but had returned to chance in VMTL (p = 0.94). Notably, a time × region (LPC vs VMTL) interaction was not present when considering incidental reactivation (F_(5,125) = 1.04, p = 0.40). Thus, whereas VMTL target reactivation was relatively transient, LPC target reactivation persisted throughout the trial (Curtis and Lee, 2010; Vilberg and Rugg, 2012). Considering PFC versus VMTL, time × region interactions were not significant either for target or incidental reactivation (all F < 1).

The current LPC reactivation findings build on prior observations that LPC is activated when mnemonic information is successfully retrieved (Wagner et al., 2005; Cabeza et al., 2008; Vilberg and Rugg, 2008). However, although our LPC classifier used a broad mask, successful remembering of episodic details tends to be associated with increased responses within ventral aspects of LPC—particularly in angular gyrus (Hutchinson et al., 2009). Thus, to better characterize responses within LPC subregions we trained and tested four additional classifiers representing subregions of LPC [temporoparietal junction (TPJ), supramarginal gyrus (SMG), angular gyrus (ANG), and superior parietal lobule (SPL); Figure 3B]. We compared the overall magnitude of category reactivation (i.e., across category and location trials) as well as evidence for goal modulation as a function of subregion (TPJ, SMG, ANG, SPL). These analyses were based on the trial-level measures of reactivation (collapsing across TRs). The main effect of subregion was significant (F_(3,75) = 5.54, p = 0.002; Fig. 3B), with reactivation most robust in ANG and SPL. The difference in reactivation between ANG and SPL was not significant, however (p = 0.50). The main effect of condition (reflecting goal modulation), remained significant (F_(1,25) = 7.88, p < 0.01), as in the broader LPC mask. A trend toward a condition by subregion interaction (F_(3,75) = 2.36, p = 0.08), reflected somewhat greater goal modulation in SMG and SPL (goal modulation in SMG and SPL: p < 0.05), particularly relative to TPJ. An additional set of classification analyses that separated the four LPC subregions by hemisphere (eight total subregions) did not reveal a main effect of hemisphere (F_(1,25) = 1.65, p = 0.21) nor a condition by hemisphere interaction (F < 1).

Figure 3.

Reactivation by subregions. A, Top, Subregions of PFC; bottom, trial-level classification accuracy for category and location trials by PFC subregions. B, Same as A, but for LPC. C, Same as A, B, but for VMTL. Error bars represent SEM; *p < 0.05, **p < 0.01, ***p < 0.005.

Classifier performance from PFC and VMTL subregions was also considered. In VMTL, evidence for target and incidental reactivation was comparable across fusiform gyrus (FG), parahippocampal gyrus (PHG), and hippocampus (HIPP) (condition by subregion interaction: F< 1; Fig. 3C), and closely resembled the pattern in the broader VMTL ROI; namely, there was no evidence for stronger target than incidental reactivation when collapsing across subregions (F_(1,25) = 1.37, p = 0.25). In PFC, across five subregions [inferior frontal gyrus (IFG), middle frontal gyrus (MFG), superior frontal gyrus (SFG), medial prefrontal cortex (MPFC), orbitofrontal cortex (OFC)], the subregion by condition interaction was not significant (F_(4,100) = 1.34, p = 0.26; Fig. 3A); however, the main effect of condition (target > incidental reactivation) was significant (F_(1,25) = 4.54, p < 0.05). Thus, whereas the goal modulation effect was not significant in the broad PFC mask (p = 0.31), it was significant when averaging across PFC subregions. Similarly, whereas the goal-modulation effect was not significantly greater in PFC than VMTL when considering the broad masks (p = 0.11), this difference was significant when considering data from the subregions (p = 0.02). These apparent contradictions between data from the broad masks versus subregions can be explained in terms of the number and influence of voxels that contributed to the classifiers in each case. The goal modulation effect was, numerically, greatest in OFC (effect of condition: t₍₂₅₎ = 2.93, p = 0.007); however, in the broad PFC mask, OFC comprised only 9.4% of the total voxels. In contrast, the goal modulation effect was weakest in IFG (effect of condition: t₍₂₅₎ = − 0.37, p = 0.72), a subregion that comprised 26.4% of total voxels in the broad PFC mask. Thus, these additional analyses indicate that whereas goal-modulated reactivation was not evident across all PFC subregions, goal modulation was (1) present in PFC, and (2) stronger in PFC than VMTL.

Decoding retrieval goals

As an alternative to testing for goal-modulated reactivation of patterns of encoding activity, we also tested whether retrieval goals could be directly decoded from patterns of activity elicited at retrieval. That is, rather than decoding the category (face or scene?) of a retrieved image, we sought to decode the feature (category or location?) that was goal-relevant. The 12 test rounds were thus divided into six groups (folds), with each fold containing one category test round and one location test round. Using a cross-validation approach, classifiers for each region of interest were trained on 5/6 of the folds and applied to the remaining fold, with this process repeated until all trials contributed to both training and testing. Relative to the preceding classification analyses, one additional preprocessing step was included to minimize potential confounds: for each voxel, linear regression analyses were applied at the individual participant level to remove the influences of response time and response accuracy (Todd et al., 2013). Specifically, linear regression models were generated with these behavioral factors as predictors of a voxel's activity; trial-level residuals from these models (i.e., voxel activity for a participant unexplained by that participant's responses times or accuracy) were then used to train the retrieval goal classifiers. Classification was performed on a TR-by-TR basis for each trial (even though trial type was blocked).

Averaging across the six TR's for each trial, “goal decoding” was above chance in both LPC and PFC (p < 0.05), but not VMTL (p = 0.68). An ANOVA with factors of TR and region revealed that accuracy varied significantly across time (main effect of TR: F_(5,125) = 5.26, p < 0.001), peaking at the third and fourth TRs (4–8 s; Fig. 4A). When focusing specifically on the peak TR's (3 and 4), goal decoding was still not above chance in VMTL (p = 0.44), but was well above chance in LPC (p = 0.00002) and PFC (p = 0.0003; Fig. 4B). Moreover, goal decoding in both LPC and PFC was greater than in VMTL (p < 0.01). Thus, complementing the analyses above that specifically focused on reactivation of encoding activity, these results similarly indicate that retrieval goals influenced patterns of activity in PFC and LPC, but not VMTL. Notably, the fact that goal decoding peaked at TR's 3 and 4 indicates that goal classification was based on activity elicited by retrieval cues (i.e., based on what was retrieved), as opposed to a tonic goal state.

Figure 4.

Decoding of retrieval goals. A, TR-by-TR (shown in seconds) classification accuracy for the category versus location trial classifier for each of the three regions of interest. Shaded area of the time course represents mean ± SEM. B, Same as A, except collapsing across data from TRs 3 and 4 (4 s/6 s; gray rectangle shown in A). Error bars represent SEM; ***p < 0.0005, ****p < 0.00005.

Cross-region correlations

The preceding analyses indicate that frontoparietal reactivation was modulated by retrieval goals to a greater degree than VMTL. However, goal-directed remembering likely involves interactions between regions. To explore potential interactions, the output of the category classifier for each trial in region X was correlated with the output of the classifier for each trial in region Y (Kuhl et al., 2012b). [Note: here classifier output (a continuous measure of reactivation strength; see Materials and Methods) was used instead of classifier accuracy (a binary measure) because of the requirements of correlation analyses; however, the key results reported above (which used classifier accuracy) remained significant if classifier output was used instead: region (PFC, LPC, VMTL) × goal interaction (p = 0.003) and goal modulation effect in LPC (p = 0.008)]. Correlations were separately computed within the location and category test trials. All correlation coefficients were z-transformed (Fisher's z) and statistical analyses were applied to the resulting Z-scores.

Correlations were first computed for classification analyses that were performed on a trial-by-trial level (i.e., collapsing across TRs). Averaging across category and location trials, correlations were significantly >0 for all pairings (all < 0.005); however, the PFC–LPC correlation was markedly stronger than either the PFC–VMTL or LPC–VMTL correlations (p < 001; Fig. 5A). Correlations between regions were not modulated, however, by trial type (location vs category, all p > 0.07). Considering subregions within PFC and LPC, the correlation between SFG and ANG was particularly robust (Fig. 5B). When region-to-region correlations were calculated separately for each time point in a trial, the PFC–LPC correlation was stable throughout the course of a trial (effect of time: F_(5,125) = 1.24, p = 0.25), whereas the PFC–VMTL and LPC–VMTL correlations peaked during the time when VMTL reactivation was maximal (4–8 s; Fig. 5C; effect of time, PFC: F_(5,125) = 18.35, p < 0.001; LPC: F_(5,125) = 2.76, p = 0.02). Direct comparison of PFC–VMTL versus LPC–VMTL correlations indicated that PFC–VMTL correlations were more temporally selective than LPC–VMTL correlations (region × time interaction; F_(5,125) = 8.26, p < 0.001) and were weaker than LPC–VMTL correlations, when considered across the entire trial (F_(1,25) = 7.21, p = 0.01). Thus, content in PFC and LPC “coupled” with content in VMTL, particularly at the point in time when VMTL feature reactivation was strong, but as VMTL reactivation subsided; PFC and LPC content became less similar to VMTL while remaining highly similar to each other.

Figure 5.

Cross-region correlations. A, Mean z-transformed trial-level correlations between output from category classifiers for each pair of regions; *p < 0.05, **p < 0.01. B, Correlation matrix showing mean z-transformed correlations for all pairs of PFC, LPC, and VMTL subregions. C, Same as A, except showing correlations at each time point in a test trial (TR-by-TR, shown in seconds) and for each pair of regions. Note: all correlations were computed separately for category and location trials and then averaged across conditions. Shaded area of the time course represents mean ± SEM.

Reactivation and response time

The analyses reported thus far characterize how patterns of activity during retrieval were modulated by goals and whether information content was correlated across regions. We next sought to relate test phase reactivation to behavioral measures of response speed and success. Although subjects were not instructed to respond quickly during the test rounds in the present study, response times were recorded and presumably relate to the speed with which mnemonic decisions were reached. Thus, we compared reactivation across regions for trials associated with fast versus slow response times. For each participant, a median split of test trials associated with correct responding was performed. The median split was performed separately within each condition (category vs location trials) and for each image category (faces vs scenes). Because reactivation may be graded, and trial-level gradations in reactivation strength may be important for predicting behavior (Kuhl et al., 2011, 2012a), here we used classifier output (a continuous measure of classifier confidence; see Materials and Methods) as the dependent variable. Additionally, because fast versus slow trials may be associated with a shift in the time course of reactivation, as opposed to a difference in overall mean levels, we compared reactivation strength at TR's 3 versus 4 (the time points at which reactivation peaked) as a function of response time.

As shown in Figure 6, in VMTL, the interaction between TR (3 vs 4) and speed (fast vs slow) was significant, both for category trials (target reactivation; F_(1,25) = 8.13, p = 0.009) and location trials (incidental reactivation; F_(1,25) = 13.28, p = 0.001). Main effects of speed were not significant for either condition (F <1). In each case, the interaction reflected fast trials being associated with earlier-peaking reactivation of category information. That is, earlier-peaking category reactivation in VMTL predicted faster behavioral responding on category trials and location trials.

Figure 6.

Category reactivation and response time. Classifier output on a TR-by-TR basis (shown in seconds) for PFC, LPC, and VMTL (columns), separately for category trials (top row) and location trials (bottom row) associated with fast and slow response times. Main effect of response time, *p < 0.05. Interaction between TR (TRs 3/4 = onset of 4 s/6 s; gray rectangles) and response time: p̂ < 0.05, ^p̂ < 0.01, ^p̂ < 0.005. Shaded area of the time course represents mean ± SEM.

In PFC, for category trials there was both a significant interaction between TR and speed (F_(1,25) = 5.48, p = 0.03) as well as a main effect of speed (F_(1,25) = 6.98, p = 0.01): that is, faster responding on category trials was associated with earlier-peaking and overall greater category reactivation. In contrast, on location trials, there was no interaction between speed and TR (F_(1,25) = 1.68, p = 0.21), nor a main effect of speed (F_(1,25) = 1.12, p = 0.30). Thus, PFC reactivation was related to response speed, but only when considering behaviorally relevant reactivation.

Finally, in LPC, neither the TR by speed interaction nor the main effect of speed was significant when considering category trials (F <1). Likewise, neither the interaction (F_(1,25) = 1.35, p = 0.26) nor main effect of speed (F<1) were significant for location trials. Thus, LPC reactivation was unrelated to response speed either when the reactivated information was goal-relevant or incidental.

Reactivation and subsequent memory

Behaviorally, we found that an immediate memory test powerfully benefited longer-term memory not only for target (tested) but also for incidental (untested) features (e.g., category memory at post-test was improved if location had initially been tested; Fig. 1C). We next assessed whether these target and incidental testing benefits observed at post-test were related to trial-by-trial variability in the magnitude of reactivation during initial test trials. As with the response time analysis, above, here we used classifier output (a continuous measure) as the variable of interest. To assess the relationship between target reactivation and subsequent target memory, we compared, within-subjects, the strength of classifier output for category reactivation during category test trials as a function of whether the corresponding category was later remembered with high confidence at post-test (subsequently remembered) versus all other post-test outcomes (subsequently forgotten). Similarly, to assess the relationship between incidental category reactivation and subsequent category memory, we compared category reactivation during location test trials as a function of category memory at post-test. Thus, for both the category and location trials, we tested for a relationship between category reactivation during initial test and subsequent category memory. For both analyses, only trials associated with an accurate behavioral response during initial test trials (i.e., selection of either the appropriate category or location) were included (results were highly similar when all trials were included). Two participants failed to contribute data to at least one cell and were thus excluded from all subsequent memory analyses. Note: for each subject, classifier output for subsequently remembered and forgotten trials was first computed separately for face and scene trials; these means were then averaged across categories (face/scene).

A significant positive relationship between target reactivation (category reactivation on category trials) and subsequent category memory was observed in PFC (t₍₂₃₎ = 4.07, p = 0.0005) and LPC (t₍₂₃₎ = 2.23, p = 0.04), but not in VMTL (t₍₂₃₎ = − 0.68, p = 0.5; Fig. 7). That is, stronger frontoparietal target reactivation predicted better subsequent target memory, even when controlling for initial behavioral accuracy. Incidental reactivation (category reactivation on location trials) in PFC and LPC did not predict later category memory (p > 0.10). In contrast, incidental category reactivation in VMTL was significantly greater when category information was subsequently remembered versus forgotten (t₍₂₃₎ = 2.91, p = 0.008; Fig. 7). Thus, the strengthening that nontested features experienced was related to incidental reactivation in VMTL. Confirming the differential contribution of frontoparietal cortex versus VMTL to target versus incidental strengthening, the interaction between the subsequent memory effect as a function of feature (target vs incidental) and region (PFC, LPC, VMTL) was significant (F_(2,46) = 5.91, p = 0.005; Fig. 7A).

Figure 7.

Category reactivation during initial test trials as a function of subsequent category memory at post-test. A, Trial-level category reactivation (classifier output) during test trials for which category information was subsequently remembered–subsequently forgotten (at post-test). Category reactivation during category trials (target reactivation; orange) in PFC and LPC predicted subsequent category memory. Category reactivation during location trials (incidental reactivation; blue) in VMTL predicted subsequent category memory. Data only represent trials associated with behavioral accuracy during initial in-scanner test trials. Error bars represents SEM; *p < 0.05; **p < 0.01; ***p < 0.001. B, Category reactivation on a TR-by-TR basis (shown in seconds) as a function of condition (category trials, top row; location trials, bottom row) and subsequent memory (orange/blue = category information subsequently remembered; gray = category information subsequently forgotten). Shaded area of the time course represents mean ± SEM.

Discussion

We measured reactivation of visual category information according to its relevance to current retrieval goals. We found that category reactivation was overall robustly distributed across frontoparietal cortex and VMTL. However, category reactivation within frontoparietal cortex was greater when category information was goal-relevant (target reactivation) than when it was goal-irrelevant (incidental reactivation). In contrast, incidental reactivation was as strong as target reactivation in VMTL. The dissociation between reactivation in frontoparietal versus VMTL was further evidenced by differential relationships to subsequent memory. Target reactivation in frontoparietal cortex during initial test trials predicted strengthening of the tested (target) feature, whereas incidental reactivation in VMTL predicted strengthening of the nontested (incidental) feature.

Incidental versus automatic reactivation in VMTL

Here we have used the term incidental reactivation to refer to reactivation of feature information that was orthogonal to the tested feature and therefore not directly required for the participants' behavioral response (Nyberg et al., 2000; Wimber et al., 2012). However, this does not mean that incidental reactivation was either involuntary (Hall et al., 2008; Kompus et al., 2011) or implicit (Gratton et al., 1997; J.D. Johnson et al., 2009; Turk-Browne et al., 2010; Wimber et al., 2012). Indeed, participants indicated (in a postexperimental questionnaire) that category information often “came to mind” during location trials (mean frequency reported = 78.0%) and that this was “somewhat” deliberate (response options: 1 = “not at all deliberate,” 2 = “somewhat deliberate,” 3 = “very deliberate”; mean response = 2.1). Thus, even when category information was incidental to retrieval goals, it may have been strategically retrieved, which potentially accounts for the fact that incidental reactivation was present in frontoparietal cortex (cf. Wimber et al., 2012). It is striking, however, that additional strategic processing directed toward category information when it was goal relevant did not translate into goal-modulated reactivation in VMTL. Although it is clear that in some situations VMTL activity is modulated by retrieval goals (Dudukovic and Wagner, 2007; Kuhl et al., 2011, 2012a), the lack of VMTL modulation here may reflect strong binding of event features (Squire, 1992; Johnson and Chalfonte, 1994; Davachi and Wagner, 2002; Shimamura, 2010).

Frontoparietal content representations

The present observation that frontoparietal cortex preferentially reactivated behaviorally relevant category information is consistent with evidence of goal-modulated coding of perceptual stimuli in monkey PFC (Rainer et al., 1998; Duncan, 2001) and LPC (Toth and Assad, 2002; Assad, 2003). Although human neuroimaging studies have found that activity in frontoparietal regions is modulated by behavioral goals during memory retrieval (Dudukovic and Wagner, 2007; Quamme et al., 2010), these studies have not specifically focused on modulations in the content represented by these regions. Indeed, whereas several studies have reported reactivation that is broadly distributed across the brain, including in frontoparietal cortex (Khader et al., 2005; Polyn et al., 2005; Buchsbaum et al., 2012; Jost et al., 2012), human neuroimaging studies have more typically argued that the frontoparietal regions that guide memory retrieval are distinct from those that actively represent mnemonic content (Buckner and Wheeler, 2001; Badre and Wagner, 2007; Cabeza et al., 2008). Similarly, successful versus unsuccessful retrieval of episodic memories has consistently been associated with activation increases in PFC (Buckner and Wheeler, 2001) and LPC (Wagner et al., 2005; Cabeza et al., 2008), but it is not clear that regions signaling successful retrieval also represent mnemonic content (Vilberg and Rugg, 2008). Thus, the present results are notable in that they highlight robust content sensitivity distributed across frontoparietal cortex.

Content sensitivity in frontoparietal cortex raises the question of what, precisely, is being represented by these regions. Here, decoding of category information in frontoparietal cortex cannot be explained in terms of decoding the motor response that participants made because we trained our category classifier based on data from the study phases and participants did not know during the study phases which event feature (and therefore response) would later be relevant. Moreover, if only the response and not the mnemonic representation were being decoded, there should have been comparable classification of category (face/scene) and location (left/right), but we confirmed that location reactivation was markedly lower than category reactivation (<53% for each ROI). Similarly, it is unlikely that participants simply encoded (and retrieved) verbal tags representing each trial (e.g., “Word A = face, left”). If so, again, we should have seen comparable decoding of category and location information. In addition, participants did not report such a strategy nor is it likely that such a strategy would be optimal given the potential for interference (i.e., many cue words would be associated with the same verbal codes; e.g., “face, left”). Finally, a verbal tag account does not explain frontoparietal representations of visual stimuli that have been observed (1) in monkeys (Constantinidis et al., 2001; Toth and Assad, 2002; Freedman and Assad, 2006), where verbal labels are unlikely to be generated or (2) in humans when using nonverbalizable stimuli (Christophel et al., 2012).

We believe the frontoparietal reactivation observed here is most likely explained either in terms of content representations (Constantinidis et al., 2001; M.K. Johnson et al., 2003) or process representations (M.K. Johnson et al., 2003; J.D. Johnson et al., 2009; Danker and Anderson, 2010). According to a content account, frontoparietal regions represent perceptual details of an experience during both encoding and retrieval. These representations could be a fixed property of the functional organization of PFC and/or LPC (M.K. Johnson et al., 2003) or may develop purely based on task demands (M.K. Johnson et al., 2003; Freedman and Assad, 2006; Tosoni et al., 2008). According to a process account, patterns of activity in PFC and LPC differ according to visual category because distinct categories invoke distinct control processes. However, these possibilities are not mutually exclusive (Wood and Grafman, 2003) and the most likely account may be that task demands determine both the category representations that are relevant (Freedman and Assad, 2006) and the particular processes that will be engaged.

Prefrontal versus lateral parietal contributions

We observed strong trial-level correlations between reactivation (or classifier output) in PFC and LPC, suggesting functional communication between these regions. Reactivation in each of these regions also correlated with reactivation in VMTL. Although reactivation should initially be triggered by the hippocampus, it is unclear how and when information propagates to PFC versus LPC. It is possible, for example, that modulation in LPC was the result of bias signals originating in PFC (Tomita et al., 1999; Miller and Cohen, 2001). However, in at least some contexts, category representations in LPC precede those in PFC (Swaminathan and Freedman, 2012). Thus, reactivation in LPC may not be a simple reflection of PFC biases; rather, LPC may make distinct contributions or PFC and LPC may reciprocally interact to achieve biased reactivation (Chafee and Goldman-Rakic, 2000). Interestingly, we found that response speed was related to target reactivation (but not incidental reactivation) in PFC; that is, response speed was related to representations of behaviorally relevant information in PFC. This result contrasted with LPC, where reactivation (target or incidental) was unrelated to response time. Although caution is warranted in interpreting a null result, it is possible that PFC is more directly involved in the memory-based decision making process whereas LPC may support sustained attention to the target feature (Curtis and Lee, 2010; Guerin et al., 2012; Vilberg and Rugg, 2012) that may persist after behavioral responses are made (for review, see Mitchell and Johnson, 2009).

It is also of note that we observed functional heterogeneity within PFC and LPC. In PFC, for example, goal modulation was fully absent in IFG, whereas in LPC, modulation was absent in TPJ. Interestingly, in the context of visual attention, IFG and TPJ are thought to, collectively, comprise a ventral frontoparietal network that is involved in bottom-up reorienting of attention, whereas a dorsal frontoparietal network is thought to support goal-directed attention (Corbetta and Shulman, 2002). It has been argued that this dorsal/ventral attentional framework can also explain dorsal/ventral LPC dissociations that have been observed in memory retrieval (Cabeza et al., 2008). Although targeted investigations have indicated that the neural mechanisms of visual attention and memory retrieval are not isomorphic (Hutchinson et al., 2009, 2013), a general dorsal/ventral framework does appear consistent with the present findings. Specifically, when considered collectively, IFG and TPJ (the ventral network) failed to show any evidence of goal-modulation (p = 0.66), whereas SFG and SPL (the dorsal network) displayed robust goal-modulation (p = 0.009). Although potential dissociations within PFC and LPC is a topic that will require further investigation, the present results are consistent with the view that frontoparietal regions jointly support selective reflective attention to behaviorally relevant reactivated content (Chun and Johnson, 2011).

Reactivation and subsequent memory

Our behavioral results provide clear evidence that the initial memory test, for which no feedback was provided, powerfully benefitted future memory for both target and incidental features, relative to a no-retrieval baseline condition (Chan et al., 2006). Moreover, target reactivation in frontoparietal cortex predicted the observed benefit for target features, indicating that the biased reactivation observed in frontoparietal cortex was related to strengthening of goal-relevant features. Somewhat surprisingly, target reactivation in VMTL did not predict target strengthening. Although not anticipated, one account of this null result is that frontoparietal target reactivation, which persisted longer than VMTL target reactivation (Fig. 2C), “masked” a potential relationship between VMTL target reactivation and subsequent target memory. Notably, incidental reactivation in VMTL did predict subsequent memory for the nontested (incidental) feature. Thus, VMTL category reactivation on Location trials may have predicted subsequent category memory because frontoparietal regions did not continue processing category information. In any case, these findings clearly indicate a behavioral significance of both frontoparietal target reactivation and VMTL incidental reactivation. More broadly, these findings are consistent with the argument that reactivation plays a causal role in memory strengthening (Johnson, 1992; Rasch and Born, 2007; Rudoy et al., 2009; Dupret et al., 2010; Tambini et al., 2010), indicating that when a memory test occurs, the benefit derived from that test is related to the degree of reactivation (target or incidental) that is elicited.

Conclusions

In summary, the present results indicate that although memory reactivation is broadly distributed across temporal, parietal, and prefrontal regions, retrieval goals differentially influence the selectivity of reactivation across these regions. Whereas, frontoparietal regions preferentially represent mnemonic information relevant to current behavioral goals, reactivation may be less selective in ventral/medial temporal lobe regions. Finally, reactivation across these regions not only supports remembering in the present, but also predicts memory in the future.