Cortical and subcortical contributions to sequence retrieval: schematic coding of temporal context in the neocortical recollection network

Abstract

Episodic memory entails the ability to remember what happened when. Although the available evidence indicates that the hippocampus plays a role in structuring serial order information during retrieval of event sequences, information processed in the hippocampus must be conveyed to other cortical and subcortical areas in order to guide behavior. However, the extent to which other brain regions contribute to the temporal organization of episodic memory remains unclear. Here, we examined multivoxel activity pattern changes during retrieval of learned and random object sequences, focusing on a neocortical “core recollection network” that includes the medial prefrontal cortex, retrosplenial cortex, and angular gyrus, as well as on striatal areas including the caudate nucleus and putamen that have been implicated in processing of sequence information. The results demonstrate that regions of the core recollection network carry information about temporal positions within object sequences, irrespective of object information. This schematic coding of temporal information is in contrast to the putamen, which carried information specific to objects in learned sequences, and the caudate, which carried information about objects, irrespective of sequence context. Our results suggest a role for the cortical recollection network in the representation of temporal structure of events during episodic retrieval, and highlight the possible mechanisms by which the striatal areas may contribute to this process. More broadly, the results indicate that temporal sequence retrieval is a useful paradigm for dissecting the contributions of specific brain regions to episodic memory.

INTRODUCTION

The ability to temporally organize sequences of events that occur within a given context plays a central role in episodic memory (Tulving, 1972; 1984; Eichenbaum, 2013; Polyn and Kahana, 2008). Tulving's original definition emphasized the importance of temporal context, when he suggested that, “Episodic memory receives and stores information about temporally dated episodes or events, and temporal-spatial relations among these events (Tulving, 1972, p.385).” Despite the consensus that temporal organization is a defining feature of episodic memory, the neural mechanisms that underlie this ability remain largely unexplored.

Recent attempts to examine temporal organization of episodic memory in the human brain have focused on the role of the hippocampus. Indeed, hippocampal activity is increased during successful encoding (Jenkins and Ranganath, 2010; Tubridy and Davachi, 2011) and retrieval (Ekstrom and Bookheimer, 2007; Lehn et al., 2009) of temporal order information between studied items. Furthermore, recent functional magnetic resonance imaging (fMRI) studies have shown that multivoxel activity patterns in the hippocampus carry information about temporal sequences of letters (Kalm, Davis, & Norris, 2013) and about the positions of items in object sequences (Hsieh et al., 2014). In contrast to the hippocampus, activity patterns in the perirhinal cortex and parahippocampal cortex seem to encode information about objects and serial positions within a sequence, respectively (Hsieh et al., 2014).

Although the available evidence converges on hippocampal encoding of temporal sequences, information carried by the hippocampus during memory retrieval should be conveyed to other brain areas in order to guide behavior (cf. Eichenbaum et al., 2007; Ranganath and Ritchey, 2012). Anatomical studies in animals (Wyss and Van Groen, 1992; Kondo et al., 2005; Aggleton, 2012) and functional and structural connectivity studies in humans (Libby et al., 2012; Vincent et al., 2006; Kahn et al., 2008; Greicius et al., 2009; Uddin et al., 2010) have indicated that the hippocampus closely interacts with a small set of brain regions, including the medial prefrontal cortex (PFC), retrosplenial cortex (RSC), and angular gyrus (ANG). Interestingly, this network of cortical areas is consistently co-active with the hippocampus during recollection-based episodic retrieval that entails conscious reinstatement of contextual details of a past event (Spaniol et al., 2009; Kim, 2010; Rugg and Vilberg, 2013). Co-activation of these areas during episodic retrieval might indicate that these cortical areas work in concert to support the recovery of the temporal and spatial context of a past event (King et al., 2015; Schedlbauer et al., 2014). Put another way, neocortical recollection network regions might structure temporal as well as spatial processing in episodic memory. Indeed, studies have shown that regions constituting the neocortical recollection network are collectively engaged in spatial navigation in virtual environments (Burgess et al., 2001; Hartley et al., 2003; Spreng et al., 2009), despite likely functional differences between the medial PFC (Euston et al., 2012), RSC (Vann et al., 2009) and ANG (Vilberg and Rugg, 2008). It is possible that the same network of brain regions also contributes to the coding of temporal information in episodic memory (cf. Eichenbaum 2014; Ritchey, Libby, and Ranganath, in press), but this possibility remains largely unexplored. The present study thus focused on examining the role of neocortical recollection network regions in processing of temporal sequences.

In addition to neocortical areas, subcortical areas also contribute to memory for temporal sequences. Specifically, the caudate nucleus and putamen have been implicated in sequence learning (Rauch et al., 1995; Grafton et al., 1995; Packard and Knowlton, 2002; Schendan et al., 2003; Fletcher et al., 2005; Kumaran and Maguire, 2006), and dysfunction of these striatal regions is detrimental to the acquisition of sequence knowledge (Knopman and Nissen, 1991; Vakil et al., 2000; Brown et al., 2003; Deroost et al., 2006; Muslimovic et al., 2007; Wilkinson et al., 2009). Although most studies examining the role of the striatum have focused on learning of visuomotor sequences, it has been argued that striatal functions are not exclusively motoric (Cohen et al., 1990; Vakil et al., 2000; Willingham et al., 2000), and that they are important for memory behavior (for review, see Pachard and Knowlton, 2002; Foerde and Shohamy, 2011; Liljeholm and O'Doherty, 2012). To the extent that the neural mechanisms supporting motor sequence learning might also facilitate memory retrieval for object sequences, we examined the role of striatal areas in memory for sequential information. Specifically, based on neuropsychological evidence that damage to the putamen (e.g., Parkinson's disease) impairs the perception of temporal intervals between sequential events (Artieda et al., 1992; Gibbon et al., 1997; Malapani et al., 1998; Riesen and Schnider, 2001; Nobre and O'Reilly, 2004; Lewis and Miall, 2006; Meck et al., 2008), we hypothesize that the putamen would exhibit pattern similarity effects that are consistent with coding of timing-related information. With respect to the caudate nucleus, we do not have a strong prediction. However, its involvement in motor sequence learning (Packard and Knowleton, 2002; Schendan et al., 2003) suggests that it may also contribute to sequence retrieval. The present study aimed to examine these hypotheses.

To characterize the roles of neocortical and striatal regions in temporal sequence retrieval, we reanalyzed a recent fMRI dataset from our lab (Hsieh et al., 2014). In this study, participants learned five constant and one “Random” object sequences, each of which consisted of five distinct objects (see Figure 1A). For the constant sequences, the order of the five objects was always fixed. The “Random” sequence, in contrast, consisted of the same five objects but the order of the objects was randomly varied across sequence repetitions. Thus, participants became highly familiar with each object in the “Random” sequence, but they could not consistently associate an object with any serial position. Immediately after the learning phase, participants were scanned during exposure to multiple repetitions of these sequences (see Figure 1B). To examine the extent to which a particular brain region represents serial position or item information during retrieval of object sequences, we adopted a multivoxel pattern analysis strategy, on the basis of the idea that that the relative activation pattern among voxels in a given region is informative with regard to the type of information represented by that brain region (Kriegeskorte et al., 2008). Accordingly, if a region represents a particular type of information, we expect to see higher similarity in multivoxel activity patterns between pairs of trials that share this information. As demonstrated in our previous study (Hsieh et al, 2014), the current design allows us to separately examine the contribution of object, temporal position, or object-in-position in activity patterns in the neocortical recollection network and striatal sequence processing areas.

Figure 1.

Illustration of object sequences, sequence retrieval, and pattern analysis. A. The six types of objects sequences used in the current study. Each sequence consisted of five distinct objects organized in a specific order, except for the “Random” sequence in which the temporal order between the five distinct objects was randomly determined on each repetition. As a result, participants could not associate an object with a specific serial position in the “Random” sequence. B. Schematic of object sequence retrieval. Each object sequence was repeated three times in an fMRI run (five runs in total). The order in which object sequences appeared was pseudo-randomized such that there was no back-toback repetition of an object sequence and all six object sequences must have been presented before subsequent repetitions. The green brackets below the object sequence are to highlight repetitions of an object sequence in an fMRI run. In the real experiment, participants simply saw a continuous stream of objects without explicit cues to mark divisions between object sequences. Inter-stimulus-intervals (ISIs) were kept constant across trials (i.e., 5 s) throughout retrieval of object sequences. The matrix shown above each trial is to depict hypothetical multivoxel activation pattern associated with an object trial within an anatomically defined ROI. C. Pattern analyses associated with learned and “Random” sequences. Pattern similarity was computed by correlating multivoxel activity patterns between every possible pair of trials between repetitions of an object sequence (see also Methods for details), and could be summarized by a 5 by 5 similarity matrix. Similarity matrix for “All learned sequences” was computed by averaging together five 5 by 5 similarity matrices, each of which was associated with a learned, constant object sequence (i.e., “Fixed”, “X1”, “X2”, “Y1”, and “Y2”). Diagonal elements of the similarity matrix for “All learned sequences” reflected pattern similarity across repetitions of the same objects in the same serial position (i.e., “same obj.+pos.”). Offdiagonal elements reflected pattern similarity between trials that didn't share the same object information and were one position (i.e., “lag 1”) or two or more than two positions apart (i.e., “lag 2+”). Similar analysis procedures were applied to compute pattern similarity across repetitions of different learned sequences (excluding X1, X2 and Y1, Y2 similarity matrices). However, because each learned sequence consisted of a distinct set of objects, diagonal elements only reflected pattern similarity between trial pairs that shared the same position information (i.e., “same pos.”). Coding of position information could also be accessed by correlating between repetitions of the “Random” sequence (“Random” sequence (i.e., “Random (based on position)”). Because each serial position was not associated with a specific object in the “Random” sequence, similar to the analyses across learned sequences, diagonal elements only reflected pattern similarity between trial pairs that share the same position information (i.e., “same pos.”). Data associated with the “Random” sequence could also be rearranged, such that diagonal elements of the similarity matrix were computed from repetitions of the same objects but occupied different serial positions (i.e., “same obj.” in “Random (based on object)”).

METHODS

Most of the methods reported here have been previously published in Hsieh et al. (2014), and are reprinted here for readers’ convenience. However, the analyses and results reported in this paper are novel and original, and have not been published elsewhere.

Participants

Twenty healthy individuals (11 females) from the student community of the University of California at Davis were recruited in this study. All participants had normal or correct-to-normal vision. Functional MRI data from two subjects (both females) were excluded due to problems with image acquisition set-up; however, behavioral data from these two participants were still included for analysis. Due to errors in response acquisition device, one participant's (female) behavioral responses were not recorded. Therefore, the fMRI and behavioral results reported here are based on data from 18 and 19 participants, respectively. The study was approved by the Institutional Review Board of the University of California at Davis, and written informed consent was obtained from each subject before the experiment.

Task Procedures

The study consisted of a sequence learning session and a sequence retrieval session. Functional MRI data were only acquired during the sequence retrieval session. During the sequence learning session, participants learned five object sequences. Each object sequence consisted of five distinct visual objects, and the order of the objects was always constant. The “Fixed” sequence consisted of five objects that were not used in any of the other sequences, whereas the remaining “overlapping” sequences had objects in common (see Figure 1A). Specifically, the second and the third objects were identical in two “X” (i.e., “X1” and “X2”) sequences such that they partially overlapped with one another. Two “Y” (i.e., “Y1” and “Y2”) sequences were designed such that they shared the same first three objects. For comparison purposes, we also included a “Random” sequence, which always consisted of the same five objects, but the order of the five objects was always random. Therefore, participants could not form a consistent temporal sequence representation between items in the “Random” sequence. Accordingly, comparisons between the fixed and random sequences allowed us to assess memory for temporal order information, while controlling for overall object familiarity.

The sequence learning session consisted of multiple study-test cycles that were repeated until the participant learned the five constant object sequences (i.e., the “Fixed”, “X1”, “X2”, “Y1” and “Y2” sequences) to criterion (see below). During each study phase, each sequence was repeated to participants three times before proceeding to the next sequence. The order in which object sequences were studied was randomized across study blocks. Objects in each sequence were presented on the screen for 1000 ms each with a 1500 ms inter-stimulus fixation. Objects within each sequence were always presented in the same order, except for the “Random” sequence, in which the five objects were always presented in a different randomized order on each repetition. To constrain learning strategies and also keep participants actively engaged, they were also required to make a semantic judgment on each presented object. A semantic question was provided at the beginning of each study phase (e.g., “Is the presented object living?”), and each study phase was associated with a different semantic question. During the test phase of a study-test cycle, each object sequence was tested three times, with the constraint that a sequence was not tested consecutively and all six sequences must have been tested before the second and the third tests. On each test trial, five objects from a specific sequence were presented simultaneously on the screen, and participants had to reconstruct the temporal order in which these five objects appeared during study. For the “Random” sequence, participants were instructed to make up their responses. After participants made their responses for a test trial, the correct order of the object sequence was presented. The study-test cycles continued until the participant was able to reconstruct the order of objects in each of the five constant object sequences in three consecutive tests. On average, participants reached this criterion after 5 (SD: 1.57) study-test cycles.

Immediately after learning the object sequences, participants completed an MRI scan session (i.e., the sequence retrieval session). FMRI data were collected over 5 consecutive scanning runs. During each scanning run, participants made semantic decisions on a continuous stream of objects. Each object stream consisted of contiguous presentations of the 5 learned sequences and one “Random” sequence, such that there were no obvious boundaries between distinct object sequences (see Figure 1B). Each object was presented at the center of the screen for one second, followed by a five-second inter-stimulus fixation. The inter-stimulus-intervals (ISI) between objects that were within a temporal sequence were identical to ISIs between objects that belonged to two nearby, different temporal sequences. Within each fMRI scanning run, each sequence was repeated three times with the constraint that there was no back-to-back repetition of a specific sequence, and that all six sequences must have been presented before the second and the third repetitions. For each repetition of the “Random” sequence, the temporal order of the five objects was randomly varied such that each repetition had a unique temporal order between the five objects that was not repeated in the entire experiment. To keep participants actively engaged throughout the experiment, a different, unique semantic task was used for each functional run, and these tasks were distinct from the semantic tasks used during the sequence learning session.

fMRI Data Acquisition and Preprocessing

Whole-brain imaging was conducted at the Center for Neuroscience of the University of California at Davis on a 3T Skyra (Simens, Erlangen, Germany) MRI system with 32 channel head-coil. T1-weighted structural images were acquired with magnetization-prepared rapid acquisition gradient echo (MPRAGE) pulse sequence (208 slices; voxel size = 1×1×1 mm; TR = 1800 ms; TE = 2.96 ms; flip angle = 7°; FoV = 256 mm). Functional images were collected with gradient echo planar imaging (EPI) sequence (280 time points; voxel size = 3.2×3.2×3.0 mm; TR = 2000 ms; TE = 25 ms; FoV = 205 mm; 34 slices, ascending¹). Experimental stimuli were presented on a custom-made computer screen positioned in the back of the scanner, which could be viewed by participants via a mirror mounted on the head-coil. Collected brain images were analyzed using FMRI Expert Analysis Tool in the FMRIB Software Library (FSL version 5.0.2.1; www.fmrib.ox.ac.uk/fsl). Brain volumes were extracted using Brain Extraction Tool (BET) to remove non-brain tissues and skull. Functional images were slice-time corrected using sinc interpolation to account for differences in slice acquisition times. Image signal was high-pass filtered with the cut-off of 0.01 Hz. A rigid-body motion correction was performed with normalized correlation cost function (using MCFLIRT). Functional images were then coregistered (with FLIRT) to the participant's MPRAGE image via a rigid-body transformation, which generated a transformation matrix that was used to affine transform anatomically-defined region-of-interests (ROIs) back to each individual participant's native-space fMRI data (see also fMRI Pattern Analysis for details).

fMRI Pattern Analysis

Analyses of fMRI data were performed by assessing patterns of activity across voxels within anatomically defined ROIs evoked during single trials. Parameter estimates (beta weights) indexing the magnitude of activity evoked during each stimulus event (i.e., each presented object) within individual voxels were estimated with the Least-Square2 (LS2) method as described in Turner et al. (2012). Parameter estimates associated with each presented object were computed by setting up a general linear model (GLM) that was dedicated to estimate the beta weights associated with that object. Each fMRI run was associated with 90 (5 objects/sequence × 6 temporal sequences × 3 repetitions) GLMs, with each GLM aiming to extract the beta weights associated with a specific stimulus event. The resulting 450 beta maps (90 beta maps/run × 5 runs) subsequently underwent an outlier exclusion procedure in which beta maps whose signal intensity within individual ROIs lied in the extreme 1% of all 450 beta maps were excluded from further analysis.

ROI identification

Cortical brain regions constituting the recollection network (i.e., medial prefrontal cortex, retrosplenial cortex, and angular gyrus) were identified using an automated parcellation procedure implemented in Freesurfer (http://surfer.nmr.mgh.harvard.edu/), which defines regions based on individual participant's gyral and sulcal anatomy as revealed in the MPRAGE structural image. Cortical parcellation was based on the Destrieux cortical atlas (Fischl et al., 2004; Destrieux et al., 2010). Six cortical parcellation labels (three in each hemisphere) were selected to best capture medial prefrontal cortex (PFC), retrosplenial cortex (RSC) and angular gyrus (ANG) — key brain regions of neocortical recollection network. Bilateral striatal ROIs (i.e., caudate nucleus and putamen) were obtained using Freesurfer's subcortical automatic segmentation algorithm (Fischl et al., 2002; Fischl et al., 2004; Han and Fischl, 2007). Individual participant's cortical and striatal ROIs were then binarized and aligned with the corresponding native-space functional data by applying the affine transformation parameters obtained in the coregistration preprocessing step. Table 1 shows the averaged number of voxels included in pattern analysis for individual cortical and subcortical ROIs.

Table 1.

The averaged number of voxels included for pattern analyses in each cortical and subcortical ROI.

Cortical ROIs	Number of voxels (SD)
Medial PFC
R.	282 (36)
L.	233 (37)
RSC
R.	35 (9)
L.	37 (11)
ANG
R.	387 (49)
L.	310 (39)
Striatal ROIs
Caudate
R.	149 (22)
L.	157 (18)
Putamen
R.	209 (26)
L.	219 (26)

Pattern similarity measure

Pearson's correlation coefficient was used to quantify similarity between activation patterns evoked during different trials. Pearson's r was chosen because it estimates voxel pattern similarity between pairs of trials irrespective of overall activation magnitude. The beta weights associated with each trial for each voxel in the ROI were extracted and arranged into a column vector. Pattern similarity between presented objects was estimated by computing the correlation coefficient between vectors of beta weights across pairs of trials. The resulting correlation coefficient was then Fisher transformed and averaged within particular bins prior to conducting statistical tests.

Pattern analyses associated with learned sequences

Activation pattern similarity across serial positions within each learned sequence (i.e., “Fixed”, “X1”, “X2”, “Y1”, and “Y2”) was quantified by correlating between repetitions of each learned temporal sequence, which yielded three 5 by 5 similarity matrices for each temporal sequence (i.e., a total of three possible correlation combinations between three repetitions of a temporal sequence, see also Hsieh et al., 2014). The three 5 by 5 similarity matrices were then averaged together to yield a single 5 by 5 similarity matrix for each learned temporal sequence. The resulting five similarity matrices (each associates with a learned temporal sequence) were further averaged together to yield a single 5 by 5 similarity matrix that represents activation pattern similarity associated with all learned sequences (see the similarity matrix for “All learned sequences” in Figure 1C). Diagonal elements of the resulting 5 by 5 similarity matrix reflect pattern similarity between pairs of trials that share the same object and position information (i.e., “same obj.+pos.”). Immediate off-diagonal elements (i.e., “lag 1”), in turn, reflect pattern similarity between pairs of trials that are one position apart and have different object information. The remaining off-diagonal elements (i.e., “lag 2+” elements) also have different object information, and reflect pattern similarity between trial pairs that are two or more than two positions apart. Elements corresponding to different types of trial pairs (i.e., “same obj.+pos.”, “lag 1”, and “lag 2+”) were averaged together to characterize pattern similarity changes in different conditions.

We also quantified activation pattern changes across serial positions by correlating between repetitions of different learned sequences. In this analysis, each of the learned sequences was correlated with all other learned sequences (e.g., “Fixed” sequence was correlated with “X1”, “X2”, “Y1”, and “Y2” sequences), with the exception that X1, X2 and Y1, Y2 sequences were not correlated with each other. The resulting 5 by 5 similarity matrices were then averaged together to yield a single 5 by 5 similarity matrix. Given that each of the learned sequences consisted of a distinct set of five objects that did not overlap with other sequences (except for the excluded X1, X2 and Y1, Y2 similarity matrices), diagonal elements of the resulting 5 by 5 averaged similarity matrix would reflect pattern similarity between trial pairs that shared the same position but different object information (i.e., “same pos.”, see also similarity matrix for “Between learned sequences” in Figure 1C). This is different from the above analysis where object and serial position information was not separable (i.e., trial pairs that shared the same object information always appeared in the same serial position). All off-diagonal elements (i.e., “lag1” and “lag2+”) of the similarity matrix have different object information in common. Immediate off-diagonal (i.e., “lag 1”) elements reflect pattern similarity between pairs of trials that are one position apart. The remaining off-diagonal elements (i.e., “lag 2+”), in turn, reflect pattern similarity between trial pairs that are two or more than two positions apart.

Pattern analyses associated with the “Random” sequence

Several pattern similarity analyses were conducted on trials associated with the “Random” sequence. The manipulation that objects constituting the “Random” sequence were presented in a different randomized order on each repetition enabled us to separately quantify activity patterns across pairs of trials that shared either serial position or object information alone. Pattern similarity associated with serial position information was computed by correlating between repetitions of the “Random” sequence. Because each serial position was occupied by distinct objects on different repetitions of the “Random” sequence, similar to the correlations between repetitions of different learned sequences, diagonal elements of the resulting 5 by 5 similarity matrix would reflect pattern similarity between trial pairs that only shared the same position information (i.e., “same pos.”). Immediate off-diagonal (i.e., “lag 1”) and the remaining off-diagonal elements (i.e., “lag 2+”) would reflect pattern similarity between trial pairs that were one position and two or more than two positions apart, respectively, and had different object information (see the similarity matrix for “Random (based on position)” in Figure 1C). To quantify pattern similarity associated with object information, we reorganized the data in each repetition of the “Random” sequence such that the diagonal elements of the resulting 5 by 5 similarity matrix reflected pattern similarity between pairs of trials that share the same object information (i.e., “same obj.”), and all of the off-diagonal elements were associated with pattern similarity between different object pairs (see similarity matrix for “Random (based on object)” in Figure 1C). However, there were a few instances in which the same object happened to occupy the same serial position between repetitions of the “Random” sequence (e.g., “camel” presented at the first position in the first and the second repetition of the “Random” sequence). In these rare cases, object information would contaminate position-related pattern similarity results and vice versa. To prevent these rare instances from adding noise to position- or object-related pattern similarity effects associated with the “Random” sequence, we excluded those correlation coefficients from further analysis.

Statistical tests

Permutation tests were used to control for overall family-wise Type-I error rates across multiple ROIs in individual pattern similarity contrasts. This approach is parallel to controlling for multiple comparisons across brain voxels (which are analogous to ROIs in the current study) for a specific contrast in conventional voxel-based fMRI analyses (Nichols and Holmes, 2001). Separate permutation procedures were conducted to separately control for the family-wise error rate across bilateral cortical and bilateral subcortical ROIs, respectively. For instance, to determine the significance of a specific contrast (e.g., “same pos.” vs. “”lag2+) in the core recollection network ROIs, we used the following permutation procedures: (1) Compute the original t-statistics associated with a pattern similarity contrast in individual recollection network ROIs (six ROIs in total; three in each of the hemispheres); (2) Randomly shuffle condition labels and re-compute t-statistics in individual ROIs; (3) Extract the maximum of the t-statistics associated with individual ROIs in Step 2; (3) Repeat Steps 2 and 3 two thousand times and use the obtained maxima to create a null distribution of the maximal t-values; (4) Assess each of the original t-statistics obtained in Step 1 (each associated with an ROI) relative to the null distribution and determine the probability of obtaining a t-statistic that is equal or larger than the original t-value under the permutation (null) distribution; (5) If the probability (i.e., the reported p-values) is less than 0.05, then the contrast is considered significant with a family-wise error rate (i.e., across recollection network ROIs) less than 0.05. Each pattern similarity contrast underwent the same procedures to determine its significance level in individual recollection network ROIs. Similar permutation procedures were also applied to striatal ROIs to control for family-wise rate across these ROIs for individual pattern similarity contrasts.

It should be noted that the statistical tests described above used one-tailed thresholds, because all pattern similarity analyses were performed to test directional hypotheses. For instance, to test whether a region shows position coding, one needs to demonstrate that pattern similarity is higher for trials that share the same temporal position (i.e., “same pos.”) than for trials that do not (e.g., “lag2+”). Hypothesis tests were directional because in each test, we would expect higher pattern similarity for pairs of trials that share a particular attribute than for pairs of trials that do not. In this experiment, if we were to observe higher pattern similarity between trials that do not share a common attribute than for trials that do share a common attribute (e.g., higher pattern similarity for trials that do not share object or position information than for trials that share position information), the result would be difficult, if not impossible, to interpret. Thus, in all cases, the statistical analyses were aimed at testing directional hypotheses.

RESULTS

Behavioral performance during sequence retrieval

Behavioral results reported here focus on characterizing response differences between all learned sequences (i.e., collapsed across “Fixed, “X1”, “X2”, “Y1”, and “Y2”) vs. the “Random” sequence during object sequence retrieval in the scan session. Detailed behavioral analyses for individual object sequence types can be found in Hsieh et al. (2014).

We directly compared accuracies and reaction times (RTs) on semantic judgments for objects in all learned sequences vs. objects in the “Random” sequence, collapsing across all serial positions. We expected that, if participants utilized learned sequence knowledge to facilitate semantic judgments during sequence retrieval, accuracy and particularly RTs for semantic judgments should be facilitated for all learned relative to “Random” sequences. Indeed, participants were significantly more accurate (t(18) = 2.334, p < 0.05) and faster (t(18) = 5.999, p < 0.001) in making semantic judgments for objects in the learned sequences (RT (ms): Mean, 571; SEM, 45; Accuracy: Mean, 0.93; SEM, 0.01) than for objects in the “Random” sequence (RT (ms): Mean, 707; SEM, 49; Accuracy: Mean, 0.90; SEM, 0.01). Breaking down RTs into individual serial positions showed that semantic judgments for the first position objects were significantly slower than objects in other serial positions in both the learned (815 vs. 514 ms) and “Random” (807 vs. 685 ms) sequences (All learned sequences: F(1,18) = 43.981, p < 0.001; “Random” sequence: F(1,18) = 45.170, p < 0.001). This may be associated with the fact that, during sequence retrieval, the order of sequence types was randomly shuffled and participants could not anticipate what object sequence would appear next. Closer examination of the RT profiles revealed that RTs gradually declined at successive serial positions in the “Random” sequence, whereas for the learned sequences there was an abrupt reduction in RTs for objects following the first serial positions. The qualitatively different RT profiles were confirmed by a significant sequence type (i.e., All learned vs. “Random”) by serial position (i.e., five serial positions) interaction (F_{(2.688,48.390)} = 18.114, p < 0.001). The gradual reduction in RTs across serial positions in the “Random” sequence suggested that even though object order was randomly shuffled, participants still learned the temporal structure of object sequence (i.e., tracking the serial positions) and were able to use it to anticipate possible upcoming objects as the “Random” sequence was unfolded (e.g., by the end of the sequence, they could know what object would appear). With respect to the abrupt RT transition for all learned sequences, this suggests that participants utilized their learned sequence knowledge to guide semantic judgments after seeing the first object in a learned sequence.

Multivoxel activity patterns in the neocortical recollection network and striatal regions are sensitive to sequence retrieval

To investigate whether activity patterns in the neocortical recollection network and striatal regions carry information about temporal sequences, we first focused on quantifying pattern similarity across repetitions of learned sequences in individual ROIs, separately for the right and the left hemispheres. If multivoxel activity patterns in these regions are sensitive to sequence retrieval, then we would expect that activation patterns would change as a function of the information overlap between trials. Specifically, pairs of trials that share the same object and position information should be more similar to each other than pairs of trials that have different object and position information. Moreover, object pairs that are two or more than two positions apart should show further reduction in pattern similarity as compared to pairs of trials that are one position apart. As shown in Figure 2 (bar graphs for “All learned sequences”), all of the cortical recollection ROIs elicited significantly higher similarity values for “same obj.+pos.” than for “lag 1” (for all left and right cortical ROIs, t(19) > 3.662, p < 0.005) or “lag 2+” pairs of trials (for all left and right cortical ROIs, t(19) > 4.608, p < 0.001). Furthermore, “lag 1” pairs of trials were associated with higher pattern similarity than “lag 2+” pairs of trials (for all left and right cortical ROIs, t(19) > 2.641, p < 0.05). These results suggest that cortical brain regions constituting the recollection network are sensitive to coding of either object or serial position information.

Figure 2.

Multivoxel pattern similarity across repetitions of object sequences in the cortical recollection network. A. Pattern similarity metrics computed based on data from the right (top row) and the left (bottom row) medial PFC. Pattern similarity estimates associated with individual conditions (mean, +− 1 SEM) were computed by averaging together similarity values that fell into the same conditions (see also Figure 1C). B and C are organized in the same way as in A, but show data from the retrosplenial cortex and angular gyrus, respectively.

The same analyses were also performed on data from the striatal regions (including the caudate nucleus and putamen) that have been reported to be associated with sequence learning. Similar to the results for cortical ROIs, “same obj.+pos.” trial pairs were associated with higher pattern similarity than “lag 1” (for all left and right subcortical ROIs, t(19) > 3.662, p < 0.01) or “lag 2+” pairs of trials (for all left and right subcortical ROIs, t(19) > 4.206, p < 0.001; see bar graphs for “All learned sequences” in Figure 3). In contrast to the results for cortical ROIs, “lag 1” trial pairs elicited higher pattern similarity than “lag 2+” trial pairs only in the putamen (for both the left and right ROIs, t(19) > 2.785, p < 0.05), but not in the caudate nucleus (for both the left and right ROIs, t(19) < 0.673, p > 0.67). Overall, the activity patterns in the striatal areas are also sensitive to coding of object or position information for learned sequences.

Figure 3.

Multivoxel pattern similarity across repetitions of object sequences in subcortical striatal areas. A. Pattern similarity metrics (mean, +− 1 SEM) computed based on data from the right (top row) and the left (bottom row) caudate nucleus. B. Pattern similarity results associated with the putamen.

Temporal position coding in the neocortical recollection network

The above analyses indicate that activity patterns in the neocortical recollection network and striatal sequence processing regions carry information about objects in learned sequences. However, it remains unclear what type of information contributed to the differences in pattern similarity between “same obj.+pos.” and “lag 1” or “lag 2+” pairs of trials in these ROIs. The fact the pattern similarity was quantified by correlating between repetitions of learned sequences suggest that the increased pattern similarity associated with “same obj.+pos.” could be due to the overlap of object (e.g., “banana”), position (i.e., the first object in the sequence), or object-in-position (i.e., “banana” at the first position) information. To further tease apart the contribution of position information to the lag-dependent pattern similarity effects reported above, we conducted pattern similarity analyses on trial pairs that quantified pattern similarity between repetitions of different learned sequences, as well as on trial pairs from the “Random” sequence.

Quantifying pattern similarity across repetitions of different learned sequences allowed us to examine pattern similarity effects that were solely driven by position information without being contaminated by object information, as each learned sequence consisted of a distinct set of objects that did not overlap with other learned sequences (except for the excluded X1-X2 and Y1-Y2 correlations; see also Methods for details). We performed this analysis on data from individual cortical and striatal ROIs. The results showed that object pairs that share the same position information (“same pos.”) were associated with higher pattern similarity than object pairs that were two or more than two positions apart (“lag 2+”) in all neocortical recollection ROIs (all t(19) > 3.649, p < 0.005), except for the left RSC (t(19) < 1.973, p > 0.17). In contrast, none of the striatal ROIs showed evidence of position coding (for all left ROIs, t(19) < 2.255, p > 0.06; for all right ROIs, t(19) < 1. 417, p > 0.30; see also summary statistics for “Between learned sequences” in Table 2 and similarity metrics associated with “Between learned sequences” in Figures 2 and 3).

Table 2.

Summary of statistical results for pattern analyses on learned and “Random” sequences in individual cortical and subcortical ROIs.

Cortical ROIs	Contrasts
	Between learned sequences	Random sequence		All learned sequences vs. Random sequence
	“same pos.” vs. “lag 2+”	“same pos.” vs. “lag 2+”	“same obj.” vs. “different obj.”	“same obj.+pos.” vs. “same pos.”	“same obj.+pos.” vs. “same obj.”
Medial PFC
R.	p < 0.001****	p > 0.335	p > 0.946	p > 0.277	p > 0.060
L.	p < 0.005***	p > 0.064	p > 0.900	p > 0.888	p < 0.035*
RSC
R.	p < 0.005***	p < 0.016*	p > 0.751	p > 0.900	p > 0.600
L.	p > 0.171	p > 0.235	p > 0.652	p > 0.990	p > 0.809
ANG
R.	p < 0.001****	p > 0.148	p < 0.018*	p > 0.986	p > 0.735
L.	p < 0.001****	p > 0.059	p > 0.686	p > 0.995	p > 0.072
Striatal ROIs
Caudate
R.	p > 0.300	p > 0.844	p > 0.500	p > 0.192	p > 0.395
L.	p > 0.209	p > 0.708	p < 0.029*	p > 0.124	p > 0.534
Putamen
R.	p > 0.331	p > 0.900	p > 0.900	p < 0.041*	p < 0.005***
L.	p > 0.066	p > 0.724	p > 0.981	p > 0.203	p < 0.029*

^****p < 0.001

^***p < 0.005

**p < 0.01

^*p < 0.05

Position coding was also assessed by computing pattern similarity across repetitions of the “Random” sequences. The fact that the same serial positions were occupied by different objects on different repetitions of the “Random” sequence ensured that the pattern similarity effects were solely associated with position information (see also Methods for details). However, unlike the above analyses, the “same pos.” vs. “lag 2+” contrast associated with the “Random” sequence might have weakened sensitivity to detect reliable position coding, which might be associated with the fact that there were only three repetitions of the “Random” sequence in each fMRI run, and that some correlation coefficients were excluded due to the overlap of object and position information (see also Methods for details). Indeed, the results showed that “same pos.” object pairs were associated with higher pattern similarity than “lag 2+” pairs only in the right RSC (t(19) > 3.217, p < 0.05), with trends that did not reach significance for all other cortical ROIs (t(19) < 2.450, p > 0.059). For the subcortical ROIs, no significant effect was observed (t(19) < 0.618, p > 0.70; see also summary statistics in Table 2 and similarity metrics associated with “Random (based on position)” in Figures 2 and 3).

The two sets of analyses above provided evidence that the cortical ROIs coded for serial position information, independent of object identity, although the effects were weaker in analyses associated with the “Random” sequence. Nonetheless, the overall pattern of results suggest that core regions of the cortical recollection network carried information about the temporal positions in object sequences, whereas this type of pure position coding was not observed for striatal regions.

Coding of object information

We next examined the extent to which voxel patterns in the cortical recollection network and subcortical sequence processing regions carry information about objects, irrespective of their temporal positions. We computed voxel patterns between trials for which the same object was presented, but occupied different serial positions across repetitions of the “Random” sequence. This allowed us to examine pattern similarity that is solely driven by object information (see Methods for details). Analyses on cortical and striatal ROIs indicated that the right ANG (t(19) > 3.203, p < 0.05) and the left caudate nucleus (t(19) > 2.729, p < 0.05) showed significantly higher pattern similarity for “same obj.” than for “different obj.” pairs (see Figures 2-3 and Table 2), suggesting that activity patterns in these two brain regions are sensitive to object information. The finding of object coding in the right ANG is interesting, given its role in position coding as demonstrated in the above analyses. The significant object coding in the caudate nucleus, on the other hand, seems to suggest that the pattern similarity differences between “same obj.+pos.” and “lag 1” or “lag 2+” for learned sequences (see the above analyses) might be driven by object information.

Coding of objects in temporal position

To examine the extent to which these cortical and striatal ROIs conjunctively represent object and temporal position information, we conducted analyses that directly compared pattern similarity effects associated with learned sequences (i.e., “same obj.+pos.”) vs. pattern similarity effects associated with the “Random” sequence (i.e., “same obj.” and “same pos.”). Specifically, in each ROI, we compared pattern similarity for “same obj.+pos.” trial pairs with pattern similarity for “same obj.” and “same pos.” trial pairs. The idea behind these comparisons is that if activity pattern within a specific ROI represents objects in temporal context, then trial pairs that share the same object and position information (i.e., “same obj.+pos.”) should be associated with higher pattern similarity than trial pairs that only share the same object (i.e., “same obj.”) or position (i.e., “same pos.”) information. Thus, a region must exhibit higher pattern similarity for “same obj.+pos.” than for both “same obj.” and “same pos.” in order to be considered that this region carries information about object-position bindings, over and above information about temporal positions and individual objects. This analysis only revealed a significant result in the right putamen (“same obj.+pos.” > “same pos.”, t(19) > 2.496, p < 0.05; “same obj.+pos.” > “same obj.”, t(19) > 3.529, p < 0.005; see also Figure 3 and Table 2).

DISCUSSION

The present study used fMRI and multivoxel pattern analyses to examine how neocortical and striatal brain regions represent sequences of objects. Specifically, we focused on activation patterns in neocortical areas constituting the recollection network (i.e., the medial PFC, RSC, and ANG) and striatal regions (i.e., the caudate and putamen) implicated in sequence processing. We found that activity patterns in the neocortical recollection network carried information about the serial position of each object in a sequence, regardless of whether or not serial positions are associated with specific objects. These findings strongly parallel results reported for the parahippocampal cortex in our previous paper (Hsieh et al., 2014). By contrast, none of the striatal regions examined showed evidence of position coding as seen in the neocortical recollection network – the putamen carried information specific to objects in learned sequences whereas the caudate nucleus carried information about objects irrespective of the sequence context. Below, we describe these findings in further detail and discuss the insights that they provide into memory retrieval processes.

Temporal position coding in the core recollection network

Episodic retrieval entails recovering information about an event in order to reconstruct the associated temporal-spatial contexts. It has been proposed that, during retrieval, activation of representations in the hippocampus that associate item and context information may lead to activation of separate item- and context-based information in distinct cortical and subcortical areas (Diana et al., 2007; Eichenbaum et al., 2007; Ranganath and Ritchey, 2012). Brain regions constituting the cortical recollection network appear to be particularly involved in the processing of context information, as these regions are consistently activated only when a studied event is retrieved along with its associated context (Henson et al., 1999; Woodruff et al., 2005; Cansino et al., 2002; Wheeler and Buckner, 2004; Yonelinas et al., 2005; Guerin and Miller, 2009; for reviews, see Spaniol et al., 2009; Kim, 2010).

Although it is likely that there are important functional differences between the retrosplenial cortex (Vann et al., 2009), angular gyrus (Vilberg & Rugg, 2008), and medial PFC (Euston et al., 2012), there is reason to think that they collectively contribute to certain cognitive processes (Ranganath & Ritchey, 2012; Ritchey et al., 2014). These regions, along with the parahippocampal cortex, are structurally interconnected via the cingulum bundle (Mufson and Pandya, 1984; Kravitz et al., 2011) and show highly correlated activity fluctuations during the resting state (Kahn et al., 2008; Vincent et al., 2006; Buckner et al., 2008; Greicius et al., 2009; Uddin et al., 2010; Libby et al., 2012; Ritchey et al., 2014). In addition to being co-activated during episodic retrieval tasks, functional connectivity between regions in the core recollection network is increased during successful recollection (King et al., 2015). Beyond recognition tasks, regions in the core recollection network are also reliably co-activated during tasks that require autobiographical memory retrieval (see Svoboda et al., 2006), prospective thinking (e.g., Addis et al., 2007; Schacter et al., 2007), imagination/scene construction (Hassabis and Maguire, 2007, 2009), or spatial navigation in virtual environments (Burgess et al., 2001; Hartley et al., 2003; Spreng et al., 2009). Moreover, direct electrophysiological recordings in humans have shown that activity in this network is associated with theta oscillations (Ekstrom et al., 2008; Foster and Parvizi, 2012; Hsieh and Ranganath, 2014), indicating common neural signatures across regions within the network.

One hypothesis, proposed by Ranganath and Ritchey (2012), is that regions of the core recollection network are recruited during tasks that require the construction of a “situation model”—a schema that specifies the spatial, temporal and causal relationships that apply within a particular context. The results reported here are in accord with the idea that regions of the core recollection network contribute to the representation of temporal schemas. In the present study, participants quickly learned (either implicitly or explicitly) that the stream of objects presented during the scan phase could be broken up into sequences of five objects. Learning of this temporal structure is evident in the behavioral data, which demonstrate that, even in the random sequence, reaction times declined across successive serial positions. This result indicates that participants used their knowledge of the five-object sequence structure to predict upcoming objects, even in the random sequence. Activity patterns in the recollection network also reflected the temporal structure of the paradigm, in that they carried information about the serial position of each object within a five-object sequence, irrespective of its identity. Thus, regions of the recollection network (including the parahippocampal cortex; see Hsieh et al., 2014) appear to schematically encode sequences, conveying information about temporal relationships that generalize across specific object sequences.

Temporal coding in the recollection network might also reflect processes that support episodic simulation (e.g., Schacter et al., 2007). According to episodic simulation hypothesis, the recollection network plays a role in imagining, simulating, and predicting possible future events. It is possible that the recollection network could represent the temporal structure of events in a schematic manner (akin to a “situation model”) that is necessary in order to simulate a sequence of events. The results do not suggest, however, that the network carries highly specific simulations, because in this case we would expect that the network would show sequence-specific patterns (i.e., coding of items in context).

The current results are also pertinent to an emerging literature suggesting parallels between the processing of temporal and spatial information (Manns et al., 2007; MacDonald et al., 2011; Eichenbaum, 2014). As noted above, the cortical recollection network has also been implicated in spatial navigation (Spreng et al., 2008), possibly supporting the construction and maintenance of a coherent mental model of a spatial context (Burgess et al., 2001; Hassabis and Maguire, 2007). The large degree of overlap between neural substrates of spatial navigation reported in previous studies (Ekstrom et al., 2014) and the regions that exhibited temporal coding in the present study suggest that the same core network of brain regions could be attuned to optimally process temporal or spatial regularities of experience. Our finding of schematic coding of temporal information in the core recollection network thus complements accounts that have focused more on the role of the core recollection network in representing spatial context (e.g., Hassabis and Maguire, 2009). By facilitating the construction of a model for the spatial context and temporal sequence of events that occurred during a past event, this network might facilitate the recovery of specific details surrounding that event. The integrated activation of these details (i.e., sensory, motor, and cognitive representations of event-specific information) and the corresponding mental representation of the contextual gist of the event may be associated with the experience of recollection (cf. Mitchell and Johnson, 2009; Diana et al., 2007; Eichenbaum et al., 2007).

An active debate revolving around the functional significance of recollection network is whether it represents “content-general” or “content-specific” information (cf. Johnson and Rugg, 2007; Johnson et al., 2013). The invariant involvement of the recollection network in studies using a wide range of test materials (e.g., Guerin and Miller, 2009; Duarte et al., 2011) and memory paradigms (e.g., Wheeler and Buckner, 2004; Woodruff et al., 2005; Cansino et al., 2002; Hayama et al., 2012) have led to the conclusion that it might represent content-general information (Johnson et al., 2013; for review, see Rugg and Vilberg, 2013). However, a recent study showing that the recollection network's activity is modulated by the amount of recollected information could potentially be interpreted as evidence supporting “content-specific” property of the network (Leiker and Johnson, 2014; see also Vilberg and Rugg, 2007; Vilberg and Rugg, 2009; Guerin and Miller, 2011). Our finding of schematic coding of sequence structure within the recollection network, irrespective of specific object identity (i.e., position coding across learned sequences in which a serial position is not associated with a specific object), could suggest that both of these hypotheses have some merit. That is, the observation of position-related activity pattern changes in the neocortical recollection network (i.e., distinct activity patterns for individual serial positions) is consistent with the idea that the recollection network processes contextual “content” that differentiates classes of events (i.e., different serial positions), but the lack of object-specific coding in some regions (i.e., the medial PFC and RSC) might indicate a lack of item-specific information.

One exception may be the ANG, for which we observed evidence for both coding of serial position information and coding of objects regardless of their serial positions (see Figure 2 and Table 2). The ANG therefore seems to be in a unique position to represent both contextual and item-specific information during memory retrieval (see also Kuhl and Chun, 2014 for evidence of item-related coding in the ANG). This idea is compatible with the proposal that ANG acts as an “episodic buffer” that holds multi-modal features of an event during episodic retrieval (Vilberg and Rugg, 2008). One caveat is that, unlike what we previously observed in the hippocampus (Hsieh et al., 2014), we did not find a disproportionate increase in pattern similarity for repetitions of the same object in the same sequence contexts, relative to pairs of trials that shared either temporal position or object information alone. Although one should be careful not to interpret null results, the overall pattern of results might indicate that the ANG activated information about objects and sequence structure in parallel during retrieval, rather than binding item and context information together as would be predicted by some models (e.g., Shimamura, 2011). An alternate possibility is that the ANG binds item and context information primarily in paradigms that explicitly require retrieval of an integrated episodic memory (e.g., source or Remember-Know recognition tasks; cf. Kuhl & Chun, 2014), and that parallel coding might be more evident during indirect expressions of memory. Future studies can address this question by comparing voxel pattern information during active recollection and during implicit retrieval of item-context bindings.

Striatal coding of object identity

As mentioned in the Introduction, in addition to the cortical recollection network, we were also interested in examining activity patterns in striatal areas that have been implicated in motor sequence learning (Rauch et al., 1995; Grafton et al., 1995; Packard and Knowlton, 2002; Schendan et al., 2003; Fletcher et al., 2005). Although the current study investigated sequence retrieval, rather than sequence learning, as in prior neuropsychological studies, results from prior sequence learning studies are clearly relevant to our results. Interestingly, in contrast to the recollection network, none of the striatal ROIs examined showed heightened pattern similarity for pairs of trials across learned sequences that shared the same serial position. Instead, the putamen showed some evidence of conjunctive coding for object-position pairs. Pattern similarity in the caudate nucleus, in turn, was enhanced for pairs of trials that shared object information.

Studies on patients with Parkinson's disease (for which putamen is affected) have unequivocally pointed to the critical role of putamen in sequence learning (Knopman and Nissen, 1991; Vakil et al., 2000; Brown et al., 2003; Deroost et al., 2006; Muslimovic et al., 2007; Wilkinson et al., 2009). Although most of these studies used variants of visuomotor sequence tasks, sequence learning in these tasks transferred to a different sequence of finger movements, suggesting that learning was not exclusively motoric (Cohen et al., 1990; Vakil et al., 2000; Willingham et al., 2000). Patients with Parkinson's disease also have problems in memory for the temporal order of events (Vriezen and Moscovitch, 1990), and their ability in temporal perception and production is compromised (Artieda et al., 1992; Gibbon et al., 1997; Malapani et al., 1998; Riesen and Schnider, 2001; Nobre and O'Reilly, 2004; Lewis and Miall, 2006; Meck et al., 2008; Lucas et al., 2013). Consistent with the role of the putamen in processing of temporal information, a recent human fMRI study using a classical conditioning paradigm in which a light predicted the time of reward delivery showed that activity in the putamen is particularly sensitive to temporal differences between expected and actual rewards (McClure et al., 2003; see also O'Doherty et al., 2003, den Ouden,et al, 2009, and Sutton, 1988). McClure et al. (2003) showed that when the reward was delivered at an unpredicted time, activity in the putamen was increased at the time upon receiving the unexpected reward. Conversely, at the time when the reward was supposed to be delivered but was not, activity in the putamen was decreased. A subsequent study by de Ouden et al. (2009) also showed evidence along these lines, but found that the putamen activity increased to both unexpectedly presented and unexpected absence stimulus events (de Ouden et al., 2009). Accordingly to these findings, the putamen has been proposed to play a role in supporting the temporal predictability of events (McClure et al., 2003; O'Doherty et al., 2003; den Ouden et al., 2009).

Considering all these results together, the putamen may function in a manner similar to the “core timer” proposed by Meck et al. (2008) that keeps track of the time that has elapsed between a recent event and an anticipated, upcoming event. Damage to the putamen may compromise the precision of the “timer” and, therefore, lead to memory problems for temporal information. Relating the “timer” metaphor to our current results, it is possible that, in the present study, high pattern similarity in the putamen for pairs of trials that shared object and temporal position information reflected the demand to anticipate upcoming specific objects with specific temporal regularity (i.e., an object every 6 seconds). This idea could be tested by randomly varying the time interval between objects in learned sequences. If timing prediction (McClure et al., 2003; O'Doherty et al., 2003; den Ouden et al., 2009), rather than serial position information, is critical for coding in the putamen, we would expect that this manipulation should eliminate evidence for object-in-position coding in the putamen.

If our interpretation is correct, then we would expect that the role of the putamen in sequence retrieval can be distinguished from that of the hippocampus (see Hsieh et al., 2014). We speculate that although the putamen and hippocampus are critical for temporal sequence retrieval, the two regions might contribute in different ways. This distinction could be conceptualized as the differences between the ability to judge temporal intervals or form temporal predictions, for which the striatum is central, and the binding of items to a global sequence context, for which the hippocampus is critical (Eichenbaum, 2014; MacDonald, 2014). If our analysis is correct, then it will be important for future research to clarify when and how the two systems interact to support memory for temporal information and/or the perception of time across different scales (MacDonald, 2014; Barnett et al., 2014; see also Jacob et al., 2013).

As pointed out in the Introduction, we did not have strong predictions about the role of the caudate nucleus in object sequence processing. Nonetheless, we found that the caudate nucleus was sensitive to object identity, irrespective of temporal position. We can only speculate about the significance of this finding, based on what is known about the role of the caudate in spatial processing. Studies using virtual navigation tasks, for instance, have shown that activity in the caudate nucleus is associated with learning of landmark-related information, even when participants did not follow a specific route (Doeller et al., 2008; Doeller and Burgess, 2008). Interestingly, lesions of dorsal striatum in rats impair approach to landmarks (O'Keefe and Nadel, 1978; McDonald and White, 1994; Packard and McGaugh, 1992, 1996). These results are consistent with the idea that the caudate nucleus might be representing information about salient landmarks in order to facilitate memory-guided spatial navigation. It is possible that the caudate nucleus might play a similar role in temporal processing. For instance, in the present study, objects might have been treated as distinct “temporal landmarks”, thereby leading to coding of specific objects in the caudate nucleus. One way to test this idea is to see if object coding gradually emerges in the caudate nucleus (so as objects are more landmark-like) during the course of learning of object sequences (cf. Doeller et al., 2008).

Overall conclusions

In summary, the current study shows that the neocortical recollection network schematically codes information about temporal structure. This finding might relate to a broader role for this network of brain regions in the representation of temporal context information during episodic memory retrieval. The putamen, in contrast, showed evidence for coding of objects in specific learned sequences. The caudate carried information about objects, irrespective of sequence context. The results suggest that temporal sequence retrieval may be a useful paradigm for dissecting the relative contributions of cortical and subcortical circuits to memory and cognition. Future studies can build on these findings by exploring the interaction of temporal, object, spatial, and motor representations in these regions.

Highlights.

• The recollection network and striatal areas contribute to sequence retrieval

• The recollection network represents temporal structure of event sequences

• The striatal areas make distinct contributions to retrieval of sequence information