Cortical Networks Involved in Memory for Temporal Order

Abstract

We examined the neurobiological basis of temporal resetting, an aspect of temporal order memory, using a version of the delayed-match-to-multiple-samples task. While in an fMRI scanner, participants evaluated whether an item was novel, or whether it had appeared before or after a reset event that signified the start of a new block of trials. Participants responded “old” to items that were repeated within the current block, and “new” to both novel items and to items that had last appeared before the reset event (pseudonew items). Medial temporal, prefrontal and occipital regions responded to absolute novelty of the stimulus – they differentiated between novel items and previously seen items, but not between old and pseudonew items. Activation for pseudonew items in the frontopolar and parietal regions, in contrast, was intermediate between old and new items. The posterior cingulate cortex extending to precuneus was the only region that showed complete temporal resetting and its activation reflected whether an item was new or old according to the task instructions regardless of its familiarity. There was also a significant Condition (old/pseudonew)-by-Familiarity (second/third presentations) interaction effect on behavioral and neural measures. For pseudonew items, greater familiarity decreased response accuracy, increased response times, increased anterior cingulate cortex (ACC) activation and increased functional connectivity between the ACC and the left frontal pole. The reverse was observed for old items. Based on these results, we propose a theoretical framework in which temporal resetting relies on an episodic retrieval network that is modulated by cognitive control and conflict resolution.

Introduction

Remembering an event involves knowing not only what happened, but also where and when (Nyberg et al., 1996; for a review, see Tulving, 2002). The latter temporal aspect of episodic memory depends on being able to judge when events occurred relative to meaningful boundaries or markers. This ability underlies routine tasks, causal reasoning, and social behaviors. For example, when preparing for work, people do not ask themselves whether they have ever brushed their teeth or taken their medicine, but whether they have done so that day. Additionally, to infer that a certain food caused an allergic reaction, one must remember that the food was consumed before the illness occurred. Furthermore, it is customary to greet one’s co-workers upon first seeing them each day, but not to greet them repeatedly. As these examples illustrate, the ability to make temporal judgments is an important part of normal life.

Previous lesion (e.g. Corkin, 2002; Downes et al., 2002; Freed & Corkin, 1988; Freed, Corkin, & Cohen, 1987; Hurst & Volpe, 1982) and neuroimaging (DuBrow & Davachi, 2014, 2016; Eichenbaum, 2013) studies of humans have shown that remembering temporal aspects of episodic experiences relies heavily on functioning of the hippocampus. Yet the neurobiological basis of judgments that require determining the temporal position of an event relative to a boundary (Friedman, 1993) remains poorly understood. One of the few studies to explore this issue in humans measured event-related potentials (ERPs) while subjects performed a delayed-match-to-multiple-samples (DMMS) task (Walsh, Paynter, Zhang, & Reder, 2016). In that task, participants responded to pictures of objects presented continuously within blocks of trials. Blocks were separated by a special “reset” screen and participants had to respond “old” to pictures that were repeated within a block, and “new” to both novel pictures and to pictures that had last appeared during an earlier block. The study showed that the timing of images relative to when the reset screen appeared, affected the magnitude of the FN400, a component thought to reflect the absolute familiarity of stimuli (Curran, 2000; Curran &Cleary, 2003; Düzel et al., 1999; Rugg & Yonelinas, 2003; Rugg & Curran, 2007; Tsivilis, Otten, & Rugg, 2001; Yovel & Paller, 2004). Specifically, the FN400 was most negative for new items, least negative for items that were repeated since the last reset screen, and intermediate for items that had last appeared before the reset screen. In other words, the FN400 reflected whether an item had previously occurred, as well as when it had occurred relative to the reset screen.

The ERP methodology, however, lacks the spatial resolution to resolve the neural basis of relative temporal judgment in humans. For example, it is unclear which brain regions contributed to the differences in the FN400 familiarity signal for old items that appeared before and after the reset screen. One potential source of the FN400 is the perirhinal cortex (PRc; Curran, Tepe & Piatt, 2006; but see Rugg & Curran, 2007 for an alternate view). The PRc plays an important role in visual processing and memory and usually decreases in activation when a stimulus is repeated (Mayes, Montaldi, & Migo, 2007; Murray, Bussey, & Saksida, 2007; Suzuki & Naya, 2014). However, functional magnetic resonance imaging (fMRI) studies suggest that the PRc is sensitive to absolute familiarity and insensitive to context in humans (Henson, Cansino, Herron, Robb, & Rugg, 2003; Curran et al., 2006).

The original support to the idea that the PRc might be involved in relative temporal order judgment comes from studies of the DMMS task with primates (Hölscher & Rolls, 2002; Yakovlev Bernacchia, Orlov, Hochstein, & Amit, 2005; Yakovlev, Amit, Romani, & Hochstein, 2008; Yakovlev, Amit, & Hochstein, 2013). For example, a subset of primate PRc neurons showed reduced firing rates when an image repeated within a block, but not when the image repeated between blocks and after the “reset” boundary (Hölscher & Rolls, 2002). The fact that some PRc neurons showed no decrease in activation for the images repeated after the “reset” boundary suggests that, similarly to the FN400 results in humans, this region does not simply detect whether an item has previously occurred (i.e., absolute familiarity), but also when it occurred (i.e., relative temporal judgment). The PRc involvement in temporal order judgments was also shown in rat studies, where targeted deactivation of the PRc disrupted performance in a relative recency task (Hannesson, Vacca, Howland & Philips, 2004). Those authors argued that the rat PRc is involved in the calculation of temporal order through interactions with the medial prefrontal cortex, which it supplies with the necessary familiarity information.

Despite the partial similarity between the PRc firing rate reduction in primates and the FN400 negativity reduction in humans during the DMMS task, task differences complicate direct comparison of the neural results. Monkeys need extensive training to learn how to perform the DMMS task (Yakovlev et al., 2013), while humans can perform it immediately (Hochstein & Yakovlev, 2015; Walsh et al., 2016). This suggests that temporal resetting in humans may rely on more advanced strategies and, in turn, more complex neural networks than the PRc alone. Prime candidates include prefrontal, anterior cingulate and parietal cortical regions involved in encoding and retrieval of episodic memories, cognitive control, and resolution of proactive interference (Fletcher et al., 1995; Kim et al., 2009; Koechlin, Ody & Kouneiher, 2003; Konishi, Chikazoe, Jimura, Asari & Miyashita., 2005; Lundstrom, Ingvar, & Petersson, 2005; Nee, Jonides & Berman, 2007; Okada, Vilberg & Rugg, 2012).

A related, but distinct question concerns how familiarity with the stimuli affects humans’ ability to make relative temporal judgments. In the Hölscher & Rolls (2002) study, the PRc neurons showed the same level of activation for the novel stimuli and for stimuli that last appeared before the reset screen, suggesting that temporal resetting may be independent of subjects' familiarity with the stimuli. On the other hand, Hölscher & Rolls' study was not specifically designed to test the relationship between familiarity with the stimuli and temporal resetting because monkeys’ familiarity with the stimuli was not varied. In summary, the role of PRc, as well as other brain regions, in the service of temporal resetting in humans, remains unclear.

The current neuroimaging study examines, for the first time using fMRI, the neural underpinnings of the temporal reset in humans. We focused on the relationship between stimulus familiarity and humans’ ability to treat repeated items as new following a reset cue. We scanned subjects as they performed a continuous DMMS task (Hölscher and Rolls, 2002; Walsh et al., 2016). Subjected viewed a sequence of pictures of everyday items. A colored reset screen signifying the end of a block occasionally appeared. Subjects were instructed to respond “old” if a picture had already been seen in that block (i.e., since the last relevant colored screen), and “new” otherwise. Some trials involved pictures presented for the first time in the experiment (new items), others involved pictures that had last appeared before the reset screen (pseudonew items), and others involved pictures that had last appeared after the reset screen (old and old-repeat items). Old-repeat items, by definition, had previously appeared both before and since the reset screen.

We included old-repeat items so that we could vary the number of times that an item was presented as well as when it was presented relative to a reset screen. Presenting an item more frequently increases its relative familiarity (Mandler, 1980; Reder et al., 2000; Xiang & Brown, 1998) which can benefit or hurt memory performance, depending on the task. Repeated presentations and preexisting familiarity facilitate encoding (Diana & Reder, 2006; Reder, Liu, Keinath, & Popov, 2016; Reder, Paynter, Diana, Ngiam, & Dickison, 2007), which may increase the accessibility and probability of subsequently freely recalling such items. Alternatively, repeated presentations and pre-existing familiarity may be a liability during single-item recognition tests where participants must judge whether an item has appeared recently (Reder et al., 2000; Reder et al., 2007). In such cases, participants are more likely to wrongly endorse highly familiar (i.e., high-frequency) items not presented during an experiment’s study phase as “old” during the test phase.

Based on these considerations and previous findings, we tested two hypotheses about the neural correlates involved in temporal resetting in humans. First, we hypothesized that the PRc, prefrontal, anterior cingulate and parietal cortices, which are involved in episodic memory, cognitive control and resolution of proactive interference, would show differential activation patterns for old, new and pseudonew items. For example, activation in these regions might be greater for new and pseudonew items than for old items. Second, we hypothesized that stimulus familiarity, as controlled by number of presentations, would differentially affect both behavioral and neural responses to old versus pseudonew items. Specifically, participants might have greater difficulty correctly identifying pseudonew items that have appeared more often due to their greater familiarity. This effect might be manifested in decreased accuracy and increased response time (RT) to more familiar pseudonew items. Interference between item familiarity and the task requirements to respond “new” to pseudonew items might affect activation in regions involved in conflict detection and monitoring, such as the anterior cingulate gyrus (ACC; e.g., Botvinick, Cohen, & Carter, 2004; Kerns et al., 2004), as well as the functional connectivity between these regions and the rest of the brain.

Methods

Participants

Thirty-nine right-handed individuals from the Pittsburgh community participated in the study (18 males, 21 females, mean age = 22.3). All participants were fluent in English and had normal or corrected to normal vision. All were treated in accordance with Carnegie Mellon University’s IRB guidelines. Six participants were excluded from the data analyses due to excessive head movement (3 mm in any direction) or technical problems during scanning. This yielded a total of 33 participants.

Design and Materials

Stimuli included 136 pictures of everyday items (e.g., squirrel, hammer, etc.). They were approximately 9.5 degrees of visual angle in size and were displayed against a solid white background. Pictures were randomly selected to create a unique trial sequence for each participant. A stimulus list was divided into four higher-order sets, each consisting of three short blocks, two medium blocks, and two long blocks. A block was defined as the period between two appearances of the reset screen. Short, medium and long blocks contained twelve, twenty or thirty picture trials, respectively. There were four types of picture trials in a list: (1) New when a picture appeared for the first time in the experiment; (2) Old when a picture was repeated within the same block; (3) Pseudonew when a picture that had previously appeared in the experiment before the reset screen was presented; and (4) Old-repeat when a pseudonew picture was repeated within the same block.

Each picture was repeated several times over the course of the study, allowing us to examine RT, accuracy and brain activation as a function of stimulus repetition. Old items appeared up to 3 times, pseudonew items appeared up to 6 times, and old-repeat items appeared up to 7 times. By definition, new items appeared only once. There was a difference in the number of repetitions for old, pseudonew and old-repeat items because the old items had to be repeated within one block. Increasing the number of repetitions for old items would significantly increase the block length. This would either make the task much longer, or significantly decrease the number of reset screens. The “pseudonew” and “old-repeat” items could be distributed within and across the blocks, so it was possible to increase the number of repetitions for these items to make the task less predictable and more difficult.

In addition to viewing pictures, participants occasionally saw blank reset or distractor screens. The reset screen denoted the start of a new set of trials. The distractor screen served as a control condition to validate that the behavioral and neural changes related to temporal position of an item relative to the reset screen are not due to the screen presentation itself. The background color of the reset and distractor screens were red and blue, with the colors counterbalanced across participants. In total, the experiment contained 136 new pictures, 136 old pictures, 136 pseudonew pictures, 136 old-repeat pictures, 28 reset screens, and 28 distractor screens, for a total of 600 stimuli. Figure 1 shows a diagram of the task and conditions.

Figure 1.

Simplified task diagram. The number after the trial specifier (i.e., Old, Pseudonew, Old-repeat) indicates the number of times the specific image had appeared, thus far, in the task.

Procedure

Participants were told to respond as quickly as possible without sacrificing accuracy. They were instructed to respond “new” if the picture had not appeared since the reset screen (i.e., new and pseudonew pictures), and “old” if the picture had appeared since the reset screen (i.e. old and old-repeat pictures). While in the scanner, they responded by pressing either the left or right index finger button of response gloves. The assignment of response type (old/new) to response hand (left/right) was counterbalanced across participants. Participants were instructed not to respond to the reset or distractor screens. Additionally, they were told to ignore the distractor screens and continue with the task as if those screens did not appear. Prior to scanning, participants completed a 21-trial practice version of the task using a distinct set of pictures that did not repeat during the experiment proper.

For each trial, a picture appeared for 750 ms, followed by a black fixation cross against a white background for 1250 ms. A total of 164 jitter periods during which a fixation cross was shown for two seconds were included to perform event-related analysis of the data (Dale, 1999). Jitter periods were randomly distributed throughout the stimulus list, with the constraint that no more than five jitter periods appeared consecutively. After the final stimulus, the fixation cross was shown for an additional 16 seconds so that the full hemodynamic response could be determined for the final stimuli.

Data acquisition and analysis

Behavioral data analysis

To examine the effects of item familiarity and trial type on behavior, we analyzed response accuracies and RTs for new, old, pseudonew, and old-repeat items as a function of number of times that the item had appeared. We used logistic mixed effects regression to analyze accuracy, and linear mixed effects regression to analyze RTs. Participants and items were treated as random intercept effects (Baayen, Davidson, & Bates, 2008). RTs were analyzed only for correct responses. We used likelihood ratio tests to compare models with and without each of the main effects and interactions of interest. All effects were added to the models in the order in which they are reported. Given that old items were only presented up to 3 times, while pseudonew items were presented up to 6 times, we focused our analyses on the interaction between item familiarity (second versus third presentation) by Condition (old versus pseudonew).

MRI acquisition parameters

Scanning was performed using a Siemens 3T Verio MR system and a 32-channel RF coil. At the start of the scanning session, high-resolution structural images were acquired for each participant using an MPRAGE (i.e. a magnetization-prepared rapid acquisition in gradient echo) sequence (TR = 1.8 s, TE = 2.22 ms, FOV = 205 mm, FA = 9 degrees, number of slices = 256). Functional images were acquired using a gradient echo, echo-planar imaging pulse sequence (TR = 2 s, TE = 30 ms, FOV = 205 mm, FA = 79 degrees, slice thickness = 3.2 mm, number of slices = 36, interleaved slice acquisition order, isotropic voxels dimensions = 3.2 mm, and 780 volumes).

Functional MRI preprocessing

The fMRI images were preprocessed and analyzed using FSL 5.0.8 (www.fmrib.ox.ac.uk/fsl). Preprocessing included motion correction with MCFLIRT (Jenkinson, Bannister, Brady & Smith, 2002), non-brain removal using BET (Smith, 2002), spatial smoothing with a Gaussian kernel of FWHM 6 mm, grand-mean intensity normalization of the entire 4D dataset by a single multiplicative factor, and highpass temporal filtering (Gaussian-weighted least-squares straight line fitting, with cutoff period of 60 s [sigma = 30.0 s]). We did not apply a slice-timing correction, because the TR interval is short relative to the hemodynamic response delay, and because we included stimulus temporal derivatives in the model. The first-level and group-level analyses were conducted using FEAT version 6.00.

Functional MRI analyses

The first-level model included the following explanatory variables: New, Old2, Old3, Pseudonew2, Pseudonew3, Pseudonew4–5, Old-repeat3, Old-repeat4–6, Old-repeat7 trials, Errors (across all conditions), Reset screen, and Distractor screen (the numbers after the trial names represent the number of times that the items were presented in the experiment). The Old2-Old3 and Pseudonew2-Pseudonew3 contrasts were modeled at the first-level analysis. A time-series statistical analysis was carried out using FILM with local autocorrelation correction (Woolrich, 2001). The hemodynamic response was modeled using FSL’s standard double-gamma function. Motion outliers were computed for each participant using the motion_outliers script and were included in the model as covariates of no interest. FLIRT and FNIRT were used to register the BOLD images, first to the individual participant’s high-resolution T1-weighted anatomical image, then to the Montreal Neurological Institute (MNI) standard space template. All group-level analyses were conducted using FLAME1 (FMRIB’s local analysis of mixed effects). Featquery was used to extract mean percentage change for significantly activated voxels. For the purposes of plotting significant activation results, and to conduct several post-hoc t-tests, we extracted mean percentage change in BOLD activation within significantly activated clusters separately for each participant and condition.

Neural correlates of temporal resetting

Comparison of brain activation for New vs. Old2 vs. Pseudonew2

First, we examined our hypothesis about the PRc involvement in temporal resetting by conducting an ROI one-way ANOVA (New vs. Old2 vs. Pseudonew2) in a bilateral PRc mask that was generated from the perirhinal label in Freesurfer’s FsAverage brain (Augustinack et al., 2013), binarized, and then registered to FSL’s 2 mm MNI152 space brain. After that, we examined neural correlates of temporal resetting in the whole brain, using the one-way ANOVA, described above (New vs. Old2 vs. Pseudonew2). In both analyses, we focused on the second presentation of items in old and pseudonew trials (ignoring all subsequent presentations) to limit interference effects between the increasing relative familiarity of pseudonew items and the task requirement to respond “new” to them. In both analyses, Z-statistic images were thresholded using Gaussian random field theory-based maximum height thresholding. For these comparisons, we used a more stringent voxel-wise correction with a significance threshold of p=.05 (Worsley, 2001) in the corresponding mask.

Post-hoc t-tests were conducted to compare New vs. Old2, Old2 vs. Pseudonew and New vs. Pseudonew trials in the voxels where the F-test was significant. For the PRc, we extracted mean percent signal changes from the activation clusters in the right and left PRc separately and conducted these post-hoc t-tests with Bonferroni-corrected p-values (.05/6=.0083) in R. The R software was used because all voxels were localized to the PRc ROI, and there was no need for further spatial localization. For the whole brain, we conducted these post-hoc t-tests in FSL. Significant clusters of activation were determined by thresholding Z-statistic images in the New vs. Old2 vs. Pseudonew2 mask at z > 3.09 and a family-wise error-corrected cluster significance threshold of p =.05 (Worsley, 2001) to account for multiple comparisons.

Effect of familiarity on temporal resetting

Familiarity by Condition interaction effect

To examine the interaction effect between Familiarity and Condition (i.e., Old2-Old3 vs. Pseudonew2- Pseudonew3) on brain activation and functional connectivity, we conducted a whole brain analysis. Significant clusters were determined by thresholding Z-statistic images at z > 2.3 and a family-wise error-corrected cluster significance threshold of p = .05 (Worsley, 2001). This less stringent threshold was chosen because we expected that effect size in the interaction analysis to be the smallest.

Functional Connectivity Analysis using PPI

Functional connectivity reflects the statistical relationships between measures of neural activity in various brain regions. We used the psycho-physiological interactions (PPI) analysis (Friston et al., 1997) to examine functional connectivity between the region(s) showing significant Familiarity*Condition interaction (a seed region) and the regions that responded differentially to New vs. Old2 vs. Pseudonew2 items (a target region). The mean time series extracted from the seed region was averaged across all voxels in the region of interest, and served as a repressor in the first-level PPI model. The other regressors included New, Old2, Old3, Pseudonew2, Pseudonew3, Pseudonew4–5, Old-repeat3, Old-repeat4–6, Old-repeat7, Errors, Reset screen, Distractor screen, and the interaction terms between activation in the ROI and other regressors. The higher-level analyses contrasted Old2-Old3 vs. Pseudonew2- Pseudonew3 to identify the voxels that showed significant Familiarity by Condition connectivity interaction effects in the regions that responded differentially to New vs. Old2 vs. Pseudonew2.

Results

Behavioral Results

Figure 2 shows accuracy and RTs for correct responses. The statistical analyses were conducted only for Old2/Old3 versus Pseudonew2/Pseudonew3 trials. Responses were overall faster for old items than for pseudonew items, AIC = −8, LLR χ²(1) = 10.434, p = .001. Response times also decreased from the second to the third presentation, AIC = −13, LLR χ²(1) = 14.850, p < .001. The interaction between the two factors was significant, AIC = −62, LLR χ²(1) = 63.801, p < .001. Response times from the second to third presentation decreased for old items ( = - 48 ms, z = −7.965, p < .001), but they increased for pseudonew items ( = 31 ms, z = 3.973, p < .001).

Figure 2.

Response times and accuracy as a function of the number of times the item was presented during the experiment and the trial type

The accuracy data mirrored the response time results. Accuracy was greater for old items than for pseudonew items, AIC = −10, LLR χ²(1) = 12.136, p < .001. Response accuracy also increased from the second to the third presentation, AIC = −35, LLR χ²(1) = 36.799, p < .001. The interaction between the two factors was significant, AIC = −34, LLR χ² (1) = 36.231, p < .001, owing to the increased accuracy from the second to the third presentation of old items ( = .09, z = 7.805, p < .001), but not pseudonew items ( = −.001, z = −0.062, p = .998).

Neural correlates of temporal resetting

PRc ROI activation differences for New vs. Old vs. Pseudonew items

There were significant F-test results in the right (Z-max=4.57, n-vox=176, [22, −10, −28]) and left (Z-max=4.04, n-vox=92, [-26, −32, −20]) PRc. Pairwise comparisons of mean percent signal changes extracted from the activated voxels revealed that the PRc was less active when a stimulus was repeated (Figures 3a and 3b; left New>Old2, t(32) = 5.989, p < .0001; left New>Pseudonew2, t(32) = 4.759, p < .001; right New>Old2, t(32) = 6.836, p < .0001; right New>Pseudonew2, t(32) = 5.121, p < .0001). However, there was no evidence for a reset effect in the bilateral PRc, which was equally deactivated after Old2 and Pseudonew2 items (left Old2 vs Pseudonew2, t(32) = −0.270, p = 1.000; right Old2 vs Pseudonew2, t(32) = 0.821, p = 1.000).

Figure 3.

Brain activation for new items, old items on the second presentation (Old2) and pseudonew items on the second presentation (PN2) in (a) Left perirhinal cortex (PrC), (b) right PrC, (c) the right hippocampus (HPC), (d) Left intraparietal sulcus (IPS), (e) right posterior cingulate cortex (PCC)/precuneus. Error bars represent 1 standard error.

Whole brain activation differences for New, Old and Pseudonew items

The results of the ANOVA (New vs. Old2 vs. Pseudonew2) are reported in Table 1, and the results of the pairwise comparisons are reported in Table 2. The summary of the pairwise comparisons is also reported in Table 1 in the column “Comparison”. Figure 4 depicts the activation patterns corresponding to these effects. Various regions were differentially sensitive to whether the picture had appeared before or after the reset screen. We divided results into three groups (no evidence of resetting (New>Pseudonew2 and Old2, Pseudonew2=Old2), partial resetting (Old2>Pseudonew2>New), and complete resetting (Old2>New and Pseudonew2, New=Pseudonew2). These categories are also listed in Table 1.

Table 1.

Results of the whole brain ANOVA for comparisons among conditions.

				MNI coordinates (mm)
	Region	Voxels	Z-MAX	X	Y	Z	Comparison	Temporal Resetting
R	Lateral occipital cortex (LOC) extended to fusiform gyrus (FFg)	2825	7.49	40	−86	10	N > O2; N > PN2	No
R	Precuneus	2052	8.07	12	−64	24	O2 > PN2 > N	Partial
L	Lateral occipital cortex (LOC) extended to (fusiform gyrus) FFg	1854	6.48	−38	−88	8	N > O2 > PN2	No
L	Frontal Pole (FP)	508	6.34	−26	58	0	O2 > PN2 > N	Partial
L	Intraparietal sulcus (IPS)	382	7.12	−36	−60	40	O2 > PN2 > N	Partial
R	Intraparietal sulcus (IPS)	362	5.91	44	−50	48	O2 > N	No
R	Hippocampus (HPC)	131	5.49	24	−18	−20	N>O2; N>PN2	No
L	SFG (medial)	117	5.3	−6	56	28	N>O2; N>PN2	No
L	Orbital frontal cortex (OFc)	116	6.46	−38	34	−18	N>O2; N>PN2	No
R	Orbital frontal cortex (OFc)	99	6.16	32	26	−10	O2>N	No
R	PCC/Precuneus	71	5.79	12	−50	32	O2>N; O2>PN2	Complete
R	Frontal Pole (FP)	58	6.14	36	36	−18	N>O2; N>PN2	No
R	Paracingulate/ACC	53	5.23	6	24	44	O2>N	No
L	Medial Prefrontal cortex (mFC)	37	5.52	−4	54	−14
R	Parietal oppercular cortex	23	4.96	58	−28	18
L	Insula, ant	16	4.91	−34	20	0
L	Caudate	15	5.1	−8	12	4
R	Inferior frontal gyrus (IFG)	8	5.12	50	34	8
R	Middle frontal gyrus (MFG)	5	4.88	36	8	52
R	Occipital pole	4	4.82	28	−100	16
L	Hippocampus (HPC)	4	4.77	−18	−8	−22

Note: R = right hemisphere, L = left hemisphere, all results (except those reported in the column “Comparison”) are corrected for multiple comparisons at the voxel level of p<05. The right-most column “Comparison” shows the results of the pair-wise comparisons corrected for multiple comparisons at the cluster level (z>3.09, p<05) in the ROI mask consisting of the regions listed in the second column. N = New trials, O2 = Old2 trials, the second presentation of an old stimulus, PN2 = Pseudonew trials, the second presentation of a pseudonew stimulus. Only significant results are reported in the column “Comparison”. The “Temporal resetting” column reflects the relationship between activation for New, Old2 and Pseudonew2 stimuli. “Complete” resetting refers to greater activation for Old stimuli then for New and Pseudonew stimuli whose activation is not significantly different. “Partial” resetting refers to the activation pattern with greater activation for Old2 stimuli, lower activation for Pseudonew2 stimuli and the lowest activation for New stimuli. Regions with other activation patterns are referred as “No” resetting regions

Table 2.

Pairwise comparison of New, Old (second stimulus presentation) and Pseudonew (second stimulus presentation) trials within regions that showed significant ANOVA differences.

				MNI coordinates (mm)
	Region	Voxels	Z-MAX	X	Y	Z
New > Old2
R	LOCsup/inf extended to FFg	2823	6.43	36	−84	10
L	LOCsup/inf extended to FFg	1852	5.94	−30	−94	16
R	HPC	131	6.07	24	−12	−20
L	OFc	116	5.28	−38	34	−18
L	SFG (medial)	112	4.55	−10	52	32
R	Frontal Pole	58	5.95	38	36	−20
Old2>New
R	Precuneus	2052	7.14	12	−64	24
L	Frontal Pole	508	5.71	−36	56	4
L	IPS	382	6.35	−36	−60	40
R	IPS	362	5.77	44	−50	48
R	OFc	99	5.81	30	26	−10
R	PCC/Precuneus	71	5.42	12	−50	32
R	Paracingulate/ACC	53	5.09	8	24	44
New>Pseudonew2
R	LOCsup/inf extended to FFg	2814	5.94	38	−82	10
L	LOCsup/inf extended to FFg	1854	5.9	−38	−88	6
L	SFG (medial)	117	5.06	−6	54	30
L	OFc	116	5.11	−36	36	−14
R	HPC	102	4.64	30	−20	−20
R	Frontal Pole	58	4.78	34	36	−16
Pseudonew2>New
R	Precuneus	879	5.16	12	−64	22
L	Frontal Pole	126	4.03	−26	58	2
L	IPS	97	4.05	−34	−54	34
Old2>Pseudonew2
R	Precuneus	1386	4.23	−8	−74	28
L	IPS	190	4.03	−38	−60	40
L	Frontal Pole	112	4.21	−34	54	−2
R	PCC/Precuneus	71	4.4	8	−54	32
L	LOCsup/inf extended to FFg	48	3.52	−46	−80	10

Note: R = right hemisphere, L = left hemisphere, all results are corrected for multiple comparisons at the voxel level of p<.05. ROI abbreviations follow the definitions in Table 2.

Figure 4.

Activation patterns for New (N) vs Old2 (O2) vs Pseudonew2 (PN2) ANOVA, showing pair-wise comparisons, as labeled.

No resetting (New>Pseudonew2 and New>Old2)

There was no evidence of resetting in the right hippocampus (HPC), right lateral occipital cortex (LOC) extended to fusiform gyrus (FFg), right frontal pole (FP), left medial superior frontal gyrus (SFG), and left orbitofrontal cortex (OFc). These regions all showed greater activation to new stimuli than to stimuli that had been previously seen, regardless of whether the latter had appeared before or after the reset screen (Figure 3c).

Partial resetting (Old2>Pseudonew2>New)

Evidence for partial resetting was revealed in the left IPS and right precuneus, which showed a parametric change in activation that was the greatest for the old trials, lower for the pseudonew trials, and the lowest for the new trials (Figure 3d). In other words, these regions were sensitive to whether the stimulus had appeared before and whether the stimulus had repeated within the block. In addition, the contrasts Old2>Pseudonew2, Pseudonew2>New and Old2>New all elicited activation in the left FP, and the three did not overlap.

Complete resetting (Old2>New and Old2>Pseudonew2)

The right PCC/Precuneus showed greater activation for stimuli that were repeated within a block (i.e. old stimuli), compared to the stimuli that were presented in the block for the first time. This occurred regardless of whether the stimuli appeared for the first time in the task (New) or were repeated from an earlier block but appeared for the first time in the current block (Pseudonew; Fig. 3e).

Effect of familiarity on temporal resetting

The 2 (Old vs. Pseudonew) × 2 (second vs. third item presentation) ANOVA revealed that activation in the ACC [-6, 40,18; z-max = 3.25, nvox = 629] decreased from the second to the third stimulus presentation during the old trials, but increased from the second to the third presentation during the pseudonew trials (Figure 5a). The functional connectivity between ACC and the left frontal pole (LFP) decreased from the second to the third presentation of old items, but increased from the second to the third presentation of pseudonew items (n-vox = 185, z-max = 3.47 [-28 62 0]).

Figure 5.

(a) The interaction between the trial type (Old/Pseudonew) and the presentation number (second/third) in the bilateral ACC (anterior cingulate gyrus); (b) The interaction in functional connectivity between ACC and left frontal pole (LFP), with ACC serving as a seed region.

Discussion

In this study, we examined the neural correlates of temporal order judgments using a modified delayed-match-to-multiple-sample (DMMS) task (Hölscher & Rolls, 2002; Walsh et al., 2016). Participants viewed a continuous sequence of pictures separated by occasional reset screens. They were instructed to respond “new” to items that had last appeared before the reset screen (i.e., pseudonew items), and “old” to items that had last previously appeared after the reset screen. Consistent with our first hypothesis, we identified a brain region (R PCC/precuneus) that showed greater activation for old relative to both new and pseudonew items, which did not differ from each other (complete reset region). Other brain regions showed intermediate activation for pseudonew items relative to old and new items (partial reset regions), consistent with the findings of the ERP study (Walsh et al., 2016). Still other brain regions showed no evidence for temporal resetting (no-reset regions), notably the PRc, contrary to the findings of Hölscher & Rolls (2002) that used primates. The present study, using humans, provides support for previous conclusions that the human PRc responds to absolute familiarity. Finally, consistent with our second hypothesis, we found a significant interaction between the task condition (old vs. pseudonew) and relative familiarity with the stimuli on behavioral and neural measures. Specifically, accuracy decreased, while RTs, ACC activation and functional connectivity between the ACC and the LFP increased for more familiar pseudonew items as compared to more familiar old items. Below, through a consideration of each of these results, we present a cognitive control theoretical framework that accounts for the behavioral and neural pattern of temporal resetting.

Neural correlates of temporal resetting

PRC findings

In contrast to the findings from primate research (Hölscher & Rolls, 2002), the PRc showed no evidence of resetting in our fMRI study. PRc activation was identical for old and pseudonew stimuli and was significantly lower compared to new stimuli. It is important to note that in Hölscher & Rolls’ (2002) study only a subset of PRc neurons showed a resetting effect. The PRc contains a heterogeneous population of neurons: in addition to showing a resetting effect, individual neurons are differentially sensitive to item novelty, recency, and/or familiarity (Xiang & Brown, 1998). Since each fMRI voxel reflects the hemodynamic changes from averaged neural activity over hundreds of thousands of neurons, it is possible that fMRI might be unable to detect evidence for resetting in the PRc in humans if such neurons were distributed throughout that region. A higher resolution study targeting the PRc in combination with multivariate analysis techniques might be necessary to identify temporal resetting in the PRc. Alternatively, the difference in findings might be due to inter-species differences: since humans have better developed frontal cortex, they might rely on a frontal-parietal network to perform the reset task, while monkeys have to rely on simpler circuits in the PRc.

Novelty/familiarity processing regions with no resetting

In addition to the PRc, a network of prefrontal, parietal, occipital and medial temporal lobe regions closely resembled previous findings from the old/new recognition paradigms (e.g., Okada et al., 2012). Among those regions, right hippocampus, right frontal polar cortex, left superior frontal gyrus, left orbital frontal cortex and bilateral lateral occipital cortex extending to the fusiform gyrus showed greater activation for new items than for old and pseudonew items. The finding of a novelty-related signal in the hippocampal, frontal and occipital regions is consistent with previous studies that suggest that novelty signals are automatically generated and distinguish between new and old stimuli (e.g. Kumaran & Maguire, 2007; Ranganath & Rainer, 2003). Our finding that activation in these regions did not differ between old and pseudonew items extends the current understanding of the neural basis of novelty detection in these regions by showing that this fundamental mechanism is not affected by changes in temporal context and the cognitive demands imposed by the DMMS. This result also suggests that these regions track absolute novelty/familiarity, rather than whether the participant is preparing to respond “old” or “new”. Lastly, this result supports the idea that stimuli are not simply forgotten after the reset screen – otherwise neural and behavioral responses to new and pseudonew items would be identical.

Regions with evidence for partial resetting

Several regions showed partial temporal resetting, similar to our recent ERP study in which the FN400 was most negative for new items, less negative for pseudonew items, and the least negative for old items (Walsh et al., 2016). Specifically, the left frontal polar cortex (LFP), left intraparietal sulcus (LIPS), and right posterior precuneus showed greatest activation during processing of old items, lower activation for pseudonew items, and the lowest activation for new items (old > pseudonew > new).

The partial reset pattern in the LIPS and the right posterior precuneus can be understood with respect to their previously proposed roles in episodic memory. Previous studies have suggested that the posterior parietal cortex is a likely episodic buffer involved in the online maintenance of recollected memories (e.g., Rugg, Otten, & Henson, 2002; Sohn, Goode, Stenger, Carter & Anderson, 2003; Vilberg & Rugg, 2009; although for a different view, see Cabeza, Ciaramelli, Olson, & Moscovitch, 2008). Similarly, the right precuneus is involved in searching among a pool of potentially relevant items in memory (Makino, Yokosawa, Takeda, & Kumada, 2004). Based on these findings, we propose that the partial reset effect in parietal regions likely reflects differences in the probability of recollecting the previous encoding trace for pseudonew and old items.

This proposal is based on existing theories and empirical results concerning how changes of context affect the probability of episodic retrieval (DuBrow & Davachi, 2013; Ezzyat & Davachi, 2014; Norman, Newman & Detre, 2007; Polyn & Kahana, 2008; Reder et al., 2001). In order to perform the DMMS task, participants have to encode each item and bind it to the context of the current block of items. Furthermore, distinct boundaries, such as the reset screen in the current study, can cause a shift to a new encoding context or sub-context (DuBrow & Davachi, 2013; Ezzyat & Davachi, 2014;). Thus, to decide whether a repeated item appeared before or after the last reset screen participants could depend on recollecting the context in which the item was previously experienced. The current context and the item itself can serve as cues to retrieve the previous encoding trace of that item (Reder et al., 2000). When an item is presented a second time in the same block (i.e., an old item), both the item and the current block’s context are already associated with the episodic trace for the previous occurrence, thereby facilitating retrieval of that trace. On the other hand, the current context is not associated with the previous episodic trace of a pseudonew item and it will not facilitate its retrieval. As a result, when the current item is pseudonew, participants are less likely to recollect the episodic trace and the associated context in which the item was previously presented. In summary, we suggest that the differences in activation in parietal regions during processing of pseudonew and old stimuli may indicate the differential success in memory search for previous encodings of those items¹, and the subsequent reactivation of representations following successful retrieval (Johnson & Rugg, 2007; Manelis, Hanson, & Hanson, 2011; Vaidya, Zhao, Desmond, & Gabrieli, 2002; Wheeler et al., 2006; Wheeler, Petersen, & Buckner, 2000).

When it comes to the LFP, researchers have suggested that it integrates the outcomes of several separate cognitive operations to achieve a higher behavioral goal (Ramnani & Owen, 2004). Specifically, the LFP sits at the top of a hierarchical cognitive control network that determines eventual responses in the premotor cortex. This network is responsible for representing task-, stimulus- and context-specific behavioral rules, which allows it to select and execute appropriate actions depending on the nature of the stimuli, the perceptual context and the temporal episode (Koechlin et al, 2003). Within this control network, the LFP is supposedly involved in the controlled retrieval of episodes (Allan, Dolan, Fletcher, & Rugg, 2000; Koechlin et al, 2003) and in selecting the appropriate task-stimulus-response rule for each condition (Koechlin et al, 2003). In support of this model, previous studies have demonstrated that activation in the LFP increases with higher demand for controlling episodic retrieval (Koechlin et al, 2003), and that its activation decreases when the cause of episodic interference is removed (Konishi et al, 2005). Based on the results of Koechlin et al (2003) and Konisi et al (2005), one might expect that we would find activation in the LFP to be highest for pseudonew trials, if there was strong interference between familiarity and the task goal to respond “new”. However, that was not the case for pseudonew items in this study, at least on their second presentation. It is possible that two repetitions did not produce sufficient familiarity to cause competition of responses for pseudonew items, as evidenced by the behavioral results and ACC activation results. Presenting the pseudonew items for a third time, however, did increase conflict and interference as demonstrated by the increased ACC activation and the increased ACC-LFP connectivity. We return to this issue below, where we discuss the effects of familiarity on temporal resetting.

Regions with evidence for complete resetting

Finally, the only region that showed a complete reset effect was the right PCC/precuneus. Activation in that region was greater for old items than for new and pseudonew items, which did not differ from each other. Thus, the right PCC/precuneus was the only region whose activation corresponded not to absolute novelty of the stimulus, but rather to whether the item was to be considered as being new or old according to the task instructions. Remarkably, the location of the right PCC/precuneus region identified in our study [12, −50, 32] was almost identical to that identified by Tomasi and Volkow's (2010) [6, −48, 33], who showed that this specific region had the highest local functional connectivity density among all regions in the brain. Other researchers have also noted that the right PCC/precuneus is an energy-efficient convergence hub (Cavana & Trimble, 2006; Chua, Schacter & Sperling, 2009; Fransson & Marrelec, 2008; Utevsky, Smith & Huettel, 2009) that supports higher-level cognitive functioning by integrating information through its dense connections with other brain regions (e.g., Schedlbauer, Copara, Watrous, & Ekstrom, 2014; Tomasi & Volkow, 2010). Despite these findings, the specific function of the PCC/precuneus is unclear and it is role in memory is a matter of debate (Leech & Sharp, 2014). For example, while some researchers believe that it is involved in representing successfully recollected information (Maddock, Garrett & Buonocore, 2001), others have suggested that it is instead signaling the retrieval success or even evaluating the retrieved content (Yonelinas, Otten, Shaw & Rugg, 2005).

We believe that the evaluative explanation is more consistent with our findings and we extend it in the following way. We propose that the PCC/precuneus not only integrates retrieved mnemonic information across the cortex, but it also evaluates information with respect to the task-defined response rules, which are possibly represented in the LFP (Koechlin et al. 2003). In the current study, this evaluation might correspond to a rule such as “If the recollected encoding context of the previous occurrence matches the current context, then respond that it is old”. Thus, during temporal resetting specifically, the PCC/precuneus might be evaluating whether the recollected encoding trace, which is likely represented in the posterior parietal cortex (Vilberg & Rugg, 2009), was originally encoded with current block’s context. This results in a positive signal only for old items. Finally, it is possible that the PCC/precuneus informs the LFP and the ACC of the on evaluation outcome through its intrinsic connections with them (Park & Friston, 2013). It is worth nothing that while this account for the complete temporal resetting in the PCC/precuneus is consistent with our data and with some previous proposals for its functional role in memory (Yonelinas, Otten, Shaw & Rugg, 2005), our study was not explicitly designed to test it or to contrast it with alternative accounts.

The effect of familiarity on temporal resetting

Both the behavioral and the neuroimaging findings of this study, together with our previous ERP findings (Walsh et al., 2016), reveal for the first time that increases in stimulus familiarity make temporal resetting more difficult. Specifically, increasing familiarity made behavioral responses to old but not to pseudonew items faster and more accurate. In fact, RTs for the third presentation of the pseudonew items were slower than for their second presentation. Our neuroimaging findings parallel the behavioral results by showing that both ACC activation and ACC-LFP functional connectivity decreased from the second to third presentation of old items, but increased from the second to third presentation of pseudonew items. These findings are consistent with previous research showing that ACC is involved in conflict monitoring, and that conflict detection increases ACC activation and functional connectivity with the prefrontal cortex (Barber & Carter, 2005; Botvinick, Nystrom, Fissell, Carter, & Cohen, 1999; Sylvester et al., 2003; also see Bush, Luu, & Posner, 2000 for a review).

Yet exactly how the ACC detects the conflict on pseudonew trials remains an open question. One possibility is that during pseudonew trials the ACC directly detects the two opposing memory signals. One signal is based on the PCC/precuneus evaluation of the recollected information, as discussed above. The other signal reflects absolute familiarity and it comes from the PRc, hippocampal, ventrolateral prefrontal and lateral orbitofrontal regions, none of which showed resetting. These two signals create a potential response conflict for highly familiar pseudonew items. When the ACC detects potential interference after the third repetition of pseudonew items, it might attempt to modulate LFP activation in order to allow the LFP to direct behavior more efficiently (as previously discussed, LFP regions also usually increase in activation when the amount of interference in the task increases; Koechlin et al., 2003). This explanation is consistent with our finding of increased ACC activation and increased ACC-LFP functional connectivity for repeated (and, consequently, more familiar) pseudonew items. This proposal and the empirical results reviewed above potentially extend the hierarchical cascade model of cognitive control (Koechlin et al, 2003) by suggesting that the ACC could be involved in modulating the LFP in the face of interference.

Summary

The behavioral and neuroimaging results of this study suggest an account of how humans make accurate judgments about relative temporal order of the events in a temporal resetting paradigm. We propose that temporal resetting relies on an episodic retrieval network that is modulated by cognitive control and conflict resolution regions. Repeated/new items cause a familiarity/novelty signal in prefrontal, medial temporal and occipital brain regions involved in episodic memory. This signal indicates the absolute familiarity of the stimulus; that is, whether it had appeared earlier in the experiment. For familiar items, participants likely attempt to recollect the temporal context in which the previous occurrence of the item was experienced. The current context and the item itself are used as cues for this memory search. As a result, the episodic traces for the previous occurrences of items are more likely to be recollected for old than for pseudonew items. This results in a partial reset pattern in parietal areas involved in the retrieval and temporary reinstatement of the recollected information. The PCC/precuneus, in turn, possibly integrates mnemonic information across the cortex and evaluates whether the recollected contextual information matches the current block context. Finally, the LFP might select task-specific response rules and it might initiate responses based on these rules and the signals it receives from the familiarity/novelty network and the PCC/precuneus. On highly familiar pseudonew trials these two signals are in conflict. The ACC monitors for and resolves this conflict for highly familiar pseudonew items by increasing its own activation and by modulating the LFP through their increased functional connectivity. While various aspects of the framework proposed here might require further empirical testing, the framework accounts for the full pattern of behavioral, neuroimaging results in the current study, and it also extends our understanding of the role of cognitive control in temporal order judgements.