Temporal binding within and across events

Abstract

Remembering the order in which events occur is a fundamental component of episodic memory. However, the neural mechanisms supporting serial recall remain unclear. Behaviorally, serial recall is greater for information encountered within the same event compared to across event boundaries, raising the possibility that contextual stability may modulate the cognitive and neural processes supporting serial encoding. In the present study, we used fMRI during the encoding of consecutive face and object stimuli to elucidate the neural encoding signatures supporting subsequent serial recall behavior both within and across events. We found that univariate BOLD activation in both the middle hippocampus and left ventrolateral prefrontal cortex (PFC) was associated with subsequent serial recall of items that occur across event boundaries. By contrast, successful serial encoding within events was associated with increased functional connectivity between the hippocampus and ventromedial PFC, but not with univariate activation in these or other regions. These findings build on evidence implicating hippocampal and PFC processes in encoding temporal aspects of memory. They further suggest that these encoding processes are influenced by whether binding occurs within a stable context or bridges two adjacent but distinct events.

Introduction

Recalling details from the past is a core function of episodic memory yet it remains relatively understudied in cognitive neuroscience. Research on the dynamics of free recall has led to influential models of episodic memory (Raaijmakers and Shiffrin, 1981; Anderson et al., 1998; Howard and Kahana, 2002; Polyn et al., 2009; Farrell, 2012; Lehman and Malmberg, 2013). One well-established pattern in recall is the tendency to transition to items that were presented in close proximity to the just-recalled item (Kahana, 1996). This contiguity effect is biased forward such that transitions are more often made to items that followed rather than preceded the just-recalled item. Thus, even when people have no constraints on the order of recall, they tend to adopt forward serial recall, or recall of items in the order in which they were presented at study. The spontaneous tendency to adopt forward serial recall is particularly strong for short lists (Ward et al., 2010; Grenfell-Essam and Ward, 2012).

The mechanisms supporting serial recall have long been debated with one major theory positing associative chaining, or direct inter-item associations, and the other major theory advocating for positional coding, or item-position associations (Young, 1968). Recent behavioral analyses have lent support to associative chaining theories by showing that shuffling the position of items while keeping relative order intact does not interfere with serial recall (Kahana et al., 2010) and that temporal clustering is more prominent than positional clustering when deconfounding the two (Solway et al., 2012). Thus, serial recall is thought to involve the mnemonic binding of one item to the next across a temporal gap. However, the mechanisms that support this temporal binding remain unknown. One possibility is that a stable context can provide a scaffold for linking events, for example through the binding of items to their temporal context (Howard and Kahana, 2002; Polyn et al., 2009; Farrell, 2012). Another possibility is that recently encountered items are more actively retrieved or refreshed in order to integrate consecutive items (Johnson, 1992; Murray and Ranganath, 2007; Hales and Brewer, 2011). Indeed, both may contribute to serial encoding depending on the properties of the environment, such as stability.

Paradigms that manipulate environmental stability by introducing event boundaries, or changes to the current context or goal state, have been shown to modulate access to associative information in memory (Zwaan et al., 1995; Zwaan, 1996; Swallow et al., 2009, 2011). Recent work has shown that cued and serial recall across event boundaries is lower compared to recall within the same event (Ezzyat and Davachi, 2011; DuBrow and Davachi, 2013; Horner et al., 2016; Heusser et al, in revision), presumably due to greater difficulty in bridging temporal gaps that contain boundaries. Consistent with this, people tend to remember items separated by boundaries as being farther apart than those that occurred within the same event (Ezzyat and Davachi, 2014), suggesting that boundaries influence the organization of associative memories.

Much of the neuroimaging data examining memory recall has focused on free recall. These studies have implicated the hippocampus and other medial temporal lobe (MTL) and lateral prefrontal cortical (PFC) regions during encoding in supporting later free recall (Alkire et al., 1998; Fernández et al., 1998; Strange et al., 2002; Schott et al., 2004; Staresina and Davachi, 2006; Dickerson et al., 2007; Long et al., 2010). When examining recall specifically in contrast to item and associative recognition, Staresina and Davachi (2006) found that BOLD activation in the hippocampus and left ventrolateral PFC (vlPFC) at encoding showed a graded effect such that later free recall was associated with the highest level of activation and item-only recognition with the lowest. By contrast, activation in left dorsolateral PFC (dlPFC) was specifically higher for items subsequently recalled but did not differentiate between items recognized with or without associated detail. Interestingly, Long and Badre (2010) showed that subsequent recall effects in the MTL and left vlPFC were correlated across subjects, suggesting that these two regions may interact to promote strong encoding. Schott and colleagues (2013) provided more direct evidence for this view in their finding that hippocampal-vlPFC functional connectivity during deep encoding was associated with successful subsequent recall.

In contrast to free recall, serial recall has primarily been studied in verbal short-term memory paradigms (see Marshuetz, 2005 for review) and, as a result, neuroimaging studies of immediate serial recall have largely implicated areas in and around the auditory and prefrontal cortices (Chein and Fiez, 2001; Acheson et al., 2011; Kalm et al., 2012; Kalm and Norris, 2014). However, in one such study of letter sequence learning, the hippocampus was shown, across repetitions, to exhibit increasing pattern similarity within the same sequence and reduced pattern similarity between different sequences (Kalm et al., 2013). This suggests that hippocampal processes may contribute to serial learning over repetitions. However, it is still unclear whether the same system is involved in single-shot serial encoding.

In the current study, we sought to investigate the neural mechanisms that support episodic encoding of stimuli that are later serially recalled, both within events and across event boundaries. Given prior behavioral work, we predicted that serial transitions to event boundaries would be reduced compared to serial transitions within events. If serial recall is simply a more difficult form of free recall, we would expect the same network of regions (MTL and left lateral PFC) would be involved in single-shot encoding for serial recall. Furthermore, if bridging temporal gaps across boundaries is simply a more difficult form of within-event sequential binding, we would expect the same mechanisms supporting within-event binding to be more strongly engaged in binding across a boundary. However, an alternative possibility is that the mechanisms that support serial encoding across boundaries may differ from those that support binding within events. The stable context within events, for example, may promote coupling between regions involved in episodic encoding and context maintenance. By contrast, integrating consecutive items across event boundaries may require more active retrieval of preboundary representations. Here, we investigated these possibilities by examining fMRI univariate activation as well as functional connectivity at the time of associative binding.

Materials and Methods

Participants

25 right-handed native English speakers (17 female; age range: 18–28, mean = 22) participated for pay ($25/hour). Participants were recruited from New York University and the broader community. Informed consent was obtained in a manner approved by the University Committee on Activities Involving Human Subjects. 4 participants were excluded from the recall analyses because we did not have their verbal response data due to equipment failure. An additional 4 participants were excluded due to poor serial recall performance that averaged less than 4 items per list (the number we discarded for primacy and recency). One additional participant was excluded from any analyses that compared serial recall to item only (nonserial) recall due to the latter having only one trial in the boundary condition.

Procedure

The experimental procedure was previously reported in DuBrow and Davachi, (2014). Briefly, stimuli consisted of color images of celebrity faces and nameable objects. 16 study-test rounds were performed in the scanner and a final test was given outside of the scanner. During encoding, participants were presented with a series of 25 trial-unique images (faces and objects, Fig 1a) with their corresponding label (full name or object name, respectively) each for 2 seconds. Participants were instructed to memorize the order of the images and were encouraged to use an associative encoding strategy. Additionally, participants were prompted to make a category-specific judgment for which they rated on a scale of 1 to 4 the likability of each celebrity face and the commonality of each object during a 2-second response period following stimulus presentation. We define context as the combination of the stimulus category and the category-specific task and consider the first items of a new context “event boundaries.” Boundaries occurred semi-predictably every two or seven items such that conditions were position-matched across lists. This resulted in list constructions in which a long event at each end of the list flanked 4 short events in the middle or 2 short events at either end flanked 2 long events in the middle. The inter-trial interval (ITI) during list encoding was pseudorandomized to be 4, 6 or 8 seconds in order to orthogonalize stimulus category and event position.

Figure 1.

(A) Encoding task schematic. Participants were presented with lists of 25 images and were instructed to remember their order for subsequent recency discrimination and serial recall. Images were celebrity faces, for which likability judgments were made, and common objects, for which commonality judgments were made. Images were presented with their corresponding label (not depicted) for 2 sec each followed by the response prompt for another 2sec. Boundary items were those for which the category/judgment changed, in this example the broccoli. (B) Behavioral serial recall data. Left) Serial transitions to boundary items were reduced compared to transitions to preboundary items when controlling for item recall of the pre-transition and post-transition items. * p < .05, one-tailed. Right) Nonserial transitions, or “jumps,” were made more often to boundary than preboundary items. * p < .05. Error bars reflect SEM.

After each study list, a 45-second arrow distractor task was presented followed by the recency test period. During this test, 12 image pairs were presented and participants indicated which of the two images was more recent. Results from this recency test are reported in DuBrow and Davachi (2014). Following the recency discrimination test, the scan ended and participants were instructed to recall the list in serial order. It is important note here that because recall occurred after recency discrimination, most recalled items had been seen twice. However, because neighboring items made up only 1/6 of the tested item pairs and the order of test trials and within-pair items was randomized, it is unlikely that the recency test could directly induce serial memory. The instructions for recall were to start at the beginning of the list and continue to the end, skipping items that could not be recalled. Participants were given up to 90 s to complete serial recall for each list. Verbal responses were recorded via the intercom in the MRI scanner on a handheld recording device.

Behavioral Data

Recall data were transcribed using Penn TotalRecall (http://memory.psych.upenn.edu/TotalRecall) and transcription was performed blind to condition. The transcribed data were then imported into MATLAB (Mathworks, Sherborn MA) and recall accuracy was computed. Items were considered recalled if at least part of the item’s name was said and there was no possibility of reference to another item in the experiment. An item was considered serially recalled if it was recalled immediately after the item that preceded it in the list. The first and last two items were considered primacy and recency items, respectively, and were discarded from analysis. The remainder of each list contained five boundary items and five preboundary control items, which were those that immediately preceded boundaries. Thus, across the 16 lists there were a total of 80 items in each condition. To test whether recall was performed serially above what would be expected by chance item recall, we shuffled the output order of each participant’s recalls and recomputed their serial recall rate. To get the recall rate expected by chance based on each participant’s recall, we averaged the result of 1000 permutations and compared this to the actual serial recall rate. To compare serial recall behavior between the boundary and preboundary conditions, item recall was controlled by considering only those trials in which both consecutive items were recalled and computing the percentage of those in which the serial transition was made accurately. To examine the frequency of incorrect serial transitions in each condition, the proportion of items recalled out of serial order was computed within condition and compared between conditions controlling for item recall.

FMRI Data

Scanning was performed on a 3T Siemens Allegra head-only scanner. A high-resolution anatomical scan (magnetization-prepared rapid-acquisition gradient echo sequence, 1 x 1 x 1 mm voxels) was collected and functional data were acquired using an echo-planar (EPI) pulse sequence (34 contiguous slices oriented parallel to the AC-PC axis; TR = 2000 ms; TE = 15 ms; flip angle = 82°, 3 x 3 x 3 mm voxels). The first four volumes of each run were discarded to allow for T1 stabilization.

Preprocessing was performed in fMRI Expert Analysis Tool (FEAT) version 6.00 as implemented in FSL version 5.0.8. Functional images were brain-extracted, realigned within run to correct for head motion, high-pass filtered (100 s cutoff) and smoothed (5 mm FWHM kernel). The functional images were then registered to the high-resolution anatomical using FMRIB's Linear Image Registration Tool (FLIRT) and then concatenated to create a single time series for the entire scanning session. Subsequent general linear models (GLMs) of the functional data included a regressor for each run as well as an additional regressor with timepoints identified by FSL’s motion outliers tool. For whole-brain analyses, fMRI Non-Linear Registration Tool (FNIRT) was used to transform high-resolution anatomical scans to Montreal Neurological Institute (MNI) standard space using a nonlinear transformation with a 10-mm warp resolution. All clusters are reported in MNI space with a voxel dimension of 2x2x2 mm.

Hippocampal ROI Univariate Analysis

To investigate the role of the hippocampus in associative binding across boundaries for subsequent serial recall, we ran an ROI analysis. The bilateral hippocampi of each subject were segmented using FSL’s automated segmentation tool FIRST (Patenaude et al., 2011) and then split into thirds along its long axis by slice number using custom software in MATLAB. Functional data were extracted from each subregion and averaged across voxels. A finite impulse response (FIR) was used to model the hippocampal response to boundary and preboundary items that were either recalled in serial order or recalled but not in serial order (nonserial recall). Six TRs were included in the model beginning at the onset of the recalled items. Each TR for each of the four conditions of interest was modeled as a separate regressor. As we were interested in the effects of serial versus nonserial recall and boundary condition, we ran 2-way repeated measures ANOVAs on the average of the parameter estimates from 2–4 TRS (4–8 seconds) after stimulus onset with memory and condition as factors. Items that were not recalled at all (misses) were excluded from this analysis, thus providing a stringent control on recall status.

Whole-brain Univariate Analysis

To investigate whether other regions in the brain show activation related to subsequent serial recall, we ran a whole-brain analysis. At the subject level, we ran a GLM with 4 conditions of interest: boundary and preboundary items crossed with subsequent recall type (serial versus nonserial). Condition regressors were created by convolving a 2- TR boxcar, corresponding to both the stimulus presentation and response periods, with a double-gamma hemodynamic response function. As mentioned above, nuisance regressors included runs and motion outliers. This fixed effects model produced whole brain maps of parameter estimates for each condition. Contrasts between serial and nonserial recall within each of the boundary conditions were performed at the group level using FSL’s mixed effects analysis (FLAME 1). We chose to focus on this simple effect measure of within-condition memory rather than a conjunction or an interaction because we did not have strong a priori hypotheses about whether the effects would be similar or distinct in the two conditions. Multiple comparisons correction was performed using FSL’s cluster correction with a primary threshold of p < .01 and a whole-brain false discovery rate (FDR) of p < .05.

Whole-brain PPI Analysis

To investigate whether interactions between the hippocampus and other regions might play a role in binding within and across events, we conducted a psycho-physiological interaction (PPI) analysis. We chose this measure of functional coupling to avoid single trial estimation of events with minimal temporal separation. For this analysis, we used the entire hippocampus as the seed region, in line with recent evidence that hippocampal connectivity patterns are homogenous along its long axis (Wang et al., in press). Thus, hippocampal timecourses were extracted from the bilateral hippocampi in each subject’s native space and averaged across voxels. GLMs were constructed with the 4 condition regressors included above with the addition of the physiological hippocampal regressor and 4 PPI regressors that reflect the interaction between the hippocampal timecourse and each condition regressor. As in the whole-brain univariate analysis, within-condition simple effect contrasts were performed at the group level and whole-brain FDR correction was applied.

Results

Behavioral Results

Overall, participants recalled 65.9% (SD = 17.4%) of list items. 67.2% (SD = 20.6%) of boundary items and 69.4% (SD = 19.8%) of preboundary items were recalled, and no significant difference between conditions was observed (t(16) = 1.38, p = .186). Serial recall averaged 42.9% (SD = 23.2%), which was significantly higher than the average percentage expected by chance (3.6%, SD = .6%; t(16) = 7.03, p < .001). Controlling for item recall of the pre and post transition items, we found that serial versus nonserial transitions were made more frequently to preboundary items (M = 53.5%, SD = 25.6%) than to boundary items (M = 50.2% SD = 27.5%; Fig 1b, left; t(16) = 1.78, p = .047, one-tailed as it replicates DuBrow and Davachi, 2013). This provides behavioral evidence for a reduction in sequential binding across event boundaries.

To further examine how recall dynamics led to a reduction in serial transitions to boundary items in the absence of evidence for item memory differences, we analyzed the nonserial transition data. Specifically, we compared the conditions in terms of how frequently nonserial “jumps” were made to boundary versus preboundary control items. We found that nonserial jumps accounted for more of the transitions to boundary items (M = 53.4%, SD = 23.9%) versus preboundary items (M = 47.4%, SD = 21.7%; t(16) = 3.44, p = .003, Fig 1b, right). Thus, boundary items were more likely to be recalled out of order relative to control items.

Hippocampal Univariate Results

To test whether hippocampal activation was related to serial encoding, we compared the FIR model results for our four conditions of interest -- boundary versus preboundary crossed with serial versus nonserial recall. We first ran a repeated measures ANOVA on parameter estimates corresponding to 4–8 seconds after stimulus onset from the whole hippocampus. While we observed a main effect of condition (F(1,15) = 10.41, p = .006), there was no effect of recall type (F(1,15) = 0.30, p = .592) or interaction between condition and memory (F(1,15) = 1.86, p = .193). However, follow-up t-tests revealed that the boundary condition effect was driven by greater activation in the boundary versus preboundary condition for items later serially recalled (t(15) = 2.70, p = .017) but not for those nonserially recalled (t(15) = 0.60, p = .558).

Given recent emphasis on functional segregation along the hippocampal long axis related to the granularity of information (Poppenk et al., 2013; Collin et al., 2015), we reasoned that the fine level of detail needed to perform serial recall might preferentially recruit mid and posterior regions of the hippocampus in contrast to the anterior hippocampal involvement in coarse temporal memory (Jenkins and Ranganath, 2010). Thus, we ran more exploratory follow-up analyses splitting the hippocampus into anterior, mid and posterior portions. In the mid hippocampus (Fig 2), we found a significant effect of boundary condition (F(1,15) = 9.41, p = .008) and a significant condition by recall type interaction (F(1,15) = 4.61, p = .048). The interaction was driven by a marginally significant serial recall effect in the boundary condition (t(15) = 1.98, p = .066) that was not significant in the preboundary condition (t(15) = -1.31, p = .209). The posterior hippocampus also showed a significant boundary condition effect (F(1,15) = 6.19, p = .025) driven by higher responses in the boundary versus preboundary condition (t(15) = 2.49, p = .025), but no interaction with recall type (F(1,15) = 2.51, p = .134). Neither the main effect of condition nor the interaction were present in the anterior hippocampus (main effect: F(1,15) = 1.62, p = .222; interaction: F(1,15) = 0.18, p = .674). These results suggest that the mid hippocampal region may be important for sequential binding across event boundaries.

Figure 2.

Mid-hippocampal activation by boundary condition and subsequent serial recall. Timecourses were extracted from the middle third of individuals’ hippocampi. Responses were estimated using an FIR model. Left) Preboundary condition. No differences were found between encoding responses for subsequent serial recall versus nonserial recall. Right) Boundary condition. A marginal difference was found between encoding responses for subsequent serial recall versus nonserial recall in the boundary condition. There was a significant interaction between boundary condition and subsequent recall type. Ribbons reflect SEM at each TR.

Whole-brain Univariate Results

We next performed an exploratory whole-brain analysis to test whether other regions in the brain show activation related to subsequent serial recall across boundaries as well as within events. In the boundary condition, the contrast of serial recall versus nonserial recall revealed a large region in left vlPFC extending into the anterior temporal lobe with two peaks (X = −50, Y = 10, Z = 12, 342 voxels; X = −56, Y = 8, Z = −6, 79 voxels, Fig 3). However, the same contrast in the preboundary condition did not reveal any significant clusters. We additionally probed the region that showed a recall effect in the boundary condition, but no effect was observed for the preboundary condition (t(15) = 1.17, p = .260). Thus, while we found that univariate activation in left vlPFC evoked during the encoding of boundary items is related to successful binding of across-event information, no significant univariate effect was observed for items within events.

Figure 3.

Statistical parameter map of the whole-brain contrast of serial versus nonserial recall in the boundary condition. Surface rendering (left) and coronal slice (right) on the group averaged normalized brain. P < .05, whole-brain corrected.

Whole-brain PPI Results

The previous results revealed that activation in mid hippocampus and left vlPFC at event boundaries is related to whether boundary items would be serially recalled. However, the univariate analyses did not find any regions that showed significant recall effects within events (i.e., serially recalling a preboundary item).

One possibility is that the more stable within-event context might promote a greater role for functional coupling between relevant encoding-related and context or goal maintenance regions. To test this, we performed a PPI analysis with the hippocampus as a seed region and contrasted the interaction terms between serial and nonserial recall within each condition separately. For the boundary condition, no significant clusters emerged. Conversely, the same contrast for the preboundary condition revealed clusters in the ventromedial PFC (vmPFC) and ventral cingulate regions (X = 2, Y = 36, Z = −16, 110 voxels, Fig 4; X = −12, Y = 28, Z = −10, 95 voxels; X = 12, Y = 24, Z = −6, 38 voxels; X = −6, Y = 18, Z = − 12, 27 voxels). We additionally probed whether these clusters would show a recall effect in the boundary condition, but no difference between serial and nonserial recall was observed (t(15) = 0.09, p = .933).

Figure 4.

Seed based PPI functional connectivity results. Top) Probabilistic bilateral hippocampal mask used to define anatomical hippocampus in each subject’s native space to be used as the seed region. Bottom) Statistical parameter map of the PPI contrast for serial versus nonserial recall in the preboundary condition displayed on sagittal slices on the group averaged normalized brain. P < .05, whole-brain corrected.

Thus, within-event binding was associated with greater functional coupling between the hippocampus and vmPFC, but binding across boundaries was not associated with any differences in coupling with the hippocampus. Together with the univariate data, these results suggest that contextual stability may bias associative binding to rely more on functional interactions between encoding-related regions, whereas binding across boundaries may preferentially recruit encoding-related regions operating independently. However, it is important to note that while each analysis type only showed significant effects in one of the two conditions (i.e., univariate effects in the boundary condition and connectivity effects in the preboundary condition), the null effects in the other conditions do not imply that the conditions differed statistically. Thus, it is unlikely that the univariate and connectivity effects represent two entirely distinct encoding mechanisms.

Discussion

In the present study, we investigated the neural mechanisms supporting the encoding of contiguous representations that are later recalled in the correct serial order. We compared conditions in which consecutive items’ encoding context either changed or remained the same. Behaviorally, we found that serial transitions to boundary items, or items at which the category and task context changed, were reduced compared to within-event transitions. Neurally, univariate activation in the mid hippocampus and left vlPFC was related to later serial recall for boundary items. While no significant effects were found for the same contrast in the within-event condition, functional connectivity analyses revealed that coupling between the hippocampus and vmPFC was associated with later within-event serial recall. This suggests that activation evoked at the time of the boundary in the mid hippocampus and left vlPFC may support mnemonic binding with prior event representations, whereas stable, within-event contexts may promote a role for network interactions, specifically between the hippocampus and vmPFC, in serial encoding.

Many recent studies have demonstrated hippocampal involvement in encoding and retrieval of sequential associations (see Davachi & DuBrow, 2015 for review). Thus, we predicted that the hippocampus would show subsequent serial recall effects in the present study. However, rather than finding a main effect of memory in hippocampal activation across both conditions, we found a specific effect for the boundary condition. This is consistent with prior work showing that hippocampal pattern similarity is related to mnemonic judgments of temporal proximity only when the items spanned a boundary (Ezzyat and Davachi, 2014). Relatedly, studies have found that introducing a temporal gap between to-be-associated features enhances hippocampal subsequent memory effects (Staresina and Davachi, 2009; Hales and Brewer, 2010). Thus, similar mechanisms in the hippocampus may support binding across temporal gaps and event boundaries.

Studies investigating levels of encoding have consistently implicated the hippocampus in encoding that supports more associative or contextual memory retrieval (see Davachi, 2006; Eichenbaum, Yonelinas, & Ranganath, 2007 for review). In one study that compared free recall to associative and item recognition, regions in the bilateral mid hippocampus, similar to the anatomical ROI used in the present study, were found to show graded encoding effects such that free recall was associated with the highest activation and item recognition with the lowest (Staresina and Davachi, 2006). In addition to the hippocampus, two regions in the left vlPFC also showed this graded effect, consistent with the whole-brain results reported here. Thus, one possibility is that the hippocampus and left vlPFC show strength of encoding effects and are implicated in the present study because stronger encoding is required for successful serial recall. An alternative possibility is that both regions, which have also been implicated in retrieval (Spaniol et al., 2009), refresh (Johnson et al., 2005) and recall specifically (Kragel and Polyn, 2015), may be involved in retrieving the preceding across-event item such that the preboundary and boundary items may become linked. Consistent with this notion, it has been shown using a Sternberg working memory task that retrieval of just-presented items out of focal attention is associated with hippocampal activation (Oztekin et al., 2009). This suggests that the attentional shifting that occurs at event boundaries may necessitate a hippocampal retrieval operation to recover preboundary information (Swallow et al., 2011). It is important to note that the focus of the present paper is on neural processes that occur specifically at the time of binding the transitioned-to item (also see Murray and Ranganath, 2007; Hales and Brewer, 2010, 2011). We focused on this period because it corresponds to the time at which the boundary is signaled and thus the condition manipulation is relevant. However, while these results are consistent with a retrieval mechanism at the time of binding across a boundary, they do not preclude a role in serial encoding for the maintenance of the prior item during the delay period (e.g., Hales et al., 2009).

The lateral PFC has been implicated in episodic encoding by a wealth of patient and neuroimaging studies, and one major subdivision that has emerged is between dorsal and ventral lateral PFC (Blumenfeld and Ranganath, 2007). In particular, the vlPFC has been shown to support both item and associative encoding in a graded manner whereas the dlPFC has been linked with the manipulation and organization of items in episodic memory. Consistent with this, Long & Badre (2010) found that vlPFC activation during encoding was associated with later successful free recall while dlPFC activation was associated with semantic clustering in recall. In Staresina and Davachi (2006), the dlPFC was the only region that showed specific subsequent free recall effects while the vlPFC showed graded subsequent memory effects. In the current study, unlike in these prior studies, we identified subsequent serial recall effects, which involve strong temporal organization, that were limited to the vlPFC. While this may appear inconsistent with the previous studies, it is not clear that temporal and semantic organization would be mediated by the same mechanisms (Morton and Polyn, 2016). Furthermore, there is some evidence suggesting that the vlPFC in particular contributes to episodic recall. For example, in one DTI study, connectivity between medial temporal lobe cortex and vlPFC but not dlPFC was correlated with recall performance (Schott et al., 2011). Moreover, in an fMRI study of event segmentation in memory, the left vlPFC showed boundary-evoked activation that correlated with cued recall of boundaries (Ezzyat and Davachi, 2011), consistent with the present findings. Thus, the vlPFC may be particularly important for encoding salient events. In line with this idea, vlPFC has been implicated in deeper levels of processing during encoding (Schott et al., 2004, 2013).

Interestingly, we did not observe a univariate activation signal in hippocampus, nor elsewhere in the brain, that was related to subsequent serial recall within events. While caution is warranted in interpreting a null effect, one possibility is that the stable context within events provides encoding support such that overall less engagement of the encoding network, as measured by univariate BOLD signal, is required. Consistent with this notion, we did find significant functional connectivity effects whereby hippocampal-vmPFC coupling was associated with serial encoding within events. In a potentially related line of work, connectivity with the vmPFC has been implicated in schema-related processing. For example, functional connectivity with the mPFC has been shown to be enhanced for schema-congruent retrieval, possibly mediating the integration of new information into existing schemas (van Kesteren et al., 2010). Moreover, recent work has shown that mPFC coupling with the hippocampus during new encoding is related to integration with prior associations (Schlichting and Preston, in press). Our present findings extend this work by showing that hippocampal-vmPFC coupling within a stable context may promote sequential binding. Together, this raises the possibility that the vmPFC may be involved in maintaining and integrating new information into the current active context, or event model, for facilitated encoding (see Ranganath & Ritchey, 2012). It is important to note, however, that these neural dynamics may change over the course of learning. Indeed, a statistical learning study of community structure found that, after repeated exposures, hippocampal-vmPFC coupling was enhanced at community boundaries despite both regions showing reduced overall activation at boundaries (Schapiro et al., 2015). Thus, further investigation is needed to understand the roles of the hippocampus and vmPFC in encoding under conditions that manipulate novelty, stability and predictability.

In the present study, event boundaries were defined by a change in both the stimulus category and encoding task. Thus, either the category or task switch may be sufficient to drive the effects of boundaries on memory. For example, the domain dichotomy view of associative memory posits that MTL cortex can support within-domain associations, but the hippocampus is necessary to bind between domain associations (Mayes et al., 2007). Thus, the hippocampal effects observed here may be related to the across-category nature of the boundary transitions. On the other hand, the task switch could be the primary source of the boundary effects reported here. Indeed, a prominent model of episodic memory posits that source features, including one’s task when encoding a stimulus, are critical components of the context representation that becomes bound to that item (Polyn et al., 2009). Thus, the model predicts that task shifts perturb context such that items encountered in different tasks would be more weakly associated in memory, consistent with the present results. Neurally, fMRI and patient work has implicated the lateral frontal and anterior cingulate cortices in task switching (Badre and Wagner, 2006; see Sakai, 2008 for review). Interestingly, updating task sets has been associated with posterior vlPFC activation, particularly when the switch is exogenously cued (Brass and von Cramon, 2004; Forstmann et al., 2005). This is consistent with the present results, which show that a similar region of left posterior vlPFC is activated during exogenously cued event boundaries specifically when they are bound to the preceding item or event. It is important to note that the task switching and memory goals were made very explicit in the present study, and thus the cognitive and neural processes engaged may differ from naturalistic encoding. Future work will be needed to determine whether the same processes are engaged during implicit memory tasks with more incidental context shifts.

Highlights.

• Serial recall transitions are reduced across context boundaries versus within events.

• Hippocampus and vlPFC show activation at boundaries related to later serial recall.

• Within-event serial recall is associated with enhanced hippocampal-vmPFC coupling.