The influence of context boundaries on memory for the sequential order of events

Abstract

Episodic memory allows us to re-experience the past by recovering the sequences of events that characterize those prior experiences. Interestingly, although experience is continuous, we are able to selectively retrieve and re-experience more discrete episodes from our past, raising the possibility that some elements become tightly related to each other in memory, while others do not. The current series of experiments was designed to ask how shifts in context during an experience influence how we remember the past. Specifically, we asked how context shifts influence our ability to remember the relative order of past events, a hallmark of episodic memory. We found that memory for the order of events was enhanced within, as compared to across, context shifts, or ‘boundaries’ (Experiment 1). Next, we showed that this relative enhancement in order memory was eliminated when across-item associative processing was disrupted (Experiment 2), suggesting that context shifts have a selective effect on sequential binding. Finally, we provide evidence that the act of making order memory judgments involves the reactivation of representations that intervened, or bridged, the tested items (Experiment 3). Together, these data suggest that boundaries may serve to parse continuous experience into sequences of contextually related events and that this organization facilitates remembering the temporal order of events that share the same context.

Introduction

The ability to “jump back in time” and relive events from our past has been named episodic memory (Tulving, 2002). Episodic memories contain rich detailed representations of the what, where and when of our past experiences, allowing us to retrieve the content and context of these prior events as well as to remember the order of events. However, it is also clear that when we remember the past, we can recreate some aspects of the temporal structure of our experiences while other aspects remain harder to retrieve. This raises the important question about how our experiences are organized in memory such that some elements are represented together while others are not. Since experience unfolds in a continuous manner over time, how does this continuous flow of information become discretized such that distinct episodes can later be selectively retrieved?

One account describing how experiences are segmented during online processing is called Event Segmentation Theory (EST; Zacks, Speer, Swallow, Braver, & Reynolds, 2007). EST proposes that, as we experience the world, we are guided by mental models that allow us to make predictions about the immediate future (Reynolds, Zacks, & Braver, 2007; Zacks et al., 2007; Zacks, Tversky, & Iyer, 2001). According to EST, these mental models, or event models, are updated at transition points called ‘boundaries’ that result from environmental changes. Interestingly, prior work has shown that event boundaries can have a lasting influence on how those events are later remembered. For example, a temporal shift in a narrative can result in the reduced mnemonic binding of narrative content that spanned the shift (Ezzyat & Davachi, 2011; Zwaan, 1996; Zwaan, Langston, & Graesser, 1995). Likewise, an influential set of associative memory models proposes that items become bound in memory through shared context, such that temporal distance and changes in other contextual features, such as task set, can reduce associative binding between items (Howard & Kahana, 2002; Polyn, Norman & Kahana, 2009a; Polyn, Norman, & Kahana, 2009b). Thus, in these models, context shifts essentially separate elements of experience, much like boundaries in EST.

Whereas boundaries have been shown to have an impact on later associative memory when probed using free recall or cued recall, the consequences of event segmentation on other components of episodic memory remain unclear. In particular, temporal order memory, or memory for when events occurred relative to one another, may, in some cases, benefit from the distinctiveness in memory afforded by context boundaries. Specifically, research across various paradigms and species has shown that recency discrimination, or memory for which of two items was presented most recently, improves with greater temporal distance between the tested items (Fortin, Agster, & Eichenbaum, 2002; Fuhrman & Wyer, 1988; Gower, 1992; Lockhart, 1969; St. Jacques, Rubin, LaBar, & Cabeza, 2008; Templer & Hampton, 2013; Yntema & Trask, 1963). This suggests that recency discrimination can be based on a comparison between the strength of individual items in memory, which should be more distinct, and thus easier to discriminate, the farther apart two items are encountered (e.g., see Friedman, 1993). Thus, if boundaries induce a shift in context, items encountered on either side of the boundary may be represented with greater distance in memory, and recency decisions on these items should be facilitated. However, an alternative mechanism that might support order memory judgments is the recovery of direct associations between neighboring items that are laid down during encoding (e.g., Lewandowsky & Murdock, 1989). This ‘chaining’ account predicts that memory for the temporal order of two presented items would be better within an event than for items encountered across a context shift, since boundaries have been shown to weaken item-item memory (Ezzyat & Davachi, 2011; Zwaan, 1996; Zwaan, Langston, & Graesser, 1995).

In the present series of behavioral experiments, we examined the influence of context boundaries on later temporal order memory by testing recency discrimination within and across boundaries. On the one hand, if recency discrimination is facilitated by the recovery of sequential associations, or item-item links, then boundaries that segment the tested events should be disruptive. On the other hand, if order memory is supported by direct comparisons of item strength, then segmentation should create greater mnemonic ‘distance’ between those representations and, thereby, facilitate recency discrimination. Thus, in essence, we aimed to test what type of mnemonic information -- individual item information or sequential associations -- is used to support judgments of the temporal order of previously experienced events, a critical component of episodic memory.

Experiment 1

Experiment 1 was designed to investigate the effect of boundaries on temporal order memory when explicit item-to-item associative encoding was encouraged. Participants studied lists consisting of famous faces and common objects and were instructed to make different decisions for the two categories of stimuli. Thus, boundaries were defined as a shift from one category to another and also required a shift in the task performed on each item. Temporal order memory was tested in two different ways in order to test both direct item-item sequential associations (via serial recall) and relative order memory (via recency discrimination). We predicted that serial recall across boundaries should be relatively impaired based on prior work showing reduced cued recall to narrative event boundaries (Ezzyat & Davachi, 2011). However, we hypothesized that the effect of boundaries on recency discrimination would depend on whether item strength or sequential associative information was used to resolve temporal order. To address this, we compared performance on recency discrimination for pairs of items that were presented within boundaries versus across boundaries. Thus, the effect of intervening boundaries on order memory was always relative to order memory for items that were presented within surrounding boundaries.

Method

Participants

23 right-handed native English speakers with normal or corrected vision participated in the study. Consent was obtained in a manner approved by the Institutional Review Board at New York University. One participant was excluded due to chance-level performance (thus, N = 22).

Design and materials

280 color images each of celebrity faces and nameable common objects were modified from stimulus sets used in previous studies (Kuhl, Rissman, Chun, & Wagner, 2011; Polyn, Natu, Cohen & Norman, 2005), and obtained from various online sources. Familiar stimuli were used to maximize temporal order memory and promote the role of temporal context in the task (Polyn, Norman & Kahana, 2009a). The same stimulus set is used for all the experiments reported here.

16 study lists were each composed of 25 unique images including two fillers on the ends of the list. Each list contained five boundaries in which both the category and task switched, and category ‘trains’ that were either short (2 items) or long (7 items). The recency discrimination task consisted of five conditions. The two main conditions of interest were test pairs (items from study list) that had been presented with three intervening images all from the same category train (‘lag-3 no switch’) or had been presented across two category switches (‘lag-3 switch’, see Fig. 1). The switch condition consisted of two switches in order to always test pairs drawn from the same stimulus category (i.e. either two faces or two objects). In addition to the no switch and switch conditions, the effect of distance on recency performance was also assessed. Neighboring items from the same category train (‘neighbors’) and same-category items presented across 7 to 9 intervening images that included switches (‘long’) were also tested. Finally, two-thirds of the test pairs only had one intervening switch (between-category), and were included so as not to discourage sequential binding across switches.

Figure 1.

Experimental procedure for a single study-test round in the three reported experiments. During encoding subjects studied lists of images of faces and objects and made category-specific judgments. Following a short distractor task, subjects performed recency discrimination on pairs of images from the preceding list. The main conditions of interest were no switch (depicted in blue) and switch (depicted in orange). In Experiment 3, a recognition probe immediately followed each recency discrimination trial. The main conditions of interest were intervening probes (dark color), which occurred between the just discriminated images in the study list, and preceding control probes (light color), which occurred just prior to the more recent image of the just discriminated pair.

With the exception of the long-lag and the between-category filler condition, boundary items were not tested in recency discrimination to prevent potential differences in their item strength from influencing temporal order judgments. For the long-lag condition, half of the items tested were boundary items. In this condition, recency discriminations were always made between two boundary or nonboundary items, and no difference in accuracy was observed between them (t(21) = 0.70, p = .49). The lag-3 no switch and switch conditions were matched for serial position across lists. For the other distance conditions, which could not be perfectly matched for list position due to their different lengths, mean list positions of the image pairs were matched within list fifths. Each condition of interest was tested twice in each list, and the order of test trials was randomized.

A final recognition memory test was also administered. It contained five boundary and five preboundary items (items immediately preceding boundary items) from each list that had each been tested in recency discrimination. An equal number of novel foils were also selected. Stimulus assignment to each condition was randomized for each subject.

Procedure

The experiment was composed of 16 study-test rounds. During each study phase, participants were presented with a series of 25 images with their corresponding verbal label. Participants were instructed to create associations between consecutive images in the list, either through narrative construction or vivid mental imagery. Additionally, after each 2 s image presentation, a prompt appeared for an additional 2 s instructing participants to make a simple, category-specific response. After each celebrity face, the prompt displayed “male/female?” and participants made a gender discrimination. For each object, the prompt displayed “bigger/smaller?” and participants were instructed to make a size judgment relative to a 13-inch shoebox. Thus, as noted above, every boundary event was a category switch as well as a task switch event. The category-specific task judgments were used to maximize contextual differences between the image categories. Indeed, prior work has shown that changes to both the stimulus class and task influence the organization of memory recall (Polyn et al., 2005).

After each study phase, participants performed an odd/even task for approximately 45 s before the recency test phase. Each recency test trial consisted of two images randomly assigned to each side of the screen. The prompt “more recent?” appeared, and participants were instructed to select the image that appeared more recently and indicate their confidence in a single judgment by making one of four responses (high and low confidence for each of the two images) with their right (dominant) hand. Participants had a maximum response time of 8 s per trial. Each study and test trial was followed by a variable inter-trial interval (ITI) ranging from 6 – 10 s.

Following recency discrimination, participants were instructed to perform serial recall. Their responses were recorded on a laptop running the audio program Audacity (http://audacity.sourceforge.net, © 2005 Audacity development team, distributed under the GNU General Public License). Specifically, participants were instructed to recall the names of the images from the preceding list in the order in which they had been presented. If they failed to recall an image, they were instructed to proceed to the next remembered image. They were given up to 90 s to complete serial recall for each list.

After the last study-test round, participants performed a final recognition memory test. 320 images were tested, half of which were novel items and half of which had been seen twice – once during study and once during recency discrimination. Participants were given up to 6 s to make one of six responses – “high confidence”, “low confidence” or “guessing” for each “old” or “new” judgment.

Results

Encoding

The average response time (RT) on the encoding task was 945 ms (SE = 59 ms). RTs to boundary items were significantly slower than those to nonboundary items during encoding (t(21) = 4.46, p < 0.001). There was no overall effect of judgment/stimulus type on RTs (faces versus objects, t(21) = 0.08, p = 0.94).

Recency discrimination

Overall performance on recency discrimination was 0.76 (SE = 0.02). A comparison between the switch and no switch pairs revealed that recency discrimination was significantly lower for the switch pairs (lag-3 no switch > switch, t(21) = 2.62, p < 0.05, Cohen's d = 0.04; see Fig. 2). We also examined the effect of distance and found a significant benefit for greater distance (long > neighbors, t(21) = 4.38, p < 0.001, Cohen's d = 0.07). Furthermore, each condition pair matched for whether there were intervening context boundaries also revealed a distance enhancement (lag-3 no switch > neighbors, t(21) = 3.49, p < .01; long > lag-3 switch, t(21) = 4.03, p < .001). A repeated-measures ANOVA with condition and confidence as factors revealed no interaction between condition and confidence (F(3,63) = 0.15, p = 0.93). The average RT for correct trials was 3.4 s (SE = 0.1 s), and there was no significant modulation of response time by condition (F(3,63), p = 0.37).

Figure 2.

Proportion correct on recency discrimination in Experiment 1. Error bars represent the standard error of the mean across subjects. * p < 0.05

Serial recall

On average, participants recalled 63% of the items from each study list (SE = 6%). There was no difference in the average number of objects versus faces recalled (t(21) = 0.27, p = 0.63), and no difference in boundary versus preboundary item recall (t(21) = 0.05, p = 0.96) nor for boundary versus other nonboundary control items (t(21) = 0.15, p = 0.88).

Next we adopted a strict serial scoring of the recall data in which only direct transitions between consecutive items are considered accurate. Of those items that were recalled, 39% (SE = 6%) were recalled in the correct order, that is, directly following recall of the preceding item. Interestingly, while no difference was observed in overall recall for boundary and non-boundary items, accurate serial transitions to boundary items were significantly less likely. Specifically, when controlling for the item recall status of the pre-transition and post-transition items, we observed significantly fewer transitions to boundary items from preboundary items compared to transitions from other nonboundary items to preboundary items (t(21) = 2.37, p < 0.05, Cohen's d = 0.014; t(21) = 2.37, p < 0.05, Cohen's d = 0.018, respectively; see Fig. 3). Thus, subjects were less likely to recall a boundary item immediately after the preboundary item even when both are recalled at some point during the recall test.

Figure 3.

Proportion correct transitions in serial recall contingent on item recall of the pre-transition and post-transition items in Experiment 1. Error bars represent the standard error of the mean across subjects. * p < 0.05.

Final recognition

The average hit rate on the final recognition memory test was .90 (SE = .02) with an average false alarm rate of 0.08 (SE = 0.01). We observed no effect of boundary status on hit rates (boundary vs nonboundary items, t(21) = 0.49, p = 0.63).

Discussion

The results of Experiment 1 show that when instructed and given sufficient time to link successive items, recency discrimination is relatively impaired across boundaries. Additionally, correct serial recall transitions were more likely within single-category trains than across boundaries. These data suggest that boundaries parse continuous sequences into discrete segments within which order is better maintained in memory. Furthermore, the parallel effects of context boundaries on recency discrimination and serial recall support the notion that associations between items in a sequence may be used in resolving relative temporal order.

Our results, also, suggest that nonassociative information was also used in recency discrimination. Specifically, the lag distance manipulation revealed that temporal order memory performance increased with greater temporal lags. That is, participants performed best in the long-lag condition and worst in the neighbors condition – an effect that cannot be accounted for by sequential linking. Instead, the distance effect suggests that comparisons of item strength may also be used in this task to make recency judgments, perhaps specifically when intervening associative links are not accessible, which may be more likely as the lag, or the number of intervening associations, increases.

Thus, from the current results we can conclude that, under conditions of associative encoding where sequential links between studied items are being forged, recency discrimination of pairs with three intervening items is better for those items studied in the same category train compared to those with intervening boundaries. However, we do not know if the same effect would be evident when associative links between sequential items are difficult to form and retrieve. The next experiment was designed to test this.

Experiment 2

The goal of Experiment 2 was to assess whether the influence of context boundaries on order memory is specific to judgments based on associative information or, instead whether the same effect would be evident when item-focused encoding is emphasized. To test this, we modified our paradigm to interfere with sequential binding during encoding by inserting a highly demanding arithmetic task between each study item in a continual distractor design (e.g., Bjork & Whitten, 1974; Poltrock & MacLeod, 1977). We additionally encouraged item-level processing by instructing participants to make more complex item-oriented encoding judgments and shortened the amount of time allotted to make recency judgments to limit the ability to engage in associative retrieval.

Method

Participants

27 right-handed native English speakers with normal or corrected vision participated in the study. Consent was obtained in a manner approved by the Institutional Review Board at New York University. Five participants were excluded for chance-level performance on either the memory or the arithmetic task (thus, N = 22).

Design and materials

Same as Experiment 1

Procedure

The task was similar to that of Experiment 1 except for several modifications intended to reduce participants’ reliance on sequential associations. First, we increased the encoding task difficulty by requiring participants to make a 4-point subjective judgment on each item. Specifically, participants were instructed to judge the likability of celebrity faces and the frequency of the objects. Second, a difficult arithmetic task was added between image presentations during the study phase to reduce continued processing and elaboration of study items between item presentations. The arithmetic task consisted of two division problems and the subtraction of their quotients. Participants were given a single problem between every image with up to 5 s to complete it. This same task was also performed during the 45 s study-test intervals, after which participants received feedback on their arithmetic performance. Third, the maximum response time for recency discrimination was reduced to 3 s, just under the average correct response time from Experiment 1. Finally, no associative encoding strategy was encouraged in the instructions and the serial recall phase was removed altogether, since we expected that a serial recall test would encourage associative encoding strategies in subsequent lists.

Results

Encoding

Average RTs on the encoding task were 512 ms (SE = 37 ms) and no significant RT differences between boundary and nonboundary items were found at encoding (t(21) = 1.46, p = 0.16). There was also no effect of judgment/stimulus type on RTs (faces vs objects, t(21) = 0.03, p = 0.98). On average, accuracy on the intervening arithmetic task was .75 (SE = .03).

Recency discrimination

Overall performance on recency discrimination was 0.70 (SE = 0.01). Interestingly, a direct comparison revealed no significant difference between switch and no switch pairs (t(21) = 1.15, p = 0.26, see Fig. 4). Again, we examined the distance effect by comparing the ‘long’ condition to the ‘neighbors’ condition, and found a significant benefit for distance (long > neighbors, t(21) = 6.59, p < 0.001, Cohen's d = 0.13). Each pair matched for switch status also showed a distance enhancement (long > switch, t(21) = 4.46; no switch > neighbors, t(21) = 3.47; p's < 0.005).

Figure 4.

Proportion correct on recency discrimination in Experiment 2. Error bars represent the standard error of the mean across subjects. * p < 0.05.

Unlike the first experiment, here we found that recency discrimination RTs were significantly modulated by condition (F(3,63) = 13.32, p < 0.001). Paired t-tests revealed no effect of switches (t(21) = 0.31, p = 0.76), but a significant effect of distance on RTs was observed. This was driven by an overall reduction in RTs to the ‘long’ pairs (long < neighbors, t(21) = 4.50, p < 0.001; long < lag-3 switch, t(21) = 2.60, p < 0.05; no significant difference between lag-3 no-switch and neighbors, t(21) = 1.51, p = 0.15).

To directly examine the influence of encoding orientation (item versus associative) on boundary effects in temporal order memory, we compared accuracy across the two experiments, for which the list lengths and lag-distances tested were matched. A two-sample t-test revealed a significant reduction in overall recency discrimination in Experiment 2 compared to Experiment 1 (t(42) = 2.57, p < .05, Cohen's d = .05). This decrement in performance suggests that the availability of precise temporal information was reduced. Interestingly, the distance effect, as measured by the difference in performance between the ‘long’ and ‘neighbors’ conditions, was significantly larger in Experiment 2 (t(42) = 2.21, p < .05, Cohen's d = .05). This increase in the magnitude of the distance effect is consistent with a shift from the use of sequential associations to item strength information.

Final recognition

Due to equipment failure, data from the final recognition memory test was lost for one subject, thus N = 21 for this analysis. The average hit rate on the final recognition memory test was 0.95 (SE = 0.01) with an average false alarm rate of 0.06 (SE = 0.01). As with Experiment 1, we observed no effect of boundary status on hit rates (boundary vs nonboundary items, t(20) = 1.06, p = 0.30).

We compared corrected recognition scores between experiments to test whether the difference in task and encoding orientation had an effect on item memory. Interestingly, a two-sample t-test revealed a significant improvement in recognition memory for Experiment 2 (t(41) = 2.30, p < 0.05, Cohen's d = 0.04). Thus, although temporal order memory was reduced in this experiment, item recognition was actually enhanced.

Discussion

The results of Experiment 2 indicate that boundaries did not produce a significant reduction in recency discrimination when associative processing was disrupted. Furthermore, the reduction in overall recency discrimination performance between Experiments 1 and 2 suggests that the accuracy of temporal order memory is related to the availability of sequential associations in memory. This result supports the notion that temporal order memory benefits from sequential binding. By contrast, the increased distance effect observed in Experiment 2 suggests the absence of associative information may result in an increased reliance on comparisons based on item strength. To bolster this interpretation, we observed that item recognition memory was significantly better here than in the previous experiment, suggesting that participants indeed focused on item-level processing and, importantly, that the reduction in temporal memory performance was not simply a result of generally worse memory. This pattern of results is consistent with the interpretation that boundaries selectively disrupt sequential associations while leaving item memory intact.

One point to note is that in this design, we attempted to reduce associative processing both at encoding, by interfering with binding across a delay, and at retrieval, by reducing the allotted response time for recency discrimination. Thus, we are not able to disentangle the effect of boundaries on associative processing at encoding versus retrieval. The last experiment was designed to partially address this question by testing whether intervening items are reactivated during order memory decisions.

Experiment 3

The results of the first two experiments suggest that, under encoding conditions designed to promote sequential binding, context boundaries appear to have a lasting influence on the way in which items on either side of the boundary are linked. Specifically, participants are less likely to be able to discriminate the temporal order of items encountered across boundaries compared to those encountered between boundaries. One implication of this finding is that, sequential binding is enhanced for those representations encountered within the same context, which, in turn, may facilitate the reactivation of those intervening representations from memory during recency discrimination of the two test items.

In the current experiment, we aimed to directly test this hypothesis by asking whether intervening representations are indeed reactivated, or more accessible, when they were encoded as part of the same category train or event than when a boundary intervened. Specifically, we measured the priming of intervening representations immediately after each recency discrimination trial. This would allow us to measure if non-presented items were made more accessible simply by making recency judgments on the test items and, if so, whether boundaries encountered during encoding modulate the later priming of these representations.

Method

Participants

30 right-handed native English speakers with normal or corrected vision participated in the study. Consent was obtained in a manner approved by the Institutional Review Board at New York University. Seven participants were excluded for chance-level performance on the recency discrimination memory test, one participant for falling asleep and one participant for reporting having previously seen the stimuli (thus, N = 21). Equipment failure for two additional participants resulted in the loss of one block for each participant. There were no differences in the reported effects when excluding the latter data.

Design and materials

The design was similar to that of the previous experiments with some modifications to test for recognition priming of intervening items following each recency discrimination attempt. The main conditions of interest were no switch and switch recency trials immediately followed by either an intervening recognition probe or a control recognition probe. Intervening probes were items that came from the list position immediately preceding the more recent item of the test pair, and thus occurred in the intervening sequence that spanned the test items. Control probes came from the list position immediately preceding the less recent item of the test pair. The list position and boundary status of both the recency test pairs and recognition probes were matched. In order for the intervening recognition probes in the switch condition to be matched in terms of their boundary status to the no switch condition, we reduced the length of the switch category train to a single item.

In addition to testing an old intervening and control probe for each switch condition once in each test phase, we also tested recognition memory for novel probes following three recency discrimination trials. Finally, we also included two different-category recency test pairs so as not to discourage across-category binding during subsequent study lists.

Procedure

The task was similar to that of Experiment 1 with the critical addition of a recognition probe presented after each recency discrimination trial. We also eliminated the subsequent serial recall test. Participants were instructed to make recency responses with their left (non-dominant) hand. The recognition probe then appeared .5 s after each recency decision and participants were instructed to respond “old or new” with their right (dominant) hand as quickly as possible. Then a fixation cross was presented for 1 s before the next recency trial. Each test phase consisted of eight of these alternating recency discrimination and recognition test trials.

Results

Encoding

The average RT on the encoding task was 658 ms (SE = 68 ms). No significant RT differences between boundary and nonboundary items were found (t(20) = 0.66, p = 0.51). There was also no effect of judgment/stimulus type on RTs (faces vs objects, t(20) = 0.87, p = 0.39).

Recency discrimination

Overall performance on recency discrimination was .70 (SE = .02). Planned comparisons revealed no significant effect of context switching when including all trials (t(20) = 0.27, p = 0.79). However, there was a marginal enhancement of the no switch condition for high-confidence recency discrimination (t(20) = 1.93, p = .07, Cohen's d = .05). A two-way repeated measures ANOVA revealed a significant interaction between condition and response confidence (F(1,20) = 6.84, p < .05) that was driven by the enhancement of the no switch condition for high-confidence responses and a non-significant enhancement of the switch condition for low-confidence responses (switch > no-switch, t(16) = 1.43, p = 0.17). Thus, boundaries appear to impair order memory judgments here as in Experiment 1, but only for the subset of trials for which subjects reported high confidence in their temporal order memory.

The average RT for recency discrimination was 3.13 s (SE = 0.27 s). Importantly, there was no significant effect of condition on RTs (F(1,20) = 1.88, p = 0.19).

Immediate recognition

The mean overall hit rate for the recognition memory probes was 0.96 (SE = 0.01) and the false alarm rate was 0.02 (SE = 0.01). The mean RT to all correct hits on recognition probes was 1.56 s (SE = 0.10 s). Importantly, we focused our analysis of RTs on recognition trials that followed only accurate and high confidence recency discrimination on both the no switch and switch conditions to avoid the possibility that any differences seen in the priming of those probes are due to differences in objective and subjective measures of memory for the preceding, temporal order memory judgment.

As predicted, planned comparisons revealed a significant reduction in recognition RTs for intervening probes relative to preceding probes following no switch recency discriminations (t(20) = 2.34, p < 0.05, Cohen's d = 0.04; see Fig. 5). This suggests that the accessibility of an intervening item was facilitated compared to a control item. Importantly, priming of intervening representations was not seen for the switch condition (t(20) = 0.98, p = 0.34), suggesting that recency discrimination of items encountered across boundaries was not accompanied by the same level of reactivation of intervening items.

Figure 5.

Response times on recognition decisions immediately following recency discrimination in Experiment 3. Bars represent response times for trials following high confident correct recency discrimination. Error bars represent the standard error of the mean of the difference between intervening and preceding items within condition. * p < 0.05.

Discussion

The results of Experiment 3 demonstrate that events become structured during encoding in a manner that not only influences our ability to actively remember and resolve temporal order but further appears to influence what representations become reactivated, perhaps more automatically, during retrieval. Specifically, the increased accessibility of intervening representations that came from the same encoding event suggests that sequential representations are reactivated even when not necessarily needed during order memory judgments. This reactivation, as measured by priming, was specific and was only observed when temporal order memory was successful and confident, a result that complements earlier work showing a recognition priming effect for neighboring items specifically following high-confident recognition (Schwartz, Howard, Jing, Kahana, 2005). Furthermore, this reactivation was not seen for intervening representations that did not share the same encoding context (i.e., the switch condition), even when those decisions were made accurately and with high confidence.

In the current experiment, the overall effect of boundaries was only seen for high confidence trials, while in the first experiment the effect was seen irrespective of subjective confidence. This may be due to a slight change made in order to match the category and boundary status of the intervening probes that involved reducing the shortest ‘event’ to a single item rather than two items. Thus, the effectiveness of the boundary manipulation may be dependent on the length or the extent of the context shift. The way in which the length and content of boundaries influences segmentation in memory is poorly understood and future work will be needed to explore these questions.

General Discussion

In this set of experiments, we examined the influence of context boundaries during encoding on subsequent temporal order memory. Taken together, our results show that recency discrimination is more accurate for items encountered as part of the same context when associative processing is encouraged (Exp 1). Importantly, however, this effect is dependent on the availability of item-item associations in memory (Exp 2), which we showed may come online during successful order memory retrieval when tested items shared a context (Exp 3). Thus, event segmentation may modulate temporal order memory to the extent that those memory judgments are based on the reconstruction of sequential associations, perhaps by structuring events into related segments within which temporal order information is better maintained.

The reduction of both recency discrimination accuracy and serial recall across boundaries is consistent with prior work that has found impaired cued recall, reduced recognition priming and lower proximity ratings across narrative event boundaries (Ezzyat & Davachi, 2011; Zwaan, 1996; Zwaan, Langston, & Graesser, 1995), providing further support for a role for boundaries in structuring the relationship between items in long-term memory. One point to note is that in our data, we are only able to compare the across-boundary switch condition to the no switch, within-boundary condition. Without a third ‘baseline’ condition, our effects can only be interpreted relative to each other. Thus, we cannot say whether segmentation impairs across-boundary order memory or enhances within-boundary order memory in absolute terms, but rather that order memory is superior for items that occur within boundaries compared to items that occur across boundaries.

In our design, boundaries involved both a stimulus category and a task shift. Changes in stimulus category between faces and objects are likely to induce significant changes in neural activity, particularly in inferotemporal cortex (e.g., Logothetis & Sheinberg, 1996). Such changes in neural activity are hypothesized to be incorporated into one's representation of context (Polyn and Kahana, 2008), and thus may function as context boundaries. Additionally, task switching likely also contributes to the boundary effects reported here. However, while task switching has typically been associated with a reduction in many aspects of cognitive processing (e.g., Liefooghe, Barrouillet, Vandierendonck, & Camos, 2008), we found that context boundaries in our task were specifically associated with changes in the way items become bound to each other, but not in how well the boundary items themselves are later recognized or recalled. Since task switches occurred every time there was a category switch, we cannot disentangle the effects of category shifts from those of task switching in the current series of experiments. However, it is noteworthy that prior work has shown that task switching alone is sufficient to influence the organization of free recall (Polyn, Norman & Kahana, 2009b).

Importantly, and adding to the prior work on boundaries and long-term memory, we demonstrate that the effects of boundaries on temporal order memory are specific to conditions that encourage the construction of sequential associations during encoding. Specifically, in Experiment 2, the effect of boundaries on recency discrimination was eliminated when associative processing was disrupted. However, we found that the distance effect in recency discrimination and overall item recognition was enhanced compared to Experiment 1, suggesting that participants may have relied more on item-based memory when associative information was unavailable. Thus, our findings suggest that the effect of boundaries on the organization of memory representations may be specific to conditions in which the representations consist of sequential associations. Indeed, if one function of event segmentation on episodic memory is to parse long sequences into shorter, more meaningful subunits, binding the events that make up the sequence in an associative manner may be a precondition to segmenting the sequence into discrete episodes.

Beyond supporting the role of event segmentation in long-term temporal order memory, our findings additionally suggest that item-item sequential associations may play a more general role in order memory. Whereas some research has challenged the role of item-item associations in sequence memory (Botvinick & Plaut, 2006; Henson, Norris, Page, & Baddeley, 1996), recent serial learning studies have found patterns of recall that suggest sequential associations are utilized (Kahana, Mollison, & Addis, 2010; Solway, Murdock, & Kahana, 2012). Furthermore, a correlation between overall episodic memory performance and strong temporal associative clustering in free recall has been previously described (Sederberg, Miller, Howard, & Kahana, 2010). Thus, our findings that disrupting associative processing reduced order memory accuracy and, moreover, that sequential associations may be reactivated during recency discrimination add support to the role of item-item associations in temporal order memory.

The sequential associations that are utilized for making recency discriminations may be direct item-item associations as proposed by chaining theory (Lewandowsky & Murdock, 1989). However, our findings are also consistent with an associative account of order memory that links items in a sequence indirectly via their connections to a separate, dynamic construct such as temporal context (Howard, Fotedar, Datey, & Hasselmo, 2005; Howard & Kahana, 2002; Polyn, Norman, & Kahana, 2009a). Since retrieval of an item's associated context can theoretically facilitate the recognition of items bound to a similar state of context, temporal context models can account for our results from Experiment 3. One hint, however, that the associations retrieved during recency discrimination are more specific than purely context-based retrieval is the main finding from Experiment 3 that following no switch recency discriminations, recognition of the intervening item is facilitated more than the preceding item despite being associated with the same context, or category train. Nevertheless, future work will be needed to adjudicate between these two associative accounts.

In addition to associative information, the current data suggest that distance-based information may also be used in temporal order judgments. Each experiment showed a significant enhancement for recency discrimination with increasing temporal lags, which should only reduce associative memory strength between items. Additionally, under conditions of limited associative encoding in Experiment 2, distance effects were greater over the same temporal lags as Experiment 1, suggesting that participants may use coarse estimates of order, such as item strength, when more precise information is unavailable. It is worth noting, however, that while we approximately equated for the mean list position across lists, long pairs will always include items in earlier or later positions than short pairs. This is a concern because there is evidence that distance effects can be attributed to the increase in recency or primacy of long-lag items (Hacker, 1980; McElree & Dosher, 1993; Muter, 1979). Thus, the distance effect reported here might also be due to differences in the distance to the beginning or end of the list rather than the distance between the items per se. List positions were perfectly matched, however, for switch and no switch pairs. This and the equivalent memory for boundary and nonboundary items suggest that item strength differences are unlikely to account for the switch effect reported here.

Another important question addressed here is at what stage associative processing is important for temporal order memory. The recognition priming effect found in Experiment 3 suggests that intervening items were incidentally reactivated during recency discrimination, consistent with associative retrieval. Thus, our results support a retrieval-based account whereby sequential relationships can be reconstructed to resolve temporal order. Of course, these associations would need to be formed during encoding, and in the present studies we found that the effect of context boundaries on order memory may have been modulated by the difference in encoding orientation between Experiments 1 and 2. Thus, the effect of context boundaries on temporal order memory likely involves interactions between encoding and retrieval. Future work using functional imaging will help to elucidate the effects of boundaries on encoding and retrieval, as fMRI allows the examination of each stage separately (Davachi & Dobbins, 2008).

While we cannot address the neurobiological mechanisms of temporal memory in the present studies, the hippocampus has been shown to be important in relational memory encoding (Davachi, 2006; Davachi, Mitchell, & Wagner, 2003; Davachi & Wagner, 2002), and initial evidence points to a role for the human hippocampus in sequence encoding (Tubridy & Davachi, 2011), learning (Kalm, Davis, & Norris, 2013; Kumaran & Maguire, 2006; Paz, Gelbard-Sagiv, Mukamel, Harel, Malach, & Fried, 2010; Schapiro, Kuster, & Turk-Browne, 2012) and retrieval (Dudukovic & Wagner, 2007; Lehn, Steffenach, van Strien, Veltman, Witter, & Haberg, 2009). Furthermore, the necessity of the hippocampus for order memory tasks has been demonstrated in rodents (DeVito & Eichenbaum, 2011; Fortin et al. 2002; Kesner, Gilbert, & Barua, 2002). Thus, the hippocampus is likely important for building and later resolving the order of fine temporal sequences such as those employed in the present experiments. Future work will be needed to address important mechanistic questions about how hippocampal processing is altered in response to boundaries and how hippocampal and other neural mechanisms contribute to remembering the order of distinct events that occur across boundaries.