Memory’s penumbra: Episodic memory decisions induce lingering mnemonic biases

Abstract

How do we decide if the people we meet and the things we see are familiar or new? If something is new, we need to encode it as a memory distinct from already stored episodes, using a process known as pattern separation. If familiar, it can be used to reactivate a previously stored memory, using a process known as pattern completion. In order to orchestrate these conflicting processes, current models propose that the episodic memory system uses environmental cues to establish processing biases that favor either pattern separation during encoding or pattern completion during retrieval. To assess this theory, we measured how people’s memory formation and decisions are influenced by their recent engagement in episodic encoding and retrieval. We found that the recent encoding of novel objects improved subsequent identification of subtle changes, a task thought to rely on pattern separation. Conversely, recent retrieval of old objects increased the subsequent integration of stored information into new memories, a process thought to rely on pattern completion. These experiments provide behavioral evidence that episodic encoding and retrieval evoke lingering states that influence subsequent mnemonic processing.

When you walk into a café for the first time, your memory system can both encode the details of this new environment and allow you to remember a similar one where you recently dined with a friend. The often effortless way in which we can encode the present and remember the past belies the complexity of the underlying processes, however. While decades of theoretical and empirical research (1–5) have improved our understanding of both the neural systems and the computations that underlie episodic memory, the convergence of these lines of research reveals a paradox: Neuroscience research shows that both encoding new memories and retrieving old ones depend on the same specific brain region – the hippocampus (6–8), but computational models propose that encoding and retrieval are differentially supported by computationally incompatible network processes (9). Specifically, encoding is thought to rely on pattern separation, a process that makes overlapping representations more distinct, whereas retrieval is thought to depend on pattern completion, a process that increases overlap by reactivating related memory traces. Since a process that emphasizes overlap cannot simultaneously operate on the same representation as one that deemphasizes it, a potential resolution to the paradox is that the hippocampus can establish processing biases that favor either pattern separation or completion depending on the current context. This suggests that your likelihood of remembering your earlier lunch with a friend may be biased by the specific hippocampal processing you were engaged in, even before walking into the café. In fact, neurocomputational models have long hypothesized that neuromodulatory systems may dynamically bias hippocampal processing toward either pattern completion or separation (10–13). Here, we test a prediction derived from these models and provide empirical support for these biases in human behavior.

The crux of our approach lies in the relatively slow action of neuromodulators in the hippocampus (14). If switching between pattern completion and separation biases is, in fact, mediated by hippocampal neuromodulatory input, it follows that a processing bias should linger in time and, thus, influence subsequent mnemonic processing. To test this, we presented participants with pictures of novel and familiar objects and asked them to make old/new recognition decisions. According to models (10–13), detecting novelty should bias the memory system towards pattern separation to support distinctive encoding of the novel information, whereas recognizing that a stimulus was previously experienced should induce a pattern completion bias that supports retrieval of stored representations. We measured lingering biases by presenting subjects with critical test trials immediately following unrelated old and new memory decisions. We designed test trials across three different experiments to measure biases both at the time of retrieval decisions and during periods of encoding.

In the first two experiments, we measured biases in retrieval decisions using a modified continuous recognition paradigm that has previously been used to study pattern separation in the human hippocampus (15, 16). Participants were presented with a series of objects that fell into three categories: novel objects, repeated objects, or objects that were similar but not identical to previously presented ones. Participants were asked to identify each as new (first presentation), old (exact repetition), or similar (not exact repetition) (Fig 1A, see SOM for more task details). Similar trials served as our critical test trials to measure pattern separation/completion biases. Specifically, although participants (N=15) were instructed to respond ‘similar’ to similar objects, the differences were often quite subtle, so similar objects were sometimes mistakenly identified as ‘old’. Thus, we reasoned that if the memory system were already biased toward pattern completion, these similar stimuli would more often be incorrectly identified as ‘old’, whereas if the system were biased toward pattern separation, the likelihood of noticing the small differences would be increased. Hence, we looked for evidence for lingering biases in memory decisions by examining whether participants were more likely to correctly identify similar trials as ‘similar’ if they followed new trials rather than following old trials.

Fig. 1.

(A) Participants are presented with a series of objects and are asked to identify whether each item is new (first presentation), old (exact repetition), or similar (a modified object). The top sequence provides a preceding new example: the similar object follows a new object. The bottom sequence provides a preceding old example: the similar object follows an old object. (B) Results from Experiment 1. The left graph plots accuracy on similar trials as a function of the preceding response (“new in blue and “old” in red) (N=15). The bars on the right graph are further divided according to perceptual similarity ratings. (C) Results from Experiment 2. The left bar graph displays the preceding new benefit (preceding “new” – preceding “old”) for similar trial accuracy at 0.5, 1.5 and 2.5 s inter-stimulus interval (ISI) (N=52). The inset figure plots accuracy on similar trials that were preceded by new responses (blue) or old responses (red) at the three ISIs. The graph to the right plots the similarity rating interaction for Experiment 2 in the same way as it is presented above. Error bars mark the standard error of the difference between preceding new and old conditions.

*p<.05 ** p<0.005 ***p<0.001

Consistent with our hypotheses, we found that similar trials were, indeed, more accurately identified as being similar when they were preceded by new trials than when they were preceded by old ones (Prec New=67.7%, Prec Old=61.7%, t(14)=3.41, p<.005). This performance benefit (6%) was even larger (9%) when we binned similar trials based on the preceding response (e.g. when subjects reported a stimulus as ‘new’) rather than the preceding trial type (when a stimulus was actually new) (Prec New=68.0%, Prec Old=59.4%, t=3.32, p<.005; Fig 1B), suggesting that the critical factor is the subjective memory decision rather than the stimulus type. Moreover, the preceding response still uniquely explained a significant portion of the variance in similar trial accuracy after adjusting for several covariates including preceding trial type, preceding accuracy and preceding response times (RT) (wald chi-squared=8.9, p<.005, see SOM for full details and additional control analyses). There was also a similar, though non-significant, trend for the preceding response’s influence on RT, with similar trials being correctly identified faster when they were preceded by new as compared to old responses (Prec New=967 ms, Prec Old=988 ms, t(14)=2.01, p=.06). Lastly, we predicted that similar trials that were difficult, with mnemonic evidence lying close to the decision boundary, would be more influenced by mnemonic biases than similar trials that were easier. We tested this hypothesis by dividing similar trials based on perceptual similarity ratings (see Exp 1a in SOM). A repeated measures ANOVA revealed a significant interaction between the preceding response (old vs. new) and similarity rating (high vs. low) (F(1,28)=5.27, p<.05; Fig 1B; also see Fig S4 for a parametric analysis). Consistent with our hypothesis, this interaction was driven by the preceding response’s influence being largest for the more difficult similar trials, those that were rated as being more similar (high similarity: t(14)=3.65, p<0.005; low similarity: t(14)=1.08, p=.30).

Experiment 1 demonstrated that memory decisions can influence subsequent ones in a manner consistent with computational models of the hippocampus (10–13). If this bias is, in fact, mediated by neuromodulator action, we reasoned that it should also be temporally-limited on the scale of seconds (12, 14). In Experiment 2, we measured the time window over which these carry-over effects exert themselves by varying the inter-stimulus time interval (ISI) that elapsed between trials (0.5, 1.5, and 2.5 seconds) with a new set of subjects (N=52). We replicated the main effect of the preceding response (old vs. new; F(1,50)=21.1, p<.001) and the interaction between similarity rating and preceding response (F(1,50)=4.41; p<.05). Critically, we also found that the preceding new benefit was time-dependent. Specifically, there was a significant interaction between ISI and preceding response (F(2,100)=3.2, p<.05) with the largest preceding response effect on trial accuracy found for similar trials that were preceded by the shortest ISI (Prec New=58.3%, Prec Old=49.4%, t(51)=4.3, p<.001; preceding old vs. new differences at longer ISIs were not significant Fig 1C). A similar interaction was also found in RTs (F(2,100)=4.2, p<.05; Fig S5B).

In Experiment 3, we tested whether old and new memory decisions can also influence subsequent memory formation (Fig 2, see Fig S6 for example trials). To this end, we modified a paradigm that was designed to assess whether related associations are integrated into newly formed memory traces (17, 18), a process that should be influenced by pattern completion/separation biases. We reasoned that the memory system would be more likely to reactivate previously encountered, but related, information following old recognition decisions compared to new decisions and that this reactivation should, in turn, lead to greater integration of the reactivated representations into newly formed memories. To test this, we had a new group of participants (N=22) alternate between making old/new object recognition decisions and encoding overlapping face-scene associations. Critically, these face-scene pairs (A-Y pairs) were associatively related to other face-scene pairs (A-X and B-X pairs) that were learned in a prior phase of the experiment. Previous research has demonstrated that participants tend to integrate these related A-X and B-X pairs into their learning of new A-Y pairs, forming a link between the indirectly associated B and Y stimuli (17–19). To assess the strength of this integration, we subsequently tested how often subjects chose to pair faces and scenes that were only indirectly related across the two sets (B-Y pairs). Consistent with our hypothesis, participants were 10% more likely to subsequently show this kind of integration when they encoded the A-Y pairs following old objects as compared to new ones (Prec New=60.1%, Prec Old=50.1%, t(21)=2.1, p<.05; Fig 2D). Moreover, when participants chose to pair indirectly related stimuli, they were 71 ms faster to do so if the stimuli had previously been learned following old objects as compared to new ones (Prec New=1487 ms, Prec Old=1558 ms, t(21)=2.2, p<.05), again consistent with the interpretation that retrieval improves subsequent integrative encoding.

Fig. 2.

Integrative encoding was tested using a multiphase design. (A) Participants first learned face-scene associations in which pairs of faces were associated with the same scene (A-X & B-X associations). (B) Next, they learned a new scene association for one face from each pair (A-Y association). These A-Y learning trials were interleaved with object recognition trials. Each A-Y association was preceded by either a novel object (preceding new condition) or one that had been studied at the beginning of the session (preceding old condition). Participants indicated whether each object was old or new. (C) The final phase tested whether they integrated the overlapping associations from the prior two phases by testing whether participants chose to pair the indirectly related faces and scenes (B-Y association). (D) The influence of preceding memory decisions on integration in Experiment 3. The graph compares how likely participants were to pair indirectly associated faces and scenes (B-Y associations) during the final test phase. Trials are divided according to whether their learned counterpart (A-Y) was encoded following new (blue) or old (red) trials during the prior phase. Error bars mark the standard error of the difference between preceding new and old conditions. *p<.05

Together, these results provide behavioral evidence that episodic encoding and retrieval can evoke biases that influence subsequent mnemonic processing, a phenomenon that computational models predict to be the consequence of a tension between episodic encoding and retrieval operations. Importantly, the evidence presented here goes beyond simply demonstrating competition between concurrent encoding and retrieval (20, 21), an effect that could be explained by a bottleneck at various cognitive stages (22); rather, we provide evidence that memory decisions can exert a temporally-extended bias on subsequent computational processes thought to support encoding and retrieval, namely pattern separation and completion. Specifically, we found that old and new recognition decisions influenced immediately-following memory decisions (Exps 1 and 2), and memory formation (Exp. 3).

Although there is agreement across several hippocampal memory models that encoding and retrieval should be temporally segregated (13, 23–25), the time required to switch between these processing biases varies widely across models from a few hundred milliseconds (24) up to 10 seconds (12). The current results presented in Experiment 2 suggest that the influence of a prior memory decision decays after a few seconds, interestingly consistent with the theoretical timescale of acetylcholine modulation in the hippocampus (12, 14). This result dovetails with the finding that acetylcholine antagonists increase proactive interference (26, 27), a potential consequence of pattern separation failure.

An intriguing open question is whether the lingering nature of the observed memory bias could be an adaptive mechanism to dynamically adjust the criterion for memory reactivation based on the nature of the environment. Rarely do our experiences rapidly switch between the familiar and novel. Instead, we tend to navigate through situations that generally contain more novel or more familiar components. It could be advantageous for our memory system to be more sensitive to change in novel environments and less sensitive to irregularities in familiar environments (25, 28–30). Regardless of the adaptive consequences, the current results shed light on fundamental computational issues of memory encoding and retrieval and highlight that our ongoing processing of the world is influenced by other preceding cognitive operations.