Memory states influence value-based decisions

Abstract

Using memory to guide decisions allows past experience to improve future outcomes. However, the circumstances that modulate how and when memory influences decisions are not well understood. Here, we report that the use of memories to guide decisions depends on the context in which these decisions are made. We show that decisions made in the context of familiar images are more likely to be influenced by past events than decisions made in the context of novel images (Exp 1), that this bias persists even when a temporal gap is introduced between the image presentation and the decision (Exp 2), and that contextual novelty facilitates value-learning whereas familiarity facilitates the retrieval and use of previously learned values (Exp 3). These effects are consistent with neurobiological and computational models of memory, which propose that familiar images evoke a lingering ‘retrieval state’ that facilitates the recollection of other episodic memories. Together, these experiments highlight the importance of episodic memory for decision-making and provides an example of how computational and neurobiological theories can lead to new insights into how and when different types of memories guide our choices.

Anyone who has ever searched for that perfect restaurant to take visiting friends probably made their choice by recalling memories of enjoyable dinners. Memory for past events, or episodic memory, appears to profoundly impact many of our behaviors: from simple choices between familiar options to complex decisions involving risk assessment, delaying rewards, and healthcare (Hertwig, Barron, Weber, & Erev, 2004; Peters & Büchel, 2010; Weber, Böckenholt, Hilton, & Wallace, 1993). Although episodic memory's potential to guide behavior is undeniable, it remains unclear why some choices are heavily influenced by memories of specific past experiences, while others appear to be made without retrieving memories.

To understand why the influence of episodic memories on choice is so variable, we turned to neurocomputational models of the hippocampus, the brain region underlying episodic memory. Specifically, computational (Hasselmo, Wyble, & Wallenstein, 1996; M. Meeter, Murre, & Talamini, 2004) and empirical (Duncan, Sadanand, & Davachi, 2012) findings suggest that the hippocampus operates in different ‘states’ to accommodate the competing computational demands of memory retrieval and encoding. Crucially, these ‘states’ are thought to be differentially evoked by the context; familiar contexts induce a state that favors retrieval, while novel contexts induce a complementary encoding state (Duncan et al., 2012). This implies that familiar contexts should increase the use of episodic memories when making decisions, even when the context is unrelated to the choice at hand.

To test the implications that the ‘memory state hypothesis’ holds for decision making, we focused on economic value-based decisions. Participants performed a monetary decision-making task in which they made a series of choices, each between two distinctive cards, to win money. Chosen cards repeated once during the experiment, so that participants could increase their winnings if they quickly retrieved a memory of the repeated card's value (Figure 1). Critically, we manipulated contextual familiarity by presenting unrelated images of either novel or familiar scenes immediately before choices were made. We predicted that familiar scenes, but not novel ones, would evoke a retrieval state, making participants more likely to use past experiences to guide decisions.

Figure 1.

Schematic of experimental design. On each trial, a familiar or novel scene was first presented to establish retrieval or encoding ‘modes’, respectively. Then participants chose between two cards for the chance to win money. Each card had an object on one side and, if chosen, flipped over to reveal a value that ranged from 0-$1. On some trials, both cards had new objects; on other trials, one of the cards had a previously chosen object on it. Because cards with the same object always had the same value, if participants remembered the value of the old card on these critical trials, they could increase their earnings (for example, remembering that the feather was worth 75¢ and choosing it). We measured whether familiar vs. novel preceding scenes influenced the degree to which choices were guided by value memories. We hypothesized that value memories would be more likely to influence choices that were made after familiar, as compared to novel, scenes. This is because familiar scenes would put subjects in a ‘retrieval state’, making relevant memories more accessible at the time of choice.

In Experiment 1, we demonstrate that familiar as compared to novel contextual scenes increase the extent to which memory for past events biases decisions. Experiment 2 replicates this effect and tests the boundary conditions, showing that the influence of familiar contexts persists when the scene is temporally and conceptually separated from the decision task. In Experiment 3, we show that contextual novelty has opposing effects on the formation of value memories and their retrieval, indicating that the observed decision-making biases were not driven by a general effect of familiarity on performance.

Experiment 1

Experiment 1 assessed whether contextual familiarity modulates the influence of episodic memory on choices. Participants chose between distinctive cards to win money. Participants could use memory to increase their earning when they were dealt previously selected cards. We manipulated contextual familiarity by having participants make choices in the context of images of familiar or novel scenes (Figure 2a).

Figure 2.

Experiment 1 design and results. (a). Participants chose between two cards, each of which had an object on one side and a value on the other. After a card was selected, it flipped over to reveal the amount of money won. Previously selected cards were re-presented once along with a ‘new’ card. Each decision was made in the context of either a novel or a familiar scene. (b). ‘Familiar’ scenes increased the influence of past experience on choices. The left graph plots how well the value of an ‘old’ card predicted participants’ choices (old vs. new card) for decisions made in the context of ‘familiar’ vs. ‘novel’ scenes. Statistical comparisons were performed by testing the interaction between ‘old’ card value and decision-scene. The right graph plots the model estimates of the likelihood of choosing ‘old’ cards of different values in the context of ‘familiar’ or ‘novel’ scenes. (c). The effect of novel scenes on value learning. The left graph plots how well the value of an ‘old’ card predicted participants’ choices (old vs. new card) for ‘old’ cards that were originally selected in the context of ‘familiar’ vs. ‘novel’ scenes. Statistical comparisons were performed by testing the interaction between ‘old’ card value and learning-scene. The right graph plots model estimates of the likelihood of re-selecting ‘old’ cards of different values that were originally selected in the context of ‘familiar’ vs. ‘novel’ scenes. Error bars represent 95% confidence intervals around the estimate. *p<.05

Participants

Twenty-four members of the Columbia University community (16 female, mean age = 23.7) participated for pay ($12/hour + bonus earnings). Prior research (Duncan et al., 2012) identified that contextual familiarity's influence on memory retrieval has a moderate to high effect size (Cohen's d .59-.86). Based on this, we selected a sample size of 20-30 participants (terminating data collection at the end of the semester) to conservatively achieve 80% power. Participants in all experiments reported normal or corrected-to-normal vision and no history of neurological or major psychiatric illness. This protocol was approved by Columbia's Morningside Institution Review Board.

Stimuli

Distinct indoor and outdoor scenes (185) served as the unrelated novel or familiar contexts. For each participant, five scenes were assigned to the ‘familiar’ condition, while the remainder were assigned to the ‘novel’ condition. Additionally, 540 virtual playing cards, each with a distinctive object on one side, served as choice options. Cards were equally likely to be preceded by a novel or a familiar scene across participants. The value and left/right position of each object card was randomly assigned. All experiments were presented on 20-inch iMacs using Psychophysics Toolbox for Matlab.

Procedure

Participants were first exposed to the five ‘familiar’ scenes. Scenes were repeated four times for 2s each, and participants indicated whether each was an ‘indoors’ or ‘outdoors’ scene. Decision-making task trials began with a novel or familiar contextual scene presented for 1s, after which two cards appeared on top of it for 1.5s (Figure 2a). Novel scenes were operationalized as the first presentation of a particular scene, whereas familiar scenes were operationalized as a previously presented scene. Importantly, in both experiments, specific cards were never presented with a particular scene more than once, so that familiar scenes could not prime specific value memories. Participants were told that scenes were ‘decorative mats’ which merely indicated that cards were about to be dealt.

Each card had an object on one side and a value (0¢, 20¢, 40¢, 60¢, 80¢, or $1; uniform distribution) hidden on the other. Participants were instructed that “each card has a different object on its back” and that they could “use memory to make more money.” Participants were given 1.5s to select a card using the ‘j’ or ‘k’ keys of a standard keyboard. The selected card then flipped to reveal the winnings while the unselected card disappeared. The outcome remained on the screen for 1.5s and was followed by a 500ms fixation cross. Missed responses were signaled with a ‘too slow’ message. Card values were independent, such that the unselected card's value could not be inferred from the outcome. Participants performed 360 trials equally divided into three blocks and were paid 5% of their total winnings after the experiment. On 180 critical trials, participants chose between a ‘new’ object card and an ‘old’ object card, which they had previously selected 5-10 trials ago.

Statistical analyses were run using Mixed Generalized Linear Models (R lme4 package). Individual participant's performance was assessed using separate logistic regressions.

Results

We found that, overall, the ‘old’ card's value was a positive predictor of choosing the ‘old’ card over the ‘new’ one (Z=8.32, p<.00001, β =2.48), indicating that participants reliably used past experience to guide their choices. There were individual differences in memory use, however, with ‘old’ card value being a non-significant negative predictor of ‘old’ card choice for 1 out of 24 participants. Because this participant's decision strategy was unclear, they were removed from subsequent analyses, though including this subject does not change the pattern of results.

We next turned to the critical question of whether memory use differed based on contextual novelty vs. familiarity. Critically, use of the ‘old’ card's value was reliably modulated by scene familiarity (β=−.52, CI₉₅=[−.94, −.07], p=.02; Table S1). This interaction was driven by participants being more likely to choose high-value ‘old’ cards while avoiding low-value ‘old’ cards in unrelated familiar as compared to novel contexts (Figure 2b). Participants were also faster to make critical choices following ‘familiar’ as compared to ‘novel’ scenes (Figure S2; Table S4). These results suggest that contextual familiarity facilitates subsequent memory retrieval, thereby increasing the influence of memory on decision-making.

An alternative explanation, however, is that ‘familiar’ scenes facilitate performance through a general mechanism, for example by freeing attentional resources, rather than specifically promoting the retrieval of relevant episodic memories. If this were the case, then ‘familiar’ scenes should similarly facilitate the encoding of value memories. To test this possibility, we assessed if the subsequent use of value memories was influenced by whether they were originally encoded in a ‘novel’ or ‘familiar’ context (learning-scene). The influence of decision-scenes and learning-scenes significantly differed from each other (p=0.01), indicating that contextual familiarity differentially influences value retrieval and encoding. In fact, values learned in the ‘novel’ contexts were numerically more likely to later guide choices (β=.44, CI₉₅=[−.11,1.07], p=.14; Figure 2c; Table S1).

Experiment 2

Experiment 1 demonstrated that memory has a greater influence on decisions made in the context of familiar as compared to novel scenes. We next tested whether this effect depends on: (1) the scene's presence while the choice is made; (2) a conceptual link between the scene and the card task (because the scenes were described as “mats” upon which the card were dealt); (3) the familiar scenes being presented repeatedly.

Participants

Twenty-eight new members of the Columbia community (21 female, mean age = 22.7) participated in the study for pay.

Stimuli

Distinctive scenes (130) and objects (360) were used as stimuli in the experiment and 110 scenes were presented twice. They were labeled ‘novel’ on their first presentation and ‘familiar’ on their second. Because the first presentation necessarily occurs before the second, this resulted in a greater probability of novel scenes early on in the experiment. To adjust for this potential confound, we removed the first 25 trials from each session (Figure S1, see supplemental materials for more details).

Procedure

The procedures were similar to Experiment 1, but changes were made to both temporally and conceptually separate the scene from the card task (Figure 3a). First, scenes and card decisions were presented in separate but interleaved tasks. In the Scene Task, participants viewed a scene for 1.5s and were asked to identify whether it was ‘indoors’ or ‘outdoors’ by pressing the ‘j’ or ‘k’ keys, respectively. Critically, the scene was not called a ‘decorative mat’, nor was it tied to the Card Task in any other explicit way, reducing the possibility that participants would believe that the scene was related to the cards. The scene then disappeared and was followed by a 500ms fixation-cross. The Card Task had the same structure as the Card Task in Experiment 1, except the cards took the values of 0¢, 25¢, 75¢, or $1 (uniform distribution). Using more extreme values allowed for trial number reduction (to 240 trials divided equally into 2 blocks) while minimizing loss in experimental power. Lastly, participants were not pre-exposed to any scenes, but instead were familiarized with each scene through a single exposure; labeled for analyses as ‘novel’ on first presentation and ‘familiar’ on second.

Figure 3.

Experiment 2 design and results. (a). In Experiment 2, preceding scenes were not presented during the decision or feedback phases and ‘familiar’ scenes were only viewed on one earlier trial. (b). Preceding ‘familiar’ scenes increased memory's influence on decision-making. The graphs in the left panel plot how well choices (old vs. new card) were predicted by the value of the ‘old’ card separately for trials that followed ‘familiar’ vs. ‘novel’ scenes. The graphs in the right panel plot the same relationship, but for the subset of trials in which participants had better memory, indexed using repetition priming in their scene indoor/outdoor judgments (see text). (c). Only primed preceding scenes influenced value learning. The graphs in the left panel plot how well choices (old vs. new card) were predicted by ‘old’ card value, separately for trials in which the ‘old’ card was originally selected following a ‘familiar’ vs. ‘novel’ scene. The graphs in the right panel plot the same relationship for the subset of trials in which participants showed repetition priming in their scene indoor/outdoor judgments. Error bars represent 95% confidence intervals around the estimate. *p<.05

Results

Overall, participants in Experiment 2 also reliably demonstrated the use of memories for past experience in their decisions (β=2.62, CI₉₅=[2.11, 3.13], p<.00001). The decisions made by 2 of the 28 participants were non-significantly negatively predicted by ‘old’ card value. These participants were excluded from subsequent analyses, though including them does not change the pattern of results.

Critically, choices made after ‘familiar’ scenes were more influenced by value memory than those made after ‘novel’ scenes (β=−.57, CI₉₅=[−1.08,−.01], p=.02; Figure 3b; Table S2; RT analyses Figure S3; Table S5). This demonstrates that familiar images can trigger a lingering state that increases the influence of relevant memories on choices even if (1) familiar images are no longer present when the choice is made, (2) participants experience the scenes as a separate task, and (3) the familiar images had only been seen once before.

As in Experiment 1, this effect cannot be attributed to general performance enhancements following ‘familiar’ scenes, as the influence of pre-decision scenes was reliably different than the influence of pre-learning scenes (p=.04), though pre-learning scenes did not reliably modulate memory formation (β=.14, CI₉₅=[−.36,.70], p=.58; Figure 3c; Table S2).

If the effects of contextual familiarity on value-based decision-making are related to recognition of the scenes, then a scene's capacity to modulate choices should depend on successfully remembering the scene. To test this prediction, we measured repetition priming of indoor/outdoor scene judgments (first presentation rt - second presentation rt) as a proxy for participants’ scene memory. Reaction times were on average 77.5ms faster on the second compared to the first presentation (t(25)=18.8, p<.00001), demonstrating repetition priming. We then selected trials preceded by scenes that showed any evidence of repetition priming (>0ms increased response speed on second presentation; 66% of scenes), and re-ran the above analyses. We found that both the pre-decision and the pre-learning scenes significantly modulated memory use on this subset of trials (pre-decision: β=−1.03, CI₉₅=[−1.94, −.06] p=.03; pre-encoding: β=1.16, CI₉₅=[.17, 2.29], p=.02; Figure 3b&c; Table S3), suggesting that memory for the contextual scene may be important for establishing the effects of context on decision making.

Experiment 3

Experiments 1 and 2 demonstrate that contextual familiarity, as opposed to novelty, facilitates the use of memories during decision making, as predicted based on models of hippocampal function. These models also predict that contextual novelty should facilitate the formation of memories. While both experiments partially support this hypothesis, the encoding modulation was less reliable than the retrieval modulation. This is consistent with prior research showing that contextual familiarity influence on encoding is less robust than its influence on retrieval (Duncan et al., 2012). One reason for this asymmetry may be that retrieval manipulations occur immediately before the behavior of interest is measured, whereas the effects of encoding manipulations must persist throughout intervening events before they can be observed. In Experiment 3, we ran a version of the task used in Experiment 1 with a larger sample size with the aim of (1) whether the effect of contextual familiarity on the retrieval of memories would replicate and (2) assess whether contextual novelty facilitates the formation of new memories.

Methods

Forty-two members of the University of Toronto community (35 female, mean age = 21.1) participated in the study for pay. Prior research (Duncan et al., 2012) identified that contextual novelty's influence on memory encoding has a moderate effect size (Cohen's d .46). Based on this, we selected a sample size of 42 participants to conservatively achieve 80% power. All procedures were approved by the University of Toronto Research Ethics Committee.

All stimuli and procedures were identical to Experiment 1.

Results

Decisions reliably reflected the influence of episodic memories across participants (β =2.33, p<.00001), however, the choices made by 1 of the 42 participants were non-significantly negatively predicted by ‘old’ card value. This participant was excluded from subsequent analyses, though including them does not change the pattern of results.

As in the prior experiments, choices made after ‘familiar’ scenes were more influenced by value memory than those made after ‘novel’ scenes (β=−.47, CI95=[−.79,−.15], p=.004; Figure 4; Table S3; RT analyses Figure S4; Table S6). Importantly, values learned in the context of novel scenes were more likely to guide later decisions (β=.36, CI95=[.04,.68], p=.03; Figure 4). This double dissociation between the influence of contextual familiarity on the retrieval of memories vs. the formation of new value memories strongly supports the hypothesis that contextual novelty shapes decisions by evoking process-specific biases in episodic memory.

Figure 4.

Experiment 3 results. The graphs in the left panel plot how well choices (old vs. new card) were predicted by the value of the ‘old’ card separately for choices made in the context of ‘familiar’ vs. ‘novel’ scenes. Preceding ‘familiar’ scenes increased memory's influence on decision-making. The graphs in the right panel plot how well the value of an ‘old’ card predicted participants’ choices (old vs. new card) for ‘old’ cards that were originally selected in the context of ‘familiar’ vs. ‘novel’ scenes. Preceding ‘novel’ scenes increased value learning in the service of later decision making. Error bars represent 95% confidence intervals around the estimate. *p<.05

Discussion

Using past experience to guide behavior is central to our ability to adapt to the demands of our environment. Although memories for distinct past episodes factor into some accounts of decision-making (Hertwig et al., 2004; Peters & Büchel, 2010; Weber et al., 1993), this research has focused on the qualities of the memories or the consequences of direct instructions to use episodic memory while making a choice. This leaves a critical question unanswered—under which conditions are episodic memories more likely to guide choices? Here we demonstrate that the novelty vs. familiarity of the context in which a decision is made is a critical factor in determining memory's influence on choice.

We found that contextual familiarity had a specific influence on memory-guided decisions: familiar contexts facilitated value-memory retrieval, whereas novel contexts facilitated value-memory encoding. This dissociation indicates that contextual familiarity is unlikely to influence decisions through a general cognitive mechanism. Moreover, it is well established that memory encoding is more dependent on attention than memory retrieval (Craik, Govoni, Naveh-Benjamin, & Anderson, 1996; Naveh-Benjamin, Craik, Perretta, & Tonev, 2000; Naveh-Benjamin, Craik, Guez, & Kreuger, 2005). Thus, if contextual familiarity influenced decisions through an attentional mechanism, one would expect the enhancing influence of familiar contexts to be even stronger during value learning—the opposite pattern to what was observed. Additionally, contextual images influenced the use of episodic memory even when the images were not directly relevant to the decision; across all experiments, specific cards were never presented with a particular scene more than once, so that familiar scenes could not prime specific value memories. Moreover, in Experiment 2, the scenes were presented in a separate task, making their familiarity incidental to the primary decision-making task.

This pattern of results is consistent with the Memory State Hypothesis (Carr & Frank, 2012; Colgin & Moser, 2010; Duncan et al., 2012; Duncan, Tompary, & Davachi, 2014; Easton, Douchamps, Eacott, & Lever, 2012; Hasselmo et al., 1996; Meeter et al., 2004), which posits that the incompatible computational demands of episodic encoding and retrieval (O'Reilly & McClelland, 1994) are accommodated by establishing modes which facilitate either retrieval or encoding within the episodic memory system. These theoretical accounts (Hasselmo et al., 1996; Meeter et al., 2004) have been confirmed empirically by recent research examining the effects of context on memory judgments. This work has revealed that novelty elicits a lingering encoding state, facilitating the computational process of pattern separation, whereas familiarity elicits a lingering retrieval state, facilitating the process of pattern completion (Duncan et al., 2012). Here, we used a similar manipulation to test the effects of familiar context on using prior experiences to guide value-based decision making. The similarities in the manipulation used along with the timescale of the effect between the current experiments and this prior work suggest that similar mechanisms may be at play here. If so, then one possibility is that familiar contexts facilitate the influence of past experience on value-based decisions, because familiar scenes bias the memory system towards pattern completion, the process by which associated details of an experience are retrieved. Being in a pattern completion state would make the value of the subsequently-presented old card more accessible and, thus, more likely to guide the choice.

Despite the clear potential of episodic memory to guide choices, research on economic decision-making has most often focused on how choices are steered by abstracted values learned incrementally over repeated experiences. Here, we demonstrate that economic decisions can also be influenced by memories for the outcomes of individual episodes. This is in line with recent proposals that memory for individual episodes plays a broader role in decision-making than previously recognized, influencing choices even on probabilistic incremental learning tasks which can be solved simply by relying on a running average of value across trials (Biele, Erev, & Ert, 2009; Erev, Ert, & Yechiam, 2008). Given that incremental learning and episodic memory depend on distinct neural and cognitive systems (Delgado & Dickerson, 2012; Knowlton, Mangels, & Squire, 1996; R. A. Poldrack et al., 2001; Russell A. Poldrack & Packard, 2003), they likely influence decisions in different ways. An important open question is how each system influences decisions and how information from both systems is integrated. By identifying factors that modulate when people use episodic memory to make choices, the work presented here provides a step towards this larger goal.

In summary, the studies presented here add to our understanding of how past experiences shape future choices by demonstrating that cognitive states, evoked prior to a choice and unrelated to the choice, can influence the information used to make that choice. Furthermore, this work highlights the importance of episodic memory for decision-making and provides an example of how computational and neurobiological theories can lead to new insights into how and when different types of memories guide our choices.