Figure 1.
Dissociating effects of RPEs, surprise and uncertainty on incidental memory encoding. a, Task: On each trial, participants were first shown the value of a successful gamble for the current trial (e.g. 100). Next, a unique image belonging to one of two categories (animate/inanimate) was shown, which indicated the probability of reward. Participants made a play or pass decision. Then, participants were shown their earnings if they played (top and middle rows), or shown the hypothetical trial outcome if they passed (bottom row). At the end of each trial a cumulative total score was displayed. b, Manipulation of RPEs before, during, and after image presentation. For the image prediction error, we show example probabilities (0.8 and 0.2), and values (100 and 20). c, Types of RPE: The value RPE signals how the current value is better or worse than what is expected on average (10). The image RPE is computed as the difference between the expected reward of the current image category (e.g., 89) and the reward prediction, computed as the average expected reward of the two image categories (e.g., 45). The expected rewards are computed using the values and probabilities for reward/punishment (value: Vrew, Vpun, probability: Prew, Ppun). The feedback RPE is computed as the difference between the expected and experienced outcomes. d, Model predictions: Reward probabilities were determined by image category, yoked across categories (i.e., p(rew | animate) = 1 – p(rew | inanimate)), and reset occasionally to require learning (solid black line). Binary outcomes (red/black dots), governed by these reward probabilities, were used by an ideal observer model to infer the underlying reward probabilities (blue line). e-g, Inputs to ideal observer model. The ideal observer learned in proportion to the surprise associated with a given trial outcome (e) and the uncertainty about its estimate of the current reward probability (f), both dissociable from RPE signals at time of feedback (g) and image presentation (h). Surprise is a probability, and uncertainty is measured in units of nats. i, Recognition memory: in a surprise recognition test, participants provided a binary answer (“old” or “new” image) and a 1–4 confidence rating.