Fig. 4. Model simulations show that forward spread of effect can emerge from the GRS.

a–d Choice simulations using different levels of w_R (−1, 0, 1, 2, respectively) are analyzed with a GLM that estimates the effect of each possible choice-reward-pair going four trials back on subsequent choices. Lighter colors indicate more positive beta weights, i.e. subsequent choice probability is increased by reward delivered for the respective conjunction. For example, when w_R is set to zero in panel b, the bright white diagonal indicates that the next choice the agent takes is influenced by the conjunction of choice and reward on the previous trial (t-1), the trial before that (t-2) and so on. However, as w_R increases (c, d), the precise history of choice-reward conjunction has less effect and conjunctions between choices made on one trial and reward occurrences on other trials become more likely to influence the next choice that is taken (lighter colors at lower left in c and d). e Effects of rewards delivered at various time points relative to time point 0 on the probability of repeating the choice made at time point zero. Effects are derived from averaging beta weights shown in panels a–d along the diagonal with different offsets. The area shaded in red corresponds to the mean effects along the red squares in panels a–d and indicates that rewards have contrasting (w_R < 0) or facilitating effects on repeating subsequent choices (w_R > 0) depending on w_R. (error bars are ±SEM around the mean across simulated monkey data sets, n = 12).