Figure 1.
A) Two-step task design, adapted from Decker et al. (2016). The fixed structure of the probabilistic transitions from 1st stage states 2nd stage states enables the distinction of model-based and model-free choices by examining the influence of the previous trial on the subsequent first-stage choice. B) A model-free learner tends to repeat previously rewarded first-stage choices (“stay”), regardless of the transition type that led to the reward (a main effect of reward on subsequent first-stage choices). By contrast, a model-based learner exploits knowledge of the transition structure and will favor the first-stage action that is most likely to lead to the same state if rewarded and the action least likely to lead to the same state if not rewarded (a reward-by-transition interaction effect on subsequent first-stage choices). C) Consistent with previous literature, our results show that the participants exhibit a mixture of model-based and model-free choice strategies.