Figure 2. Neuronal activity in the dorsomedial prefrontal cortex related to strategic decision making.

A. Monkeys deviated systematically from model-free reinforcement learning to avoid the exploitation by the computer opponent during a biased matching pennies task, in which the payoff to the animal was determined jointly by the choices of the monkey and computer (left). S and R, safe and risky target, respectively, and the number in the payoff matrix indicates the number of tokens received by the animal. The color of each box in the decision tree (middle) and the position of each circle in the scatter plot (right) indicate how much the probability of choosing the safe target following a particular choice-outcome history deviated from a model-free reinforcement learning algorithm, P (safe)-PRL(safe), and how this increased or decreased the probability of winning compared to the equilibrium strategy, P(token)-P_NASH(token). R−, R+, denote loss and gain from the risky target, whereas S0 and S+ indicate neutral outcome and gain from the safe target. B. The strength of neural signals (ordinate) related to the tendency to switch away from a model-free reinforcement learning (abscissa) across different choice-outcome history (shown in A) for a single neuron in the dorsomedial prefrontal cortex. C. Same neural-behavioral correlation for switching for the population of neurons recorded in the dorsolateral prefrontal cortex (dlPFC), dorsomedial prefrontal cortex (dmPFC), dorsal anterior cingulate cortex (ACCd), lateral intraparietal area (LIP), caudate nucleus (CN), and ventral striatum (VS). White and gray bars indicate the results obtained from a token-based biased matching pennies task [45, 86] and a symmetric matching pennies task [44, 48], respectively.