Figure 8.

Information coded in the NAc and VP. A, Time bin neuronal activity was examined as follows: for 1 s before the onset of the a nose poke at the center hole (phase 1), after the onset of the cue tone (phase 2), before initiation of action (phase 3), after the action onset (phase 4), after the onset of the reward or no-reward tone (phase 5). B, The population of neurons that showed significant selectivity (Mann–Whitney U test, p < 0.01) for each event. State-coding neurons are defined as neurons that showed a significantly different firing rate in choice and no-choice trials for 1 s after the onset of the cue tone (phase 2). The neurons coding action values for left or right choices (Fig. 7E,F) were detected for three different time bins, phases 1–3. QLn and QRn indicate the action values for left and right during phase n, respectively. Note that these action value-coding neurons were detected by simple comparisons of firing rate in different blocks, not using computational models. Action command (AC)-coding neurons are defined as neurons that showed an action selectively during the 1 s before initiation of action (phase 3). Action-coding neurons are the neurons showing action selectivity during 1 s after the action onset (phase 4) (Fig. 7A,B). Reward-coding neurons are the neurons that showed different firing rate between rewarded trials and no-reward trials during 1 s after the onset of the reward or no-reward tone (phase 5) (Fig. 7C,D). C, The population of neurons coding the action values detected by a linear regression analysis. The reward probabilities for left and right were used as regressors (a model-free analysis). The neurons with a significant coefficient for the reward probability for either left or right were defined as the action value-coding neurons for left and right, respectively. QLn and QRn indicate the action values for left and right during phase n, respectively. D, The population of neurons coding the state value and the policy detected by a linear regression analysis. The sum of the reward probabilities for both actions and the difference of them were used as regressors (a model-free analysis). The neurons with a significant coefficient for ether the sum or the difference were defined as the state value and the policy-coding neurons, respectively. Vn indicates the state value during phase n, and Pn the policy during phase n. All populations were significantly larger than the chance level (binomial test, p < 0.01). The single and double asterisks indicate significant differences in the percentages of coding neurons between the NAc and VP; p < 0.05 and p < 0.01, respectively, in Mann–Whitney U test.