Fig. 2.

Neural network model of the experiment and the brain regions associated with information processing. A, Diagram indicates our hypothesis for how the sequence of stimuli could influence dopaminergic output. In this hypothesis, we have indicated that changes in dopaminergic output could influence target neural structures in a manner detectable in a fMRI BOLD measurement. The juice and water are shown to have both sensory (projection from finite timewindow box) and reward (the r pathways) representations in their influence on dopaminergic activity. To generate an expected hemodynamic response from this hypothesis, we made a finite time window (small boxes for juice and water), which determined the value of the immediate rewardr(t) (1 if juice occurred, 0.5 if water occurred, and 0 if no stimulus occurred). This maneuver arbitrarily set juice to twice the value of water. This is not important for the main expectation generated by the model.B, Predicted dopamine effect for predictable and unpredictable sequences of juice and water delivery. Horizontal axis is scan number. Vertical axis is the expected hemodynamic response predicted by a temporal difference model. The scale on the vertical axis is arbitrary. The important point to note is that the predictable run progresses to 0, whereas the unpredictable run remains high-amplitude throughout. The traces were generated by convolving a hemodynamic response kernel with the output of a temporal difference model. This suggested that the average BOLD response would be greater when the stimuli were unpredictable.