Figure 2. Closed-loop feedback reinforcement training to volitionally link spontaneous impulses in [DA]_ex to reward.

(A) The setup for the open loop experiment (Figure 1A) is augmented. We use the measured D2-CNiFER response as a proxy for [DA]_ex and drive the delivery of a liquid reward based on the [DA]_ex signal (red) relative to an adaptive staircase threshold (green) updated every 0.25 s. If [DA]_ex exceeds the threshold, a drop (0.1 mL) of sucrose water (10% w/v) is released via the lick port and the threshold is incremented by 0.005 signal units. The value of the threshold also exponentially decreases, in discrete steps of 0.005, with a time constant of 225 s, since the last increment.

(B) Example data show the open-loop naive response on day 1 and an increase of tonic [DA]_ex with closed-loop reinforcement on day 3; the adaptive staircase threshold follows the reward to within the decay time constant. A 130-s segment of data highlighted by the beige band is expanded.

(C) Rolling average of [DA]_ex impulses for all mice. The averaging window was 235 s. On day 2, tonic [DA]_ex did not significantly increase relative to that of naive mice (p = 0.69). By day 3, tonic [DA]_ex increased significantly (p = 0.01). The increase extinguished when reward was withheld on day 4 (p = 0.73) and reinstated when reward was restored on day 5 or 7 (p = 0.02) compared to feedback withheld on day 6 or 8.