Skip to main content
PMC home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2023 Jan 13;601(3):631–645. doi: 10.1113/JP283915

Neural representation and modulation of volitional motivation in response to escalating efforts

Liping Zhang 1,✉, Chengwei Liu 1, Xiaopeng Zhou 1, Hui Zhou 1, Shengtao Luo 1, Qin Wang 1, Zhimo Yao 1, Jiang‐Fan Chen 1,2,✉
PMCID: PMC10108165  PMID: 36534700

Abstract

Abstract

Task‐dependent volitional control of the selected neural activity in the cortex is critical to neuroprosthetic learning to achieve reliable and robust control of the external device. The volitional control of neural activity is driven by a motivational factor (volitional motivation), which directly reinforces the target neurons via real‐time biofeedback. However, in the absence of motor behaviour, how do we evaluate volitional motivation? Here, we defined the criterion (ΔF/F) of the calcium fluorescence signal in a volitionally controlled neural task, then escalated the efforts by progressively increasing the number of reaching the criterion or holding time after reaching the criterion. We devised calcium‐based progressive threshold‐crossing events (termed ‘Calcium PTE’) and calcium‐based progressive threshold‐crossing holding‐time (termed ‘Calcium PTH’) for quantitative assessment of volitional motivation in response to progressively escalating efforts. Furthermore, we used this novel neural representation of volitional motivation to explore the neural circuit and neuromodulator bases for volitional motivation. As with behavioural motivation, chemogenetic activation and pharmacological blockade of the striatopallidal pathway decreased and increased, respectively, the breakpoints of the ‘Calcium PTE’ and ‘Calcium PTH’ in response to escalating efforts. Furthermore, volitional and behavioural motivation shared similar dopamine dynamics in the nucleus accumbens in response to trial‐by‐trial escalating efforts. In general, the development of a neural representation of volitional motivation may open a new avenue for smooth and effective control of brain–machine interface tasks.

graphic file with name TJP-601-631-g009.jpg

Key points

  • Volitional motivation is quantitatively evaluated by M1 neural activity in response to progressively escalating volitional efforts.

  • The striatopallidal pathway and adenosine A2A receptor modulate volitional motivation in response to escalating efforts.

  • Dopamine dynamics encode prediction signal for reward in response to repeated escalating efforts during motor and volitional conditioning.

  • Mice learn to modulate neural activity to compensate for repeated escalating efforts in volitional control.

Keywords: A2A receptor, BMIs, dopamine, efforts, motivation, NAc, volitional control

Abstract figure legend Evaluation scheme of volitional motivation and behavioural motivation.

graphic file with name TJP-601-631-g009.jpg

Introduction

The operation of brain–computer interfaces (BCIs) and brain–machine interfaces (BMIs) usually depends on the degree of volitional control of neural activity (Fetz, 2007).The volitional drive on cortical neurons can be demonstrated directly by operant training subjects to control neural activity via biofeedback (Eaton et al., 2017; Fetz, 1969; Ishikawa et al., 2014; Moritz & Fetz, 2011; Schmidt et al., 1977; Wyler & Prim, 1976). Volitional control of single or multiple neurons using biofeedback bypasses the normal biological pathways mediating volitional movements (Moritz & Fetz, 2011). Since there is no direct relationship between volitional control of neurons and their physiological functions, we can set up different criteria to reinforce neural activity via biofeedback. Just as in animal behaviour training, animals are rewarded by setting a criterion to reinforce their behaviours. The volitional control of neural activity provides a defined link between neural activity and the criteria set by the experimenter, allowing a detailed study of the neural adaptive responses for the changed criteria (Chase et al., 2012).

Motivation, defined as the energizing of behaviour in pursuit of a goal, requires the subject to weigh the costs of an action against its potential benefits (Berridge, 2004; Cook & Artino, 2016; Salamone & Correa, 2012). Motivation is represented by the rewards of maximal efforts against the costs of an action for its potential benefits (Salamone & Correa, 2012). In animal models, this is mainly evaluated by an animal's behavioural response to progressively escalating efforts, with the breakpoints representing the size of the motivation, that is when the animal stops (motor) responding to the efforts (behavioural motivation). Volitional control of neural activity is also driven by the motivational factor (volitional motivation), which is critical for improving the volitional modulation of neural activity and neuroprosthetic learning (Kleih, Riccio, Mattia, Kaiser et al., 2011; Kleih, Riccio, Mattia, Schreuder et al., 2011). How do we evaluate volitional motivation in the absence of a motor response? The volitional control of neurons directly reinforces neural activity by biofeedback. The escalating effort for volitional control can be specifically increased by predefined criteria (schedule) to progressively increase the required holding time for neural activity above a defined threshold. Finally, volitional motivation was evaluated here by the response of neuroplasticity to escalating effort, with the breakpoint (maximum plasticity of neurons) representing the size of the volitional motivation.

The imaging of neural activity using calcium indicators (Gcamp6f) has been widely used to observe neural activity based on the fluorescence intensity of the calcium indicator (Chen et al., 2013). In this study, mice volitionally controlled the neural activity of the M1 population under operational conditioning by real‐time monitoring of calcium fluorescence signals using a fibre photometry system. We first set a criterion of calcium fluorescence signal (defined threshold) in the volitionally controlled neural activity task. We then progressively increased the efforts by increasing the defined threshold‐crossing event (TCE) or holding time after a defined threshold‐crossing. We also developed a representation and quantitative analysis of volitional motivation by coupling a volitionally controlled neural task with the scheme of a progressive‐ratio task (PRT) (Bradshaw & Killeen, 2012) and a progressive hold‐down (PHD) task (Bailey et al., 2015). Specifically, we devised the calcium‐based progressive threshold‐crossing events (termed ‘Calcium PTE’) and calcium‐based progressive threshold‐crossing holding‐time (termed ‘Calcium PTH’) for quantitative assessment of volitional motivation responding to progressively escalating efforts. Using this novel representation of volitional motivation, we demonstrated that volitional motivation was similarly modulated by chemicogenetic and pharmacological manipulation of the striatopallidal pathway and shared similar dopamine dynamics in nucleus accumbens (NAc) in response to escalating efforts as with behavioural motivation. Totally, our findings established the first neural representation of volitional motivation and provided novel insights into circuit and neuromodulator control of volitional motivation that may help overcome bottlenecks in smooth and effective control of BMI tasks.

Methods

Ethical approval

Animals were handled in accordance with national and institutional guidelines. All experimental protocols were approved by the Institutional Ethics Committee for Animal Use in Research and Education at Wenzhou Medical University, China (ID Number: WYDW2020‐0299). All surgical procedures were performed under aseptic conditions. Following the completion of the protocols, all mice were killed by anaesthetic overdose and cervical dislocation. The investigators understand the ethical principles under which The Journal of Physiology operates and the work within this study fully complies with the journal's animal ethics checklist. All efforts were made to reduce the number of animals used.

Animals

Adult (8–10 weeks old) male C57B6/J mice were purchased from SPF Biotechnology Co., Ltd (Beijing, China), and A2A‐rM3Ds mice were obtained from the Jackson's labs (JAX Stock No. 017863) as described previously (Farrell et al., 2013). All mice were maintained under a 12/12 h photoperiod (lights on at 08.00 h). After surgery, the mice were individually housed under a 12 h light‐dark cycle for at least 14 days before conducting any further experiments. After completing Calcium PTE, the mice rested for half a month and then trained on Calcium PTH. rM3Ds was selectively and stably expressed in striatopallidal neurons in A2A‐rM3Ds mice and activation of the striatopallidal pathway in A2A‐rM3Ds mice was achieved by systemic injection of clozapine N‐oxide (CNO), which specifically activates rM3Ds in the striatopallidal neurons (Farrell et al., 2013). Blockade of A2ARs by KW6002 and monitoring of dopamine dynamics in NAc were performed with male C57B6/J mice.

Surgery, virus injection and optic fibre implantation

Mice were anaesthetized with pentobarbital (i.p. 60 mg/kg) and mounted on a stereotaxic apparatus. A homeothermic pad was placed below each mouse to maintain body temperature at ∼36°C. Ophthalmic gel was applied to the eyes to prevent dryness. Each animal was unilaterally injected with 200 nl of rAAV‐hsyn‐DA4.4‐WPRE‐hGH (catalogue no. PT‐1340; BrainVTA, Wuhan, China) into NAc (AP: 1.0 mm, ML: 1.2 mm, DV: −3.9 mm) and/or injected with 300 nl of AAV9‐Syn‐GCaMP6f‐WPRE‐SV40 into the left M1 cortex (AP: 1.50 mm, ML: 1.54 mm, DV: −1 mm) using a Nanojet II injector (Drummond Scientific, Broomall, PA, USA) at a rate of 60 nl/min. The mice were then implanted with an optical fibre (230 μm O.D., 0.37 NA; Shanghai Fiblaser, Shanghai, China) within a ceramic ferrule at the same virus injection sites of the NAc and M1. The ceramic ferrule was supported with a skull‐penetrating M1 and/or NAc screw and dental acrylic resin.

The volitionally controlled neural task

We used an operant volitionally controlled neural task with closed‐loop feedback system by volitional conditioning of population neurons in the M1 cortex by real‐time monitoring of calcium fluorescence signal using a fibre photometry system (the low baseline procedure) (Zhang et al., 2020). In the low baseline procedure, the baseline was defined as the lowest F 0 value within a 1 min time window and recalculated for every minute using the lowest F 0 value (Zhang et al., 2020). Briefly, mice were transfected with AAV9‐syn‐GCaMP6f‐WPRE‐SV40 to express the genetically encoded Ca2+ indicator GCaMP6f in M1 neurons and implanted with optical fibres into the same area. The mice were then conditioned to increase calcium fluorescence signal in M1 neurons above the defined threshold value within a specific time interval (30 s) to acquire a sucrose drop reward (Fig. 1A ). The defined threshold was referenced averaging M1 neural activities over 1 day of instrumental conditioning (pressure lever). This operant volitionally controlled neural task is the basis for all the training in the following task.

Figure 1. Development of Calcium PTE for detecting volitional motivation.

Figure 1

Open in a new tab

A, closed‐loop volitional control system. The calcium fluorescence signal (ΔF/F) of M1 neurons was monitored in real time by a fibre photometry system. Calcium fluorescence signals (ΔF/F) exceeding the defined threshold value triggered the operant box to deliver a drop of sucrose solution reward. B, scheme of the training procedure for Calcium PTE. The upper panel indicates the scheme of the training procedure for Calcium PTE. The lower panel indicates the number of TCEs of the sequential trial for the Calcium PTE test. C, the calcium fluorescence signal change in M1 neurons before/after the reward delivery (± 5 s) for escalating efforts (trials 1, 3, 5, 7, 9, 11 and 13) in Calcium PTE testing (n = 6). D, the breakpoint (the maximal TCEs) distribution of six mice by the Calcium PTE test (n = 6). [Colour figure can be viewed at wileyonlinelibrary.com]

In a previous study, we attempted to eliminate the overt movement in an operant volitionally controlled neural task. For example, we examined the temporal disassociation of the volitional control of M1 neural activity from movements of the right forelimb as monitored with EMG recordings (Zhang et al., 2020). Furthermore, the mice did not cross the defined threshold during free movement and foraging. Lastly, the M1 population calcium fluorescence signal in one‐lever instrumental behaviour (i.e. by pressing the lever once to get a reward in a trial) displayed different patterns compared to volitional control of neural activity.

Analysis of volitional motivation by Calcium PTE and Calcium PTH

Development of the representation and quantitative analysis of motivation involved three main steps: (1) establishing an operant volitionally controlled neural task; (2) formation of stable mapping of M1 activity responding to increasing efforts by a fixed ratio schedule; and (3) assessing motivation by Calcium PTE and Calcium PTH. The timeline of the training and testing procedures is illustrated in Figs 1B and 2A . After completing Calcium PTE, six male C57B6/J mice rested for half a month and then trained on Calcium PTH.

Figure 2. Development of Calcium PTH for detecting volitional motivation.

Figure 2

Open in a new tab

A, scheme of the training procedure for Calcium PTH. The upper panel indicates the scheme of the training procedure for Calcium PTH. The lower panel indicates the holding time of the sequential trial for the Calcium PTH test. B, the calcium fluorescence signal change in M1 neurons before/after the reward delivery (± 5 s) for escalating efforts (trials 1, 3, 5, 7, 9, 11, 13 and 15) in Calcium PTH testing (n = 6). C, the breakpoint (the maximal holding time) distribution of six mice by the Calcium PTH test (n = 6). [Colour figure can be viewed at wileyonlinelibrary.com]

Fixed‐ratio 1 (FR1) and fixed‐ratio 5 (FR5)

Mice were conditioned to exceed the defined threshold (calcium fluorescence signal, ΔF/F) once (FR1) or five times (FR5) to earn a drop of sucrose (50 rewards per session), and they earned 50 rewards in 30 min. A red light indicated the beginning of a trial, and an auditory cue would appear when the defined threshold was exceeded, after which there was a 10 s interval. FR1 and FR5 in the instrumental behaviour (PRT) were one or five presses of the pressure lever, respectively, by mice to earn a drop of sucrose

Calcium PTE

Mice underwent the volitionally controlled neural task for 10 days, and the proportion of correct trials was 85–100%. The mice then underwent FR1 training for 3 days and FR5 training for 5 days. The mice were then subjected to the Calcium PTE test, where they were required to make progressively increasing numbers of TCEs to obtain a reward. The criterion was set at one TCE for the first time, and the following TCE was calculated by the formula (TCE = 5 × e × (0.2t) − 5, t = trial number). Each session could last up to 2 h but ended early if the mouse did not cross the defined threshold for 10 min. Motivation was measured by recording the total TCEs in the session and the breakpoint (the total TCEs of the last trial).

Calcium PTH

Mice underwent the volitionally controlled neural task for 10 days, and the proportion of correct trials was 85–100%. The mice were trained to earn a reward by continuously holding the calcium fluorescence signal above the defined threshold of 200 ms for 3 days. The mice were then trained to earn a reward by continuously holding the calcium fluorescence signal above the defined threshold of 240 ms for 5 days. The mice were tested in the Calcium PTH task in which rewards could be earned by continuously holding the calcium fluorescence signal above the pre‐defined threshold. Every trial's holding time was calculated by the formula (holding time = 0.1 × 1.05( t  − 1), t = trial number). Each session could last up to 2 h but ended early if the mouse did not reach the defined holding time for 10 min. Motivation was measured by recording the total TCEs in the session and the breakpoint (i.e. the holding time of the last trial).

Calcium and dopamine fluorescence signal analysis

Photometry data were exported to MATLAB Mat files from fibre photometry for further analysis (Li et al., 2016). As in our previous study, we performed data analysis in the MatLab platform (Math Works, Natick, MA, USA) with custom‐written programs (Zhang et al., 2020). After smoothing the data with a moving average filter (20 ms span with a 10 ms moving step), we analysed the event‐related calcium fluorescence signal and dopamine fluorescence signal in relationship to the reward (with the reward as time ‘0’ point). We derived the values of fluorescence change (ΔF/F) by calculating (F − F 0)/F 0, where F 0 is the baseline fluorescence signal averaged over a 1–2 s control time window, which was typically set 1–2 s preceding reward delivery. For the dopamine fluorescence signal analysis, the baseline was defined as the average fluorescence signal within −5 to −6 s prior to reward delivery(‘0’). ‘Height’ was analysed as the highest peak of dopamine dynamics of 0 to −5 or 0 to +5 s of reward delivery (‘0’). No recording data were excluded from analysis.

Fibre photometry

To record fluorescence signals for the GCaMP6f and GRABDA sensors, a laser beam from a 488 nm laser (OBIS 488LS; Coherent) was reflected by a dichroic mirror (MD498; Thorlabs, Newton, NJ, USA) (Li et al., 2016) (Thinkertech Nanjing Bioscience Inc, Co., Ltd.).

KW6002 or CNO treatment

The specific adenosine A2AR antagonist KW6002 (5 mg/kg, Sundia, USA) for male C57B6/J mice was suspended in vehicle [15% DMSO (Sigma, St Louis, MO, USA), 15% ethoxylated castor oil (Sigma) and 75% saline] and was administered by intraperitoneal injection. The KW6002 and vehicle groups had six male C57B6/J mice each. CNO (Sigma) for A2A‐rM3Ds mice was dissolved in DMSO and then administered by intraperitoneal injection (1 mg/kg). The CNO and vehicle groups had six male A2A‐rM3Ds mice each.

Immunohistochemistry and imaging

Mice were deeply anaesthetized with an overdose of chloral hydrate (400 mg/kg). Transcardiac perfusion was conducted with saline, followed by 4% paraformaldehyde. Brains were removed and post‐fixed in 4% paraformaldehyde for 4–6 h at 4°C, and then allowed to equilibrate using a graded sucrose solution (10%, 20%, 30%). Brain slices (30 μm) were sectioned on a freezing microtome (Leica CM 307 1850). For immunohistochemistry analysis, we used the following primary antibodies A2AR (frontier, 1:500), mCherry (Clontech, Palo Alto, CA, USA; 1:500), D1 (Clontech, 1:500), together with secondary antibodies goat anti‐rabbit AlexaFluor‐594 (1:250), goat anti‐rat AlexaFluor‐555 (1:250). Neurons in the mouse brain expressing Gcamp6f in M1 or/and GRABDA sensors in NAc were post‐fixed, equilibrated, and sectioned. Brain slices were imaged by a fluorescence microscope.

Data analysis

Statistical analyses were performed using Graphpad Prism 5.01. Data are expressed as mean ± SD. Unpaired two‐tailed Student's t tests and Mann–Whitney U tests were used to compare two‐group data, as appropriate. The mean ‘height’ of the dopamine fluorescence signals analysed by one‐way ANOVA and followed by post hoc comparison with Fisher's Least Significant Difference (LSD) test. A P‐value of < 0.05 was considered statistically significant: * P < 0.05, ** P < 0.01, *** P < 0.001, **** P < 0.0001.

Results

Establishment of Calcium PTE and Calcium PTH to assess volitional motivation

Mice were transfected with AAV9‐syn‐GCaMP6f‐WPRE‐SV40 to express the genetically encoded Ca2+ indicator GCaMP6f in M1 neurons and the calcium fluorescence signal was monitored via a fibre photometry system (Zhang et al., 2020). Mice were trained to perform the volitionally controlled neural task at least 85% correct to obtain a reward (Fig. 1A ). To quantitatively assess volitional motivation at the neural level, we used the volitionally controlled neural task to establish Calcium PTE and Calcium PTH methods. These methods combined the behavioural concept of motivation (PRT and PHD) to represent volitional motivation by neural activity in response to progressively escalating efforts with the breakpoints representing the size of the motivation (Figs 1 and 2). We set a criterion (ΔF/F, defined threshold) for the calcium fluorescence signal in the volitionally controlled neural task, then escalated efforts by progressively increasing the number of the defined TCE or holding time after a defined threshold‐crossing. For the calcium PTE analysis, six mice received 1 day of instrumental conditioning and then 10 days of training of the volitionally controlled neural task, followed by 3 days of FR1 training (one TCE per drop of sucrose; 50 trials/day), followed by 5 days of FR5 training (five TCEs per drop of sucrose; 50 trials/day), and finally calcium PTE was assessed on the last day (Fig. 1B , upper panel). In Calcium PTE, TCEs were progressively increased to escalate volitional efforts in the sequential trials (Fig. 1B , lower panel). The breakpoint was defined as the maximal number of TCEs at which the subject stops responding to progressive escalation of efforts (progressive increase in TCEs). We analysed the calcium fluorescence signal locked into the reward delivery (±5 s) for trials 1, 3, 5, 7 9, 11 and 13, indicating these signals show a difference in response to the escalating efforts (trial by trial) in the calcium PTE test (Fig. 1C , n = 6). Moreover, the breakpoints of the six mice ranged from 13 to 19 (number of sessions) and 62 to 219 (total TCEs), indicating individual variation in volitional motivation (Fig. 1D , n = 6).

For Calcium PTH analysis, six mice received 1 day of instrumental conditioning and then training over 10 days for the volitionally controlled neural task. This was followed by 200 ms holding time above the defined threshold (criterion) to earn a drop of sucrose for 3 days, and then 240 ms holding time above the defined threshold to earn a drop of sucrose for 5 days, and finally a day of Calcium PTH test with progressive increasing holding time from 105 to 339 ms (Fig. 2A , upper panel). In Calcium PTH, holding time after crossing the defined threshold was progressively increased to escalate efforts in the sequential trials (Fig. 2A , lower panel). The breakpoint was defined as the maximal holding time at which the subject stopped responding to a progressive escalation of efforts. Similar to Calcium PTE, there was a difference in neural activity for the escalating efforts during the Calcium PTH test (Fig. 2B , n = 6). However, the difference between trials was relatively small for Calcium PTH. Furthermore, Calcium PTH analysis revealed the breakpoint distribution ranged from 218 to 307 ms in holding time above the threshold from 15 to 23 trials (Fig. 2C , n = 6). Taken together, we concluded that Calcium PTE and Calcium PTH analyses provided a quantitative assessment of volitional motivation at the level of M1 neural activity.

Striatopallidal pathway and adenosine A2A receptor modulate volitional motivation

We further used the Calcium PTE and Calcium PTH to evaluate the neural circuit modulation of volitional motivation by chemogenetic activation or pharmacological blockade of the striatopallidal pathway. The striatopallidal pathway has been confirmed to exert an inhibitory effect on behavioural motivation (Gallo et al., 2018b, 2018a; Soares‐Cunha et al., 2018). For this, we first used a transgenic approach with the genetically mutant acetylcholine receptor hM3Dq, which is unresponsive to endogenous acetylcholine, but can be activated by the exogenous ligand CNO. In this model, the transgenic hM3Dq receptors are preferentially expressed in striatopallidal neurons under control of the adenosine A2A receptor (A2AR) gene promoter, which promotes 20‐fold higher expression in striatopallidal neurons compared to other brain regions (Farrell et al., 2013). As shown in Fig. 3A–C , the transgenic hM3Dq receptors were preferentially expressed in the striatopallidal neurons and striatopallidal projections (Fig. 3A ). The transgenic hM3Dq receptors (red) co‐localized with A2AR (green) in the striatonigral neurons (Fig. 3B ), but not dopamine D1 receptor (D1R, green) (Fig. 3C ).

Figure 3. Striatopallidal pathway regulates volitional motivation.

Figure 3

Open in a new tab

A, sagittal whole‐brain expression pattern of A2A‐rM3Ds mice (Str: striatum; LGP: lateral globus pallidus). B, A2A‐rM3Ds (red) was co‐localized with A2A receptor (green) in the striatonigral neurons. Scale bar: 50 μm. C, A2A‐rM3Ds (red) was not co‐localized with D1 receptor (green) in the striatonigral neurons. Scale bar: 50 μm. D, the breakpoint distribution of the Calcium PTE test in individual mice for CNO (blue) and vehicle‐treated groups (red) (n = 12: CNO group = 6 and vehicle group = 6). E, chemogenetic activation of the striatopallidal pathway impaired the breakpoint in the Calcium PTE task (B; unpaired t test, P = 0.026, t = 2.609, df = 10). F, the mean calcium fluorescence signal change in M1 neurons before (+5 s) and after (+5 s) the reward delivery (0) for Calcium PTE in CNO (purple) and vehicle‐treated groups (red). G–I, calcium PTH under the same condition as A–C (E; unpaired t test, P = 0.047, t = 2.257, df = 10). [Colour figure can be viewed at wileyonlinelibrary.com]

After successfully establishing the stable mapping of the calcium fluorescence signal responding to the escalating efforts, the mice were tested for Calcium PTE (Fig. 3D–F ) and Calcium PTH breakpoints (Fig. 3G–I ) after intraperitoneal injection of saline or CNO 30 min before the test (1 mg/kg). Compared to the vehicle group, the breakpoint distribution of Calcium PTE (Fig. 3D ) and Calcium PTH (Fig. 3G ) in the CNO‐treated group was lower. Moreover, similar to previous reports (Gallo et al., 2018b; Soares‐Cunha et al., 2018), chemogenetic activation of the striatopallidal pathway reduced the breakpoint for Calcium PTE (Fig. 3E ; unpaired t test, P = 0.026, t = 2.609, df = 10) and Calcium PTH (Fig. 3H ; unpaired t test, P = 0.047, t = 2.257, df = 10). We also analysed calcium fluorescence signal for 5 s before and after reward delivery during Calcium PTE (Fig. 3F , n = 6) and Calcium PTH (Fig. 3I , n = 6) between CNO and vehicle groups. The calcium fluorescence signal was not different between CNO and vehicle groups. Consistent with previous studies (Farrell et al., 2013), activation of the striatopallidal pathway inhibited motor function but had no influence on the calcium fluorescence signal in M1 cortical neurons. These results suggested that Calcium PTE and Calcium PTH were sensitive to manipulation of the neural circuit that was known to control behavioural motivation and that activation of the striatopallidal pathway similarly suppressed volitional motivation as with behavioural motivation.

Lastly, we determined the effect of striatal A2ARs on volitional motivation by intraperitoneal injection (5 mg/kg) of the specific A2AR antagonist KW6002 30 min before the Calcium PTE or Calcium PTH test (Fig. 4A–F ). The breakpoint distribution of Calcium PTE (Fig. 4A ) and Calcium PTH (Fig. 4D ) in the KW6002‐treated group was higher than in the vehicle‐treated group. The breakpoint for the maximal TCEs increased compared to the vehicle‐treated group by Calcium PTE (Fig. 4B ; Mann–Whitney U test, P = 0.009). Similarly, the breakpoint for Calcium PTH in the KW6002 group was higher than in the vehicle group (Fig. 4E ; unpaired t test, P = 0.004, t = 3.699, df = 10). Moreover, this result also indicated that KW6002 acted indirectly at the striatal A2ARs with feedback onto the M1 neurons to regulate volitional control (Fig. 4C and  F ). Collectively, these data revealed that the striatopallidal pathway and A2AR activity similarly modulate volitional motivation.

Figure 4. Adenosine A2ARs regulate volitional motivation.

Figure 4

Open in a new tab

A, blockade of adenosine A2ARs enhanced motivation in volitional control (n = 12: KW6002 group = 6 and vehicle group = 6). The breakpoint distribution of Calcium PTE testing in individual mice for the KW6002 (blue) and vehicle‐treated groups (red) (A). B, blockade of adenosine A2ARs improved the breakpoint in Calcium PTE (B; Mann‐Whitney U test, P = 0.009). C, the mean calcium fluorescence signal change in M1 neurons before (+5 s) and after (+5 s) the reward delivery (0) for Calcium PTE in KW6002 (purple) and vehicle‐treated groups (red). D–F, calcium PTH under the same conditions as A–C (E; unpaired t test, P = 0.004, t = 3.699, df = 10). [Colour figure can be viewed at wileyonlinelibrary.com]

Escalating efforts produce diminishing dopamine signal in NAc during volitional and behavioural motivation

The dopamine projection from VTA (ventral tegmental area) to NAc is critical for reward motivation and reward‐driven learning (Mohebi et al., 2019). To determine the effect of dopamine in these two different motivation assessment methods, we separately monitored the dopamine dynamics in the NAc using GRABDA sensors (Sun et al., 2018) for PRT (behaviour motivation test) and Calcium PTE (volitional motivation test). As illustrated in Fig. 1B , the mice performed 3 days of FR1 training, followed by 5 days of FR5 training, and finally 1 day of Calcium PTE after the learning volitionally controlled neural task. Figure 5A indicates the loci for expression of GCamp6f in M1 and GRABDA sensors in NAc. We analysed the dopamine fluorescence signal (ΔF/F) before (10 s) and after (5 s) the delivery of the reward during FR5 training and Calcium PTE testing. However, we detected two dopamine signal peaks in the NAc during volitional control of neural activity: the prediction signal for the future reward (the signal detected within 5 s prior to the reward delivery, indicated by a black box) and reward value (the signal detected within 5 s after the reward delivery, indicated by a purple box) (Fig. 5B ). To verify dopamine dynamics for reward value, we programmed the time for the reward delivery with a delay of 10 s. Interestingly, the delayed reward delivery by 10 s was associated with a delayed phasic dopamine dynamics by about 10 s (Supplementary Information Fig. S1). These findings also demonstrated that the dopamine dynamics consisted of an early prediction component and a subsequent reward component in the NAc. We then analysed the ‘Height’ of the dopamine fluorescence signal before and after reward delivery for FR5, indicating there were significant changes in the prediction component [repeated‐measures (RM) one‐way ANOVA, P = 0.043, F 2.333, 11.67 = 4.007] and in the reward component (RM one‐way ANOVA, P = 0.035, F 2.263, 11.32 = 4.422). Furthermore, compared with FR5‐1 (first day of FR5 training) the reward prediction signal (height) of the dopamine fluorescence signal FR5‐5 (day 5 of FR5 training) was increased (Fig. 5C ; P = 0.029) and the reward value decreased in FR5‐5 (Fig. 5D ; P = 0.009,), indicating the mice increased their prediction of reward, but decreased their sensitivity to reward after 5 days of learning. During the Calcium PTE test, the required TCEs were progressively increased on each trial according to the formula given. However, the dopamine dynamics were analysed for the last 13 trials of the Calcium PTE test, indicating the dopamine dynamics for the prediction signal progressively disappeared (Fig. 5E and  F , RM one‐way ANOVA, P = 0.025, F 2.960, 14.80 = 4.182). Similarly, the mean reward predictions signal for total trials in the Calcium PTE test also disappeared (Fig. 5I ). When we analysed the correlation between the reward prediction signals and the escalating volitional efforts for each trial, we found that the reward prediction signal was negatively correlated with the escalating volitional efforts (Fig. 5G , r 2 = 0.06, P = 0.027). However, there was no correlation between the reward value and escalating volitional efforts (Fig. 5H , r 2 = 0.01, P = 0.34). In total, escalating efforts were negatively correlated with dopamine dynamics for reward prediction in NAc but not with the reward value in volitional control of neural activity.

Figure 5. Analysis of the dopamine dynamics in the NAc for calcium PTE.

Figure 5

Open in a new tab

A, illustration of GCaMP6f‐expressing loci in M1 (left) and GRABDA sensor‐expressing loci in NAc (right) (n = 6). The expression of GCaMP6f in M1 was only used for the volitionally controlled neural task. B, mean dopamine dynamics changes (ΔF/F) before (10 s) and after (5 s) the reward presentation for FR5‐1 and FR5‐5 training. C, mean ‘height’ before (−5) the reward presentation is higher in FR5‐5 training (P = 0.029). D, mean ‘height’ after the reward (+5) presentation is lower in FR5‐5 (P = 0.009). E, mean dopamine dynamic changes before (10 s) and after (5 s) the reward presentation in the Calcium PTE test for the last 13 trials (aligned to the last trial). F, mean ‘height’ for each trial before (5 s) the reward delivery in the Calcium PTE test (RM one‐way ANOVA, P = 0.025, F 2.960, 14.80 = 4.182). G–I, correlation analysis of the dopamine dynamics for the reward prediction (F, P = 0.027, r 2 = 0.06) and reward value (G, P = 0.34, r 2 = 0.01) with the volitional efforts in Calcium PTE. H, mean dopamine dynamic changes before (10 s) and after (5 s) the reward delivery in the Calcium PTE test. ‘0’ represents the reward delivery; ‘height’ represents the highest peak of dopamine dynamics of ±5 s of reward delivery. The black box indicates the prediction signal, and the purple box indicates the reward value signal. [Colour figure can be viewed at wileyonlinelibrary.com]

Lastly, we also analysed dopamine dynamics in the NAc in response to escalating efforts (with PRT) during motor skills (Fig. 6). Figure 6A indicates the loci for expression of GRABDA sensors in NAc. There were also two dopamine signal peaks in the NAc during motor skills: the prediction signal for the future reward (the black box) and reward value (the purple box) (Fig. 6B ). We then analysed the ‘Height’ of the dopamine fluorescence signal before and after reward delivery for FR5, indicating there were significant changes in the prediction component (RM one‐way ANOVA, P = 0.044, F 2.115, 14.80 = 3.839) and in the reward component (RM one‐way ANOVA, P = 0.027, F 2.797, 19.58 = 3.874). As with volitional control of neural activity, the reward prediction signal and reward value of the dopamine fluorescence signal changed significantly between FR5‐1 and FR5‐5 (Fig. 6C , P = 0.049; Fig. 6D , P = 0.01). The reward prediction signals of dopamine largely disappeared during trial‐by‐trial PRT testing (Fig. 6E and  F , RM one‐way ANOVA, P = 0.046, F 2.048, 10.24 = 4.184). The mean reward predictions signal for total trials also disappeared in the Calcium PTH test (Fig. 6I ). Similarly, correlation analyses revealed that dopamine dynamics for the reward prediction was negatively correlated with the escalating behavioural efforts during PRT testing (Fig. 6G, r 2 = 0.06, P = 0.018), but there was no correlation between the reward value and escalating behavioural efforts (Fig. 6H , r 2 = 0.01, P = 0.56). These results also indicated that escalating efforts were negatively correlated with dopamine dynamics for reward prediction in NAc but not with the reward value in motor skills.

Figure 6. Analysis of dopamine dynamics in the NAc for PRT.

Figure 6

Open in a new tab

A, illustration of GRABDA sensor expression and the fluorescence signal observation loci in NAc (n = 8). B, mean dopamine dynamics changes (ΔF/F) before (10 s) and after (5 s) the reward presentation for FR5‐1 and FR5‐5 training. C, mean ‘height’ before (−5) the reward presentation is higher in FR5‐5 training (P = 0.049). D, mean ‘height’ after the reward (+5) presentation is lower in FR5‐5 (pair t test, P = 0.01). E, mean dopamine dynamic changes before (10 s) and after (5 s) the reward presentation in PRT testing for the last nine trials (aligned to the last trial) (n = 6). F, mean ‘height’ for each trial before (5 s) the reward delivery in PRT test (RM one‐way ANOVA, P = 0.046, F 2.048, 10.24 = 4.184, n = 6). F and G, correlation analysis of the dopamine dynamics for the reward prediction (F, P = 0.018, r 2 = 0.06) and reward value (G, P = 0.56, r 2 = 0.01) with the behavioural efforts in PRT. H, the mean dopamine dynamic changes before (10 s) and after (5 s) the reward delivery in PRT test. ‘0’ represents the reward delivery, and ‘height’ represents the highest peak of dopamine dynamics of 0 to −5 s or 0 to +5 s of reward delivery. The black box indicates the prediction signal, and the purple box indicates the reward value signal. [Colour figure can be viewed at wileyonlinelibrary.com]

Discussion

Development of the first neural representation and quantitative assessment of volitional motivation

In this study, by utilizing the causal link between neuron activity and criteria set by the experimenter, we have adopted the concept of behavioural motivation in PRT and PHD to develop Calcium PTE and Calcium PTH tests, which allow us to directly link the calcium signal (neural activity) to the escalating effort‐related motivation (i.e. a subject is willing to expend to earn a reward) during volitional conditioning of neural activity. This quantitative analysis of volitional motivation by Calcium PTE and Calcium PTH allowed us to determine individual variations in volitional motivation at the neural level. The validity of these calcium‐based PTE and PTH analyses for quantification of the volitional motivation is supported by the chemogenetic finding that activation of the striatopallidal pathway inhibited motivation in the neuroprosthetic control, in agreement with previous studies on behavioural motivation (Gallo et al., 2018b; Ruder et al., 2021).

The development of the first neural representation and quantitative method for volitional motivation provides opportunities to address the specific contribution of the neural circuit and neuromodulator for BMI improvement. From the perspective of human subjects, the assessment and training of cognitive impairments in advanced stages of paralysis represent a challenge as the standard assessment of motivation typically involves a motor response. However, some or all motor abilities are lost in stroke patients and in other cases of severe motor loss (Carelli et al., 2017). Therefore, quantitative analysis of motivation at the neuron level in disabled patients may lead to a new therapeutic approach to enhance motivation during neurorehabilitation.

Dopamine dynamics in the NAc reflect cue‐triggered reward ‘wanting’ not escalating efforts

Motivation and reinforcement learning has been classically associated with dopamine neurons in the VTA that predominantly project to the NAc (Volkow et al., 2017). The critical role of the dopamine reward system in motivational control of behaviours is supported by the finding that disrupting dopamine transmission by pharmacological and neurotoxic approaches regulates response vigour (Hamid et al., 2016; Salamone & Correa, 2012) and work output (Salamone et al., 2018). Animals with impaired dopamine transmission reallocate their instrumental behaviour away from food‐reinforced tasks with high response requirements, and instead select less effortful food‐seeking behaviours (Nunes et al., 2013; Salamone et al., 2001). The instrumental output and effort‐related choice impaired by dopamine D2 antagonism were reversed by A2AR blockade or genetic deletion (Collins et al., 2012; Farrar et al., 2007; Mott et al., 2009; Pardo et al., 2012; Salamone et al., 2009). Indeed, dopamine dynamics in the NAc encode the reward prediction error (Schultz, 2016a, 2016b, 2016c; du Hoffmann & Nicola, 2014), efforts and delay‐related costs (Day et al., 2010) and modulate rewards through delays conferred by the escalating costs (Wanat et al., 2010). Consistent with the prediction error signal (An et al., 2019; Yao et al., 2021), we detected the development of prediction signal (i.e. calcium fluorescence signal associated with cue presentation, before the reward) in the repeated FR1→FR5 trials. Furthermore, according to the incentive salience hypothesis, a reward cue triggers ‘wanting’ and potentiates instrumental performance for that reward (Wyvell & Berridge, 2000). Applying this hypothesis, a behaviour can be designed to gradually enhance or decrease ‘wanting’ to test incentive motivation. Hamid et al. (2016) reported that the same dynamically fluctuating dopamine signal influences both current and future motivated behaviour by monitoring dopamine fluctuation in the NAc through the enhancement of ‘wanting’. In our study, Calcium PTE and Calcium PTH assessment of volitional motivation revealed that the prediction signal was negatively correlated with the breakpoint, indicating that escalating efforts caused the gradual decrease in ‘wanting’ for the same reward. Thus, the dopamine (prediction) signal in the (trial‐by‐trial) progressively escalating effort scheme encoded the ‘wanting’ but not escalating efforts. Overall, dopamine dynamics in the NAc encodes the rewards cue‐triggered ‘wanting’ and the subjective value of reward.

Volitional control of neural activity shares brain structures and learning mechanisms, including motivational control

The direct control of neural activity in BMIs may be a consequence of the integration of the cortical system, subcortical motivational areas and neurotransmitter system information, indicating that neural activity may represent integrated signals (An et al., 2019; Marsh et al., 2015; Ramkumar et al., 2016; Ramakrishnan et al., 2017; Yao et al., 2021; Zhao et al., 2018). The acquisition of neuroprosthetic learning also accompanies the creation of neural networks with distinct neural plasticity patterns (Orsborn & Carmena, 2013; Zhang et al., 2020). In contrast to natural motor skill control, BMIs involve only limited (but distinct) direct neurons that are decoded to control the neuroprosthetic device (Chapin et al., 1999; Hira et al., 2013). However, a simple task, such as pressing a lever, are known to involve bilateral control of motor programmes in different brain areas and the brainstem motor ‘centres’ (Lopez‐Huerta et al., 2021).Then, Calcium PTE only reinforced the limited population neurons in the M1 cortex to acquire reward in operant conditioning. Interestingly, dopamine dynamics in NAc were similar in the volitional Calcium PTE test and behavioural PRT. Notably, we recently demonstrated that A2AR antagonists can enhance volitional control using our current neuroprosthetic learning paradigm (Zhang et al., 2020). Our follow‐on analysis suggested that A2ARs improve BMI performance by increasing motivational control given that antagonizing A2ARs enhanced the breakpoint of Calcium PTE (Li et al., 2018; Zhang et al., 2020). Thus, dopamine dynamics and adenosine A2AR activity similarly contribute to volitional motivation control of neural activity in a similar manner to behavioural motivation. Furthermore, as learning and skilful volitional control of neural activity rely on the natural motor repertoire (Hwang et al., 2013), increasing evidence suggests that both motor and neuroprosthetic learning processes share a common circuit structure. For example, corticostriatal plasticity is also essential for learning intentional neuroprosthetic skills (Koralek et al., 2013, 2012) and the emergence of coordinated neural dynamics underlies neuroprosthetic learning. Moreover, reaching proficient control with cohesive neural firing patterns (Koralek et al., 2013, 2012; Marchesotti et al., 2017; Neely RM et al., 2018) similarly requires reinforcement learning with a lot of repetitive training to produce stable representation mapping (Athalye, 2018; Pohlmeyer et al., 2014). Our study further confirms that both behavioural and volitional conditionings are driven by motivational factors with similar modulation at the neural circuit and the neurotransmitter levels. Indeed, we found that chemogenetic activation of the striatopallidal neurons similarly suppresses volitional motivation (as evident by reduced breakpoint of Calcium PTE) and behavioural motivation (Farrell et al., 2013; Gallo et al., 2018b, 2018a; Soares‐Cunha et al., 2018). Collectively, the above‐mentioned studies together with ours suggest that the similarities between volitionally controlled neural activity and control of motor behaviours far outweigh their differences.

Conclusions

We have developed novel methods for detecting volitional motivation based on representation of the M1 population neural activity in response to progressively escalating efforts. Meanwhile, we further verified volitional control of population neural activity in shared brain structures and learning mechanisms including motivational control with sensorimotor learning.

Additional information

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Author contributions

L.Z. designed the experiments. J.‐F.C. and L.Z. conceptualized the project. C.L. performed most of the experiments. X.Z., H.Z. and S.L. provided experimental facilities, administrative assistance, implant surgery and immunohistochemistry. L.Z., Q.W. and Z.Y. analysed the data and acquired the funding. J.‐F.C. and L.Z. wrote and revised the paper. All authors read and approved the final version of the manuscript. All persons designated as authors qualify for authorship, and all those who qualify for authorship are listed.

Funding

This work was supported by Zhejiang Provincial Natural Science Foundation Grant Nos. LY22C090005 (to L.P.Z.), LQ19C090005 (to Q.W.), LQ18C090002 (to Z.M.Y.).

Compliance with Ethics Requirements

All Institutional and National Guidelines for the care and use of mice were followed.

Supporting information

Statistical Summary Document

Click here for additional data file. (46.9KB, xlsx)

Peer Review History

Click here for additional data file. (6.3MB, pdf)

Figure S1

Click here for additional data file. (202.8KB, tif)

Acknowledgements

We thank Shaomin Zhang from Zhejiang University and Yi Zhang from Wenzhou Medical University for commenting on earlier versions of the manuscript.

Biographies

Liping Zhang received her PhD degree from Wuhan University in 2013. She worked at the school of Optometry and Ophthalmology, Wenzhou Medical University, in 2013 and joined the molecular neuropharmacology lab in 2015. Currently, her research focuses on neural circuit mechanisms of volitional control of neural activity and cognitive disorders in neurodegenerative diseases.

graphic file with name TJP-601-631-g005.gif

Jiang‐Fan Chen has worked at Massachusetts General Hospital/Harvard Medical School as Instructor (1997) and Assistant Professor (1999) and at Boston University School of Medicine as Associate Professor (2003) and Professor (2010) before returning to Wenzhou Medical University as Professor (2016 to present). Professor Chen's work has contributed to our understanding of adenosine receptor neurobiology as an integrated neuromodulator of cognition and other basal ganglia functions as well as important therapeutic strategies for neurodegenerative and neuropsychiatric disorders. Currently, he is the director of the Molecular Neuropharmacology Lab, School of Optometry and Ophthalmology, Wenzhou Medical University.

graphic file with name TJP-601-631-g006.gif

Handling Editors: Richard Carson & Jing‐Ning Zhu

The peer review history is available in the Supporting Information section of this article (https://10.1113/JP283915#support‐information‐section).

L. Zhang and C. Liu contributed equally to this work.

Contributor Information

Liping Zhang, Email: lip_zhang@eye.ac.cn.

Jiang‐Fan Chen, Email: chenjf555@gmail.com.

Data availability statement

Data will be made available upon reasonable request to the corresponding author.

References

  1. An, J. , Yadav, T. , Hessburg, J. P. , & Francis, J. T. (2019). Reward expectation modulates local field potentials, spiking activity and spike‐field coherence in the primary motor cortex. eNeuro, 6(3), ENEURO.0178‐19.2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Athalye, V.R. S.F. , Carmena, J. M. , & Costa, R. M. (2018). Evidence for a neural law of effect. Science, 359(6379), 1024–1029. [DOI] [PubMed] [Google Scholar]
  3. Bailey, M. R. , Jensen, G. , Taylor, K. , Mezias, C. , Williamson, C. , Silver, R. , Simpson, E. H. , & Balsam, P. D. (2015). A novel strategy for dissecting goal‐directed action and arousal components of motivated behavior with a progressive hold‐down task. Behavioral Neuroscience, 129(3), 269–280. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Berridge, K. C. (2004). Motivation concepts in behavioral neuroscience. Physiology & Behavior, 81(2), 179–209. [DOI] [PubMed] [Google Scholar]
  5. Bradshaw, C. M. , & Killeen, P. R. (2012). A theory of behaviour on progressive ratio schedules, with applications in behavioural pharmacology. Psychopharmacology, 222(4), 549–564. [DOI] [PubMed] [Google Scholar]
  6. Carelli, L. , Solca, F. , Faini, A. , Meriggi, P. , Sangalli, D. , Cipresso, P. , Riva, G. , Ticozzi, N. , Ciammola, A. , Silani, V. , & Poletti, B. (2017). Brain‐computer interface for clinical purposes: Cognitive assessment and rehabilitation. BioMed Research International, 2017, 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Chapin, J. K. , Moxon, K. A. , Markowitz, R. S. , & Nicolelis, M. A. (1999). Real‐time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nature Neuroscience, 2(7), 664–670. [DOI] [PubMed] [Google Scholar]
  8. Chase, S. M. , Kass, R. E. , & Schwartz, A. B. (2012). Behavioral and neural correlates of visuomotor adaptation observed through a brain‐computer interface in primary motor cortex. Journal of Neurophysiology, 108(2), 624–644. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Chen, T. W. , Wardill, T. J. , Sun, Y. , Pulver, S. R. , Renninger, S. L. , Baohan, A. , Schreiter, E. R. , Kerr, R. A. , Orger, M. B. , Jayaraman, V. , Looger, L. L. , Svoboda, K. , & Kim, D. S. (2013). Ultrasensitive fluorescent proteins for imaging neuronal activity. Nature, 499(7458), 295–300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Collins, L. E. , Sager, T. N. , Sams, A. G. , Pennarola, A. , Port, R. G. , Shahriari, M. , & Salamone, J. D. (2012). The novel adenosine A2A antagonist Lu AA47070 reverses the motor and motivational effects produced by dopamine D2 receptor blockade. Pharmacology Biochemistry and Behavior, 100(3), 498–505. [DOI] [PubMed] [Google Scholar]
  11. Cook, D. A. , & Artino A. R., Jr (2016). Motivation to learn: An overview of contemporary theories. Medical Education, 50(10), 997–1014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Day, J. J. , Jones, J. L. , Wightman, R. M. , & Carelli, R. M. (2010). Phasic nucleus accumbens dopamine release encodes effort‐ and delay‐related costs. Biological Psychiatry, 68(3), 306–309. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. du Hoffmann, J. , & Nicola, S. M. (2014). Dopamine invigorates reward seeking by promoting cue‐evoked excitation in the nucleus accumbens. Journal of Neuroscience, 34(43), 14349–14364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Eaton, R. W. , Libey, T. , & Fetz, E. E. (2017). Operant conditioning of neural activity in freely behaving monkeys with intracranial reinforcement. Journal of Neurophysiology, 117(3), 1112–1125. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Farrar, A. M. , Pereira, M. , Velasco, F. , Hockemeyer, J. , Muller, C. E. , & Salamone, J. D. (2007). Adenosine A(2A) receptor antagonism reverses the effects of dopamine receptor antagonism on instrumental output and effort‐related choice in the rat: Implications for studies of psychomotor slowing. Psychopharmacology, 191(3), 579–586. [DOI] [PubMed] [Google Scholar]
  16. Farrell, M. S. , Pei, Y. , Wan, Y. , Yadav, P. N. , Daigle, T. L. , Urban, D. J. , Lee, H. M. , Sciaky, N. , Simmons, A. , Nonneman, R. J. , Huang, X. P. , Hufeisen, S. J. , Guettier, J. M. , Moy, S. S. , Wess, J. , Caron, M. G. , Calakos, N. , & Roth, B. L. (2013). A Galphas DREADD mouse for selective modulation of cAMP production in striatopallidal neurons. Neuropsychopharmacology, 38(5), 854–862. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Fetz, E. E. (1969). Operant conditioning of cortical unit activity science. Science, 163(3870), 955–958. [DOI] [PubMed] [Google Scholar]
  18. Fetz, E. E. (2007). Volitional control of neural activity: Implications for brain‐computer interfaces. The Journal of Physiology, 579(3), 571–579. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gallo, E. F. , Meszaros, J. , Sherman, J. D. , Chohan, M. O. , Teboul, E. , Choi, C. S. , Moore, H. , Javitch, J. A. , & Kellendonk, C. (2018b) Accumbens dopamine D2 receptors increase motivation by decreasing inhibitory transmission to the ventral pallidum. Nature Communications, 9(1), 1086. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Gallo, E. F. , Meszaros, J. , Sherman, J. D. , Chohan, M. O. A.‐O. , Teboul, E. , Choi, C. S. , Moore, H. , Javitch, J. A. , & Kellendonk, C. (2018a) Accumbens dopamine D2 receptors increase motivation by decreasing inhibitory transmission to the ventral pallidum. Pharmacological Reviews, 70, 747–762. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hamid, A. A. , Pettibone, J. R. , Mabrouk, O. S. , Hetrick, V. L. , Schmidt, R. , Vander Weele, C. M. , Kennedy, R. T. , Aragona, B. J. , & Berke, J. D. (2016). Mesolimbic dopamine signals the value of work. Nature Neuroscience, 19(1), 117–126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Hira, R. , Ohkubo, F. , Ozawa, K. , Isomura, Y. , Kitamura, K. , Kano, M. , Kasai, H. , & Matsuzaki, M. (2013). Spatiotemporal dynamics of functional clusters of neurons in the mouse motor cortex during a voluntary movement. Journal of Neuroscience, 33(4), 1377–1390. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hwang, E. J. , Bailey, P. M. , & Andersen, R. A. (2013). Volitional control of neural activity relies on the natural motor repertoire. Current Biology, 23(5), 353–361. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Ishikawa, D. , Matsumoto, N. , Sakaguchi, T. , Matsuki, N. , & Ikegaya, Y. (2014). Operant conditioning of synaptic and spiking activity patterns in single hippocampal neurons. Journal of Neuroscience, 34(14), 5044–5053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kleih, S. C. , Riccio, A. , Mattia, D. , Kaiser, V. , Friedrich, E. V. C. , Scherer, R. , Muller‐Putz, G. , Neuper, C. , & Kubler, A. (2011). Motivation influences Performance in SMR‐BCI.
  26. Kleih, S. C. , Riccio, A. , Mattia, D. , Schreuder, M. , Tangermann, M. , Zickler, C. , Neuper, C. , Kübler, A. , & Westbrook, A. (2011) Motivation affects performance in a P300 brain computer interface. International Journal of Bioelectromagnetism, 13, 46–47. [Google Scholar]
  27. Koralek, A. C. , Costa, R. M. , & Carmena, J. M. (2013). Temporally precise cell‐specific coherence develops in corticostriatal networks during learning. Neuron, 79(5), 865–872. [DOI] [PubMed] [Google Scholar]
  28. Koralek, A. C. , Jin, X. , Long J. D., 2nd , Costa, R. M. , & Carmena, J. M. (2012). Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills. Nature, 483(7389), 331–335. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Li, Y. , Pan, X. , He, Y. , Ruan, Y. , Huang, L. , Zhou, Y. , Hou, Z. , He, C. , Wang, Z. , Zhang, X. , & Chen, J. F. (2018). Pharmacological blockade of adenosine A2A but not A1 receptors enhances goal‐directed valuation in satiety‐based instrumental behavior. Frontiers in Pharmacology, 9, 393. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Li, Y. , Zhong, W. , Wang, D. , Feng, Q. , Liu, Z. , Zhou, J. , Jia, C. , Hu, F. , Zeng, J. , Guo, Q. , Fu, L. , & Luo, M. (2016). Serotonin neurons in the dorsal raphe nucleus encode reward signals. Nature Communications, 7(1), 10503. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Lopez‐Huerta, V. G. , Denton, J. A. , Nakano, Y. , Jaidar, O. , Garcia‐Munoz, M. , & Arbuthnott, G. W. (2021). Striatal bilateral control of skilled forelimb movement. Cell Reports, 34(3), 108651. [DOI] [PubMed] [Google Scholar]
  32. Marchesotti, S. , Martuzzi, R. , Schurger, A. , Blefari, M. L. , Del Millan, J. R. , Bleuler, H. , & Blanke, O. (2017). Cortical and subcortical mechanisms of brain‐machine interfaces. Human Brain Mapping, 38(6), 2971–2989. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Marsh, B. T. , Tarigoppula, V. S. , Chen, C. , & Francis, J. T. (2015). Toward an autonomous brain machine interface: Integrating sensorimotor reward modulation and reinforcement learning. Journal of Neuroscience, 35(19), 7374–7387. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Mohebi, A. , Pettibone, J. R. , Hamid, A. A. , Wong, J. T. , Vinson, L. T. , Patriarchi, T. , Tian, L. , Kennedy, R. T. , & Berke, J. D. (2019). Dissociable dopamine dynamics for learning and motivation. Nature, 570(7759), 65‐70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Moritz, C. T. , & Fetz, E. E. (2011). Volitional control of single cortical neurons in a brain‐machine interface. Journal of Neural Engineering, 8(2), 025017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Mott, A. M. , Nunes, E. J. , Collins, L. E. , Port, R. G. , Sink, K. S. , Hockemeyer, J. , Muller, C. E. , & Salamone, J. D. (2009). The adenosine A2A antagonist MSX‐3 reverses the effects of the dopamine antagonist haloperidol on effort‐related decision making in a T‐maze cost/benefit procedure. Psychopharmacology, 204(1), 103–112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Neely, R. M. , Koralek, A. C. , Athalye, V. R. , Costa, R. M. , & JM, C. (2018). Volitional modulation of primary visual cortex activity requires the Basal Ganglia. Neuron, 97(6), 1356–1368.e4. [DOI] [PubMed] [Google Scholar]
  38. Nunes, E. J. , Randall, P. A. , Podurgiel, S. , Correa, M. , & Salamone, J. D. (2013). Nucleus accumbens neurotransmission and effort‐related choice behavior in food motivation: Effects of drugs acting on dopamine, adenosine, and muscarinic acetylcholine receptors. Neuroscience and Biobehavioral Reviews, 37(9), 2015–2025. [DOI] [PubMed] [Google Scholar]
  39. Orsborn, A. L. , & Carmena, J. M. (2013). Creating new functional circuits for action via brain‐machine interfaces. Frontiers in Computational Neuroscience, 7, 157. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Pardo, M. , Lopez‐Cruz, L. , Valverde, O. , Ledent, C. , Baqi, Y. , Muller, C. E. , Salamone, J. D. , & Correa, M. (2012). Adenosine A2A receptor antagonism and genetic deletion attenuate the effects of dopamine D2 antagonism on effort‐based decision making in mice. Neuropharmacology, 62(5–6), 2068–2077. [DOI] [PubMed] [Google Scholar]
  41. Pohlmeyer, E. A. , Mahmoudi, B. , Geng, S. , Prins, N. W. , & Sanchez, J. C. (2014). Using reinforcement learning to provide stable brain‐machine interface control despite neural input reorganization. PLoS ONE, 9(1), e87253. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Ramakrishnan, A. , Byun, Y. W. , Rand, K. , Pedersen, C. E. , Lebedev, M. A. , & Nicolelis, M. A. L. (2017). Cortical neurons multiplex reward‐related signals along with sensory and motor information. PNAS, 114(24), E4841–E4850. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Ramkumar, P. , Dekleva, B. , Cooler, S. , Miller, L. , & Kording, K. (2016). Premotor and motor cortices encode reward. PLoS ONE, 11(8), e0160851. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Ruder, L. , Schina, R. , Kanodia, H. , Valencia‐Garcia, S. , Pivetta, C. , & Arber, S. (2021). A functional map for diverse forelimb actions within brainstem circuitry. Nature, 590(7846), 445–450. [DOI] [PubMed] [Google Scholar]
  45. Salamone, J. D. , Correa, M. , Ferrigno, S. , Yang, J. H. , Rotolo, R. A. , & Presby, R. E. (2018). The psychopharmacology of effort‐related decision making: Dopamine, adenosine, and insights into the neurochemistry of motivation. Pharmacological Reviews, 70(4), 747–762. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Salamone, J. D. , Farrar, A. M. , Font, L. , Patel, V. , Schlar, D. E. , Nunes, E. J. , Collins, L. E. , & Sager, T. N. (2009). Differential actions of adenosine A1 and A2A antagonists on the effort‐related effects of dopamine D2 antagonism. Behavioural Brain Research, 201(1), 216–222. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Salamone, J. D. , Wisniecki, A. , Carlson, B. B. , & M, C. (2001). Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement. Neuroscience, 105(4), 863–870. [DOI] [PubMed] [Google Scholar]
  48. Salamone John, D. , & Correa, M. (2012). The mysterious motivational functions of mesolimbic dopamine. Neuron, 76(3), 470–485. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Schmidt, E. M. , Bak, M. J. , McIntosh, J. S. , & Thomas, J. S. (1977). Operant conditioning of firing patterns in monkey cortical neurons. Experimental Neurology, 54(3), 467–477. [DOI] [PubMed] [Google Scholar]
  50. Schultz, W. (2016a) Dopamine reward prediction‐error signalling: A two‐component response. Nature Reviews Neuroscience, 17(3), 183–195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Schultz, W. (2016b) Reward functions of the basal ganglia. Journal of Neural Transmission (Vienna, Austria : 1996), 123(7), 679–693. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Schultz, W. (2016c) Dopamine reward prediction error coding. Dialogues in Clinical Neuroscience, 18(1), 23–32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Soares‐Cunha, C. , Coimbra, B. , Domingues, A. V. , Vasconcelos, N. , Sousa, N. , & Rodrigues, A. J. (2018). Nucleus accumbens microcircuit underlying D2‐MSN‐driven increase in motivation. eNeuro, 5(2), 0386–0318. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Sun, F. , Zeng, J. , Jing, M. , Zhou, J. , Feng, J. , Owen, S. F. , Luo, Y. , Li, F. , Wang, H. , Yamaguchi, T. , Yong, Z. , Gao, Y. , Peng, W. , Wang, L. , Zhang, S. , Du, J. , Lin, D. , Xu, M. , Kreitzer, A. C. , … Li, Y. (2018). A genetically encoded fluorescent sensor enables rapid and specific detection of dopamine in flies, fish, and mice. Cell, 174(2), 481–496.e19. e419. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Volkow, N. D. , Wise, R. A. , & Baler, R. (2017). The dopamine motive system: Implications for drug and food addiction. Nature Reviews Neuroscience, 18(12), 741–752. [DOI] [PubMed] [Google Scholar]
  56. Wanat, M. J. , Kuhnen, C. M. , & Phillips, P. E. (2010). Delays conferred by escalating costs modulate dopamine release to rewards but not their predictors. Journal of Neuroscience, 30(36), 12020–12027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Wyler, A. R. , & Prim, M. M. (1976). Operant conditioning of tonic neuronal firing rates from single units in monkey motor cortex. Brain Research, 117(3), 498–502. [DOI] [PubMed] [Google Scholar]
  58. Wyvell, C. L. , & Berridge, K. C. (2000). Intra‐accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward “wanting” without enhanced “liking” or response reinforcement. Journal of Neuroscience, 20(21), 8122–8130. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Yao, Z. , Hessburg, J. P. , & Francis, J. T. (2021). Normalization by valence and motivational intensity in the sensorimotor cortices (PMd, M1, and S1). Scientific Reports, 11(1), 24221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Zhang, L. , Zhou, Y. , Liu, C. , Zheng, W. , Yao, Z. , Wang, Q. , Jin, Y. , Zhang, S. , Chen, W. , & Chen, J. F. (2020). Adenosine A2A receptor blockade improves neuroprosthetic learning by volitional control of population calcium signal in M1 cortical neurons. Neuropharmacology, 178, 108250. [DOI] [PubMed] [Google Scholar]
  61. Zhao, Y. , Hessburg, J. P. , Asok Kumar, J. N. , & Francis, J. T. (2018). Paradigm shift in sensorimotor control research and brain machine interface control: The influence of context on sensorimotor representations. Frontiers in Neuroscience, 12, 579. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Statistical Summary Document

Click here for additional data file. (46.9KB, xlsx)

Peer Review History

Click here for additional data file. (6.3MB, pdf)

Figure S1

Click here for additional data file. (202.8KB, tif)

Data Availability Statement

Data will be made available upon reasonable request to the corresponding author.

Articles from The Journal of Physiology are provided here courtesy of Wiley

RESOURCES