Associations between aerobic exercise and dopamine-related reward-processing: Informing a model of human exercise engagement

Abstract

Endurance or aerobic exercise has many physical and mental health benefits, but less is known about the specific impact that cardiovascular activity may have on dopamine-associated brain circuits involved in reward processing and mood regulation in humans. Understanding such effects will help to explain individual differences in both exercise uptake and maintenance. This study evaluated neural response to a classical taste-conditioning reward prediction error task with the use of functional magnetic resonance imaging, along with data on self-reported aerobic exercise among healthy young adult females (N = 111). Results indicated positive associations between reported aerobic exercise and regional brain response that remained significant after multiple comparison for the right medial orbital frontal cortex response to unexpected sucrose receipt (r = 0.315, p = .0008). The medial orbitofrontal cortex is implicated in reward and outcome value computation and the results suggest that aerobic exercise may strengthen this circuitry, or reciprocally, higher orbitofrontal cortical activity may reinforce exercise behavior. The findings aid in developing a model of how exercise engagement can modify reward-circuit function and could be used therapeutically in conditions associated with altered brain salience response.

Introduction

Exercise has many well-documented benefits, including offsetting a variety of chronic diseases (Pedersen & Saltin, 2015), improving mood and cognitive function (Basso & Suzuki, 2017), and in helping to manage mental health disorders (Smith & Merwin, 2021). Given that the long-term adoption of adaptive exercise behavior holds great potential for global public health (WHO, 2020), a deeper understanding of biobehavioral mechanisms that modulate and support exercise engagement is of great interest and utility.

Just over a decade ago, it was proposed that the means by which humans might be motivated to exercise was by way of the dopamine system (Knab & Lightfoot, 2010). Animal models support the idea that dopaminergic signaling regulates engagement in physical activity (Beeler & Burghardt, 2021; Foldi et al., 2017), but to date, we know much less about the impact of naturalistic aerobic exercise on brain activation, and how this potentially dopamine-mediated process might serve to reinforce human exercise behavior. A robust association between exercise engagement and activation in reward-related brain regions would suggest a population for whom engaging in exercise might be more motivating, and therefore easier to initiate and maintain. It is also possible that engaging in regular aerobic exercise brings about alterations in neural reward response that reflexively reinforce this behavior, which could explain why some individuals exercise in a maladaptive, excessive manner (Cunningham et al., 2016). Taken together, improved understanding of the neural mechanisms that may motivate and maintain exercise engagement holds potential for promoting the adoption of sustained and adaptive exercise behavior across both healthy and clinical samples.

Furthermore, targeted exercise could be important to manipulate or modulate dopamine-related neuronal activation as dopamine signaling helps to generate learning (Graybiel & Grafton, 2015) and promote approach behaviors (Wise, 2004). A majority of this dopamine-associated learning is thought to involve the mesolimbic pathway, implicating brain regions associated with goal-directed decision-making and the ability to maintain flexible responding based on the value of a given reward, or “reinforcer” (Gourley et al., 2016). Activation of this pathway provides the foundation for behavior initiation as well as reinforcing behavior once it has been initiated, and for generating conditioned responses (Schultz, 2016; Wise, 2004). We can explain these processes in part through examining reward prediction error (RPE) (Watabe-Uchida et al., 2017), a dopamine-associated signal that is generated when evaluating the difference between an expectation and an outcome (Schultz, 2016). The absolute value of the RPE represents the extent to which a deviation from what was expected was surprising, and it is conceptualized as a reflection of motivational salience (Fouragnan et al., 2017). Expectation and outcome can also be analyzed separately, providing information on brain response to unexpected receipt or omission of a stimulus. Whether exercise behavior is related primarily to receipt or omission brain response is not known.

In summary, there may be reciprocal effects between aerobic exercise and dopamine-related brain reward processing. To examine these possibilities, the current study conducted secondary data analysis to evaluate neural response to a classic RPE task with the use of functional magnetic resonance imaging (fMRI), along with data on self-reported aerobic exercise among healthy young adult females. In the original study that specifically sought to examine RPE among individuals with transdiagnostic eating disorders, brain salience response was inversely correlated with body mass index and binge-eating behavior, and positively correlated with ventral-striatal hypothalamic effective connectivity (Frank et al., 2021). Those results suggested that food restriction and overeating may alter brain circuitry in opposite directions and reinforce an individual’s eating disorder behavior.

In the current study, we sought to examine the potential reciprocal nature of dopamine-related reward response and specifically exercise behavior, among healthy controls. Given prior work both in rodents (Beeler & Burghardt, 2021) as well as humans (Flack et al., 2021) that implicates increased activity in the dopamine system relative to physical activity, we hypothesized that we would find indication for greater salience response in the dopamine system relative to increased report of exercise. Identifying differences in neural activation between those who exercise more often compared to those who do not will help to develop a model of how exercise engagement can modify dopamine function and could be used therapeutically in conditions associated with altered brain salience response.

Methods

Participants and procedures

The current study comprises data from healthy young adult females (N = 111), drawn from a larger study (c.f., Frank et al., 2021). Participants were right-handed without history of head trauma, neurological disease, or other major medical illness; they were without history of any lifetime psychiatric disorder and were studied during the first 10 days of the menstrual cycle to reduce hormonal confounds. Psychiatric diagnoses were excluded using the Structured Clinical Interview for DSM-5 (doctoral-level interviewer) (First et al., 2015).

All procedural details are available elsewhere (Frank et al., 2021). In brief, all subjects participated in a classic sucrose taste-conditioning paradigm to evoke the dopamine-related RPE response. We asked participants to report their weekly minutes of endurance or aerobic exercise activities (Plowman & Smith, 2014). Those exercise behaviors had to be stable for at least three months. The activities were further defined as those that increase breathing and heart rate and are usually associated with sweating; examples were provided such as running, cycling, cardio exercises on devices such as elliptical or treadmill machines.

The Colorado Multiple Institutional Review Board approved the study. All participants provided written informed consent.

Brain Imaging Methods

Functional Magnetic Resonance Imaging (fMRI).

Between 0700 and 0900 hours, participants ate a provided breakfast (see Frank et al., 2021 for detail). FMRI of the brain was performed between 0800 and 0900 hours (3T GE Signa or Siemens Skyra 3T scanner).

Taste Reward Task.

The design was adapted from (O’Doherty et al., 2003). Participants learned to associate three unconditioned taste stimuli (US: 1 molar [M] sucrose solution [100 trials], no solution [100 trials], or artificial saliva [80 trials]) with paired conditioned visual stimuli (CS) during scanning (total task duration = 28 minutes). Each CS was probabilistically associated with its US such that 20% of sucrose and no solution CS trials were unexpectedly followed by no solution or sucrose US, respectively.

fMRI Analysis.

Image preprocessing and analysis were performed using SPM12 (http://www.fil.ion.ucl.ac.uk/spm/software/spm12/). Full information is available in the main report (Frank et al., 2021). In brief, images were realigned to the first volume, normalized to the Montreal Neurological Institute template, and smoothed at 6mm full-width-at-half-maximum Gaussian kernel. Data were preprocessed with slice-time correction and modeled with a hemodynamic response convolved function using the general linear model, including temporal and dispersion derivatives. A 128-second high-pass filter (removing low-frequency BOLD signal fluctuations), motion parameters (as first-level analysis regressors), and SPM’s FAST (pre-whitening attenuation of autocorrelation effects) were applied (Olszowy et al., 2019).

Prediction Error Analysis.

Each participant’s prediction error signal was modeled based on trial sequence and regressed with brain activation across all trials (DeGuzman et al., 2017; Frank et al., 2012; O’Doherty et al., 2003). The predicted value $\left(\widehat{\mathrm{V}}\right)$ at any time (t) within a trial is calculated as a linear product of weights (w_i) and the presence of a conditioned visual stimulus (CS) at time t, coded in a stimulus representation vector x_i(t) where each stimulus x_i is represented separately at each moment in time:

$$\text{V}\left(\text{t}\right)={\sum }_{\text{i}}{\text{W}}_{\text{i}}{\text{x}}_{\text{i}}\left(\text{t}\right)$$

Predicted stimulus value at time t is updated by comparing the predicted value at time t+1 to that actually observed at time t, leading to the prediction error δ(t):

$$\text{δ}\left(\text{t}\right)=\text{ r}\left(\text{t}\right)+\text{ γ}\widehat{\text{V}}\left(\text{t}+1\right)-\widehat{\text{V}}\left(\text{t}\right)$$

where r(t) is the reward at time t. The parameter 𝛾 is a discount factor, which determines the extent to which rewards arriving sooner are more important than rewards that arrive later during the task, with $\text{γ}=0.99$. The weights w_i relate to how likely a particular unconditioned reward stimulus (US) follows the associated CS and are updated on each trial according to the correlation between prediction error and the stimulus representation:

$$\mathrm{\Delta }{\text{w}}_{\text{i}}=\text{ α}\sum _{\text{t}}{\text{x}}_{\text{i}}\left(\text{t}\right)\text{δ}\left(\text{t}\right)$$

where α is a learning rate. A slow α=0.2 was applied. Initial reward values were 1 for Sucrose Receipt and 0 for No Sucrose. Trial-to-trial prediction error was regressed with brain activation across all trials within each subject. The prediction error calculated for each trial was modeled as an absolute (reflecting degree of deviation of the outcome from the expectation) without separating positive or negative prediction error trials. Model prediction error values were then regressed against the fMRI data for each individual subject, to identify brain regions correlating with the model-predicted time series (O’Doherty et al., 2007).

Group-by-Condition Analysis.

We developed first-level models to predict the response in each voxel as a function of each of five stimulus conditions: expected sucrose, unexpected sucrose, expected no-solution, unexpected no-solution, and expected artificial saliva. Three contrasts of interest were computed per subject: (1) unexpected sucrose receipt: trials with CS for no-solution followed by unexpected US sucrose contrasted against trials with CS for no-solution, followed by expected no-solution; (2) unexpected sucrose omission: trials with CS for sucrose solution followed by unexpected US no-solution contrasted against trials with CS for sucrose solution, followed by expected sucrose solution; (3) expected sucrose receipt: trials with CS for sucrose solution followed by expected US sucrose contrasted against trials with CS for artificial saliva solution followed by expected US artificial saliva.

Region of Interest (ROI) Data Extraction.

We extracted parameter estimates (prediction error analysis) and beta values (group-by-condition analyses) from predefined regions of interest bilaterally (http://marsbar.sourceforge.net/, automated anatomical labeling Atlas, AAL (Tzourio-Mazoyer et al., 2002): superior, middle, medial and inferior orbitofrontal cortex (OFC); dorsal anterior insula, ventral anterior insula, posterior insula; caudate head; putamen; as well as ventral striatum (J. O’Doherty et al., 2004) and nucleus accumbens (Breiter et al., 1997).

Statistical Analysis

Data were tested for normality (Shapiro-Wilk test) and ranked and normalized using the Rankit procedure if they were non-normally distributed (Soloman & Sawilowsky, 2009). Pearson correlations and partial correlations were used to test associations between behavior and brain activation and results were multiple comparisons, controlled using a False Discovery Rate (Benjamini & Hochberg, 1995). We tested partial correlations among multiple ROIs with exercise in a temporal difference (RPE) model, as well as in conditions specific to unexpected reward receipt (RSUU) and unexpected reward omission (RNOU). The original sample was evaluated using two scanners (see Frank et al., 2021); a scanner variable, BMI, and age were controlled for in the partial correlation analyses. We used a False Discovery Rate correction to adjust for multiple comparisons. SPSS 27 software was used for statistical analyses (IBM, Armonk, N.Y.).

Results

Participants (100% female) had mean age (SD) = 25.28 (5.05), mean BMI (SD) = 21.40 (SD = 1.63) and reported aerobic exercise an average of 159 minutes (SD = 164) per week. Controlling for scanner, age, and BMI, significant positive associations were evidenced between minutes of aerobic exercise and prediction error response (bilateral posterior insula, right medial OFC, left nucleus accumbens, left ventral striatum), activation to unexpected sucrose stimulus receipt (bilateral medial OFC, left nucleus accumbens, left ventral striatum) and activation to unexpected stimulus omission (right medial OFC). However, after adjustment for multiple comparisons, only the correlation with right medial OFC response to unexpected stimulus receipt remained significant (Table 1 and Figure 1).

Table 1.

Partial correlations among aerobic exercise and brain regions of interest, controlling for age, BMI and scanner

ROI	RPE		RSUU		RNOU
	r	p	r	p	r	p
R Dorsal Anterior Insula	.136	.156	.186	.051	.085	.376
L Dorsal Anterior Insula	.166	.081	.203	.033	.122	.201
R Ventral Anterior Insula	.090	.345	.142	.136	.068	.479
R Ventral Anterior Insula	.124	.193	.156	.102	.089	.352
R Posterior Insula	.188	.048	.143	.135	.051	.594
L Posterior Insula	.209	.028	.160	.093	.049	.613
R Superior Orbital Frontal	.089	.355	.129	.177	.033	.729
L Superior Orbital Frontal	.006	.950	.123	.197	.051	.597
R Mid Orbital Frontal	.092	.337	.121	.206	.022	.816
L Mid Orbital Frontal	.114	.232	.133	.165	.113	.239
R Medial Orbital Frontal	.213	.025	.315	.0008	.211	.026
L Medial Orbital Frontal	.162	.089	.258	.006	.183	.055
R Inferior Orbital Frontal	.094	.328	.122	.202	.003	.977
L Inferior Orbital Frontal	.147	.125	.177	.062	.144	.132
R Caudate Head	.147	.123	.120	.209	.002	.985
L Caudate Head	.143	.135	.127	.185	.029	.761
R Nucleus Accumbens	.141	.141	.133	.165	.032	.736
L Nucleus Accumbens	.191	.045	.196	.039	- .019	.845
R Ventral Striatum	.106	.268	.168	.078	.072	.452
L Ventral Striatum	.196	.039	.188	.049	.067	.484

Note: N = 111. ROI = region of interest; RPE = reward prediction error; RSUU = unexpected receipt; RNOU = unexpected omission. Bolded text indicates values that are significant at p < .05.

Figure 1.

Partial correlation plot, controlling for age, BMI and scanner

Discussion

The current study sought to identify potential reciprocal effects between exercise and dopamine-related brain reward processing. Amount of aerobic exercise was significantly positively correlated with right medial OFC response across all three reward conditions tested but remained significant after multiple comparison correction only for the unexpected stimulus receipt condition.

The current study suggests that engagement in aerobic exercise is associated with heightened motivational salience response in the right medial OFC during unexpected receipt of reward. Whether higher exercise drives higher brain response or whether greater medial orbitofrontal brain response facilitates higher engagement in exercise cannot be determined from this study. The right medial OFC is specifically associated with goal-directed decision making, it is implicated in reward and outcome value computation, and aids in regulating sensitivity to the value of a given outcome (Gourley et al., 2016). It is therefore possible that individuals who engage in more aerobic activity may be intrinsically more responsive to salient stimuli and especially stimulus receipt, or alternatively, engagement in aerobic exercise has modulated brain activity and dopamine signaling, which may then reflexively reinforce and functionally maintain the exercise behavior. These two possibilities may each be true, and be additionally related to unique individual-level factors (e.g., temperamental traits) that increase the likelihood that a regular and adaptive program of exercise is upheld (Laborde et al., 2020).

Animal studies suggest that antagonizing the dopamine system via dopamine D2/D3 receptor blockers reduces physical activity, implicating the dopamine circuitry (Hillebrand et al., 2005; Klenotich et al., 2015; Verhagen et al., 2009). A recent study in mice demonstrated that genetic knockdown of dopamine transporters increased wheel running for some, but not all animals, suggesting individual variability in this process (Beeler & Burghardt, 2021). In humans, propensity for physical activity is variable and heritable (de Geus et al., 2014; Flack et al., 2019; Herring et al., 2014; Klimentidis et al., 2018) and in addition to modulating the pleasure and reward system (Matta Mello Portugal et al., 2013), physical activity modulates major neurotransmitters (Matta Mello Portugal et al., 2013). Contrary to our hypothesis, we did not find the strongest aerobic exercise correlations with the RPE contrast, but rather for the unexpected receipt condition, suggesting that it is in particular the unexpected receipt or possibly the better than expected outcome condition brain response that is associated with aerobic exercise. This has not been described before to our knowledge. It is possible that exercise may in particular enhance the ability to value or enjoy stimuli or experiences, which could be important for intervening on psychiatric disorders.

Altered brain salience response is characteristic of many psychiatric illnesses (e.g., depression; (Heshmati & Russo, 2015)), for which exercise has been proposed as generally effective in managing these disorders (Smith & Merwin, 2021). For example, some adults with generalized anxiety disorder demonstrate deficits in reinforcement-based decision-making and reduced RPE (White et al., 2017). For these individuals, determining if an exercise-induced improvement in motivational salience mediates response to standard psychotherapy treatment may serve to inform future treatment adaptations.

Limitations.

Although longitudinal work with a more sophisticated analytic approach is needed to confirm the directionality of associations, our findings offer an important foundation for a model of understanding the therapeutic potential of endurance or aerobic exercise in intervening on or enhancing salience response. However, less vigorous activity has been implicated in improving depression (Morres et al., 2019), and the impact of other non-aerobic activity on brain salience response also warrants examination. Exercise was self-reported, and findings from the current study may reflect general activity level moreso than effects of aerobic exercise itself. Further, some data on self-reported activity in healthy controls suggests that vigorous activity can be over-reported (Tomaz et al., 2016); future work might include objective measures of activity. Given possible confounds resulting from neuro-modulatory factors that are associated with anaerobic exercise (de Sousa et al., 2020), future work might include measurement of anaerobic exercise as well. Exercise has demonstrated association with the modulation of a variety of neurotransmitters (e.g., serotonin, norepinephrine) and neurohormones (e.g., brain derived neurotrophic factor) (Brellenthin et al., 2017; Heijnen et al., 2016; Szuhany et al., 2015). Therefore, while the RPE task is a measure of reward salience (Fouragnan et al., 2017), it is possible that results may reflect less specificity to the dopamine system, and instead, be a reflection of other biomarker activity in the studied ROIs. Of note, we focused our investigation on the study of healthy female controls; while this approach adds benefit in improving the generalization of our findings, our results do not specifically inform understanding across gender, or of addictive (Cook et al., 2014) or compulsive (Meyer et al., 2011) exercise.

Conclusions

In summary, our findings support the potential that aerobic exercise intervenes on reward-based processing such that there are reciprocal effects between exercise and possibly dopamine- and other neurotransmitter related brain activity in the medial OFC. While our study is cross-sectional, it lays a preliminary foundation in developing a model of adaptive exercise engagement and how it might modify reward response, and could be considered in intervening therapeutically in the OFC in psychiatric illnesses that include altered brain salience response.