Lithium modulates striatal reward anticipation and prediction error coding in healthy volunteers

Abstract

Lithium is one of the most effective mood-stabilizing medications in bipolar disorder. This study was designed to test whether lithium administration may stabilize mood via effects on reward processing. It was hypothesized that lithium administration would modulate reward processing in the striatum and affect both anticipation and outcome computations. Thirty-seven healthy human participants (18 males, 33 with suitable fMRI data) received 11 (±1) days of lithium carbonate or placebo intervention (double-blind), after which they completed the monetary incentive delay task while fMRI data were collected. The monetary incentive delay task is a robust task with excellent test-retest reliability and is well suited to investigate different phases of reward processing within the caudate and nucleus accumbens. To test for correlations with prediction error signals a Rescorla–Wagner reinforcement-learning model was applied. Lithium administration enhanced activity in the caudate during reward anticipation compared to placebo. In contrast, lithium administration reduced caudate and nucleus accumbens activity during reward outcome. This latter effect seems related to learning as reward prediction errors showed a positive correlation with caudate and nucleus accumbens activity during placebo, which was absent after lithium administration. Lithium differentially modulates the anticipation relative to the learning of rewards. This suggests that lithium might reverse dampened reward anticipation while reducing overactive reward updating in patients with bipolar disorder. This specific effect of lithium suggests that a targeted modulation of reward learning may be a viable approach for novel interventions in bipolar disorder.

Introduction

Lithium is one of the most effective mood stabilizers for maintenance and treatment of bipolar disorder [1]. Clinical data support a role both as an antidepressant and in reducing the occurrence of manic episodes. Despite lithium having a long history in the treatment of psychiatric disorders, its mechanisms of action are poorly understood. Leading neurochemical theories of the mechanism of lithium include effects on inositol signaling and inhibition of GSK-3β but also support an involvement of the monoamine and glutamate systems [2]. However, it remains unclear how these biochemical effects translate into lithium’s mood-stabilizing action. Understanding the profile of lithium on core psychological processes relevant to bipolar disorder has the potential to enhance our search for candidate mood stabilizers by providing a surrogate marker for treatment action. One candidate process with obvious relevance to the experience of both depression and mania, relates to how rewards are processed. The behavioral approach system model of bipolar disorder argues that hypersensitivity in reward processing and incentive motivation plays an important role in the pathophysiology of bipolar disorder particularly during mania (see [3, 4]). In line with this model, increased self-report reward sensitivity is associated with onset, severity and recurrence of hypomanic/manic episodes [3]. At a neurobiological level, however, the results have been more mixed and may depend on clinical state, medication usage and reward paradigm. In addition, differences in reward seeking and impulsivity at a behavioral level have been more broadly related to both increases and decreases in neural reward circuitry response in fMRI studies [5]. These differences are often conceptualised as excessive reward seeking either representing increased reward reinforcement or conversely an attempt to compensate for reduced experience of everyday reward [5].

The response to reward can be broadly split into an anticipatory and consummatory phase. Both reward anticipation and consummation (i.e., with positive outcome) have been reliably linked to brain activation of the caudate nucleus and nucleus accumbens (NAcc) [6]. Dopamine dependent signaling within these areas is believed to code reward prediction error (RPE) signals [7, 8]. Thus, initially a dopamine response is seen during reward receipt but this transfers to anticipation once reward associations are learnt. A reward that is greater than expected then leads to a positive RPE, while lower reward leads to negative RPEs (i.e., a dip in dopamine signaling). These RPEs are crucial for updating expectation on rewards in the future, and thus may play a critical role during episodes of mood disturbance and stabilization.

Depression and more specifically anhedonia have been reliably associated with reduced striatal responses during the anticipation of reward [9, 10]. In bipolar disorder mixed results on ventral striatum activity during reward processing have been reported [3, 11–16]. These differences may be explained partly by characteristics of the sample, including clinical state, since mood elevation may be expected to lead to a different profile to depression. In addition, the majority of studies in bipolar disorder have included patients taking mood-stabilizing medication, which may have affected the pattern of results [17]. Critically, Yip et al., reported that unmedicated patients with bipolar disorder showed blunted sensitivity of the caudate nucleus during anticipation of reward [16]. Such observations contrast with those in medicated patients [12, 15] and suggest a potential effect of medication to enhance reward sensitivity during anticipation. Furthermore, unmedicated young men at increased risk for bipolar disorder showed reduced subjective psychostimulant response to acute ethanol administration [18], consistent with a reward seeking as compensation hypothesis. These observations suggest that it is critical to consider the effects of mood stabilising medication on reward processing to disambiguate the pathophysiology of bipolar disorder from potential treatment action.

The current study was designed to address how reward responding might be modified through lithium administration thus addressing part of the translational gap between the well-characterised molecular effects of lithium and its clinical efficacy. More specifically, healthy participants completed a well-validated reward-processing task, the monetary incentive delay (MID) task [19], within a double-blind randomized-controlled between-subject design. The MID task elicits reward anticipation, reward outcome processing as well as RPEs that are informative to characterise the value of upcoming rewards. Healthy volunteer studies are a useful way of characterizing direct effects of drug treatments unconfounded by unstable clinical state or other medication use. On the assumption that lithium reverses deficits in reward anticipation in unmedicated bipolar disorder [16], we predicted that caudate-related anticipation effects in healthy participants would be enhanced during lithium administration. Conversely, we predicted that responses during the outcome phase, as well as PE signaling, would be reduced by lithium administration in line with its mood-stabilizing properties.

Materials and methods

Participants

In total, 37 right-handed participants (18 males) were recruited from the general population, written, and oral consent was obtained according to the guidelines of the local ethics committee (NRES committee South Central—Oxford REC B). Participant recruitment occurred through posters, web ads, ads in local newspapers and a local participant pool. The participants were reimbursed for their time. The participants were 18–55 years old and physically fit (physical examination by a medical doctor) with normal laboratory values for thyroid and renal function, and had a BMI of 19–30. Females additionally scored negative on a pregnancy test and used two forms of effective contraception. Exclusion criteria were: taking psychotropic medication, any past or current Axis 1 psychiatric disorder on DSM-IV (as assessed by structured interview for DSM-IV), current pregnancy or breastfeeding, current or past history of drug or alcohol dependency, participation in the last 3 months in a medication research study, smoking >5 cigarettes per day, dyslexia, and any contra-indication to MR scanning. Participant drop-out/exclusions during the experiment/analyses occurred due to not starting intervention (1×), an unexpected adverse effect (1×), incomplete MRI session (1×), and excessive movement during fMRI (1×; >6 mm, 2× voxel size). This resulted in 33 participants for the reported results (Table S1).

Study design and intervention

The study was a double-blind randomized design with both experimenter and participant blind to the intervention. Randomization was performed by a qualified researcher not involved in the study. The randomization programme included a minimization algorithm to ensure balanced allocation of participants across groups, stratified by gender, using a block design of 4. Allocation ratio was 1:1 for treatment (lithium vs placebo) and 1:1 for gender (male, female). The sequence was concealed from the experimenter until completion of the study. Participants came into the lab (Psychiatry/OCMR, Oxford) four times, namely for an initial assessment, at the start of the intervention and 2 days at the end of the intervention.

At initial assessment a medical and psychiatric screening was performed (including SCID-IV), and blood levels were taken measuring thyroid stimulating hormone and creatinine. At treatment start, several questionnaires were completed assessing mood, anxiety, and personality characteristics (see questionnaire measures) and females took a pregnancy test. Participants were given blinded bottles of capsules to take home as well as questionnaires to complete daily (not included here). Participants were contacted at day 3 and 5 to check for adverse side effects and to ensure they followed the dosage regimen.

Participants received lithium carbonate (as ‘Priadel’ prolonged release tablets) or placebo intervention for 11 (±1) days to take at night. The lithium carbonate dose was encapsulated and increased in a gradual fashion with day 1: 400 mg, day 2: 600 mg, day 3–11: 800 mg (based on [20, 21]). The placebo intervention was 200 mg Rayotabs placed in identical capsules.

At the end of the treatment period the participants completed a morning behavioral and MR session (on separate days, nonfixed order). During the behavioral session, blood was drawn for lithium levels, participants completed a battery of tasks (not included here), and the questionnaires described below. During the MR test session, the participants underwent an anatomical scan, fMRI while completing the MID task, a visual checkerboard task, an emotional reappraisal task (not included here), and a MR spectroscopy scan (not included here).

Questionnaire measures

At baseline and follow-up the following questionnaires were assessed: the Beck Depression Inventory [22], the state–trait anxiety inventory [23] (only STAI-state at follow-up), the Mood Disorder Questionnaire [24], and the Emotional Blunting questionnaire (part 1 only) [25]. Additionally, only at baseline the National Adult Reading test (NART IQ scale [26]) and the Eysenck Personality Questionnaire [27] were completed. For each treatment day participants completed the Befindlichskeit scale of mood and energy [28], the positive and negative affective scale [29], the Bond and Lader visual analog scales [30] and a side effects questionnaire.

Monetary incentive delay task

The MID task [19], programmed in E-Prime, contained 54 trials, with 18 trials per condition and an additional 12 practice trials outside the scanner. At each trial (Fig. 1) participants saw a cue, waited a variable interval while viewing a cross-hair, then made a response when seeing a white square (target). Outcome was given indicating response correctness (i.e., within time limit), if a reward was obtained and current task winnings. Initially, target duration was set by the practice. Then, target duration was reduced by 20 ms if the previous trial was correct and overall accuracy >66%. In total, 20 ms was added if the previous trial was incorrect and overall accuracy <66%. The range of the target duration was restricted between 160–260 ms. Different cues reflected the different conditions of the task: (1) circle: reward could be obtained when responding within time limit (reward anticipation condition), (2) square: participants were asked to respond but could not obtain a reward (no reward anticipation condition), and (3) triangle: do not make a response condition. If a response before target was made (early response), a fixation cross was presented for the remaining part of the trial. FMRI volume acquisitions were time-locked to cue offset [19]. Please see the Supplementary materials for analysis of behavior.

Fig. 1. MID task.

Example trial. A cue signals a reward (shown here), no reward, or no movement trial. After an inter-trial interval, a target is presented during which participants have to respond. Outcome indicates if the response was within the target period (win) and possible reward obtained. When a response was too slow it is coded as a loss.

Checkerboard control task

To control for possible confounding effects of treatment on general brain activation, a visual checkerboard task was used. Participants viewed blocks of alternating checkerboards (black and white squares switching at a frequency of 8 Hz) for 16 s or stationary fixation cross for 15 s. In total, participants viewed 10 blocks of each, while instructed to lie quietly with their eyes open [31].

MRI acquisition

The fMRI volumes were acquired on a 3T MRI scanner (Magnetom, Siemens Medical Systems) with a 32 channel head coil using a sequence from [32], (repetition time (TR) = 3000 ms, echo time (TE) = 30 ms, 45 slices (no gap), a slice angle of 15°, interleaved acquisition, voxel size = 3 × 3 × 3 mm, flip angle = 87°, field of view (FOV) = 192 mm, with local z-shimming). Field maps were obtained using a dual echo 2D gradient-echo sequence (TR = 488 ms, TE = 7.65 ms, and 5.19 ms, grid = 64 × 64 × 40). High-resolution anatomical images were acquired (TR = 2040 ms, TE = 4.7 ms, 192 transversal slices, voxel size = 1 ×1 × 1 mm, FOV = 192 mm).

Computational model of reward prediction errors

The RPE for each reward trial was estimated to use as parametric modulation on brain activation [7]. A Rescorla–Wagner algorithm based reinforcement-learning model was used to generate estimates of RPE and expected value (EV) from reward trials [33, 34]. EV reflects the estimated probability of receiving a reward on a given trial. RPE reflects the difference between this expectation and the actual reward.

$$\begin{array}{c}{\mathrm{EV}}_{\mathrm{1}}=\mathrm{0}\mathrm{.5},\hfill \\ {\mathrm{RPE}}_{\mathrm{t}}={\mathrm{R}}_{\mathrm{t}}-{\mathrm{EV}}_{\mathrm{t}},\hfill \\ {\mathrm{EV}}_{\mathrm{t}+\mathrm{1}}={\mathrm{EV}}_{\mathrm{t}}+\mathrm{\eta }\times {\mathrm{RPE}}_{\mathrm{t}},\hfill \end{array}$$

R is the actual reward received, t is the trial, η is the learning rate. RPE_t is determined by the difference between the received reward and EV. The EV for the next trial (EV_t+1) is updated based on the EV of the current trial (EV_t) and the prediction error of that trial (RPE_t) times the learning rate (η). EV was initialized at 0.5 and η was fixed at 0.7 [7]. Varying the value used for the learning rate did not significantly influence the prediction error regressors generated from this procedure ([33]; see Supplemental results).

Functional MRI data analysis

fMRI volumes were analyzed using FSL FEAT ((37, 44, 45); version: FinalFive). First, bias correction and brain extraction was performed on the anatomical and functional volumes and, if present, the magnitude volume [35, 36]. The magnitude volume was eroded one voxel to exclude any skull. A fieldmap volume was estimated based on the gradient-echo magnitude and phase volumes. The functional timeseries were high passed filtered at 100 s, motion and fieldmap distortion corrected [37], spatially smoothed with 5 mm FWHM, and Melodic ICA data exploration [38] was performed. Registration of the functional to T1 volume was done using linear boundary-based registration and then to a standard T1 MNI brain (isometric voxel size: 2 mm) using linear transformation with 12 degrees of freedom and nonlinear warp with 10 mm resolution [37, 39, 40]. The melodic components were manually checked for noise and when identified removed from the timeseries [38].

A first level model for reward anticipation and outcome per participant was modeled with a double-gamma HRF and its temporal derivatives on the onset times of the anticipation and outcome cues. FILM prewhitening and temporal filtering was applied. Three different types of anticipation cues were modeled, signaling a potential reward, no reward, or no movement. Five different types of outcome were modeled, namely indicating a win or miss on reward trial, a win or miss on no reward trial or a no movement trial in which a movement was correctly withheld. Error outcome was modeled when a participant responded too early or responded while instructed not to (i.e., on no response trials). Additional confound regressors were included capturing the white matter timeseries and motion. The white matter mask for the timeseries was created using the HCP pipeline with Freesurfer ([41], http://surfer.nmr.mgh.harvard.edu/). The white matter mask was warped to functional space, eroded one voxel to exclude partial-voluming effects and restricted to the largest cluster of voxels. The denoised functional data were used within this white matter mask to obtain the white matter timeseries. To account for residual signal related to head movement, the six movement parameters from the motion correction were included as regressors of no interest as well as regressors capturing volumes with excessive motion (applied for participants with >1 mm mean movement displacement and artifacts in slices as observed by visual inspection). Motion outliers were identified using the FSL Motion Outliers tool.

Two contrasts were calculated. (1) Reward anticipation was modeled by comparing brain activity during the anticipation phase of reward trials to no reward trials. (2) Reward outcome was modeled on the outcome phase when participants were informed of a win versus a miss on reward trials.

A second first level model including the prediction error was created similarly to the model above with a few critical differences. First, the reward expectation and prediction error regressors from the computational model were added as a parametric regressor modeled at the onset of reward anticipation or outcome, respectively. Second, outcome regressors were modeled per cue option, without taking specific feedback into account to avoid overfitting [7]. A single contrast was calculated for this analysis that represented the prediction error.

For the visual checkerboard control task, a first level model was created following the same procedure with FSL FEAT [42, 43] modeling the double-gamma HRF and its temporal derivatives on the onset times of the checkerboard and fixation blocks. One contrast was calculated, namely when brain activity is larger for checkerboard blocks compared to fixation blocks.

The contrasts were separately fed into a two-sample t-test within randomise, FSL’s tool for nonparametric permutation inference (5000 permutations; [44]), to assess general effects of task relevant contrasts on both groups, as well as test for group differences. Statistics were assessed using threshold free cluster enhancement [45] method with family-wise error correction of 0.05 (or 0.95 threshold within randomise). Cohen’s d was estimated following [46]. The Caudate/NAcc mask used for the ROI analyses and grey matter mask used for the whole brain analyses were created using the HCP pipeline with Freesurfer ([41], http://surfer.nmr.mgh.harvard.edu/). First, for each participant the T1-weighted structural image was processed following the HCP pipeline. Second, these segmented images were warped to standard space where the regions of interest were isolated and a summary mask was calculated over all participants.

Significant brain areas were extracted for visualization using the fslmaths and cluster tool, with a threshold of 0.95 (based on 1/p thresholding from randomise) [47] and [48] were used for localisation. We visualised blood oxygen level-dependent (BOLD) time-courses underlying the significant interactions between reward processing and intervention, using BOLD signal extracted from ROIs per participant [49]. ROIs with 5 mm radius restricted to the Caudate/Nacc ROI were centered on the peak coordinates of the group differences. Extracted timeseries were up-sampled using b-spline fitting and signal covarying with the original confound regressors was removed. To obtain the RPE associated time-course a GLM model was created including a constant (or mean activation) and the RPE values as parametric modulation. The standardized beta of the RPE parametric modulation reflects the correlation strength of RPE with the time-course. The time-courses are averaged over trials and subjects for visualization.

Results

There were no significant differences in the demographic characteristics of the two groups before the intervention, nor on measured subjective state or brain volume as a result of the intervention (Table S1).

fMRI results

Anticipation phase

There was a significant main effect and a group difference in the left and right caudate when comparing trials in which participants anticipated a reward versus trials in which they did not anticipate a reward (Table 1). As hypothesized, the group difference was driven by increased activity in the lithium group during reward anticipation compared to no reward anticipation (Fig. 2, MNI coordinates (x, y, z): 16, 16, 10; cluster size = 583, z-score = 4.88), while no significant difference was observed in the placebo group.

Table 1.

MID results.

Anatomical region	BA	Side	x, y, z—MNI coordinates	K	z-score	Cohen’s d
ROI on caudate/NAcc
Anticipation: reward > no reward, lithium > placebo
Caudate head		R	12, 2, 12	23	4.21	1.72
Caudate body		L	−18, 2, 20	19	3.81	1.51
Caudate head		L	−6, 14, 0	1	3.79	1.50
Anticipation: lithium group: reward > no reward
Caudate & NAcc		L & R	16, 16, 10	583	4.88	2.11
Anticipation: over both groups: reward > no reward
Caudate head		R	16, 6, 16	153	4.36	1.80
NAcc/caudate (head)		L	−4, 10, −6	67	4.17	1.70
Outcome on reward trials: win > loss, placebo > lithium
Caudate head		R	16, 20, 4	9	3.19	1.22
Caudate head		R	18, 20, 14	7	3.36	1.30
Caudate head		R	12, 16, 4	6	3.21
Outcome on reward trials: placebo group: win > loss
Caudate/NAcc		L & R	−8, 18, 0	652	4.97	1.23
NAcc		R	14, 22, −10	8	3.18	1.22
Outcome on reward trials: over both groups: Win > loss
NAcc/caudate (head)		L & R	6, 12, −4	356	4.79	2.05
Whole brain effects
Outcome on reward trials: over both groups: win > loss
mPFC: dorsal ACC/frontal pole	32d, 10 m	L & R	−2, 50, 12	487	5.75	2.71
Retrosplenial cortex	29, 30	L & R	−6, −52, 12	399	5.47	2.50
NAcc/caudate		L & R	6, 12, −4	135	4.79	2.05
PCC	23	L	−6, −38, 36	131	4.33	1.79
Precuneus	7/31	L	−2, −64, 36	42	4.76	2.03
PCC	23	R	4, −36, 46	11	3.87	1.54

See Table S5 for subthreshold effects.

BA Brodmann area, k number of voxels in a cluster, ACC anterior cingulate cortex, PCC posterior cingulate cortex.

Fig. 2. Lithium modulations on brain activation.

A Brain maps showing group interaction during the anticipation phase (reward > no reward trials). B Time-course of the strongest cluster in (A). C Brain maps showing group interaction during outcome phase on reward trials (win > loss). D Time-course of the strongest cluster in (C). SE = standard error.

Outcome phase

A significant main effect was observed in several regions including the medial prefrontal cortex (mPFC), left and right caudate (Table 1) during outcome on reward trials comparing wins versus losses. A group difference in the opposite direction as above was seen in the caudate during the outcome contrast (Table 1, Fig. 2). This effect was driven by increased activity in the placebo group for a win compared to a loss in rewarding trials (MNI coordinates (x, y, z): −8, 19, 0; cluster size = 652, z-score = 4.97).

Reward prediction error

The RPE was regressed against the individual brain data (parametric modulation) to test if lithium intake changed the strength of RPE modulation on striatal activity. There was a significant group difference in the left and right caudate and NAcc, which was driven by a relatively higher positive correlation between RPE and caudate/NAcc activity in the placebo group relative to the lithium group (Table 2, Fig. 3).

Table 2.

Reward prediction error.

Anatomical region	BA	Side	x, y, z—MNI coordinates	K	z-score	Cohen’s d
ROI on caudate/NAcc
Placebo > lithium
Caudate/NAcc		R	12, 16, 4	124	3.63	1.43
Caudate head/NAcc		L	−12, 18, 6	15	2.95	1.11
Placebo group
Caudate head/NAcc		R & L	−10, 4, 2	433	4.12	1.67
Whole brain effects
Over both groups
Retrosplenial/PCC	30/31	L	−8, −62, 20	166	6.25	3.12
Precuneus	7/31	L	−8, −68, 30	13	4.36	1.80

BA Brodmann area, k number of voxels in a cluster, PCC posterior cingulate cortex.

Fig. 3. RPE modulation on brain activity.

A Brain maps showing group interaction on the RPE parametric modulation. B Time-course underlying group interaction.

Visual checkerboard control task

No group differences were found for the visual checkerboard task, suggesting that the observed effects during the MID task do not reflect general BOLD changes. There was an overall highly significant effect of visual stimulation in the visual cortex (see Table S2), as previously reported [50]. The main effect in visual cortex was also present in the individual groups with exactly the same coordinates (MNI: x = 12, y = −86, z = −12; placebo group: z-score = 17.3, cluster size = 19,430; lithium group: z-score = 16.6, cluster size = 23,585).

Discussion

This study tested the effects of lithium administration on reward processing in healthy participants. We found that lithium enhances striatal reward anticipation, while it dampens striatal reward outcome and associated prediction error signals. This provides a potential mechanism by which lithium treatment might stabilize reward responsivity in bipolar disorder.

The MID task has been extensively used to measure different phases of reward processing. Reward anticipation is triggered by cues on the prospect of receiving a reward. During the reward outcome phase participants are informed on whether they have received that reward. The reward outcome phase is relevant to learning about potential future rewards and salience of the rewarding cue. Knutson et al. showed a clear involvement of the caudate and NAcc during reward anticipation as well as outcome, an effect confirmed by multiple meta-analyses [6, 19, 51, 52]. Increased activation in striatal regions during reward outcome has typically been related to unexpected reward, so called RPE [6]. Here we model each of these effects, looking at the effect of lithium administration on reward anticipation, outcome, and prediction errors.

Depression has been associated with reduced activity of caudate and NAcc during anticipation of reward [51]. Similarly, unmedicated euthymic patients with bipolar disorder showed blunted sensitivity of the caudate nucleus during anticipation of reward [16]. The results in medicated patients have been mixed, with studies often showing increased activity within the ventral striatum [3, 11–14]. The current investigation illustrates what might underlie this discrepancy since lithium administration itself increases activity of the caudate during reward anticipation, suggesting it increases sensitivity for potential reward. Critically, this anticipatory effect within the caudate showed no relation with the height of the trial-dependent EV value from the prediction error model which reflects sensitivity for the amount of reward (see Supplementary material). This suggests that it was not a greater sensitivity to the value of reward that drove lithium-related increases in striatal anticipation, but the expectation of any potential reward.

In contrast to the effects of lithium on reward anticipation, we found reduced caudate and NAcc responses during the receipt of reward and reduced RPE signals following lithium administration. Indeed, the volunteers taking lithium were relatively insensitive during the outcome phase as to whether they received a win or a loss. It seems that the striatal drive to reinforce rewards is reduced as a function of lithium administration, thereby providing a potentially important distinction between the ability to maintain anticipation and motivation toward reward but perhaps, preventing escalation or reinforcement while engaging in potentially pleasurable but risky behaviors.

The theoretical account provided by the behavioral approach system model [4] suggests that bipolar disorder is characterized by excessive reward seeking and incentive motivation, underpinned by increased reactivity of the fronto-striatal system to reward [3]. The current findings suggest that examining the neural basis of reward seeking in bipolar disorder requires a focus on unmedicated patients since mood-stabilizing medication can have key effects on reward response. Furthermore, differences between the effects of lithium on reward anticipation and receipt suggest a more complex pattern of action which requires further investigation.

The current pattern of results appears at least partly distinct from previous studies exploring the effects of antidepressant medication on the response of the NAcc and RPE in the MID task. In particular, Scholl et al. [49] and Graf [53] reported an increase in RPE signals following SSRI administration, though NAcc response to reward (erotic images) was reduced in the latter study. McCabe et al. [54] also reported decreased ventral striatal responses to chocolate reward after administration of the SSRI citalopram. Studies exploring the effects of the noradrenaline and dopamine reuptake inhibitor bupropion have also tended to report increased NAcc responses during reward anticipation [55] and a similar pattern was seen with low dose amisulpride in the same task but extending to both anticipation and outcome phases [56]. Further work is needed to characterize these effects, both between antidepressants with different pharmacological properties and between agents with mood-stabilizing actions.

The current study has implications for reward studies involving medicated bipolar patients as it shows that lithium administration can significantly influence the pattern of results. It also provides a candidate marker for exploring the effects of putative mood-stabilizing agents in drug development pipelines. The identification of novel treatments in bipolar disorder has been slow and with a high failure rate. Use of a mechanistic biomarker as a way of screening and characterizing novel treatments can double the success of drug development programmes across medicine [57]. Further validation of the effects of mood stabilisers in bipolar disorder using tasks, which separate reward anticipation and outcome, is therefore urgently needed.

Other fMRI studies have been performed testing for lithium effects in healthy participants (e.g., [58]), using a variety of paradigms, though not reward. Attention has mostly focused on changes in structural measures including grey and white matter volumes, finding differential effects after several weeks of lithium administration [20, 59–61]. Crucially, we found no differences in grey or white matter volume (Table S1). This might be related to the length of the treatment, i.e., 11 days versus 4 weeks, as rodent work has shown increased frontal cortex volume after 5 weeks but not after 11 days of lithium treatment [62].

Further work in this area may wish to include tasks with differential magnitudes of reward and punishment conditions to characterize the effects of lithium further. As our task did not include a punishment condition, it is not possible to draw a conclusion with respect to the specificity of effect on reward vs punishment anticipation and receipt. Our sample size was relatively small, with a final sample of 33. However, the MID task is a very robust task with good reliability [52, 63, 64]. Indeed, previous studies suggest significant power for individual difference effects (13 participants needed for power of 0.80 [63]) and our results have large effect sizes (Tables 1 and 2). The coherence of our findings, despite the smaller sample size, is amplified by the close match between the pattern of activity observed in this study compared to a recent meta-analysis [6]. Future studies could investigate potentially altered connectivity between prefrontal and striatal brain regions which might underlie the observed effect [65]. We did not include a measure of impulsivity in our study which may have been a useful way of understanding the effects of lithium on different reward components characterized here. Post hoc correlation analyses with lithium levels, depression, anxiety and mood instability questionnaires indicate potential correlations with the prediction error signal (Tables S7–10), which might be relevant for future meta-analyses but are difficult to interpret in this relatively small sample size. Finally, lithium has been associated with general changes in MRI signalling [66], however, our checkerboard control task shows this was not the case with 11 days administration in a healthy control sample.

To conclude, our study shows a potential mechanism by which lithium stabilizes reward processing in bipolar patients. Lithium administration in healthy participants increased striatal responsivity to reward anticipation, while striatal prediction error signals and outcome-related activity were reduced, thereby shifting neural processing from outcome to anticipation. This shifted balance in reward processing might increase initial sensitivity to reward, while it reduces overresponding to positive reinforcement, both key neuropsychological processes in the pathophysiology of bipolar disorder.

Funding and disclosure

We are grateful to the John Fell Fund for research funding. This research was supported by the NIHR Oxford Health Biomedical Research Centre. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health. CJH has received consultancy fees from P1vital, Lundbeck, Sage Therapeutics, and Johnson and Johnson. MB is supported by an MRC Clinician Scientist Fellowship (MR/N008103/1), has received travel expenses from Lundbeck for attending a conference and has acted as a consultant for J&J and CHDR. PJC was supported by an MRC grant MR/S003037/1. LV was supported by a Wellcome Trust Senior Investigator Award (WT100973AIA) awarded to Matthew Rushworth, and the Wellcome Centre for Integrative Neuroimaging (203139/Z/16/Z). IV, AB, LV, and PJC declare no potential conflicts of interest.

Author contributions

Substantial contributions to the conception or design of the work; or the acquisition, analysis, or interpretation of data for the work: IV, AP, LV, MB, PJC, CJH—drafting the work or revising it critically for important intellectual content: IV, AP, LV, MB, PJC, CJH—final approval of the version to be published: IV, AP, LV, MB, PJC, CJH—agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved: IV, CJH.

Supplementary information

Supplementary Information accompanies this paper at (10.1038/s41386-020-00895-2).