Skip to main content
NCBI home page
Elsevier Sponsored Documents logoLink to Elsevier Sponsored Documents
. 2016 Jul 1;80(1):73–81. doi: 10.1016/j.biopsych.2015.07.018

A Neurocomputational Account of How Inflammation Enhances Sensitivity to Punishments Versus Rewards

Neil A Harrison a,b,c,⁎, Valerie Voon d,e, Mara Cercignani f, Ella A Cooper a, Mathias Pessiglione g, Hugo D Critchley a,b,c
PMCID: PMC4918729  PMID: 26359113

Abstract

Background

Inflammation rapidly impairs mood and cognition and, when severe, can appear indistinguishable from major depression. These sickness responses are characterized by an acute reorientation of motivational state; pleasurable activities are avoided, and sensitivity to negative stimuli is enhanced. However, it remains unclear how these rapid shifts in behavior are mediated within the brain.

Methods

Here, we combined computational modeling of choice behavior, experimentally induced inflammation, and functional brain imaging (functional magnetic resonance imaging) to describe these mechanisms. Using a double-blind, randomized crossover study design, 24 healthy volunteers completed a probabilistic instrumental learning task on two separate occasions, one 3 hours after typhoid vaccination and one 3 hours after saline (placebo) injection. Participants learned to select high probability reward (win £1) and avoid high probability punishment (lose £1) stimuli. An action-value learning algorithm was fit to the observed behavior, then used within functional magnetic resonance imaging analyses to identify neural coding of prediction error signals driving motivational learning.

Results

Inflammation acutely biased behavior, enhancing punishment compared with reward sensitivity, through distinct actions on neural representations of reward and punishment prediction errors within the ventral striatum and anterior insula. Consequently, choice options leading to potential rewards were less behaviorally attractive, and those leading to punishments were more aversive.

Conclusions

Our findings demonstrate the neural mediation of a rapid, state-dependent reorientation of reward versus punishment sensitivity during inflammation. This mechanism may aid the adaptive reallocation of metabolic resources during acute sickness but might also account for maladaptive, motivational changes that underpin the association between chronic inflammation and depression.

Keywords: Depression, Imaging, Inflammation, Insula, Reward, Striatum

Inflammation rapidly reorients motivational state; pleasurable activities are avoided, sensitivity to negative stimuli is enhanced, and feelings of depression, fatigue, and irritability are common (1, 2). Mediated by the host immune response, this motivational shift efficiently prioritizes whole organism responses to the infecting agent (1, 2). However, how inflammation mediates these rapid shifts in behavior currently remains unclear.

To address this, we used computational modeling of a reinforcement-learning task to dissect effects of inflammation on reward- and punishment-related decision-making processes. Importantly, this approach allows computation of hidden prediction error signals (δ), the teaching signal embodied in contemporary computational reinforcement learning theory, critical to updating estimates of the value of available options and consequent biasing of behavioral choice (3). Data from rodents and primates suggest that midbrain dopaminergic cells may provide this teaching signal at least in the context of reward learning (4, 5, 6, 7), with actions on corticostriatal synaptic efficacy providing a mechanism for flexible reward learning and behavioral optimization. Dopamine-dependent modulation of striatal reward prediction error has also been linked to human reinforcement learning to reward (8). It is therefore noteworthy that inflammation has been observed to modulate striatal dopamine uptake (9) and efflux (10), as well as ventral striatal responses to both reward outcome (9) and cues predicting reward (11), suggesting that the rapid changes in reward-related behavior induced by inflammation may be mediated via an action on striatal reward prediction error encoding.

However, behavioral effects of inflammation are not limited to changes in reward-related behavior. In both rodents and humans, experimentally induced inflammation has also been shown to enhance sensitivity to punishment, at least when experienced as musculoskeletal pain (12, 13). Though proinflammatory mediators can sensitize peripheral nociceptors (14), lipopolysaccharide-evoked hyperalgesia does not typically develop until 3 hours (13, 14, 15), suggesting a likely role for central sensitization processes, an interpretation supported by the characteristic pattern of mechanical but not thermal hyperalgesia (16). Interestingly, studies investigating reinforcement learning to punishment have identified a punishment specific prediction error signal within insula cortex (8, 17), a region implicated in signaling a range of aversive events (18, 19, 20, 21) including pain (22) and peripherally induced inflammation (23, 24, 25). Correspondingly, patients with insula lesions show impairment in punishment but not reward-based learning (26). Whether previously observed actions of inflammation on insula reactivity additionally modulate punishment prediction error signals, proving a mechanism for enhancing sensitivity to punishment, was a second focus of the current study.

To investigate the behavioral and brain mechanisms mediating this inflammation-induced motivational reorientation (expressed as enhanced punishment sensitivity and simultaneously impaired reward sensitivity), we studied 24 healthy individuals (18 during functional magnetic resonance imaging [fMRI]) on two separate occasions, one 2.5 to 3.5 hours after a standard inflammatory challenge (typhoid vaccination) and one 2.5 to 3.5 hours after control (saline injection). We applied a reinforcement-learning model to a probabilistic learning task and restricted our primary hypotheses to ventral striatum and insula regions previously shown to encode reward and punishment prediction error, respectively (8). We hypothesized that inflammation would impair sensitivity to gains (win £1) (manifest as an acute reduction in ventral striatal positive δ and a consequent reduction in the propensity to choose the most rewarding action on a reinforcement-learning task) and simultaneously enhance sensitivity to punishment (observed as an enhancement in insula negative δ on loss trials and a consequent increase in the propensity to avoid the punishing [lose £1] choice).

Methods and Materials

Inclusion and Exclusion Criteria

Twenty-four healthy nonsmokers (9 male subjects, mean 27.6 ± 7.0 years) were recruited and screened for relevant physical or psychiatric illness. One was later excluded after failure to complete the second scanning session. Volunteers who had received typhoid vaccine within 3 years or other vaccine within 6 months were excluded. All were medication free and rated their general health as good, very good, or excellent. Participants were advised to not consume alcohol, avoid high-fat meals, and refrain from excessive exercise for 24 hours before testing and avoid nonsteroidal anti-inflammatory drug medications, steroids, and antibiotics for 7 days before testing. Written informed consent was obtained after complete description of the study and study procedures were approved by the Brighton-East National Research Ethics Committee.

Study Design

We adopted a randomized, repeated-measures, cross-over design with both participant and researcher blind to intervention. Participants underwent two separate testing sessions 7 days apart. In the first session, participants were randomly assigned to one of two experimental conditions (typhoid vaccine or saline injection) with 12 participants receiving typhoid vaccination in the first session. Baseline blood samples were taken and then injections of .025 mg Salmonella typhi capsular polysaccharide vaccine or .5 mL normal saline placebo were administered intramuscularly into the deltoid muscle. Behavioral testing was performed 2.5 to 3.5 hours after injection in a 60-minute session (23); 18 participants completed testing during fMRI and 6 completed testing in a behavioral testing suite. Immediately after testing, a second blood sample was taken for repeat cytokine measurement. Body temperature and Profile of Mood States (POMS) questionnaire with four extra items (fever, aching joints, nausea, and headache) added to assess somatic symptoms associated with mild infection (27) were completed at baseline and after 3.5 hours. The second testing session was identical except that participants received the alternate injection (i.e., typhoid vaccination if they previously received saline and vice versa).

Reinforcement Learning Task

Participants completed three runs of the same instrumental learning task, each using three new pairs of abstract stimuli on each testing session (Figure 1A). Each pair of stimuli (gain, loss, neutral) was associated with a pair of outcomes (gain £1/nil, lose £1/nil, look £1/nil), and the two stimuli corresponded to reciprocal probabilities (.8/.2 and .2/.8). On each trial, one pair was randomly presented with the two stimuli presented left and right of a central fixation cross; relative positions were counterbalanced across trials. The participant chose the right-sided stimulus with a button press (go response) and the left-sided stimulus with an absence of a response (no-go response). The choice was then circled in red and the outcome displayed on the screen after a 4-second delay. To maximize winnings and minimize losses, participants had to use trial and error to learn stimulus-outcome associations. They were told that they would be remunerated their winnings, though all left with the same fixed amount. Effects of inflammation on behavioral performance were assessed using repeated-measures analysis of variance (ANOVA).

Figure 1.

Figure 1

Open in a new tab

Experimental task and behavioral responses. (A) Participants chose between stimulus pairs from three conditions: gain (upper images), loss (lower images), and neutral (not shown) associated with the corresponding pairs of outcomes: gain £1/nothing, lose £1/nothing, and look £1/nothing. The two stimuli forming each stimulus pair had reciprocal probabilities (.8/.2 and .2/.8) of receiving the corresponding outcome. For example, in the gain condition, one of the stimulus pairs had an 80% chance of winning £1 and a 20% chance of winning nothing; the other option had a 20% chance of winning £1 and an 80% change of winning nothing. Stimulus pairs were presented randomly, with the high probability win/loss/look stimulus presented on the right on 50% of trials and on the left on 50% of trials. (B) Observed behavioral choices for gain and loss conditions following placebo (blue) or typhoid vaccine induced inflammation (red). The learning curves (moving average) depict trial by trial the percentage of times participants chose the correct stimulus (probability = .8 of winning £1) upper graph and the incorrect stimulus (probability = .8 of losing £1). (C) Modeled behavioral choices for placebo (blue) and inflammation (red). The learning curves represent the probabilities predicted by the computational model. (D) Proportion of the last 50% of trials in which participants chose the correct stimulus for both gain (left) and loss (right) conditions. (E) Modeled behavioral choices for the proportion of the last 50% of trials in which participants chose the correct stimulus. Obs., observed.

Computational Model

A standard algorithm of action-value learning that combines the Rescorla-Wagner learning rule (which updates chosen option values in proportion to reward prediction errors) and a softmax decision rule (which estimates choice probability as a sigmoid function of the difference between the two option values Qa and Qb) (8, 28) was fitted to the observed behavior. For each pair of stimuli (A and B), the model used each individual’s sequences of choices and outcomes to estimate the expected values of choosing A (QA) and B (QB). Expected values (QA and QB) were initialized at zero and the value of stimulus chosen at each trial (e.g., A) was updated according to the rule QA(t + 1) = QA(t) + α × δ(t), with outcome prediction error δ(t) defined as the difference between the actual and expected outcome, δ(t) = R(t) – QA(t). Given the expected values, the probability of the observed choice was estimated using the softmax rule PA(t) = exp(QA(t) / β) / {exp[QA(t) / β] + exp[QB(t) / β]}. The free parameters alpha (learning rate), beta (temperature), and R (subjective value) were adjusted to maximize the likelihood of each participant’s observed choices under the model.

Cytokine Analysis

Blood (10 mL) was collected in ethylenediaminetetraacetic acid vacutainer tubes (Becton Dickinson and Company, Franklin Lakes, New Jersey) and centrifuged at 1250g for 10 minutes; then plasma was removed, aliquoted, and frozen at −80°C. Plasma interleukin-6 (IL-6) was assessed using high-sensitivity enzyme-linked immunosorbent assays (R&D Systems, Abingdon, United Kingdom). The limit of detection of the IL-6 assay was .039 pg/mL, with intra-assay and interassay coefficients of variation of 7.4% and 7.8%. Cytokine analysis was performed using repeated-measures ANOVA in SPSS 22 (IBM Corp., Armonk, New York).

Image Acquisition and Analysis

T2*-weighted echo planar images (EPIs) were acquired on a 1.5T Siemens Avanto magnetic resonance scanner equipped with a 12-channel head coil (Siemens Healthcare, Erlangen, Germany) using a −30° tilted acquisition to reduce orbitofrontal dropout (29). Each volume provided whole-brain coverage (40 interleaved ascending 2 mm slices with 1 mm interslice gap, echo time 40 ms: repetition time 3.3 s, spatial resolution 3 mm3). High-resolution inversion-recovery echo planar images were additionally acquired, segmented, and then normalized in SPM8 (Wellcome Trust Centre for Neuroimaging, Institute of Neurology, United College London, United Kingdom; http://www.fil.ion.ucl.ac.uk/spm) to aid group-level anatomical localization. EPIs were analyzed in an event-related manner using SPM8. Preprocessing consisted of spatial realignment, segmentation, and normalization of the mean EPI image to a standard EPI template and then spatial smoothing with an 8 mm full-width at half maximum Gaussian kernel. Subject-specific realignment parameters were modeled as covariates of no interest to correct for motion artifacts. Stimulus and outcome onsets were modeled as separate delta functions and convolved with a canonical hemodynamic response function. Prediction errors and Q-values calculated by the computational model were used as additional regressors that parametrically modulated outcome and cue onsets, respectively. Linear contrasts of regression coefficients were computed at the individual subject level and then taken to group level repeated-measures ANOVA (factors: inflammation [vaccine, placebo], condition [gain, loss]). Activation maps for reward prediction error (rPE) and punishment prediction error (pPE) in ventral striatum and anterior insula reported in the original article (8) using this task were obtained and used as region of interest masks. All group-level statistical parametric maps are reported with a whole-brain or region of interest familywise error correction threshold of p < .05.

Results

Participant Characteristics

Behavioral outcomes were derived from 24 healthy nonsmokers (9 male subjects, mean 27.6 ± 7.0 years) screened for a history of relevant physical or psychiatric illness. Of these, 18 were scanned (one did not complete the second scanning session due to technical difficulties). All were medication free and rated their general health as good, very good, or excellent.

Response to Typhoid Vaccination

Typhoid vaccination evoked a robust inflammatory response with an approximately 250% increase in plasma IL-6 from mean (± SE) .89 ± .23 pmol/L at baseline to 2.17 ± .26 pmol/L at 3½ hours (t22 = 5.21, p < .001) (Figure 2A). The placebo condition evoked a smaller nonsignificant rise in IL-6 from .73 ± .15 pmol/L at baseline to .94 ± .16 pmol/L at 3.5 hours (t15 = 1.60, p = .13). This was confirmed by a significant treatment (inflammation, placebo) by sample (baseline, 3.5 hours) interaction for IL-6 (F1,22 = 17.13, p < .001).

Figure 2.

Figure 2

Open in a new tab

Effects of vaccination on model parameters. Behavioral outcomes for model parameters: (A) alpha (learning rate), (B) beta (choice randomness or temperature), (C) R (subjective value), and (D) log likelihood of model fit. Data are represented as solid bars for gain and shaded bars for loss conditions. Higher values represent faster learning, greater choice randomness, reward (and punishment) subjective value, and model fit, respectively. Error bars represent standard error of the mean.

There was no significant effect of vaccination on core body temperature (treatment × sample interaction: F1,22 = .81, p = .38) (Figure 2B) or somatic symptoms (interaction: F1,22 = .28, p = .60), confirming that effects were not driven by pain, temperature, or subjective discomfort. POMS measured fatigue significantly increased following inflammation (time × inflammation interaction: F1,22 = 5.02, p = .036). Though there was a larger drop in POMS total mood score in the inflammation compared with placebo condition (4.8 vs. 2.2 points), this was not statistically significant (F1,22 = 1.20, p = .28).

Behavioral Outcomes

Inflammation was associated with a shift in reward versus punishment sensitivity, expressed as reduced selection of high probability reward, yet increased avoidance of high probability punishment stimuli (Figure 1B). This was supported by a significant inflammation (placebo, vaccine) by valence (reward, punishment) interaction (F1,22 = 5.48, p = .029) (Figure 1D). Of note, post hoc t tests for reward and punishment conditions were p = .195 and p = .071, respectively, indicating that inflammation induced a relative increase in sensitivity to punishment versus reward. Importantly, there was no significant main effect of inflammation or inflammation by valence interaction for go versus no-go responses, confirming equal task engagement across conditions (p > .10), and no significant main effect of session or session by condition (reward, punishment) interaction (F1,22 = .32, p = .58, and F1,22 = .63, p = .44, respectively). There was no significant main effect of time (session 1/session 2) or time by condition (gain/lose) interaction (F1,22 = .50, p = .49, and F1,22 = 1.26, p = .27, respectively).

To analyze this effect of inflammation on reward versus punishment sensitivity in more detail, we next fitted our reinforcement-learning model (3) to the observed choices. The three free model parameters, learning rate (α), choice randomness (β), and subjective value (R) were adjusted to optimally fit the model to the learning curves and maximize the likelihood of the observed choices. This was done separately for gain and loss conditions under both placebo and inflammation for each participant. The adjusted free parameters were then tested for condition effects (inflammation/placebo, reward/punishment) in repeated-measures ANOVAs.

Analysis of the model data confirmed the inflammation (placebo, vaccine) by valence (reward, punishment) interaction observed in the behavioral responses (F1,22 = 4.33, p = .049) (Figure 1C, E). Interestingly, inflammation was not associated with a change in either learning rate (α) or choice randomness (β) (F1,22 = 3.258, p = .082, and F1,22 = 1.781, p = .19, respectively) (Figure 2C, D). However, it was associated with a change in subjective value (R) (inflammation × valence interaction: F1,22 = 4.694, p = .041; Figure 2E), specifically an increase in the (negative) subjective value of the punished stimulus (paired t22 = −2.107, p = .047). There was no significant effect on the subjective value of the rewarded stimulus (paired t22 = .938, p = .359). Of note, participant behavior was equally well modeled across placebo and vaccine conditions (mean log likelihood = −9.17; interaction: F1,22 = .246, p = .624; Figure 2F).

Imaging

Modeling of gain versus neutral (look £1) cues was associated with significant activation within ventral striatum and left posterior putamen (Figure 3A) and loss versus neutral cues with bilateral ventral striatum and insula activation (Figure 3B; Supplemental Table S1), as described previously for this task in an independent population (8). We next used the reinforcement-learning model to extract trial by trial δ and predicted outcome, which were then used as parametric modulators of outcome and stimulus phases, respectively. Examination of the representation of outcome prediction error across both conditions (placebo, inflammation) demonstrated positive correlation with bilateral ventral striatum activity with an additional negative correlation with punishment prediction error in the left insula (Figure 3; Table 1), as previously reported with this task.

Figure 3.

Figure 3

Open in a new tab

Brain regions correlating with reward and punishment cues and prediction error. Upper rows of both panels denote data from the current study at an uncorrected statistical threshold of p < .001. Lower rows show comparable contrasts from Pessiglione et al. (8). (A) Statistical parametric maps resulting from the main contrasts between stimuli conditions. Go and NoGo refer to stimuli requiring or not requiring a button press to get the optimal outcome. Gain, neutral, and loss correspond to the different pairs of stimuli. Activations are shown on slices comparable with Pessiglione et al. (8) located in the posterior putamen (green), left ventral striatum (blue), and bilateral insula (red). (B) Brain activity correlated with prediction errors (PE) derived from the computational model. Reward prediction errors (positive correlation) are shown across both gain and loss conditions (left and center panels); punishment prediction errors (negative correlation) are found in the loss condition alone (right panel). As above, activations are shown on slices comparable with Pessiglione et al. (8) located in the posterior putamen (green), left ventral striatum (blue), and bilateral insula (red). [Reproduced with permission from Pessiglione et al. (8).]

Table 1.

Significant Clusters Correlating With Reward and Punishment Prediction Error

Side Region Coordinates Z Score k p FWE (ROI)
Reward Prediction Error
L Ventral striatum [−16 8 −8] 3.97 142 <.001 .063 (.006)
R Ventral striatum [12 8 −8] 5.01 232 <.001 .007 (.001)
L Posterior putamen [−26 −6 8] 4.12 118 <.001 .118 (.004)
R Sensory-motor [6 −32 68] 5.05 520 <.001 .001
R STS [56 −8 0] 4.76 292 <.001 .002
R Inferior parietal [52 −28 40] 4.58 254 <.001 .004
R Cerebellum [42 −66 −42] 4.41 321 <.001 .001
R Occipital pole [24 −100 12] 4.32 280 <.001 .003
Punishment Prediction Error
R Anterior insula [42 −28 −10] 4.33 275 <.001 .002 (.023)
Mid Striate cortex [4 −78 −6] 6.85 1435 <.001 .001
L Fusiform [−24 −48 −14] 5.17 198 <.001 .013
R Fusiform [24 −64 −8] 5.01 456 <.001 .001
R TPJ [62 −40 28] 4.81 216 <.001 .009
Open in a new tab

Only clusters surviving whole brain or region of interest (reported in brackets) familywise error correction are reported. k denotes cluster extent; [x y z] are Montreal Neurological Institute coordinates.

FWE, familywise error; L, left; R, right; ROI, region of interest; STS, superior temporal sulcus; TPJ, temporoparietal junction.

To further investigate the basis of the behavioral effects of inflammation, specifically increased sensitivity to punishment compared with reward, we next investigated effects of inflammation on ventral striatal and insula encoding of reward and punishment prediction error. Ventral striatum and insula regions of interest were first defined using clusters correlating with reward and punishment prediction error in an independent population (8). We then investigated effects of inflammation on reward and punishment prediction error within each region using the contrasts vaccine < placebo and placebo > vaccine, respectively. This demonstrated a significant reduction in ventral striatal encoding of reward prediction error following inflammation and conversely a significant increase in right insula encoding of punishment prediction error (Figure 4; Supplemental Table S2). Bayesian model selection (Supplement) supported mediation via actions on prediction error rather than outcome value (Supplemental Figure S1).

Figure 4.

Figure 4

Open in a new tab

Effects of inflammation on ventral striatal and insula responses to reward and punishment prediction error (PE). (A) Bottom right panel: right ventral striatal region demonstrating significantly reduced correlation with reward prediction error following inflammation (compared with placebo). Remaining panels illustrate the same contrast (yellow) overlaid on the right ventral striatal region of interest mask (correlation with reward prediction error) from Pessiglione et al. (8) (cyan). (B) Bottom right panel: right insula region demonstrating significantly increased correlation with punishment prediction error following inflammation (compared with placebo). Remaining panels illustrate the same contrast (yellow) overlaid on the right insula region of interest mask (correlation with punishment prediction error) from Pessiglione et al. (8) (cyan). P, placebo; V, vaccine.

Discussion

Theories of instrumental learning highlight a central role for prediction error signals in updating the values associated with available choices, aiding learning from success and failure and ultimately improving future decisions (30). Using a probabilistic instrumental learning task, we showed that experimentally induced inflammation significantly enhances sensitivity to punishments versus rewards. Modeling of individual choices using our reinforcement-learning model accurately reflected this pattern of behavioral effects and demonstrated a significant interaction between inflammation and model parameters for the subjective value of rewards versus punishments. Across conditions, we replicated and extended previous findings of correlations between ventral striatal and anterior insula activity and computationally determined reward (rPE) and punishment (pPE) prediction errors, respectively. However, we also showed that both were significantly modulated by inflammation, with inflammation prompting a significant reduction in the encoding of rPE within ventral striatum and a converse enhancement of insula encoding of pPE. Bayesian model selection (Supplement) further supported this mechanistic interpretation. These findings suggest that actions of inflammation on ventral striatal and insula regions encoding rPE and pPE together mediate the motivational reorientation characteristic of sickness behaviors (1, 2), differentially modulating how values associated with available choices are updated and ultimately enhancing sensitivity to punishments compared with rewards. They also provide further evidence for differential neural encoding of reward and punishment prediction error signals in humans.

Impairment in reward-related behavior is a core feature of the motivational reorientation characteristic of sickness behaviors and can be indexed in animals by reduced saccharin preference (31, 32, 33), anhedonia (2), and reduced rewarding electrical self-stimulation (10, 34). Previous human fMRI studies note inflammation-induced reductions in ventral striatal reactivity to both reward cues (11) and reward outcomes (9). Our fMRI data support and develop this literature by suggesting that impairments in reward-related behavior, which can be observed within hours of inflammatory challenge, may be mediated via specific actions on ventral striatal rPE encoding. Computational analyses captured this shift in plateau response across gain and loss conditions as a significant condition (gain, loss) by inflammation (vaccine, placebo) interaction for the subjective value of rewards compared with punishments. Nevertheless, it should be noted that, unlike the loss learning condition, this reduction in reward magnitude did not reach statistical significance for post hoc t test (p > .05). Further, though our behavioral data demonstrated a significant increase in relative sensitivity to punishments compared with rewards, there was no statistically significant reduction in reward (or punishment) sensitivity per se. Together, these data reveal that ventral striatal encoding of rPE, considered critical for reward learning, is sensitive to inflammatory state and affords one element of an efficient mechanism for the rapid reorientation of behavior in the face of an acute infection.

Though we did not measure dopamine activity directly, a similar reduction in striatal reward prediction error magnitude and propensity to choose the most rewarded action has previously been reported on this task after haloperidol (a dopamine receptor 2 antagonist) (8). This suggests that our observed changes in striatal prediction errors were likely mediated by actions of inflammation on dopamine release. It is therefore noteworthy that inflammation has also been linked to altered nucleus accumbens dopamine efflux in rodents (10) and reduced presynaptic dopamine synthesis or release in humans (9). Supporting this, monkeys showing behavioral impairment after inflammatory challenge with lipopolysaccharide exhibit significantly lower cerebrospinal fluid concentrations of the dopamine metabolite homovanillic acid (35). How inflammation modulates dopamine function is currently unclear. However, individual cytokines such as interferon-alpha have been shown to inhibit dopamine synthesis by reducing central nervous system tetrahydrobiopterin, an essential cofactor for tyrosine hydroxylase, the rate-limiting step in dopamine synthesis (36). Inflammation can also decrease synaptic dopamine by increased expression of the monoamine reuptake transporter (36, 37, 38, 39). Inflammation may further influence dopamine neurotransmission via activation of the tryptophan-degrading enzyme indoleamine 2,3-dioxygenase and resultant formation of neurotoxic kynurenine metabolites (2, 37).

Inflammation significantly enhanced sensitivity to punishments compared with rewards, suggesting a coordinated biasing of behavior toward avoidance of punishment yet decreasing sensitivity to reward. Computational analysis of the loss task revealed a distinct effect of inflammation, with greater avoidance of the punishing option (reflected as a lower plateau) specifically captured by a greater (negative) punishment subjective value. This was also reflected in the larger effect size of the right anterior insula correlation with negative pPE. Increasing pPE is one way to increase the subjective value of punishment, theoretically aiding discrimination of the two cues. This may serve as the computational mechanism by which the anterior insula drives the improvement in avoidance behavior. This is in line with theories proposing that brain areas involved with somatic affective representations (including inflammation) are causally responsible for making a choice (24, 40, 41, 42). This characteristic pattern of behavioral change is noteworthy as it complements an earlier study showing impaired sensitivity to punishment (with a higher plateau) in patients with selective insula lesions (26). Interestingly, it also suggests that relative sensitivity to reward versus punishment is a state rather than a trait-dependent attitude, flexibly enhancing loss minimization in the context of a threat to the organism (such as an infection) yet maximizing responses to gains when in good health.

Bayesian model selection suggested that pPE (as opposed to punishment outcomes) drove effects observed within the whole anterior insula region of interest, including the subregion showing sensitivity to inflammation. However, rPE only drove effects for the discrete ventral striatal subregion that showed sensitivity to inflammation, with effects within other regions of the ventral striatum being driven more by reward outcome. Interestingly, this region is consistent with effects of interferon on reward outcomes (9) but lies slightly more dorsal to a region previously shown to be sensitive to lipopolysaccharide-induced effects on reward cues (11).

Low-level systemic inflammation similar to that induced using the typhoid challenge model is increasingly implicated in the etiology of depression (2, 43), a condition itself characterized by impaired sensitivity to reward yet increased sensitivity to punishment (13, 44). Indeed, one in three patients given weekly injections of the pro-inflammatory cytokine interferon-alpha for hepatitis C develop major depression (45). Dysfunctional responses to negative feedback were among the earliest cognitive changes observed in depression, as predicted by models of learned helplessness (46). More recently, meta-analysis of computationally modeled reinforcement learning tasks has reported, similar to our own findings, a selective reduction in subjective reward value rather than reward learning rate in individuals with depression or a past history of depression (47). Relatively selective actions on reward/punishment magnitude, rather than learning rate or choice temperature, have also been reported following dopamine manipulation and insular damage (8, 26). Our findings of a rapid cognitive adaptation following inflammation heightening relative sensitivity to punishment versus reward raise the intriguing possibility that while this may be beneficial in the context of an infective challenge when metabolic resources are diverted to fighting the infecting organism, when chronic, they may predispose to developing the maladaptive changes in motivation observed in depression.

Acknowledgments and Disclosures

This study was funded by the Wellcome Trust (WT) Fellowship Grant for NAH (Grant No. WT093881MA). NAH and VV are WT Intermediate Clinical Fellows. MC has received grants from the Italian Ministry of Health and the Science & Technology Facilities Council. EAC is supported on the WT Fellowship Grant (Grant No. WT093881MA). MP is funded by the European Research Council (RCN: 98088). HDC is supported by a European Union Advanced Grant (RCN: 108306).

The authors report no biomedical financial interests or potential conflicts of interest.

Footnotes

Appendix A

Supplementary material cited in this article is available online at http://dx.doi.org/10.1016/j.biopsych.2015.07.018.

Appendix A. Supplementary Materials

Supplementary Material

mmc1.pdf (1.4MB, pdf)

References

  • 1.Hart B.L. Biological basis of the behavior of sick animals. Neurosci Biobehav Rev. 1988;12:123–137. doi: 10.1016/s0149-7634(88)80004-6. [DOI] [PubMed] [Google Scholar]
  • 2.Dantzer R., O’Connor J.C., Freund G.G., Johnson R.W., Kelley K.W. From inflammation to sickness and depression: When the immune system subjugates the brain. Nat Rev Neurosci. 2008;9:46–56. doi: 10.1038/nrn2297. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Sutton R.S., Barto A.G. The MIT Press; Cambridge, MA: 1998. Reinforcement Learning: An Introduction. [Google Scholar]
  • 4.Bayer H.M., Glimcher P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron. 2005;47:129–141. doi: 10.1016/j.neuron.2005.05.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
  • 6.Steinberg E.E., Keiflin R., Boivin J.R., Witten I.B., Deisseroth K., Janak P.H. A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci. 2013;16:966–973. doi: 10.1038/nn.3413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Tsai H.-C., Zhang F., Adamantidis A., Stuber G.D., Bonci A., de Lecea L., Deisseroth K. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science. 2009;324:1080–1084. doi: 10.1126/science.1168878. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Pessiglione M., Seymour B., Flandin G., Dolan R.J., Frith C.D. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature. 2006;442:1042–1045. doi: 10.1038/nature05051. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Capuron L., Pagnoni G., Drake D.F., Woolwine B.J., Spivey J.R., Crowe R.J. Dopaminergic mechanisms of reduced basal ganglia responses to hedonic reward during interferon alfa administration. Arch Gen Psychiatry. 2012;69:1044–1053. doi: 10.1001/archgenpsychiatry.2011.2094. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Borowski T., Kokkinidis L., Merali Z., Anisman H. Lipopolysaccharide, central in vivo biogenic amine variations, and anhedonia. Neuroreport. 1998;9:3797–3802. doi: 10.1097/00001756-199812010-00006. [DOI] [PubMed] [Google Scholar]
  • 11.Eisenberger N.I., Berkman E.T., Inagaki T.K., Rameson L.T., Mashal N.M., Irwin M.R. Inflammation-induced anhedonia: Endotoxin reduces ventral striatum responses to reward. Biol Psychiatry. 2010;68:748–754. doi: 10.1016/j.biopsych.2010.06.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Watkins L.R., Maier S.F. Immune regulation of central nervous system functions: From sickness responses to pathological pain. J Intern Med. 2005;257:139–155. doi: 10.1111/j.1365-2796.2004.01443.x. [DOI] [PubMed] [Google Scholar]
  • 13.Wegner A., Elsenbruch S., Maluck J., Grigoleit J.-S., Engler H., Jäger M. Inflammation-induced hyperalgesia: Effects of timing, dosage, and negative affect on somatic pain sensitivity in human experimental endotoxemia. Brain Behav Immun. 2014;41:46–54. doi: 10.1016/j.bbi.2014.05.001. [DOI] [PubMed] [Google Scholar]
  • 14.Watkins L.R., Hutchinson M.R., Ledeboer A., Wieseler-Frank J., Milligan E.D., Maier S.F. Glia as the “bad guys”: Implications for improving clinical pain control and the clinical utility of opioids. Brain Behav Immun. 2007;21:131–146. doi: 10.1016/j.bbi.2006.10.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Hutchinson M.R., Buijs M., Tuke J., Kwok Y.H., Gentgall M., Williams D., Rolan P. Low-dose endotoxin potentiates capsaicin-induced pain in man: Evidence for a pain neuroimmune connection. Brain Behav Immun. 2013;30:3–11. doi: 10.1016/j.bbi.2013.03.002. [DOI] [PubMed] [Google Scholar]
  • 16.Ali Z., Meyer R.A., Campbell J.N. Secondary hyperalgesia to mechanical but not heat stimuli following a capsaicin injection in hairy skin. Pain. 1996;68:401–411. doi: 10.1016/s0304-3959(96)03199-5. [DOI] [PubMed] [Google Scholar]
  • 17.Voon V., Pessiglione M., Brezing C., Gallea C., Fernandez H.H., Dolan R.J., Hallett M. Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron. 2010;65:135–142. doi: 10.1016/j.neuron.2009.12.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Büchel C., Morris J., Dolan R.J., Friston K.J. Vol. 20. 1998. Brain systems mediating aversive conditioning: An event-related fMRI study; pp. 947–957. (Neuron). [DOI] [PubMed] [Google Scholar]
  • 19.Seymour B., O’Doherty J.P., Dayan P., Koltzenburg M., Jones A.K., Dolan R.J. Temporal difference models describe higher-order learning in humans. Nature. 2004;429:664–667. doi: 10.1038/nature02581. [DOI] [PubMed] [Google Scholar]
  • 20.Samanez-Larkin G.R., Hollon N.G., Carstensen L.L., Knutson B. Individual differences in insular sensitivity during loss anticipation predict avoidance learning. Psychol Sci. 2008;19:320–323. doi: 10.1111/j.1467-9280.2008.02087.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Kim H., Shimojo S., O’Doherty J.P. Vol. 4. 2006. Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain; p. e233. (PLoS Biol). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Tracey I., Mantyh P.W. The cerebral signature for pain perception and its modulation. Neuron. 2007;55:377–391. doi: 10.1016/j.neuron.2007.07.012. [DOI] [PubMed] [Google Scholar]
  • 23.Harrison N.A., Brydon L., Walker C., Gray M.A., Steptoe A., Dolan R.J., Critchley H.D. Neural origins of human sickness in interoceptive responses to inflammation. Biol Psychiatry. 2009;66:415–422. doi: 10.1016/j.biopsych.2009.03.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Harrison N.A., Cooper E., Dowell N.G., Keramida G., Voon V., Critchley H.D., Cercignani M. Quantitative magnetization transfer imaging as a biomarker for effects of systemic inflammation on the brain. Biol Psychiatry. 2015;78:49–57. doi: 10.1016/j.biopsych.2014.09.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Rosenkranz M.A., Busse W.W., Johnstone T., Swenson C.A., Crisafi G.M., Jackson M.M. Neural circuitry underlying the interaction between emotion and asthma symptom exacerbation. Proc Natl Acad Sci U S A. 2005;102:13319–13324. doi: 10.1073/pnas.0504365102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Palminteri S., Justo D., Jauffret C., Pavlicek B., Dauta A., Delmaire C. Critical roles for anterior insula and dorsal striatum in punishment-based avoidance learning. Neuron. 2012;76:998–1009. doi: 10.1016/j.neuron.2012.10.017. [DOI] [PubMed] [Google Scholar]
  • 27.Wright C.E., Strike P.C., Brydon L., Steptoe A. Acute inflammation and negative mood: Mediation by cytokine activation. Brain Behav Immun. 2005;19:345–350. doi: 10.1016/j.bbi.2004.10.003. [DOI] [PubMed] [Google Scholar]
  • 28.Voon V., Pessiglione M., Brezing C., Gallea C., Fernandez H.H., Dolan R.J., Hallett M. Mechanisms underlying dopamine-mediated reward bias in compulsive behaviors. Neuron. 2010;65:135–142. doi: 10.1016/j.neuron.2009.12.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Deichmann R., Gottfried J.A., Hutton C., Turner R. Optimized EPI for fMRI studies of the orbitofrontal cortex. Neuroimage. 2003;19:430–441. doi: 10.1016/s1053-8119(03)00073-9. [DOI] [PubMed] [Google Scholar]
  • 30.Rescorla RA, Wagner AR (1972): A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In Black AH, Prokasy WF, editors. Classical Conditioning II: Current Research and Theory. New York: Appleton-Century-Crofts, 64–99
  • 31.Shen Y., Connor T.J., Nolan Y., Kelly J.P., Leonard B.E. Differential effect of chronic antidepressant treatments on lipopolysaccharide-induced depressive-like behavioural symptoms in the rat. Life Sci. 1999;65:1773–1786. doi: 10.1016/s0024-3205(99)00430-0. [DOI] [PubMed] [Google Scholar]
  • 32.Yirmiya R. Endotoxin produces a depressive-like episode in rats. Brain Res. 1996;711:163–174. doi: 10.1016/0006-8993(95)01415-2. [DOI] [PubMed] [Google Scholar]
  • 33.Frenois F., Moreau M., O’Connor J., Lawson M., Micon C., Lestage J. Lipopolysaccharide induces delayed FosB/DeltaFosB immunostaining within the mouse extended amygdala, hippocampus and hypothalamus that parallel the expression of depressive-like behavior. Psychoneuroendocrinology. 2007;32:516–531. doi: 10.1016/j.psyneuen.2007.03.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Anisman H., Kokkinidis L., Borowski T., Merali Z. Differential effects of interleukin (IL)-1b, IL-2 and IL-6 on responding for rewarding lateral hypothalamic stimulation. Brain Res. 1998;779:177–187. doi: 10.1016/s0006-8993(97)01114-1. [DOI] [PubMed] [Google Scholar]
  • 35.Felger J.C., Alagbe O., Hu F., Mook D., Freeman A.A., Sanchez M.M. Effects of interferon-alpha on rhesus monkeys: A nonhuman primate model of cytokine-induced depression. Biol Psychiatry. 2007;62:1324–1333. doi: 10.1016/j.biopsych.2007.05.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Kitagami T., Yamada K., Miura H., Hashimoto R., Nabeshima T., Ohta T. Mechanism of systemically injected interferon-alpha impeding monoamine biosynthesis in rats: Role of nitric oxide as a signal crossing the blood-brain barrier. Brain Res. 2003;978:104–114. doi: 10.1016/s0006-8993(03)02776-8. [DOI] [PubMed] [Google Scholar]
  • 37.Felger J.C., Miller A.H. Cytokine effects on the basal ganglia and dopamine function: The subcortical source of inflammatory malaise. Front Neuroendocrinol. 2012;33:315–327. doi: 10.1016/j.yfrne.2012.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Kamata M., Higuchi H., Yoshimoto M., Yoshida K., Shimizu T. Effect of single intracerebroventricular injection of alpha-interferon on monoamine concentrations in the rat brain. Eur Neuropsychopharmacol. 2000;10:129–132. doi: 10.1016/s0924-977x(99)00067-x. [DOI] [PubMed] [Google Scholar]
  • 39.Shuto H., Kataoka Y., Horikawa T., Fujihara N., Oishi R. Repeated interferon-alpha administration inhibits dopaminergic neural activity in the mouse brain. Brain Res. 1997;747:348–351. doi: 10.1016/s0006-8993(96)01371-6. [DOI] [PubMed] [Google Scholar]
  • 40.Critchley H.D., Harrison N.A. Visceral influences on brain and behavior. Neuron. 2013;77:624–638. doi: 10.1016/j.neuron.2013.02.008. [DOI] [PubMed] [Google Scholar]
  • 41.Naqvi N.H., Bechara A. The hidden island of addiction: The insula. Trends Neurosci. 2009;32:56–67. doi: 10.1016/j.tins.2008.09.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Craig A.D. How do you feel? Interoception: The sense of the physiological condition of the body. Nat Rev Neurosci. 2003;3:655–666. doi: 10.1038/nrn894. [DOI] [PubMed] [Google Scholar]
  • 43.Dowlati Y., Herrmann N., Swardfager W., Liu H., Sham L., Reim E.K., Lanctôt K.L. A meta-analysis of cytokines in major depression. Biol Psychiatry. 2010;67:446–457. doi: 10.1016/j.biopsych.2009.09.033. [DOI] [PubMed] [Google Scholar]
  • 44.Eshel N., Roiser J.P. Reward and punishment processing in depression. Biol Psychiatry. 2010;68:118–124. doi: 10.1016/j.biopsych.2010.01.027. [DOI] [PubMed] [Google Scholar]
  • 45.Miller A.H., Maletic V., Raison C.L. Inflammation and its discontents: The role of cytokines in the pathophysiology of major depression. Biol Psychiatry. 2009;65:732–741. doi: 10.1016/j.biopsych.2008.11.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Seligman M.E. Vol. 23. 1972. Learned helplessness; pp. 407–412. (Annu Rev Med). [Google Scholar]
  • 47.Huys Q.J.M., Pizzagalli D.A., Bogdan R., Dayan P. Mapping anhedonia onto reinforcement learning: A behavioural meta-analysis. Biol Mood Anxiety Disord. 2013;3:1–16. doi: 10.1186/2045-5380-3-12. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Material

mmc1.pdf (1.4MB, pdf)

RESOURCES