Pervasive competition between threat and reward in the brain

Abstract

In the current functional MRI study, we investigated interactions between reward and threat processing. Visual cues at the start of each trial informed participants about the chance of winning monetary reward and/or receiving a mild aversive shock. We tested two competing hypothesis: according to the ‘salience hypothesis’, in the condition involving both reward and threat, enhanced activation would be observed because of increased salience; according to the ‘competition hypothesis’, the processing of reward and threat would trade-off against each other, leading to reduced activation. Analysis of skin conductance data during a delay phase revealed an interaction between reward and threat processing, such that the effect of reward was reduced during threat and the effect of threat was reduced during reward. Analysis of imaging data during the same task phase revealed interactions between reward and threat processing in several regions, including the midbrain/ventral tegmental area, caudate, putamen, bed nucleus of the stria terminalis, anterior insula, middle frontal gyrus and dorsal anterior cingulate cortex. Taken together, our findings reveal conditions during which reward and threat trade-off against each other across multiple sites. Such interactions are suggestive of competitive processes and may reflect the organization of opponent systems in the brain.

INTRODUCTION

The brain mechanisms underlying appetitive and aversive processing have been investigated, by and large, independently of each other. Although many investigators have discussed interactions between these two systems (e.g. Koob and Le Moal, 2008; Leknes and Tracey, 2008), our knowledge about how they may act simultaneously in the brain is rudimentary. Recent studies that investigated interactions between appetitive and aversive processing focused on decision making (Talmi et al., 2009; Park et al., 2011; Amemori and Graybiel, 2012). Overall, the understanding of appetitive–aversive interactions during basic perceptual and attentional processing is currently lacking.

Midbrain dopaminergic regions and their projection sites in the striatum are implicated in appetitive processing (Schultz et al., 2000; O'Doherty, 2004; Delgado, 2007; Haber and Knutson, 2010). However, these regions also participate in aversive processing, indicating that they are involved in both appetitive and aversive motivation (Salamone, 1994; Bromberg-Martin et al., 2010). Conversely, from the opposite valence, processing in the amygdala, bed nucleus of the stria terminalis (BNST) and anterior insula has been frequently linked with aversive events or stimuli (Adolphs and Tranel, 2000; LeDoux, 2000; Craig, 2002, 2009; Davis et al., 2010). Yet, these regions are engaged during appetitive processing, too (Everitt et al., 2003; Salzman et al., 2007; Liu et al., 2011; Mizuhiki et al., 2012). Finally, several brain regions, including the dorsal anterior cingulate cortex (ACC) and dorsolateral prefrontal cortex (PFC) during goal-directed behaviors, are engaged by both appetitive and aversive stimuli. Critically, in all these cases, little is known about how appetitive and aversive processing interact.

To investigate this question, we used both appetitive and aversive stimuli in a factorial design. Participants performed a variant of the monetary incentive delay (MID) task (Knutson et al., 2000) during which different types of visual cues informed them about the chance of winning monetary reward and/or receiving a mild aversive shock (Figure 1). Our goal was to investigate appetitive–aversive interactions during the anticipation period between the cue and target phases, thus allowing us to probe stimulus-independent processes. How does the simultaneous possibility of reward and shock affect brain and behavior?

Fig. 1.

Task design. (A) Subjects performed a variant of Monetary Incentive Delay task. During the threat-reward condition (shown here), a visual cue stimulus (diamond-shape overlaid with dollar sign) signaled that participants could win extra monetary reward if they respond accurately before the target display disappears and also a mild electric shock could occur at the onset of the target display (independent of the performance). Participants were instructed about the meaning of the cue stimuli before task execution. During the target phase, participants were asked to indicate whether the shape was a circle or a square. Following the target phase, participants received feedback about the monetary reward. (B) Four different types of visual cues at the start of each trial informed participants about the chance of winning extra monetary reward and/or receiving a mild aversive shock.

We tested two competing scenarios in key brain regions, including the midbrain, striatum and anterior insula. According to the ‘salience hypothesis’, appetitive and aversive stimuli are represented in terms of their motivational salience. For instance, a recent electrophysiological study in monkeys uncovered dopamine neurons that were excited by both reward-predicting stimuli and airpuff-predicting stimuli (Matsumoto and Hikosaka, 2009). In the present study, the salience hypothesis predicts that, in the condition involving both reward and threat, activation would be enhanced relative to the ‘single’ conditions because of increased salience. According to the ‘competition hypothesis’, reward and threat should trade-off against each other: in the condition involving both reward and threat, responses due to reward would be reduced during threat and responses due to threat would be reduced during reward. One reason a trade-off would be expected is because reward was task relevant, while threat might function as a ‘distractor’—thus, it might lead to a competition for limited processing resources (Pessoa, 2009). Another possibility is that, in some regions, evoked responses might trade-off if positive and negative systems are organized as ‘push–pull’, opponent systems (Konorski, 1967; Solomon and Corbit, 1974). As an example of opponency, pain reduces pleasure and reward induces analgesia (Leknes and Tracey, 2008).

MATERIALS AND METHODS

Subjects

Twenty-four volunteers participated in the study, which was approved by the Institutional Review Board of the University of Maryland, College Park. Based on self-report, subjects were free from psychiatric or neurological disease, or related past history. All participants were right-handed, had normal or corrected-to-normal vision and gave informed written consent. Three participant’s data were excluded from the analysis because of head motion exceeding 3 mm. One other participant discontinued the experiment around the half-way mark and was excluded. Thus, data from 20 participants (27.92 ± 4.61 years old; 12 females) were included in the final analysis.

Stimuli and behavioral paradigm

We used a variant of the MID task (Knutson et al., 2000). Each trial started with the presentation of a compound visual cue (1 s) that was either rectangle or diamond shaped, and overlaid with a pound or dollar sign (Figure 1A). The pound/dollar sign indicated the Reward condition (no-reward or reward), and the geometric shape (rectangle or diamond) indicated the Threat condition (safe or threat). Four different types of cues were used (Figure 1B). The dollar sign indicated the chance of winning monetary reward if the response was made correctly before the display disappeared. The geometric shape (which was counterbalanced across participants) indicated that a mild electric shock could be delivered at the onset of the target display (independent of performance). To calibrate the intensity of the electric shock, each participant was asked to choose his/her own stimulation level immediately before functional imaging, such that the stimulus would be ‘highly unpleasant but not painful’. After each run, participants were asked about the unpleasantness of the stimulus and were asked to, if needed, recalibrate it so that the shock still would be ‘highly unpleasant but not painful’. Shocks were administered with an electrical stimulator (Coulbourn Instruments, PA, USA) on the fourth (‘ring’) and fifth (‘pinky’) fingers of the non-dominant (left) hand. During the threat condition, physical shocks were administered on 50% of the trials at the onset of the target display (participants were not informed about the probability of shock).

In this study, our goal was to investigate interactions between appetitive and aversive processing during the preparatory/anticipation period between the cue and target phases, which we refer to as the delay phase. To do so, the majority (75%) of trials had a long delay period of 12 s between cue and target phases, unlike most previous studies of the MID task, which have used short intervals (2–5 s) (Knutson et al., 2001; Samanez-Larkin et al., 2007). Thus, our design provided a measure of preparatory/anticipation activity that could be largely dissociated from transient events triggered by cue stimuli. During the remaining 25% of trials, a variable delay between 2 and 10 s was used to prevent subject expectancies from developing; these trials were excluded from the data analysis (see below). The shorter-delay trials also ensured that the onset of the target display (and hence the onset of physical shock when administered) was unpredictable to the participants.

Following the delay, a target display was presented at the center of a screen (Figure 1A). Participants performed a shape-discrimination task and were instructed to press the index finger button for circles and the middle finger button for squares with the right hand as fast and as accurately as possible. The duration of the target display on each trial was adjusted dynamically (i.e. ‘staircased’) based on the participant’s performance. The initial target duration of all conditions was set to the same value and was calibrated for each participant based on a practice run (see below). For each condition, separately, if a correct response was made before the target display disappeared, the target duration on the subsequent trial of that condition was decreased by 34 ms; if an incorrect or slow response was made, the duration was increased by 34 ms. This procedure was employed so that participants would be correct and respond before the target display disappeared, on average, 50% of the time in each condition—thus the task was challenging. As noted, during reward trials, reward was based on accurate and fast performance. Consequently, participants were rewarded on ∼50% of reward trials, the same proportion of physical shocks delivered during threat trials.

One second after target onset, participants received visual feedback (1 s) indicating the outcome, as well as their cumulative earnings until that moment. During reward trials, participants won 50 cents per trial if they made a correct response before the target disappeared. On average, participants earned $24 (beyond their base pay). During no-reward trials, participants earned zero cents irrespective of their performance. During incorrect or slow-correct trials across all conditions, visual feedback containing the words ‘incorrect’ or ‘slow’, respectively, was shown. Finally, a 5–10 s variable inter-trial interval (ITI) containing a white fixation cross ended the trial. To minimize the effect of physical shock on the subsequent trial, when a shock was administered, the ITI was set to 10 s.

A practice run was performed during the anatomical scan. No cues were used (and hence, no reward or shock), and visual feedback about correct and incorrect/slow response was provided on each trial. The duration of the target display on the first trial of the practice run was set at 510 ms and was adjusted dynamically in the same fashion as mentioned previously. The final adjusted value was used as the initial duration of the target display for all the conditions in the main task. Participants were not informed that the practice run would be used to calibrate target duration for the main task.

For the presentation of visual stimuli and recording of participant’s responses, Presentation software (Neurobehavioral Systems, Albany, CA, USA) was used. Behavioral responses were collected using an MRI-compatible response box. Skin conductance response (SCR) data were also collected using the MP-150 system (BIOPAC Systems, Inc., CA, USA) with a 10 Hz low-pass hardware filter at a sampling rate of 250 Hz by using MRI-compatible electrodes attached to the index and middle fingers of the left hand.

Each participant performed 12 ‘runs’ of the main task (10 runs for 4 participants). Each run consisted of 16 trials, resulting in a total of 192 trials and 48 trials per condition (160 and 40, respectively, for 4 participants). All experimental conditions were intermixed in a pseudorandom fashion.

MR data acquisition

MR data were collected using a 3 Tesla Siemens TRIO scanner (Siemens Medical Systems, Erlangen, Germany) with a 32-channel head coil (without parallel imaging). Each scanning session began with a high-resolution MPRAGE anatomical scan (TR = 1900 ms, TE = 4.15 ms, TI = 1100 ms, 1 mm isotropic voxels, 256 mm field of view). Subsequently, for each functional run, 138 EPI volumes were acquired with a TR of 2500 and TE of 25 ms. Each volume consisted of 44 oblique slices with a thickness of 3 mm and an in-plane resolution of 3 × 3 mm (192 mm field of view). Slices were positioned ∼30 degrees relative to the plane defined by the line connecting the anterior and posterior commissures, helping to decrease susceptibility artifacts at regions such as the orbitofrontal cortex and amygdala.

General functional MRI data analysis

Preprocessing of the data was done using tools from the AFNI software package (Cox, 1996; http://afni.nimh.nih.gov/afni). The first three volumes of each functional run were discarded to account for equilibration effects. The remaining volumes were slice-time corrected using Fourier interpolation, such that all slices were realigned to the first slice to account for timing differences. Six-parameter rigid-body motion correction within and across runs was performed using Fourier interpolation (Cox and Jesmanowicz, 1999), such that all volumes were spatially registered to the first volume. To normalize the functional data to Talairach space (Talairach and Tournoux, 1988), initially, each subject’s high-resolution MRPAGE anatomical volume was spatially registered to the so-called TT_N27 template (in Talairach space) using a 12-parameter affine transformation; the same transformation was then applied to the functional data. All volumes were spatially smoothed using a Gaussian filter with a full width at half maximum of 6 mm (i.e. two times the voxel dimension). Finally, the signal intensity of each voxel was scaled to a mean of 100.

Voxelwise analysis

Each participant’s functional MRI (fMRI) data were analyzed using multiple regression in AFNI. There were four main event types in the design matrix: no-reward and reward events, separately for the safe and threat conditions. The trials that involved short delay periods (<12 s) were modeled separately using an additional regressor of no interest (pooled over all four conditions). Constant, linear and quadratic terms were included for each run separately (as covariates of no interest) to model baseline and drifts of the MR signal. To account for the signal variance related to head motion, six estimated motion parameters were included as nuisance regressors in the model. Given that all three phases (cue, delay and target) of each trial followed the same sequential order and timing (excluding short delay period trials), we estimated the ‘combined’ trial response. No assumptions were made about the shape of the hemodynamic response function. Responses were estimated starting from cue onset to 30 s post onset using cubic spline basis functions. This method is closely related to the use of finite impulses (‘stick functions’), the commonly used technique that can be considered the simplest form of basis expansion. Cubic splines allow a smoother approximation of the underlying responses, instead of the discrete approximation obtained by finite impulses. As an index of delay-phase activation, we averaged the estimated responses at 10 and 12.5 s after cue onset (as determined via the spline-based estimates) for all four main event types, separately. We used the average of these two points as the stimulus-independent delay-phase response would be maximal at these time points, while the effect of transient cue phase responses would be minimal (see Supplementary Figure S1 showing visual responses). This method of indexing delay-phase activation is similar to the commonly used method of indexing working memory maintenance—related activity in delayed match-to-sample paradigms (Ranganath and D'Esposito, 2001; Pessoa et al., 2002).

We did not exclude trials containing physical shock as the shock was delivered at the onset of the target display, and our goal was to investigate responses before that, namely delay-phase responses.

Group analysis

Whole-brain voxelwise random-effects analyses were restricted to gray-matter voxels based on the FSL automated segmentation tool [‘FAST’ (FMRIB's Automated Segmentation Tool)] (http://www.fmrib.ox.ac.uk/fsl/). A 2 × 2 repeated-measures analysis of variance (ANOVA) was run to investigate the interactions between Reward (no-reward, reward) and Threat (safe, threat) based on delay phase responses. The alpha-level for voxelwise statistical analysis was determined by simulations using the 3dClustSim program of the AFNI toolkit. For these simulations, the smoothness of the data in three directions was estimated using 3dFWHMx on the residual time series of gray-matter voxels in each participant and then averaged across participants (FWHMx = 7.61 mm; FWHMy = 7.63 mm; FWHMz = 7.40 mm). Based on a voxel-level uncorrected alpha of 0.005, simulations indicated a minimum cluster extent of 34 voxels for cluster-level corrected alpha of 0.05. We did not analyze the data from cue and target phases because they were possibly contaminated with rising and falling portions of the delay-phase signals, respectively.

Plotting effects for regions of interest

To plot the response patterns of the loci showing significant Reward × Threat interactions during the delay phase, we carried out a region of interest (ROI) analysis. For each participant, ROIs were defined in an independent fashion by using a leave-one-subject-out method. For each subject, we first created 5-mm radius spherical ROIs using the peak voxel locations of the interaction from the 2 × 2 ANOVA based on data from all subjects, except that subject. Then, for each of the four main conditions of interest, delay-phase responses of voxels that showed a significant interaction effect in the ‘left-out’ participants were averaged within the participant’s ROI. We repeated this procedure for each subject and thus were able to plot the interaction pattern for each ROI defined in a non-biased fashion.

Conjunction analysis

To identify brain areas activated during the processing of both reward and threat stimuli, we conducted a conjunction analysis (Friston et al., 2005; Nichols et al., 2005), based on delay phase responses. To do so, we initially created two statistical brain masks based on voxels that showed a significant simple effect of Reward (reward vs no-reward during the safe condition; cluster-level alpha: 0.05) and, separately, a significant simple effect of Threat (threat vs safe during the no-reward condition; cluster-level alpha: 0.05). We then created an intersection map of these two masks, which revealed voxels with significant common activation.

Skin conductance responses

Skin conductance data from one participant data were excluded owing to technical problems during data collection. Data from the remaining participants were initially smoothed with a median filter over 50 samples (200 ms) to reduce scanner-induced noise and resampled at 1 Hz. The preprocessed SCR data were analyzed using multiple regression in AFNI in a similar way as fMRI data; for related approaches, please see Bach et al. (2009) and Choi et al. (2012). No assumptions were made about the shape of the SCR function. The average response to each trial type was estimated via deconvolution. Variance related to the effect of physical shocks on SCR responses was removed before deconvolution. Responses were estimated starting from event onset to 30 s post onset using cubic spline basis functions (see fMRI analysis above for further discussion). Trials that used a short delay period (<12 s) were modeled separately using an additional regressor of no interest (pooled over all four conditions). Constant and linear terms were included for each run separately (as covariates of no interest) to model baseline and drifts of the SCR. As an index of delay response, for each condition, we averaged estimated SCRs between 10 and 13 s post cue onset (similar time range as used in the imaging data analysis) and subtracted the baseline SCR of each condition (response at cue onset). Finally, to help with normality of the data, response-strength indices were transformed by using a logarithm function [log₁₀ (1 + SCR)]. Then, a 2 × 2 repeated-measures ANOVA was run to investigate interactions between Reward (no-reward, reward) and Threat (safe, threat).

Behavioral data analysis

Trials during which actual physical shocks were delivered were discarded from the analysis, thus leaving 24 trials in the threat conditions (no-reward and reward) and 48 trials in the safe conditions (no-reward and reward). Trials in which participants made incorrect responses (18%) were excluded from further behavioral analyses, but ‘slow’ trials during which a correct response was made after the target disappeared were included. For each participant, mean reaction time (RT) data were determined as a function of Reward (no-reward, reward) and Threat (safe, threat). ANOVAs were conducted on the mean RT data, with those variables as within-subject factors. The alpha-level adopted was 0.05.

RESULTS

Skin conductance responses

SCRs during the delay phase were evaluated according to a 2 Reward (no-reward, reward) × 2 Threat (safe, threat) repeated-measures ANOVA. The main effects of Threat and Reward were significant (F_{1, 19} = 11.79, P = 0.0028 and F_{1, 19} = 5.84, P = 0.0259, respectively). SCR was greater during threat compared with safe trials, as well as during reward compared with no-reward trials. Notably, a statistically significant Reward × Threat interaction was obtained (F_{1, 19} = 6.43, P = 0.0202), such that the increased SCR during threat (vs safe) trials during the no-reward condition was reduced during reward and the increased SCR during reward (vs no-reward) during the safe condition was reduced during threat (Figure 2A).

Fig. 2.

SCR and behavioral results. (A) SCR data during delay phase revealed significant interactions between threat and reward processing, where effect of threat was reduced by reward and effect of reward was reduced by threat. (B) Reaction time data revealed a marginally significant interaction between threat and reward. Error bars in all panels denote the standard within-subject error term (Loftus and Masson, 1994) for the two-way interaction. A.U., arbitrary units.

Behavioral results

Mean RT data were evaluated according to a 2 Reward (no-reward, reward) × 2 Threat (safe, threat) repeated-measures ANOVA (Figure 2B). The main effect of Reward was significant (F_{1, 19} = 49.02, P = 0.0001). Mean RT was faster during the reward (395 ms) compared with the no-reward condition (444 ms), demonstrating the effectiveness of the motivational manipulation. The threat condition also showed faster responses (414 ms) compared with the safe (425 ms) condition, revealing a main effect of Threat (F_{1, 19} = 13.88, P = 0.0014). It is possible that, although threat was irrelevant to the task, cues signaling shock might have increased arousal (as indicated by SCR data), which might have speeded motor responses. The Reward × Threat interaction was marginally significant (F_{1, 19} = 3.52, P = 0.0761). This trend-level result was observed given that faster RTs during reward relative to no-reward trials during the safe condition (56 ms) were numerically reduced during the threat condition (43 ms).

Functional MRI results

The main goal of this study was to investigate interactions between appetitive and aversive processing during the anticipatory/delay phase. Accordingly, we ran a 2 Reward (no-reward, reward) × 2 Threat (safe, threat) voxelwise repeated-measures ANOVA based on estimated responses of the delay phase. We observed a main effect of Reward in several structures, including dorsal ACC and, bilaterally, midbrain/ventral tegmental area (VTA), caudate, putamen, nucleus accumbens and anterior insula; in all cases, responses during reward were greater than no-reward (Table 1). We also observed a main effect of Reward in ‘default’ brain regions (Raichle et al., 2001), where responses decreased during reward (vs no-reward). A main effect of Threat was observed in dorsal ACC, bilateral middle frontal gyrus (MFG), left inferior frontal cortex and right inferior fontal cortex extending into the anterior insula; in all cases, responses during threat were greater than during safe. Critically, a significant interaction between Reward and Threat was observed in the right midbrain/VTA, right caudate, bilateral putamen, bilateral thalamus, bilateral frontal eye field (FEF), bilateral anterior insula, right MFG and dorsal ACC (Figure 3). As illustrated in Figure 4A, a trade-off between reward and threat processing was observed in the right midbrain/VTA, such that the effect of reward (reward vs no-reward) during the safe condition was reduced during threat; likewise, the threat effect (threat vs safe) during no-reward was reduced during reward. Note that, although the coordinates of the midbrain site we report are consistent with the VTA (Adcock et al., 2006; Carter et al., 2009), given the spatial resolution of the fMRI signal, spatial smoothing and size of this structure, this label should be interpreted as ‘suggestive’. For additional regions, see also Figures 4B–D and 5A–D.

Table 1.

Voxelwise analysis at delay phase (Peak talairach coordinates, F_{1, 19} and t(19) values)

Peak location	Reward × Threat				Reward				Threat
	x	y	z	F	x	y	z	t	x	y	z	t
Occipital
Middle occipital gyrus
L									−40	−64	2	−5.67
R									37	−71	2	−3.87
Temporal
Fusiform gyrus
R									41	−49	−10	−4.54
Parahippocampal gyrus
L					−25	−43	−7	−4.54	−25	−37	−10	−5.45
R					20	−38	−7	−3.51	11	−46	−7	−4.78
Superior temporal gyrus
R									50	−28	20	5.19
Middle temporal gyrus
L									−64	−34	−7	5.39
R									53	−31	−7	5.45
L					−49	−67	26	−5.66
R					41	−70	23	−4.59
Parietal
Posterior cingulate cortex
L					−7	−52	26	−6.63
Precuneus
L									−13	−67	29	4.36
Supra marginal gyrus
L	−55	−52	32	28.39					−52	−49	29	5.89
R	47	−49	29	45.51					56	−58	26	5.82
Inferior parietal lobe
L	−37	−34	32	20.53	−43	−34	38	5.52
Left precentral gyrus
L					−40	−19	56	7.01	−52	−13	35	−4.07
R	47	−1	32	21.51	44	−4	50	5.35
Mid-cingulate cortex									−1	−28	29	4.99
Frontal
Frontal eye field
L	−19	−7	50	28.06	−28	−10	44	4.74
R	25	−10	44	32.82	26	−13	44	4.49
Supplementary motor area
L	−10	−10	56	51.66	−7	−7	50	5.52	−4	20	50	6.18
R	5	11	59	58.28	14	−1	56	5.62	8	18	55	5.07
Middle frontal gyrus (posterior)
L									−34	8	44	4.55
R									41	14	38	6.79
Middle frontal gyrus (anterior)
L									−22	47	26	4.95
R	23	41	17	23.11					20	41	26	6.00
Inferior frontal gyrus
L									−52	23	14	9.17
R									50	20	14	4.86
Anterior cingulate cortex (dorsal)
L					−7	5	38	4.72	−4	26	32	5.61
R	8	11	32	43.38	8	5	38	7.10
Superior medial frontal gyrus					−7	56	14	−5.24
Posterior insula
R					35	−16	17	−4.94
Mid-insula
L	−43	8	5	22.14	−43	5	8	4.07
R	47	8	−1	30.76	38	8	5	4.75
Anterior insula
L	−31	26	5	27.04	−31	20	11	6.88
R	32	20	8	40.34	29	20	11	7.17	47	20	−1	4.15
R	29	14	−4	28.46
Subcortical
Midbrain/ventral tegmental area
L					−10	−16	−13	4.72
R	8	−19	−7	22.3	11	−13	−10	5.41
Thalamus
L	−5	−8	10	16.04
R	5	−8	11	19.6	5	−13	−1	4.90
Putamen
L	−22	2	−1	21.92	−25	−1	11	6.59
R	20	2	8	12.91	23	−1	11	5.42
Caudate
L					−10	8	2	5.41
Caudate (dorsal)
R	17	−7	20	45.42	20	5	17	6.26
Caudate (ventral)
R	8	5	5	22.16	8	2	2	7.17
Nucleus accumbens
L					−10	10	2	5.15
R					10	10	2	5.84
Cerebellum
L	−19	−61	−31	32.51	−34	−43	−28	5.72
L	−13	−79	−19	25.01	17	−46	−19	6.44
R	29	−55	−25	19.71	23	−46	−46	5.86

Fig. 3.

Delay phase responses. Voxels that showed significant interaction between threat and reward (displayed at P < 0.05, cluster-level corrected). MB/VTA, midbrain/ventral tegmental area; dACC, dorsal anterior cingulate cortex; Ant. Ins, anterior insula; FEF, frontal eye field; MFG, middle frontal gyrus.

Fig. 4.

Delay phase responses. (A) Mean estimated hemodynamic responses from the right midbrain/VTA ROI (left panel), where the gray area indicates delay-phase responses. On the right panel, these responses are shown as a bar plot (B) Right ventral caudate ROI. (C) Left Putamen ROI. (D) Right thalamus ROI. Error bars in bar plots denote the standard within-subject error term (Loftus and Masson, 1994) for the two-way interaction.

Fig. 5.

Delay phase responses. (A) Mean estimated hemodynamic responses from the right anterior insula ROI (left panel), where the gray area indicates delay-phase responses. On the right panel, these responses are shown as a bar plot. (B) Dorsal anterior cingulate cortex ROI. (C) Right frontal eye field ROI. (D) Right middle frontal gyrus ROI. Error bars in bar plots denote the standard within-subject error term (Loftus and Masson, 1994) for the two-way interaction.

Control analysis

In the results reported above, delay-phase responses might have been partly confounded with motor preparatory signals associated with the target phase. One way to partially address this possibility is to investigate the role of RT in the observed responses. Accordingly, we repeated the analysis above, but now including an additional parametric RT regressor (mean corrected). We reasoned that this additional regressor would model variance related to fluctuations in motor preparation across trials as indexed by RT values. Importantly, this control analysis also revealed significant Reward × Threat interactions in all the regions reported above, minimizing the possibility that our findings were largely driven by changes in motor preparation across conditions.

Conjunction analysis

To identify brain regions commonly engaged by reward and threat processing, a conjunction analysis was run based on delay-phase responses of simple reward and simple threat effects. The conjunction analysis revealed clusters of common activation in several brain regions, notably, right midbrain/VTA, right ventral caudate, right thalamus, bilateral anterior insula, dorsal ACC and bilateral MFG (Figure 6; Table 2). We also inspected the consistency of the simple effects in individual participants as, in theory, there could be clusters of common activation at the group level without a clear counterpart in the individuals. For each region, we list the number of participants exhibiting the two simple effects (final column in Table 2), indicating that the conjunction did not originate from the group analysis process.

Fig. 6.

Conjunction analysis at delay phase. (A) Voxels that showed significant common activation during threat (vs safe during no-reward) and reward (vs no-reward during safe) are shown in yellow color. For illustrative purposes only (with no inferential interpretations), voxels that showed significant activation during threat (vs safe during no-reward) but not in reward (vs no-reward during safe) are shown in red color. In a similar fashion, voxels that showed significant activation during reward (vs no-reward during safe) but not in threat (vs safe during no-reward) are shown in green color. (B) Voxels that showed significant common activation during threat (vs safe during no-reward) and reward (vs no-reward during safe) are shown in yellow color and the border of the clusters that exhibited significant Reward × Threat interactions are shown in magenta color.

Table 2.

Conjunction analysis at delay phase (Peak talairach coordinates of minimum t_{1, 19} values)

Peak location	x	y	z	t	a
Parietal
Supra marginal gyrus
L	−52	−43	23	4.25	15
R	56	−43	20	5.11	15
Precentral gyrus
R	35	−7	38	4.67	15
Frontal
Supplementary motor area
R	2	5	50	4.82	17
Anterior cingulate cortex (dorsal)
R	5	11	35	5.59	16
R	5	29	35	4.86	13
Middle frontal gyrus
L	−34	44	29	3.94	13
R	29	47	20	4.49	15
Anterior insula
L	−31	23	11	5.49	16
R	32	20	8	6.30	17
Subcortical
Thalamus
R	8	−13	−1	4.15	15
Caudate
R	8	5	5	4.30	14
Ventral tegmental area
R	8	13	−7	4.86	14
Cerebellum
L	−25	−61	−31	4.29	15
L	−40	−49	−31	4.34	15
R	26	−67	−25	3.56	14

^aNumber of participants with both simple effects of reward and threat.

Amygdala ROI analysis

In the above analyses, we did not observe significant results in the amygdala. But given the theoretical importance of the amygdala in emotional processing, we conducted an additional ROI analysis to probe the signals of this area. Left and right amygdala ROIs were defined based on anatomy (Figure 7A). In each ROI, a representative time series was created by averaging the unsmoothed time series from all gray-matter voxels within the ROI. Then, as in the whole-brain voxelwise analysis, multiple regression was run on the representative time series data to estimate the hemodynamic response function of four main regressors of interest. A 2 × 2 repeated-measures ANOVA was then run to probe potential interactions between Reward (no-reward, reward) and Threat (safe, threat). Analysis of delay-phase data revealed only a significant main effect of Reward in the left amygdala ROI, such that responses were reduced during reward compared with no-reward (Table 3 and Figure 7B–C).

Fig. 7.

Responses in the amygdala. (A) Coronal slice of the TT_N27 template brain in AFNI showing the voxels within the anatomically defined amygdala (black outline). (B) Mean estimated hemodynamic response functions of four conditions from the left amygdala ROI. (C) Mean estimated hemodynamic response functions of four conditions from the right amygdala ROI. In B and C, the gray area indicates the response estimates related to delay phase. Error bars in bar plots denote the standard within-subject error term (Loftus and Masson, 1994) for the two-way interaction.

Table 3.

ROI analysis in amygdala based on delay-phase responses

ROI	Left amygdala		Right amygdala
	F1, 19	P	F1, 19	P
Main effect of Reward	6.63	0.019	1.36	0.258
Main effect of Threat	0.00	0.996	0.25	0.621
Reward × Threat	0.02	0.892	0.27	0.612

BNST ROI analysis

We also investigated basal forebrain sites consistent with the BNST, a structure implicated in anxiety-related mechanisms (Davis et al., 2010). In the whole-brain voxelwise analysis reported previously, the ventral caudate cluster posteriorly extended into the basal forebrain. Given that the BNST is a small region and the voxelwise analysis was done on spatially smoothed data, we conducted an additional ROI analysis. Left and right BNST ROIs were defined anatomically according to the atlas of Mai et al., 1997 (see also Alvarez et al., 2011, Figure 8A). Talairach x-values were restricted between 3 and 8 mm, y-values were restricted between −1 and 3 mm and z-values were restricted between −1 and 6 mm. For this analysis, we resampled the functional data to a finer 2 × 2 × 2 mm voxel grid and no spatial smoothing was applied. For each ROI, a representative time series was created by averaging the unsmoothed time series from all the gray-matter voxels that fell inside the anatomically defined ROI. Then, regression analysis was run to estimate condition-specific responses. A 2 × 2 repeated-measures ANOVA was subsequently run to probe potential interactions between Reward (no-reward, reward) and Threat (safe, threat). Analysis of delay-phase data revealed a main effect of Reward in both BNST ROIs, such that responses were increased during reward compared with no-reward, and a main effect of Threat in the left BNST ROI, such that responses were increased during threat compared with safe. Critically, a significant interaction between Reward and Threat was observed in the right BNST (Table 4 and Figure 8B–C), such that the effect of reward was reduced during threat and the effect of threat was reduced during reward. The right BNST ROI also showed simple effects of both Reward (reward vs no-reward during the safe condition: t(19) = 5.11, P = 0.0001) and Threat (threat vs safe during the no-reward condition: t(19) = 2.67, P = 0.014).

Fig. 8.

Responses in the BNST. (A) Coronal and axial slices of the TT_N27 template brain in AFNI showing the voxels within the anatomically defined BNST (black outline). (B) Mean estimated hemodynamic response functions of four conditions from the left BNST ROI. (C) Mean estimated hemodynamic response functions of four conditions from the right BNST ROI. In B and C, the gray area indicates the response estimates related to delay phase. Error bars in bar plots denote the standard within-subject error term (Loftus and Masson, 1994) for the two-way interaction.

Table 4.

ROI analysis in BNST based on delay-phase responses

ROI	Left BNST		Right BNST
	F1, 19	P	F1, 19	P
Main effect of Reward	7.39	0.014	22.77	0.000
Main effect of Threat	5.85	0.026	2.25	0.150
Reward × Threat	0.421	0.524	8.67	0.008

DISCUSSION

To investigate the interactions between appetitive and aversive processing, we used a task with cues signaling the chance of monetary reward and/or mild aversive shock. Our design allowed us to measure responses during the preparatory/anticipatory delay phase with minimal contamination from other task phases, thus enabling us to probe stimulus-independent processes. SCR data revealed interactions between reward and threat during the delay phase. Imaging data during this phase revealed interactions between reward and threat in several key brain regions, including the midbrain/VTA, striatum, BNST, anterior insula, right MFG and dorsal ACC. Overall, our findings support the competition hypothesis and not the salience hypothesis: when reward and threat were jointly present, reward opposed the effect of threat [(threat vs safe)_reward < (threat vs safe)_no-reward] and threat opposed the effect of reward [(reward vs no-reward)_threat < (reward vs no-reward)_safe].

Midbrain structures and the striatum are engaged by appetitive processing (Delgado, 2007; Haber and Knutson, 2010). But recruitment of these regions is not limited to appetitive conditions. They take part in the processing of aversive stimuli (Becerra et al., 2001; Roitman et al., 2005; Brischoux et al., 2009; Matsumoto and Hikosaka, 2009; Baliki et al., 2010), financial losses (Carter et al., 2009), anticipation of mild shocks (Jensen et al., 2003) and aversive learning (Delgado et al., 2008). The engagement of these regions during both positive and negative contexts has led to the idea of their role in ‘motivational salience’ (Jensen, et al., 2003, 2007; Carter et al., 2009; Metereau and Dreher, 2013). Our findings demonstrated instead that, in our task, simultaneous reward and threat information opposed each other. Together, the findings demonstrated competition during the processing of motivationally salient stimuli of opposite valence. Of note, when presented alone, appetitive and aversive cues evoked delay-phase responses in the midbrain and striatum (Figure 6A).

In the nucleus accumbens, consistent with prior studies (Schultz et al., 1992; Knutson et al., 2001), a main effect of Reward was observed during the delay phase. But, neither a main effect of Threat nor an interaction was observed. The absence of a threat effect was somewhat unexpected given the rodent literature (Schoenbaum and Setlow, 2003; Roitman et al., 2005). Nevertheless, the results are consistent with some studies in humans that did not observe accumbens activation during anticipation of mild shocks (Choi et al. 2012) and aversive pictures (Grupe et al., 2013). Future studies using other types of aversive conditions, such as monetary losses (Carter et al., 2009), are needed to clarify potential interactions between appetitive and aversive processing in this region.

The anterior insula is involved during the processing of negative events, such as cues signaling monetary losses (Knutson and Greer, 2008), as well as the anticipation and experience of aversive stimuli (Paulus and Stein, 2006; Simmons et al., 2006). The anterior insula also has been implicated in risk aversion (Kuhnen and Knutson, 2005). Yet, recent studies have observed activation in this region during appetitive processing, including to cues signaling monetary gains (Samanez-Larkin, et al., 2007; Liu et al., 2011; Padmala and Pessoa, 2011). Furthermore, anterior insula neurons increased responses when monkeys knew they would, or might receive, a reward based on performance (Mizuhiki et al., 2012). Here, we also observed the effect of reward and threat in the bilateral anterior insula during the delay phase (Figure 6A). Critically, threat and reward processing opposed each other when simultaneously presented.

We did not observe a main effect of Threat or an interaction in the amygdala during the delay phase. This null finding is not entirely surprising because, as proposed by Davis et al. (2010), responses in the amygdala may be more closely tied to phasic CS+ stimuli signaling ‘fear’ or transient cues that signal aversive stimuli (Grupe et al., 2013), as opposed to the periods of temporally extended and less predictable threats. Of note, in our previous study (Choi et al., 2012), as well as in a study by Somerville et al. (2010), greater amygdala responses were not detected during threat monitoring over a temporally extended period.

Another region that is involved in threat processing is the BNST in the basal forebrain, especially during conditions involving temporally extended and/or less predictable threat (Davis et al., 2010; Somerville et al., 2010; Alvarez et al., 2011; Somerville et al., 2013). Intriguingly, studies have also reported the involvement of BNST in appetitive processing (McGinty et al., 2011). This region, which is small and has a complex anatomy, is challenging to investigate with fMRI. Here, we investigated BNST responses based on an anatomical ROI and unsmoothed data. The right BNST was activated by both threat and reward during the delay phase. In addition, a significant interaction was detected there during the delay phase, such that reward and threat traded-off against each other.

In the context of aversive processing, the thalamus has been reported to be involved during the anticipation and experience of negative picture stimuli (Herwig et al., 2007; Goldin et al., 2008), anticipation of mild aversive shocks (Choi et al., 2012) and pain processing (Casey, 1999). At the same time, however, the thalamus participates in appetitive motivational circuits together with striatal and midbrain regions (Kalivas and Nakamura, 1999). The involvement of the thalamus in appetitive processing is further supported by human imaging studies with monetary incentives (Knutson et al., 2001; Galvan et al., 2005; Engelmann et al., 2009) and related studies in non-human animals (Gaffan and Murray, 1990; Balleine, 2005; Minamimoto et al., 2005). In the current study, delay-phase responses in the thalamus were observed during threat as well as reward (Figure 6A). In addition, we observed a significant interaction between reward and threat during the delay phase where reward and threat competed against each other when presented simultaneously.

The dorsal ACC participates in both appetitive and aversive processing, especially during goal-directed behaviors, suggesting that it plays an important function in ‘adaptive control’ (Rushworth and Behrens, 2008; Shackman et al., 2011). Many studies have reported dorsal ACC responses to reward (Shima and Tanji, 1998; Bush et al., 2002; Shidara and Richmond, 2002) and threat (Ploghaus et al., 1999; Etkin et al., 2011) stimuli. Here, dorsal ACC responses during the delay phase revealed simple effects of reward (vs no-reward during the safe condition) and threat (vs safe during the no-reward condition). Again, a competitive interaction between reward and threat was observed during the delay phase.

Whereas the dorsolateral PFC is important for cognition in general, it also has been proposed to be an important convergence site for the integration of both motivation and cognition (Watanabe, 1996; Leon and Shadlen, 1999; Kobayashi et al., 2002) and emotion and cognition (Gray et al., 2002; Erk et al., 2007). In a consistent fashion, here we observed an interaction between reward and threat processing in the right MFG during the delay phase such that reward and threat traded-off against each other.

We also observed a trade-off interaction in the FEF, bilaterally, during the delay phase. In addition, the FEF showed a main effect of Reward. These findings are intriguing because the FEF is important for attention (Kastner and Ungerleider, 2000; Corbetta and Shulman, 2002; Moore and Armstrong, 2003; Armstrong et al., 2006). We interpret the main effect of Reward in terms of attention given that, on seeing the reward cue, participants likely upregulated attention (as indicated by faster RTs). If this interpretation is correct, the counteracting effect of threat on FEF responses suggests that threat might have interfered with attention. In any case, the interaction reveals that reward and threat interact in frontal sites that are important for attention and other related cognitive functions.

COMPETITION

Our interaction results revealed a trade-off between reward and threat processing consistent with competitive interactions. Notably, voxels exhibiting the interaction overlapped with those exhibiting simple effects of both reward and threat (Figure 6B), revealing that many of the areas influenced by both reward and threat are also sites of competitive interactions.

The trade-off pattern observed here is consistent with several independent lines of studies. For example, reward and pain signals interacted in the calculation of subjective value underlying behavioral choice (Park et al., 2011); pain reduced reward sensitivity during a decision-making task (Talmi et al., 2009); stress reduced reward-related responses in medial PFC (Ossewaarde et al., 2011); and acute stress decreased reward responsiveness (Bogdan and Pizzagalli, 2006) and reward-outcome responses (Porcelli et al., 2012). In particular, the reduction of the threat effect during reward in our study is consistent with findings from the startle reflex, where blink responses are reduced during anticipation of rewards (Hackley et al., 2009; see also Lang et al., 1998).

Why is the trade-off pattern observed in the present study? A possible reason is because reward was task relevant, while threat might have functioned as a ‘distractor’. In this way, they might have acted against each other in a way that can be recast in terms of competition for limited processing resources (Pessoa, 2009). This explanation is attractive when sites such as the FEF are considered given their association with attention. The explanation is less appealing, perhaps, when regions such as the midbrain are concerned. Although some researchers have proposed that these regions can also be viewed as associated with ‘effort’ (Horvitz, 2000; Salamone et al., 2009; Boehler et al., 2011), it is possible that the trade-off reflected the organization of positive and negative systems into opponent motivational systems (Konorski, 1967; Solomon and Corbit, 1974), which would operate in a push-pull fashion. Note, however, that the current experiment was not designed to arbitrate between these two scenarios as reward was contingent on performance, but shock was not. We used this asymmetric design to evaluate the salience and competition hypotheses, and not different types of competition mechanisms.

In conclusion, using a factorial design, we investigated stimulus-independent interactions between processing of appetitive and aversive stimuli in normal healthy adult volunteers. Our results from SCR data revealed a trade-off between reward and threat processing. Paralleling the SCR data, imaging data also revealed a trade-off pattern in regions such as midbrain, striatum, BNST, anterior insula, right MFG and dorsal ACC, revealing conditions during which reward and threat compete in the brain when processed simultaneously.

SUPPLEMENTARY DATA

Supplementary data are available at SCAN online.

Conflict of Interest

None declared.