Reward anticipation and punishment anticipation are instantiated in the brain via opponent mechanisms

Abstract

fMRI investigations have examined the extent to which reward and punishment motivation are associated with common or opponent neural systems, but such investigations have been limited by confounding variables and methodological constraints. The present study aimed to address limitations of earlier approaches and more comprehensively evaluate the extent to which neural activation associated with reward and punishment motivation reflects opponent or shared systems. Participants completed a modified monetary incentive delay task, which involved the presentation of a cue followed by a target to which participants were required to make a speeded button press. Using a factorial design, cues indicated whether monetary reward and/or loss (i.e., cues signaled probability of reward, punishment, both, or neither) could be expected depending upon response speed. Neural analyses evaluated evidence of 1) directionally opposing effects by testing for regions of differential activation for reward and punishment anticipation, 2) mutual inhibition by testing for interactive effects of reward and punishment anticipation within a factorial design, and 3) opposing effects on shared outputs via a psychophysical interaction (PPI) analysis. Evidence supporting all three criteria for opponent systems was obtained. Collectively, present findings support conceptualizing reward and punishment motivation as opponent forces influencing brain and behavior and indicate that shared activation does not suggest the operation of a common neural mechanism instantiating reward and punishment motivation.

1. INTRODUCTION

Appetitive and aversive systems are often described as opponent processes - systems that result in directionally opposing changes, mutual inhibition, and/or opposing effects on shared outputs (Daw, Kakade, & Dayan, 2002; Solomon & Corbit, 1974). Nevertheless, both appetitive and aversive stimuli increase arousal, capture attention, and elicit similar cognitive and behavioral responses (Anderson, 2005; Brosch, Sander, Pourtois, & Scherer, 2008; Lang, 1995), suggesting that these systems may not always be opponent but may act instead via a common mechanism. As rewards and punishments powerfully drive behavior, understanding the extent to which motivational processes, specifically act via opponent versus common neural mechanisms is critical for understanding appetitive and aversive influences on brain and behavior. The goal of the present study is to evaluate the extent to which reward and punishment anticipation act via opponent or common neural mechanisms by overcoming limitations in previous approaches and by specifically evaluating whether reward and punishment motivation meet three criteria of opponent systems:1) directionally opposing changes, 2) mutual inhibition, and 3) opposing effects on shared outputs.

Previous fMRI investigations have explored the extent to which reward and punishment motivation are subserved by opponent or common systems primarily using two main approaches. First, studies have evaluated whether patterns of neural activation support directionally opposing changes by characterizing activation as valence-related versus salience- or arousal-related (Bissonette, Gentry, Padmala, Pessoa, & Roesch, 2014; Bromberg-Martin, Matsumoto, & Hikosaka, 2010). Regions have been classified as valence-related when they demonstrate different degrees of activation in response to reward and punishment cues, typically when they demonstrate activation in response to reward and deactivation in response to punishment. Regions have been classified as salience-related when they demonstrate common patterns of activation in response to reward and punishment. Valence-related activation has been used to support the idea of opponent, or competitive, systems, whereas salience-related activation has been used to support the idea of a common neural mechanism for reward and punishment. Support exists for both activation patterns (Garrison, Erdeniz, & Done, 2013; Kahnt, Park, Haynes, & Tobler, 2014; Pessiglione & Delgado, 2015).

Nevertheless, conclusions that can be drawn about opponent vs. common systems from such investigations are limited by a number of factors. First, the spatial resolution of typical fMRI limits the ability to conclude that common fMRI activation reflects the activity of the same neuronal populations; common activation patterns for reward and punishment cues could alternatively reflect the activity of adjacent or intermixed neuronal populations. This limitation is underscored by conflicting reports of salience vs. valence-related activation using fMRI in a number of regions implicated in affect and motivation, including striatum (Bartra, McGuire, & Kable, 2013; Carter, MacInnes, Huettel, & Adcock, 2009; Delgado, Nystrom, Fissell, Noll, & Fiez, 2000; Knutson, Adams, Fong, & Hommer, 2001; Liu, Hairston, Schrier, & Fan, 2011; Metereau & Dreher, 2013; Yacubian et al., 2006), insula (Bartra et al., 2013; Jensen et al., 2007; Kim, Yoon, Kim, & Hamann, 2015; Kurniawan, Guitart-Masip, Dayan, & Dolan, 2013; Litt, Plassmann, Shiv, & Rangel, 2011; Liu et al., 2011; Metereau & Dreher, 2013), and anterior cingulate cortex (ACC) (Fujiwara, Tobler, Taira, Iijima, & Tsutsui, 2009; Litt et al., 2011; Liu et al., 2011; Metereau & Dreher, 2013). Second, reward and punishment motivation are surely supported by networks of brain regions (Bullmore & Sporns, 2009), thus common activation for reward and punishment cues within a single region does not necessarily support the idea that reward and punishment motivation are implemented via a shared neural network (e.g., O’Doherty, Critchley, Deichmann, & Dolan, 2003; Spielberg, Heller, & Miller, 2013). Third, the neurophysiological basis of deactivations in the hemodynamic signal is still not fully understood, but available evidence indicates that a negative blood oxygenation-level dependent (BOLD) response is not necessarily the neurophysiological inverse of a positive BOLD response (Mullinger, Mayhew, Bagshaw, Bowtell, & Francis, 2014), thus some evidence presented as valence-related effects may not truly reflect opponent systems. Finally, studies that have assessed valence- and salience-related activation as described above have generally only considered reward and punishment motivation as separate conditions without evaluating concurrent presentation of reward and punishment motivation, thus only allowing for tests of directionally opposing changes but not for mutual inhibition.

Alternatively, the second main approach to study opponency of reward and punishment motivation using fMRI has been to use factorial designs to test for mutual inhibition in behavior and neural activation via interaction analyses. Interactions between reward and punishment motivation within a particular brain region indicate that these processes are opponent within that region, whereas, if reward and punishment motivation show additive effects (in other words, if these processes do not interact within a particular region), such evidence would refute the idea of opponency in that region because no evidence of mutual inhibition is observed. Prior research has reported reward and punishment opponency based on evidence of interactions between reward and punishment motivation in neural regions (Choi, Padmala, Spechler, & Pessoa, 2014; Nasser & McNally, 2012, 2013). Nevertheless, such conclusions may be limited by confounding factors. Specifically, motivational processes have been confounded by response preparation, i.e., responses required to avoid punishment and gain reward differed, and thus preparations for such responses were also in conflict. In other words, the conflict inherent in preparing for different responses may have confounded neural representations of reward and punishment motivation in such circumstances. Additionally, neither of the two approaches to evaluating motivational opponency using fMRI has evaluated opposing effects of reward and punishment motivation on shared neural outputs.

To overcome the presented limitations of prior research and evaluate whether reward and punishment motivation are subserved by opponent neural systems, in the present study participants completed a modified monetary incentive delay (MID) task during which, for a given trial, one of four cue types indicated whether both reward and punishment were possible (BOTH), only reward was possible (REW), only punishment was possible (PUN), or neither reward nor punishment was possible (NONE) based on a speeded keypress response to a target. Reinforcement depended imperfectly on performance and the magnitude of reward and punishment varied randomly between $1.80–2.35. Participants were required to make the same response to gain rewards and/or avoid punishments, thus the present study deconfounded motivational valence and response preparation to more directly determine whether activation patterns during reward and punishment motivation support opponent neural mechanisms. All neural analyses were restricted to regions previously implicated in valence and/or salience encoding, but, given discrepancies across studies, specific hypotheses regarding which regions would demonstrate specific activation patterns were not established in advance.

To determine whether reward and punishment motivation demonstrate mutual inhibition, an interaction analysis between reward and punishment motivation was conducted. An interaction between reward and punishment motivation within a particular region would support the opponency of these processes within that region. This argument is based on the assumption that the BOTH condition (when both reward and punishment are possible) should be more salient than either the REW (possible reward only) or PUN (possible punishment only) conditions separately. This assumption is based on the Pearce-Hall (1980) model, which contends that more attentional resources are devoted to cues associated with higher outcome uncertainty and thus larger prediction errors. In the present study, the range of potential reinforcement values was greater on BOTH trials, wherein both reward and punishment were possible. Thus, BOTH trials would result in the largest average prediction errors, be more salient than the other cues, and should therefore lead to larger neural responses than REW, PUN, or NONE cues.

To evaluate evidence in support of directionally opposing effects versus a common neural mechanism, a contrast between REW and PUN trials was computed. Classically, to evaluate the extent to which reward and punishment motivation are instantiated via a common neural mechanism, regions of shared activation in response to both reward and punishment have been interpreted as support for a common neural mechanism. The validity of such an interpretation was evaluated here by conducting a conjunction analysis of regions of shared activation in response to both reward and punishment and regions of increased activation in response to both reward and punishment motivation. If activation in response to reward and punishment motivation reflects a common neural mechanism, reward and punishment motivation should have additive effects (i.e., reward motivation + punishment motivation = concurrent reward and punishment motivation), and no reward x punishment motivation interaction should be observed within such regions. Alternatively, a reward x punishment motivation interaction within a region classically defined as salience-related would suggest that reward and punishment motivation are mutually inhibitory processes within such regions and would further support the opponency of reward and punishment motivation as instantiated in the brain..

Finally, to evaluate whether reward and punishment motivation have opposing effects on shared outputs, functional connectivity was leveraged to determine whether regions of common activation to reward and punishment motivation demonstrate differential connectivity during reward and punishment motivation. Differential connectivity, despite a directionally similar response to these two motivational states, would indicate that a region communicates in a valence-specific manner. Such evidence would further support reward and punishment motivation opponency by demonstrating opposing effects on shared outputs, as well as refuting the idea that regions of shared activation necessarily suggest similar mechanisms of reward and punishment motivation as regions of shared activation may participate in different patterns of network activity.

2. METHOD

2.1. Participants

Ninety-eight participants provided written consent and completed the fMRI protocol after screening for a history of serious brain injury, abnormal hearing/vision, claustrophobia, left-handedness, metal in their body, pregnancy, and nonnative English-speaking. Participants were compensated for participation. Data were excluded for participants who moved > 1 voxel (2.13 mm) between adjacent fMRI volumes or committed performance errors > 2 SDs above the group mean (> 10% of trials), resulting in a final sample of 86 participants (42 females, mean age = 19.2, SD = 1.4).

Potential participants were contacted based on their trait scores on the Positive and Negative Affect Schedule (PANAS) (Watson, Clark, & Tellegen, 1988), previously completed for psychology course credit. Individuals were recruited for participation if they scored 1) ≥ 80^th percentile on the Negative Affect subscale of the PANAS and ≤ 50^th percentile on the Positive Affect subscale, 2) ≥ 80^th percentile on the Positive Affect subscale and ≤ 50^th percentile on the Negative Affect subscale, or 3) ≤ 50^th percentile on both subscales. Group membership was a factor of no interest in the present study. All analyses were conducted with and without group membership as a covariate (Miller & Chapman, 2001) to identify impact if any. Inclusion of group membership did not substantively change results; results are reported without group membership as a covariate.

2.2. Experimental Design

Participants completed a modified MID task during separate MRI and EEG sessions. The sessions were completed on different days, with session order counterbalanced across participants; only data from the MRI session are discussed here. In the modified MID task (Figure 1A), participants were instructed that success on each trial depended on making a simple button press using their right index finger as soon as a target appeared and before it changed color. Task timing was optimized using a custom genetic algorithm based on Wager and Nichols (2003). At the beginning of each trial, prior to the target, one of four cues (1.5 s) appeared on the screen (Figure 1B). The cue characterized the possible trial outcomes as a function of response speed: 1) both monetary reward and punishment possible (BOTH); 2) only monetary reward possible (REW); 3) only monetary punishment possible (PUN); or 4) neither monetary reward nor punishment possible (NONE). A variable offset-to-onset interstimulus interval (ISI; 3, 4.5, 6, 7.5 s) indicated by a fixation dot followed cue offset. The ISI was followed by the response target – the onset of a positive, neutral, or negative word. As the present paper is concerned only with cue-related activation, the emotionality of the target words is not considered here. RTs did not vary reliably as a function of word valence or arousal. Targets were displayed for 1.5 s and changed color after an interval that varied across participants and trial blocks. The interval before the target changed color was optimized for each participant and trial block to obtain an equal number of successful and failed trials per cue type. In particular, performance on the previous block (data from the practice block was used for the first block) was used to obtain RT 15th and 85th percentiles, which were used in the subsequent block for the duration until color change for planned failed and successful trials, respectively. A small amount was randomly added/subtracted from these values on each trial to reduce predictability. If a participant response was within 50 ms of a trial’s color change (i.e., 50 ms above the cutoff for 15th percentile trials or 50 ms below the cutoff for 85th percentile trials), it was considered lost/won, respectively, in order to maintain the appropriate number of wins/losses. After a second variable offset-to-onset ISI (3, 4.5, 6, 7.5 s), visual feedback (1.5 s) indicated to participants either 1) the monetary value gained or lost on that trial, or 2) that they had made an error. Errors were defined as button press before the target appeared, pressing an incorrect button, or failing to make a button press. Potential reward and punishment values varied randomly between $1.80–2.35 (mean = $2.08) across trials. Sequential effects were systematically controlled through careful trial ordering, such that transition probabilities (e.g., the likelihood that cue A was followed by cue B) were equal.

Figure 1.

Modified monetary incentive delay task structure. A) A cue was presented indicating the motivational context of that trial. After a variable interstimulus interval, a target word appeared to which the participant was instructed to make a button press as quickly as possible and before the target changed color. After a variable interstimulus interval, feedback indicated whether the participant had responded fast enough to the target and provided the monetary change associated with that trial. B) Factorial structure of the four cue types.

Cues did not indicate reward/punishment magnitudes, only the potential for reward and/or punishment. Participants did not receive feedback about cumulative earnings until the end of the session. A variable offset-to-onset intertrial interval (ITI; 3, 4.5, 6, 7.5 s) separated consecutive trials. The task was presented in 3 blocks of 48 trials, a total of 144 trials. A practice block (24 trials) was completed in the scanner immediately before the task started. To maintain motivation on NONE trials, participants were informed that they could receive a bonus task block (in which they could only win money) at the end of the three task blocks, contingent upon their overall task performance. All participants were allowed to complete the bonus block.

Stimuli for the modified MID task were displayed using back projection. Presentation and RT measurement were controlled by locally developed Matlab code (version 2009a, The MathWorks, Natick, MA), using Psychophysics Toolbox extensions (version 2.54) (Brainard, 1997; Pelli, 1997).

2.3. Behavioral Data Analysis

Average RT was calculated for each cue type (motivational context) for each participant. A 2 × 2 repeated-measures ANOVA was conducted with reward motivation and punishment motivation as within-subjects factors. Planned comparisons between cue types were performed using two-tailed, paired-samples t-tests.

2.4. MRI Data Acquisition

MRI data were acquired using a Siemens Magnetom Trio 3T scanner. Two MPRAGE structural sequences were acquired (192 axial slices with isotropic spatial resolution of 0.9 mm) for registering each participant’s functional data to standard space while participants completed the practice block and were averaged together to increase the signal-to-noise ratio. Gradient field-maps were collected to correct for distortions caused by magnetic field inhomogeneities in the functional data (Jezzard & Balaban, 1995). During each of the three task blocks, 331 functional imaging volumes were collected using a Siemens gradient echo-planar imaging sequence (repetition time [TR] 3000 ms, echo time [TE] 25 ms, flip angle 90°, field of view [FOV] 256 mm). Each image consisted of 50 oblique axial slices (slice thickness 2.40 mm, in-plane voxel size 2.13 mm x 2.13 mm) acquired parallel to the plane containing the anterior and posterior commissures. Three volumes at the beginning of each task block were discarded to allow the scanner to reach steady state.

2.5. MRI Data Analysis

MRI data processing and statistical analysis were implemented primarily using the FSL analysis package (http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/). Raw functional data for each participant were motion-corrected using rigid-body registration implemented in MCFLIRT (Jenkinson, Bannister, Brady, & Smith, 2002), spatially smoothed using a 3D Gaussian kernel (FWHM = 5 mm), and field-map corrected. Motion-related noise components were removed using ICA-AROMA (Pruim, Mennes, Buitelaar, & Beckmann, 2015; Pruim, Mennes, van Rooij, et al., 2015). Data were then temporally filtered with a 1/90 Hz high-pass filter. First-level regression analyses were performed for each block of each participant’s preprocessed functional time series data using FILM (Woolrich, Ripley, Brady, & Smith, 2001). Statistical maps were generated via multiple regression models computed for each intracerebral voxel. A separate predictor was entered for each of the four cue types, each of the three emotion-word types (positive, neutral, negative), and each feedback type (success, failure). Given that the present study was interested in anticipation, emotion-word and feedback phases were predictors of no interest. Three predictors of no interest were included to account for performance errors, modeling each period of the task (cue, emotion word, feedback) separately for trials on which errors were made. Each explanatory variable was convolved with a gamma function (phase = 0 s, SD = 3 s, mean lag = 6 s) to approximate the temporal course of the blood-oxygen-level-dependent (BOLD) hemodynamic response to each stimulus. A per-voxel effect-size parameter estimate (β) map representing the magnitude of activation was created for each explanatory variable.

Comparisons of interest were created by contrasting β values of the relevant parameters to examine effects of reward- and punishment-related contexts and their interaction on neural activation during the cue phase (presentation of the cue + the subsequent ISI). To allow comparisons to be made between the present study and previous investigations of reward and punishment motivation that typically have not used factorial designs, simple effects of reward motivation (REW vs. NONE) and punishment (PUN vs. NONE) motivation were computed. A conjunction analysis was performed (Nichols et al., 2005) to characterize regions showing significant activation in both the REW and PUN conditions relative to the NONE condition; see Crocker et al. (2012) for further description of conjunction analysis approach.

To more fully capture reward and punishment motivation effects within the full factorial design, standard contrasts were created to determine the main effects of reward motivation ([REW + BOTH] vs [PUN + NONE]) and punishment motivation ([PUN + BOTH] vs [REW + NONE]) and reward x punishment motivation interaction ([PUN + REW] vs [BOTH + NONE]). Two-tailed, paired-samples t-tests determined the directionality and significance of these effects. A significant reward x punishment motivation in a particular region would provide evidence of opponent systems.

To test for evidence of directionally opposing effects for reward and punishment motivation, differential activation to reward and punishment motivation was assessed (REW vs. PUN).

Conjunction analyses were conducted to characterize regions showing overlapping activation in response to the main effects of reward motivation and punishment motivation. (Given limitations in interpreting BOLD deactivations, we do not report analyses evaluating the overlap between BOLD activations and deactivations for reward and punishment motivation, respectively.) The resulting region was further probed to evaluate support for a common neural mechanism interpretation by 1) a conjunction with reward x punishment interaction regions and 2) a PPI analysis as described below.

Functional activation maps for each participant were warped into the common stereotaxic space of the 2009 Montreal Neurological Institute [MNI] 152 symmetrical 1 mm x 1 mm x 1 mm template (Fonov, Evans, McKinstry, Almli, & Collins, 2009) resampled to 2 mm x 2 mm x 2 mm using FMRIB’s Non-Linear Image Registration Tool (FNIRT) (Andersson, Jenkinson, & Smith, 2007). Single-subject-level analyses were performed to average each contrast across the three task blocks for each participant using a fixed-effects model. Single-subject-level β maps were concatenated for group-level analyses.

Regions associated with motivational/affective processes (i.e., Bartra et al., 2013; Garrison et al., 2013; Laird et al., 2013; Liu et al., 2011) were selected a priori to create a region of interest (ROI) mask to help control family-wise error rate. Anatomically derived masks of bilateral cingulate, insula, nucleus accumbens, amygdala, caudate, pallidum, putamen, and orbitofrontal cortex (OFC) were based on the Harvard-Oxford probabilistic atlas available in FSL. A dopaminergic midbrain mask was derived from a probabilistic atlas (Murty et al., 2014). A right dlPFC mask was derived from a connectivity-based parcellation atlas (Sallet et al., 2013) and reflected over the y-axis to create a bilateral mask. All masks were transformed into the symmetrical standard brain template space and combined into a single ROI mask.

Nonparametric two-tailed, one-sample t-tests were conducted for regions defined by the a priori ROI mask on each contrast β map implemented in FSL’s Randomise using the threshold-free cluster enhancement (TFCE) option (Smith & Nichols, 2009) with 5,000 permutations (Winkler, Ridgway, Webster, Smith, & Nichols, 2014). The significance threshold was set at P_FWE < 0.05, corrected for multiple comparisons. Exploratory whole-brain analyses were also conducted for all analyses using a whole-brain gray-matter mask. Results from these analyses are presented in Supplementary Materials. For all analyses, only clusters greater than 10 voxels are reported for brevity. Results from smaller clusters did not alter study interpretations.

To evaluate whether reward and punishment motivation have opposing effects on shared outputs, a generalized PPI was conducted (McLaren, Ries, Xu, & Johnson, 2012) with the functionally-defined dACC region resulting from the conjunction analysis of main effects of reward and punishment motivation used as a seed. The average time course of this region was extracted and included as a regressor in first-level analyses. Four regressors modeling the interaction between the seed time course and each of the four cue types were included in addition to regressors for cue, target, and feedback types and performance errors, as described above. To examine task-specific functional connectivity differences between reward and punishment motivation, a contrast was created to compare the PPI interaction regressors for REW and PUN cue types. Single-subject-level and group-level analyses were carried out in the same manner as described above, with the ROI mask of motivation-related regions applied to group-level statistical analyses. An exploratory whole-brain analysis was also conducted using a whole-brain gray-matter mask and is included in Supplementary Materials.

3. RESULTS

3.1. Behavioral Data

Main effects of reward motivation, F(1,85) = 53.22, p < 0.001, and punishment motivation, F(1,85) = 7.75, p = 0.007, indicated that reward and punishment manipulations were associated with decreased RTs in response to the target relative to nonreward-motivated and nonpunishment-motivated conditions, respectively (Figure 2). The reward x punishment motivation interaction, F(1,85) = 35.14, p < 0.001, indicated that reward and punishment motivation had nonadditive effects on RTs. Planned comparisons indicated that RTs for REW, t(85) = −1.81, p < 0.073, and PUN conditions, t(85) = 0.74, p < 0.46, did not differ from the BOTH condition. The REW condition was associated with faster RTs than the PUN condition, t(85) = −2.30, p < 0.024, suggesting that the possibility of monetary gains was more motivating than the possibility of monetary losses. All motivated conditions (REW, PUN, and BOTH) were significantly faster than the unmotivated condition (NONE), all p’s < 0.001.

Figure 2.

Reaction time (RT) data. A) Reward motivation and punishment motivation both speeded RTs. Their influences on RTs were interactive, p < 0.001. The reward-only condition speeded RTs more than the punishment only condition, p = 0.024. BOTH, both reward and punishment possible; PUN, punishment only possible; REW, reward only possible; NONE, neither reward nor punishment possible.

3.2. fMRI Data

3.2.1. Reward and Punishment Motivation ANOVA: Evidence of Mutual Inhibition

Whole-brain analyses are reported in Supplemental Tables 1–4. Supplemental Table 5 provides simple-effects tests of REW and PUN conditions. Table 1 provides results from the reward x punishment motivation ANOVA. Reward motivation prompted greater activation in a right putamen cluster extending into NAc with local maxima in pallidum and caudate; a left NAc cluster extending into caudate and putamen; and bilateral dACC/MCC. Bilateral dlPFC, PCC, and left lateral OFC clusters showed less activation during reward motivation than nonreward-motivated conditions. Punishment motivation prompted greater activation in a left insula cluster extending into putamen and in bilateral dACC. No regions showed significantly less activation during punishment motivation.

Table 1.

Neural Activation during Cue Period for Reward, Punishment, and Reward x Punishment Motivation

Region	Cluster	Parameter	Max	Location
	Size	Estimates	t-stat	X	Y	Z
Reward [REW + BOTH] >No Reward [PUN + NONE]
R putamen, NAc, pallidum, caudate	240	5.90	5.72	13	9	−7
Bilateral dACC	106	5.46	5.41	−7	7	43
L NAc, caudate, putamen	58	5.99	4.8	−11	11	−7
Reward [REW + BOTH] < No Reward [PUN + NONE]
L dlPFC	999	−4.03	−5.05	−19	59	33
Bilateral PCC, precuneus	844	−6.80	−5.93	−13	−49	37
R dlPFC	549	−4.19	−5.17	21	29	55
L lateral OFC	95	−6.11	−4.60	−37	25	−19
L lateral OFC	26	−7.91	−4.19	−51	37	−13
R dlPFC	11	−3.51	−4.93	13	63	17
Punishment [PUN + BOTH] > No Punishment [REW + NONE]
L insula, putamen	125	4.30	5.13	−35	3	11
Bilateral dACC	86	5.74	5.38	−7	15	37
Punishment [PUN + BOTH] < No Punishment [REW + NONE]
No significant clusters
Reward x Punishment ([BOTH + NONE] > [PUN + REW])
Bilateral dACC	711	−5.05	−5.71	−7	5	43

Coordinates are for the maximum t-stat in MNI152 2009a symmetrical space; L, left; R, right; dACC, dorsal anterior cingulate cortex; dlPFC, dorsolateral prefrontal cortex; NAc, nucleus accumbens; OFC, orbitofrontal cortex; PCC, posterior cingulate cortex.

A cluster in bilateral dACC showed a reward x punishment motivation interaction, supporting mutual inhibition of reward and punishment motivation within this region. Follow-up analyses indicated greater activation for motivated conditions (REW, PUN, and BOTH) than for the unmotivated (NONE) condition, all p’s < 0.001. No other paired comparisons were significant.

3.2.2. REW vs. PUN contrast: Evidence of Directionally Opposing Effects

A direct contrast of REW vs. PUN revealed greater activation in the PUN condition in bilateral PCC and right lateral OFC (Table 2). No regions showed significantly greater activation in the REW condition.

Table 2.

Neural Activation during Cue Period for REW vs. PUN Contrasts

Region	Cluster	Parameter	Max	Location
	Size	Estimates	t-stat	X	Y	Z
REW > PUN
No significant clusters
REW < PUN
Bilateral PCC, precuneus	122	−8.46	−4.06	−1	−55	21
R lateral OFC	19	−7.89	−4.50	51	29	−9

Coordinates are for the maximum t-stat in MNI152 2009a symmetrical space; R, Right; OFC, orbitofrontal cortex; PCC, posterior cingulate cortex.

Given faster RTs in the REW than the PUN condition, control analyses were conducted to determine whether differences in the REW versus PUN conditions were associated with RT differences. Differences in brain activation in the PCC and lateral OFC for the REW vs. PUN condition did not correlate with differences in RT between these two conditions, suggesting that these differential activations were not driven by a difference in motivational intensity between reward and punishment.

3.2.3. Evaluating Evidence for a Common Neural Mechanism

A conjunction analysis revealed a significant cluster in bilateral dACC, with increased activation in response to both reward and punishment motivation (Table 3; Figure 3). Although this result would often be interpreted as evidence of a common neural mechanism, an additional conjunction analysis indicated that this cluster largely overlapped with the dACC cluster from the reward x punishment motivation interaction (Table 3; Figure 3). This overlap represents a region demonstrating significant activation to both reward and punishment anticipation, but wherein effects of these motivational cue contexts were not additive and therefore do not support a shared mechanism interpretation.

Table 3.

Conjunction Analyses

Region	Cluster	Max	Location
	Size	t-stat	X	Y	Z
(Reward > No Reward) AND (Punishment > No Punishment)
Bilateral dACC	48	4.64	−7	13	39
[(Reward > No Reward ) AND (Punishment > No Punishment )] AND (Reward x Punishment))
L dACC	37	3.74	−9	13	39

Coordinates are for the maximum t-stat in MNI152 2009a symmetrical space; L, Left; dACC, dorsal anterior cingulate cortex.

Figure 3.

A) dACC region of overlap (yellow) between Reward Motivation (green) and Punishment Motivation (magenta) and Reward x Punishment Motivation interaction (blue) B) Group means for Reward x Punishment Motivation interaction in dACC. These means are presented for descriptive purposes only.

3.2.4. PPI Analysis: Evidence for Directionally Opposing Effects on Shared Outputs

Using the dACC cluster resulting from the conjunction analysis of main effects of reward and punishment motivation as a seed, PPI analysis showed greater functional connectivity with right lateral OFC during the REW than during the PUN condition (Table 4; Figure 4), suggesting that this region of common activation has opposing effects on other brain regions. Post hoc analyses indicated that, in both OFC clusters, connectivity with dACC was negative in the PUN condition, anterior OFC cluster t(85) = −4.41, p < 0.001, and posterior OFC cluster t(85) = −2.41, p = 0.018, whereas connectivity was positive in the REW condition, though significantly so only in the more posterior cluster, t(85) = 2.76, p = 0.007.

Table 4.

PPI Results for REW vs. PUN Contrasts for dACC Seed

Region	Cluster	Max	Location
	Size	t-stat	X	Y	Z
REW > PUN
R OFC	50	4.41	43	19	−17
R OFC	14	3.94	41	29	−17
REW < PUN
No significant clusters

Coordinates are for the maximum t-stat in MNI152 2009a symmetrical space; R, Right; OFC, orbitofrontal cortex.

Figure 4.

A) Significant right OFC clusters in PPI analysis of connectivity differences during the REW and PUN condition. B) Group connectivity means for PUN and REW conditions for both OFC clusters. In the more anterior right OFC cluster, negative connectivity was greater in the PUN than REW condition. In the more posterior OFC cluster, connectivity switched from positive in the REW condition to negative in the PUN condition. These means are presented for descriptive purposes only.

To better understand the functional role of the dACC during reward and punishment motivation, an exploratory whole-brain gray-matter PPI analysis revealed a number of clusters extending outside of the ROI mask also showing greater connectivity during the REW condition (Supplemental Table 4), including bilateral inferior frontal gyrus (IFG) and bilateral dlPFC, wherein connectivity with dACC was positive during reward motivation and negative during punishment motivation. Of note, all significant clusters demonstrated positive connectivity during reward motivation and negative connectivity during punishment motivation except one lateral occipital cluster (wherein average connectivity during punishment motivation was also positive). No regions showed greater connectivity during the PUN condition in the whole-brain gray-matter PPI analysis.

Given faster RTs in the REW than the PUN condition, control analyses were conducted to determine whether connectivity differences in the REW versus PUN conditions were associated with the RT difference. No connectivity differences were positively correlated with differences in RT between REW and PUN conditions, suggesting that differential connectivity during reward versus punishment motivation was not driven by a difference in motivation between reward and punishment.

4. DISCUSSION

The goal of this study was to determine the extent to which neural activation supported opponent or shared neural mechanisms of reward and punishment motivation. Multiple lines of evidence from present neural analyses support the idea that reward and punishment motivation are associated with opponent neural mechanisms, even when they motivate the same behaviors. First, a reward x punishment motivation interaction was observed in dACC, providing evidence of mutual inhibition within this region. This interaction is consistent with evidence of competition between appetitive and aversive processing in ACC when these processes directly conflict (Choi et al., 2014; Talmi et al., 2009) and further indicates that opponency in dACC is not a byproduct of differential preparation and instead reflects competing mechanisms by which reward and punishment motivate behavior. Second, reward motivation and punishment motivation were associated with regions of differential activation, supporting directionally opposing effects. Third, the only region of shared activation between reward and punishment motivation, a dACC region, showed significant overlap with the dACC interaction cluster, suggesting that this region did not provide evidence of a shared neural mechanism, as reward and punishment motivation did not demonstrate additive effects within this region. Instead, this dACC region of shared activation showed differential connectivity during reward versus punishment motivation, providing evidence of opposing effects of shared outputs. These findings refute the idea that this overlapping activation is consistent with a shared mechanism account of reward and punishment motivation.

By testing for an interaction between reward and punishment motivation, the implementation of a factorial design allowed a fuller appreciation of the functionality of implicated regions. Present findings suggest that dACC represents reward and punishment motivation separately and in such a way that these processes interact, possibly through local inhibitory and excitatory connections (Kolling et al., 2016). This interpretation is consistent with evidence that neuronal populations that separately encode motivationally positive and negative variables are distributed throughout ACC and are largely intermixed (Amemori & Graybiel, 2012). It is also consistent with meta-analytic data suggesting that subregions of ACC differentially encode positive and negatively valenced motivational stimuli (Liu, et al., 2011).

Consistent with evidence of a neural interaction between reward and punishment motivation, a behavioral interaction between reward and punishment motivation, such that reward and punishment contributed nonadditively to speeded performance, was also observed. This interaction indicates that the combination of reward and punishment motivation did not improve performance beyond that of either reward or punishment motivation separately. Even though reward and punishment motivated the same action, they did not act synergistically to improve performance, further supporting an opponent systems interpretation. An alternative explanation for this interaction is that it was driven by a RT floor effect, such that RTs were optimally speeded in the REW and PUN conditions and therefore could not be significantly improved in the BOTH condition. Nevertheless, evaluation of RT distributions for each subject by condition confirmed this was not the case.

A major implication of the present findings is that shared activation in a region is not sufficient grounds to conclude that that region exclusively represents a shared mechanism for reward and punishment motivation. Again, this assertion depends on an assumption that BOTH cues are more salient than REW and PUN cues alone as supported by the Pearce-Hall model (Pearce & Hall, 1980) and the idea that salience/motivation is not binary. Although it could be the case that all three motivating cues (BOTH, REW, PUN) result in equal motivation and thus that the present findings support a pure salience interpretation of dACC activation, the dACC region showing shared or salience-related activation also demonstrated differential connectivity with other regions during reward and punishment motivation. This pattern further supports the conclusion that reward and punishment motivation were encoded in an opponent-like manner and suggests that valenced information still contributes to dACC function and communication within neural networks. Collectively, present results fit an opponency interpretation more parsimoniously than a shared mechanism account. Given the large number of regions showing differential coupling with dACC during reward versus punishment motivation in whole-brain analyses in addition to theories implicating dACC in integrating stimulus value and action (Hayden & Platt, 2010; Rushworth & Behrens, 2008; Rushworth, Behrens, Rudebeck, & Walton, 2007), this differential connectivity suggests that reward and punishment are associated with different neural mechanisms of motivated behavior, even when motivating the same behavioral response.

Present results add to the existing literature by demonstrating that dACC acts jointly but differentially with a number of regions, including OFC, IFG, and dlPFC, in service of translating reward versus punishment motivation into action. The role of OFC in motivating behavior according to valence is consistent with evidence implicating this region in representing value (Kahnt et al., 2014; Rothkirch, Schmack, Schlagenhauf, & Sterzer, 2012; Rushworth & Behrens, 2008; Rushworth et al., 2007) and is consistent with present evidence of differential activation in lateral OFC according to motivational valence. In addition to anatomical connections between dACC and OFC (Garcia-Cabezas & Barbas, 2017), the role of OFC in stimulus-value relationships and that of dACC in action-value relationships (Camille, Tsuchida, & Fellows, 2011) supports the idea that these regions act together to implement motivated behaviors.

Greater positive coupling between dACC and cognitive control regions, including IFG and dlPFC, during reward motivation could reflect an increase in inhibition to facilitate response preparation (Aron, Robbins, & Poldrack, 2004, 2014; Bundt, Abrahamse, Braem, Brass, & Notebaert, 2016; Duque, Lew, Mazzocchio, Olivier, & Ivry, 2010; Greenhouse, Sias, Labruna, & Ivry, 2015) or an increase in cognitive control to avoid distraction (Banich et al., 2009; Iordan, Dolcos, & Dolcos, 2013) and maintain goal pursuit (Spielberg et al., 2012), particularly given that targets were task-irrelevant words. Such a mechanism would be consistent with the idea that dACC monitors the expected value of exerting control and communicates with lateral PFC to implement such control (Shenhav, Botvinick, & Cohen, 2013; Shenhav, Cohen, & Botvinick, 2016). It would also be consistent with evidence that connectivity between these regions reflects the regulation of cognitive control in lateral PFC by dACC based on motivational incentive values (Kouneiher, Charron, & Koechlin, 2009). Further research is necessary to understand how communication between dACC and these brain regions differentially motivate behavior during reward and punishment anticipation. For example, Silton et al. (2010, 2011) found evidence of a dlPFC-dACC-dlPFC control sequence in a Stroop task, so the causal precedence between dACC and dlPFC may be complex and/or task-dependent.

In the present study, PCC and lateral OFC clusters showed differential activation to reward and punishment motivation, consistent with studies implicating these regions in valence-related activation (Litt et al., 2011). Greater lateral OFC activation to punishment than to reward motivation is consistent with a medial-lateral gradient for reward versus punishment processing in OFC (Kringelbach & Rolls, 2004; O’Doherty, Kringelbach, Rolls, Hornak, & Andrews, 2001). It has been reported that potential punishments are more motivating or salient than potential gains (i.e., Kahneman & Tversky, 1979; Sokol-Hessner et al., 2009). Thus, an alternative explanation of the greater activations we observed for PUN vs. REW could be that they reflect such a difference in motivation or salience. Nevertheless, the fact that we did not observe faster RTs on PUN trials suggests that punishment was not more motivating in the present study, possibly because participants were not in danger of losing money considered already owned (Seymour, Maruyama, & De Martino, 2015).

Two limitations of the present study can be noted. First, participants were recruited based on trait scores on the PANAS, which may have biased results. Nevertheless, histograms suggested normal distributions of variables of interest, and when analyses were conducted with PANAS as a covariate, no meaningful differences in results were observed. Thus, we do not believe that present analyses were biased by the selected recruitment of subjects. Second, the targets in the modified MID task employed in this study were emotional words, adding an orthogonal variable to the task design. Nevertheless, since emotional words were presented after the motivational cue period of interest in this study, it is unlikely that they meaningfully influenced present results. Nevertheless, the emotionality of the words was modeled in all analyses to remove the variance associated with this variable of no interest.

Across multiple lines of evidence, the present study demonstrated that reward and punishment motivation are best conceptualized as opponent forces influencing brain and behavior. The use of a factorial design and functional connectivity analyses demonstrated that common activation, observed here in dACC, does not suggest a unitary response to valenced stimuli nor the operation of a common neural mechanism instantiating reward and punishment motivation even when these motivational processes motivated the same behavior. These findings highlight how fMRI can be leveraged for advancing theories of regional function and support using advanced imaging analysis techniques to further understanding of how reward and punishment motivation are encoded in the brain. This study contributes to a growing appreciation for how reward and punishment motivation are implemented in the brain to support goal-directed behaviors.