Distinct medial temporal networks encode surprise during motivation by reward versus punishment

Abstract

Adaptive motivated behavior requires predictive internal representations of the environment, and surprising events are indications for encoding new representations of the environment. The medial temporal lobe memory system, including the hippocampus and surrounding cortex, encodes surprising events and is influenced by motivational state. Because behavior reflects the goals of an individual, we investigated whether motivational valence (i.e., pursuing rewards versus avoiding punishments) also impacts neural and mnemonic encoding of surprising events. During functional magnetic resonance imaging (fMRI), participants encountered perceptually unexpected events either during the pursuit of rewards or avoidance of punishments. Despite similar levels of motivation across groups, reward and punishment facilitated the processing of surprising events in different medial temporal lobe regions. Whereas during reward motivation, perceptual surprises enhanced activation in the hippocampus, during punishment motivation surprises instead enhanced activation in parahippocampal cortex. Further, we found that reward motivation facilitated hippocampal coupling with ventromedial PFC, whereas punishment motivation facilitated parahippocampal cortical coupling with orbitofrontal cortex. Behaviorally, post-scan testing revealed that reward, but not punishment, motivation resulted in greater memory selectivity for surprising events encountered during goal pursuit. Together these findings demonstrate that neuromodulatory systems engaged by anticipation of reward and punishment target separate components of the medial temporal lobe, modulating medial temporal lobe sensitivity and connectivity. Thus, reward and punishment motivation yield distinct neural contexts for learning, with distinct consequences for how surprises are incorporated into predictive mnemonic models of the environment.

Introduction

Individuals continually make predictions about their surrounding environment. Events that deviate from expectations are salient, attracting attention and driving memory-encoding mechanisms. Many brain regions detect and respond to surprising events, including the ventral attentional orienting system, areas coding for error prediction signals, and the hippocampus and surrounding medial temporal lobe (MTL) cortex ^1,1-4. Although the MTL is centrally implicated in novelty and surprise processing, it is clear that not all unexpected events are encoded into long-term memory. For example, when a commuter encounters an unexpected detour on her morning drive, she may or may not encode this detour and plan an alternative route the following day. Understanding the selectivity of memory for some surprising events, but not others, is critical for characterizing how salient events are encoded in service of future behavior.

Because the motivational context of an individual modulates learning and MTL neurophysiology ^5,6, motivation could be a key determinant in whether and how the brain encodes surprises. However, motivation is not a unitary construct: researchers have historically conceptualized motivation as constituting multiple components, including approaching rewards, avoiding losses and escaping punishments. These distinct states have now been investigated extensively enough to predict that, just as they elicit disparate behaviors ^7-9, distinct motivational states have disparate effects on cognition in general and memory in particular. The current study investigated neural sensitivity to and memory for surprising events encountered in two kinds of motivational contexts - pursuit of reward and avoidance of punishment, with a focus on MTL memory systems.

Surprise is an important domain in which to examine the effects of motivational context, because surprises indicate the need for an organism to update predictive mnemonic representations of the environment and potentially to alter its behavior. Neurons in the hippocampus proper encode both the goal state of an organism ¹⁰ and plans for future actions to achieve those goals ¹¹. Neuromodulatory systems associated with motivation, such as the mesolimbic dopamine system, modulate both the firing profile and plasticity of MTL neurons ^5,12-16. In humans, reward motivation enhances MTL-dependent memory ^17-22, and functional imaging studies have reported increased engagement ^17,18,23 and connectivity ^17,22,24,25 of the hippocampus and surrounding MTL cortex in response to motivationally relevant cues. These convergent findings that motivation modulates the responsivity of the MTL imply consequences for how motivation contributes to encoding, broadly speaking and including the encoding of surprise. In line with these findings, reward motivation amplifies hippocampal responses to and memory for both novel ^26,27 and surprising ²⁴ events. Such convergent results open parallel questions about how other motivational states like active avoidance impact MTL encoding of surprise.

The extant literature supports competing hypotheses about the specificity of motivational states on encoding surprising events. One line of research suggests that reward and punishment motivation should similarly affect MTL processing of unexpected events. For instance, mesolimbic dopamine neurons can support both reward- and punishment-motivated behaviors ²⁸. In humans, monetary rewards and monetary punishments both engage mesolimbic and MTL structures ^25,29 and both facilitate later memory ²⁵. On the other hand, reward and punishment motivation have also been associated with distinct behavioral states mediated by separable neural systems. For instance, anticipation of reward is associated with approach, exploration, novelty seeking, and attentional broadening ^30-32, whereas anticipation of punishment is associated with avoidance, freezing responses, and attentional narrowing ^8,33,34. In the domain of memory, reward motivation enhances and punishment motivation impairs spatial learning during spatial navigation ²⁰. These opposing behavioral patterns suggest that reward and punishment may bias neural encoding of surprising events in different ways.

Although relatively little research has investigated the impact of different motivational states on neural mechanisms of memory encoding, emerging evidence does suggest that reward and punishment engage distinct learning systems. Our own prior work has shown greater encoding-related activation in amygdala and cortical MTL under punishment motivation ³⁵ in contrast to greater ventral tegmental area (VTA) and hippocampal involvement under reward motivation ¹⁷. Similarly, rewards facilitate both cortical MTL and hippocampus-dependent encoding ^17,23,24, whereas punishments, particularly those involving shock anticipation or delivery, facilitate cortical MTL- dependent encoding ^35-37. We note, however, that these motivational states have not been directly contrasted using the same experimental paradigm to control for stimulus effects and task demands. Nonetheless, the dissimilar engagement of MTL subregions across different motivational contexts implicates differential memory outcomes, given the functional specialization of MTL subregion contributions to long-term memory ^38,39.

The hippocampus proper has a functional structure that permits the continuous comparisons of actual and expected environmental input ⁴⁰, which makes it well-suited to detect surprises. Further, the hippocampus is thought to encode broad, flexible representations of items and the contexts in which they occurred ^38,39. In contrast, cortical MTL regions are thought to encode simple, independent representations of unitized objects or scenes ^38,39. Given these functional differences between the hippocampus and cortical MTL, memory encoding for surprising events would be predicted to be facilitated especially by hippocampal encoding as a means to update contextual representations. Thus, motivational states that facilitate hippocampus- dependent encoding may uniquely support detailed, flexible memory for surprising events.

The goal of the current study was to adjudicate between these alternative hypotheses about how motivational orientation influences the neural encoding of surprising events. Prior research has mainly focused on the encoding of information that is explicitly incentivized; however, because we were especially interested in the role of MTL memory systems, we examined the encoding of events that were not explicitly incentivized but rather represented potential contextual predictors of outcomes. During the collection of fMRI data, participants performed a motivated speeded reaction time task that included occasional goal-irrelevant perceptual deviants amongst repeated object stimuli. A between-groups design was utilized to investigate differences in behavior incentivized by gaining monetary bonuses (Reward Group) or by avoiding electrical stimulation (Punishment Group). Analyses aimed to determine whether surprise was encoded by similar or different MTL substrates under reward versus punishment motivation, and further whether these motivational contexts differentially influenced MTL network connectivity. Using the identical tasks and stimuli while manipulating only the incentives allowed us to directly compare the incidental encoding of goal-irrelevant, surprising events embedded in rewarding versus punishing contexts, and thereby isolate the influence of the specific motivational state.

2.0 Methods

2.1 Participants

Fifty-three healthy, right-handed adult volunteers participated in the study. All participants gave written informed consent for a protocol approved by the Duke University Institutional Review Board. Data from four participants were excluded because of excessive head motion (> 1.5 mm, 1 participant), software malfunction during scanning (1 participant), and poor task comprehension (2 participants, i.e., participants made inappropriate responses to surprising stimuli during the task), resulting in 49 analyzed participants (median age = 25, age range = 18-36): 26 participants in the reward group (18 female) and 23 participants in the avoidance group (10 female). There were no significant differences in age, t(47) = 0.19, p = 0.85, or gender, Mann-Whitney U = 376, p = 0.13, across reward and punishment groups. Some data from the reward group was previously reported to investigate at the isolated effects of reward motivation on surprise processing ²⁴.

2.2 Task

Participants performed a speeded reaction time task to either earn monetary bonuses (reward group) or avoid electrical punishment (punishment group, Figure 1) in a modified Monetary Incentive Delay task ⁴¹. We incentivized punishment motivation with electrical shocks because prior research shows that that shocks elicit activation in neural systems associated with punishment more reliably than monetary losses ⁴². Critically, electrical punishment elicits equivalent motivational engagement as monetary rewards as assayed by self-report ²⁰ and behavioral facilitation (see Results), despite the qualitatively different nature of the reinforcers.

Figure 1.

Experimental Task: In each trial, participants first viewed either a high or low motivation incentive cue that indicated the opportunity to earn a monetary bonus (reward group) or avoid an electrical stimulation (punishment group). In control trials, the cue was followed by serial repetitions of a trial-unique, color object image. After 10 or 11 repetitions that image became gray-scale, to which participants were to make a speeded button press. In surprise trials, serial presentation of the trial-unique color object image was interrupted by a novel, yet highly similar, object image at a temporally unpredictable time. Following the button press to the target, participants were presented an outcome screen that indicated their performance (i.e., whether they earned a monetary bonus or avoided a shock).

We designed our task to manipulate two factors: participants' motivational state and the presence of surprise. To manipulate motivational state, every trial of the task began with a 500-ms cue that indicated whether a speeded button press to a target image had a high or low incentive. In the reward group, a high incentive was a $2.00 monetary bonus for a fast button press, and a low incentive was a $0.10 monetary bonus for a fast button press. In the punishment group, a high incentive was avoiding delivery of an aversive electrical stimulation for a fast button press, and the low incentive was complying with experimenter instruction to to respond as quickly as possible, but without threat of shock. Methodological limitations did not allow us to administer two different levels of shock intermittently in the scanner. Following a variable delay (5.5 - 6.0 sec), the target appeared on the screen. Targets were trial-unique, gray-scale object images. If participants were sufficiently fast at responding to targets, participants received the outcomes indicated by the cues. The target reaction time for receiving a successful outcome was determined by an adaptive algorithm, which estimated the response time threshold at which subjects would be successful on ∼65% of trials. Reaction time thresholds were calculated independently for each condition to ensure that reinforcement rates were equated across all four conditions. Given that there were only 20 trials within each condition, the algorithm approximated but did not always preceisly reach the target of 65% correct in each condition for each participant (see Table 1). Following the presentation of the target image, participants viewed an outcome screen that indicated their success on the current trial. In the reward group, to avoid the working memory demand of calculating total earnings, this screen also indicated participants' accumulated monetary bonuses.

Table 1. Behavioral performance across reward and punishment groups (mean ± SEM).

Group	Condition	Reaction Time	Accuracy
Reward
	High Incentive: Surprise	205.01 ± 4.63	73.3 ± 1.4
	High Incentive: Control	219.00 ± 5.41	71.7 ± 1.5
	Low Incentive: Surprise	214.62 ± 4.68	70.2 ± 1.8
	Low Incentive: Control	223.78 ± 3.79	70.2 ± 1.6
Punishment
	High Incentive: Surprise	205.2 ± 4.8	68.0 ± 3.1
	High Incentive: Control	211.8 ± 4.0	66.0 ± 3.3
	Low Incentive
	Low Incentive: Surprise	217.0 ± 4.0	66.6 ± 2.8
	Low Incentive: Control	225.8 ± 6.1	65.8 ± 3.0

To manipulate surprise, following the cue but prior to the target presentation, participants viewed 10 or 11 serial presentations of trial-unique, color object images for 409 msec with an inter-stimulus interval of 136 msec. During control trials, participants viewed repeated presentations of a color version of the upcoming target stimulus. During surprise trials, participants viewed repeated presentations of a color version of the upcoming target stimulus interrupted by a highly similar, but novel image, this surprising stimulus always appeared randomly between the fourth and eight object presentations.

2.3 Procedure

Prior to scanning, participants in the punishment group calibrated electrical shocks to a level that was “highly irritating but not painful” using an ascending staircase procedure with 5 V increments ⁴³. Shocks were administered using the MP-150 BIOPAC system (BIOPAC Systems, Goleta, CA). Immediately prior to scanning, participants in both groups were shown a visual schematic of the task and given verbal instructions. Further, they were instructed that on some trials a different object would interrupt the stream of objects but these interruptions were irrelevant to achieving their goals of either earning money or avoiding shocks.

After entering the scanner, participants performed a non-incentivized, practice version of the task that consisted of 10 high-incentive control trials and 10 low-incentive control trials to familiarize them with this paradigm and calibrate reaction time thresholding. Following the practice session, participants completed two runs of the incentivized version of the task. During each run of the task, participants competed 10 high-incentive control, 10 low-incentive control, 10 high-incentive surprise, and 10 low incentive-surprise trials. Trial order was pseudo-randomized across each run, with each run lasting 7 min 56 sec. Trial onsets, cue-scene intervals and trial order were optimized using Opt-seq software (http://surfer.nmr.mgh.harvard.edu/optseq).

Following scanning (approximately 30 min after the encoding session), participants performed a two-alternative forced-choice recognition memory task for objects that constituted surprise. During this test, participants saw pairs of object images, one of which was an object that constituted the surprise and the other a highly similar, novel object (Figure 2, see Bakker et al., 2008; Kirwan and Stark, 2007). For each object pair, participants had to identify which object they saw during the encoding session by pressing either the “1” or “2” button to indicate the object on the left or right, respectively. Following each memory decision, participants had to indicate their confidence in their response (i.e. 1 = Very Sure, 2 = Pretty Sure, 3 = Just Guessing). Confidence did not significantly influence the pattern of results, thus we present memory recognition data collapsed across confidence. Participants received 40 recognition memory trials (20 high incentive and 20 low incentive) in an intermixed order.

Figure 2.

Reward, but not punishment, motivation selectively enhanced memory for objects that constituted a surprise (condition*group interaction: (F(46) = 4.29, p = 0.04); reward group: t(25) = 4.89, p < 0.001; punishment group: t(21) = 0.68, p = 0.51).

2.4 Behavioral Analysis

Reaction time and hit rates to target images were submitted to separate general linear models (GLMs) with incentive level (high, low trials) and presence of surprise (surprise, control) as within-subjects factors and motivational group (reward, punishment) as a between-subjects factor. For both the reaction time and hit rate GLMs, we tested for main effects of group, incentive level, and presence of surprise a nd all interactions at a significance level of p < 0.05. Recognition memory for objects that constituted surprise was tested by submitting the proportion correct from the two- alternative forced choice test to a GLM with incentive level as a within-subjects factor and motivational group as a between-subjects factor. Post-hoc analyses were conducted to test differences in memory as a function of incentive level within each group, and differences in memory for each incentive level across groups.

2.5 MRI Data Acquisition and Preprocessing

FMRI data was acquired on a 3.0 T GE Signa MRI scanner using a standard echo-planar sequence (TE = 27 msec, flip = 77 degrees, TR = 2 sec, 34 contiguous slices, size = 3.75*3.75*3.8 mm). Each of the two functional runs consisted of 238 volumes. Prior to the functional runs, we collected a whole-brain, inversion recovery, spoiled gradient (IR-SPGR) high-resolution anatomical image (TE = 2.93 msec, flip = 12 degrees, TR = 7.58 msec, 206 axial slices, size = 1*1*1 mm) for use in spatial normalization.

FMRI preprocessing was performed using fMRI Expert Analysis Tool (FEAT) Version 5.92 as implemented in FSL 4.1.5.9. The first six scans were discarded to achieve magnetic equilibration. Blood-oxygen-level dependent (BOLD) images were skull stripped using the Brain Extraction Tool (BET). Images were realigned with run, intensity normalized by a single multiplicative factor, spatially smoothed with a 4.0 mm full-with half-medium (FWHM) kernel, and subjected to a high-pass filter (100 sec). Spatial normalization was performed using a two-step procedure on fMRIb Linear Registration Tool (FLIRT). First, mean echo-planar images (EPIs) from each run were co-registered to the high-resolution anatomical image. Then, the high-resolution anatomical images was normalized to the high-resolution standard space image in Montreal Neurological Image (MNI) space using a non-linear transformation with a 10 mm warp resolution, as implemented by fMRI Non-Linear Registration Tool (FNIRT). All coordinates are reported in MNI space.

2.6 fMRI Data Analysis

FMRI data was analyzed using FEAT version 5.92 as implemented in FSL 4.1.6⁴⁴. Time-series statistical analyses used FMRIB's Improved Linear Model with local autocorrelation correction. To investigate task-related activations, first level (i.e. within- run) GLMs included 8 regressors that modeled high incentive cues, low incentive cues, high incentive target images, low incentive target images, high-incentive surprise events, high-incentive control events, low-incentive surprise events, and low-incentive control events. The latency of control events was determined by randomly sampling from the latency of surprise events without replacement to equate the structure/timing of regressors across conditions. All trial events were modeled with an event duration of 0 sec and standard amplitude of 1. These events were then convolved with a doublegamma hemodynamic response function. Surprise and control events were orthogonalized with respect to cue and target events. Using this GLM, individual maps of parameter estimates were generated for four contrasts of interest: high incentive > low incentive cue, [high-incentive surprise + low-incentive surprise] > [high-incentive control + low-incentive control], high-incentive surprise > low-incentive surprise, and high- incentive control + low incentive control. Critically, identical GLMs were constructed for participants in the reward and punishment groups allowing for comparison across groups.

Second-level analyses (i.e. across runs, but within-subject) were modeled using a fixed effects analysis. Group-level analyses were modeled using mixed-effects analyses (FLAME 1) on the parameter estimates for contrasts of interest derived from the second level analysis. Contrasts of interest were first run within the reward and punishment groups using a one-sample t-test, and then across groups using a two- sample t-test. Statistical tests for these fMRI analyses were set to an overall p = 0.05 cluster corrected, as calculated within AlphaSim tool in AFNI with 1000 Monte Carlo simulations. The AlphaSim procedure with a voxel-wise significance of p < 0.001 yielded a cluster extent minimum of 33 voxels for the whole-brain and 15 voxels for the MTL region of interest (ROI) analyses. The MTL ROI included bilateral hippocampus and parahippocampal cortex, which includes perirhinal cortex, entorhinal cortex, and posterior parahippocampal cortex, as defined by WFU PICKATLAS (http://fmri.wfubmc.edu/-software/PickAtlas). To identify regions that were commonly activated across both groups, we conducted conjunction analyses, in which there was evidence for significant activations in both samples independently, i.e. testing the ‘Conjunction Null Hypothesis’ ⁴⁵. These maps were then thresholded for multiple comparisons as described above.

To investigate the broader networks associated with our medial temporal lobe regions of interest, we measured functional connectivity using a ‘background connectivity’ approach ⁴⁶. We first removed components of the raw timeseries that reflect responses to the cues, surprise stimuli (and corresponding control stimuli), and target stimuli using a voxelwise GLM approach (described above). Note that while this method removes the contribution of these three components of the task, the residual timeseries will still include unmodeled components of the task (such as the intervening presentations of pictures between cues, surprise stimuli, and target stimuli), as well as any components of the task-evoked response that were not captured by our canonical hemodynamic models (such as deviations in shape/shift of the HRF). We then extracted time-series from seed regions of interest from the filtered residuals. Seed regions included the MTL clusters identified in the group comparisons of how incentive levels influenced surprise processing, which included a region of hippocampus and posterior parahippocampal cortex (see 3.4, Table 4). These time-series were both entered into a whole-brain regression of the filtered residuals in all other voxels. Second-level and group level analyses were modeled as described above.

Table 4.

Interactions of incentive effects and surprise processing [High Incentive: (Surprise > Control) > Low Incentive: (Surprise > Control)] for the reward and punishment groups (p< 0.05, whole-brain corrected).

Region	x	y	z	Z	k
Reward Group
Hippocampus	-22	22	-6	3.14	16
Punishment Group
Inferior Temporal Gyrus	-48	-4	-46	4.26	96
Pre-central Gyrus	-32	-18	48	4.12	61
Cerebellum	-40	-42	-32	4.05	335
	34	-64	-60	3.65	55
	-4	-56	-64	3.47	78
	22	-54	-64	3.26	41
	44	-64	-34	3.77	386
Middle Temporal Gyrus	56	-80	28	3.98	71
	-58	-72	28	3.55	106
	54	-58	10	3.5	45
Fusiform Gyrus	44	0	-30	3.86	87
Middle Occipital Gyrus	60	-74	-14	3.68	100
Parahippocampal Cortex	14	-36	-12	3.58	45
Medial Frontal Gyrus	-8	-22	62	3.56	150
Brainstem	-4	-36	-48	3.18	61
Globus Pallidus	-10	2	-2	3.18	33

x, y, z = MNI coordinates; Z = z-score; k = cluster size

3.1 Results

3.1 Behavior: Speeded Reaction Time Task

Analyses of target reaction times revealed that both incentive level [high versus low incentive: F(47) = 21.02, p < 0.001] and the presence of surprising events [surprise versus control: F(47) = 15.46, p < 0.001] decreased participants' reaction times to target images, without any significant interaction across these factors [presence of surprise X incentive level: F(47) = 0.14, p = 0.71] (Table 1). These findings suggest that incentive cues were successful in manipulating participants' motivational state, and that surprising events were sufficient in influencing later behavior. Critically, there was no interaction of incentive level or presence of surprise by group [incentive level X group: F(47): = 1.72, p = 0.20, presence of surprise X group: F(47) = 0.62, p = 0.44, incentive level X group X presence of surprise: F(47) = 1.04, p = 0.31] (Table 1), suggesting that the motivational salience of incentives and surprising events across reward and punishment groups was equivalent. Our task was designed with an adaptive algorithm to equate feedback across condition, and thus there were no significant main effects or interactions in target success [p > 0.15, Table 1].

3.2 Behavior: Reward, but not punishment, enhances memory for surprising events

Following scanning, participants performed a recognition memory task for objects that constituted surprise (Figure 2). We found a main effect of incentive, such that memory for surprising events was greater in high versus low incentive conditions (F(46) = 10.64, p = 0.002). Further, we found an incentive*motivational group interaction (F(46) = 4.29, p = 0.04), such that recognition memory for surprising events was significantly greater in the high versus low incentive condition in the reward group (t(25) = 4.89, p < 0.001), without any memory differences across incentive conditions in the punishment group (t(21) = 0.68, p = 0.51, Figure 2). Direct comparisons of recognition memory across groups within each incentive level did not reveal any significant differences (High: Reward > Punishment: t(46) = 0.95, p = 0.35; Low: Reward > Punishment: t(46) = -1.5, p = 0.14). Thus, post-hoc analysis suggest that motivational context differentially influences memory selectivity for surprising events, such that increased selectivity is seen in rewarding but not punishing contexts. Due to the limited number of forgotten trials, we could not conduct a subsequent memory analysis to link these behavioral differences to neural measures of encoding success.

3.3 fMRI: Similar engagement of mesolimbic systems across reward and punishment

To identify brain regions modulated by motivation, independent of surprise, we compared activation in response to high versus low incentive cues. Motivation to earn monetary rewards (high vs low reward cues) and motivation to avoid punishments (shock vs no shock cues) resulted in similar patterns of activation in a broad network of regions including striatum, medial prefrontal cortex (PFC), and orbitofrontal cortex (OFC) (p < 0.05, whole-brain corrected, Table 2). The contrast of high versus low incentive cues did not yield any significant activation differences across reward and punishment groups.

Table 2. Significant activations from the contrast of high compared to low incentive cues (p< 0.05, whole-brain corrected).

Region	x	y	z	Z	k
Conjunction (Reward, Punishment)
Supplemental Motor Cortex	4	0	56	4.57	2422
Post-Central Gyrus, Middle Frontal Gyrus	-40	-22	44	4.31	2311
Lingual Cortex, Fusiform Cortex	-20	-82	0	4.13	1586
Ventral, Dorsal Striatum	18	12	0	4.09	537
	-10	12	2	4.08	334
Lateral Occipital Cortex	16	-70	54	4.25	339
	-26	-70	26	3.81	190
	40	-74	14	3.72	131
Cerebellum	-30	-58	-32	3.99	311
	12	-68	-30	3.84	199
	40	-54	-36	3.86	68
Precuneus, Lateral Gyrus, Occipital Gyrus	-8	-62	56	4.38	292
Cingulate Gyrus	-10	-24	38	4.37	250
Pre-central Gyrus, Inferior Frontal Gyrus	-52	8	30	3.75	119
Middle Frontal Gyrus	-34	32	32	3.9	113
Superior Parietal Cortex	28	-54	44	3.55	64
Thalamus	18	-18	12	3.55	54
	-16	-26	14	3.68	51
	-12	-10	8	3.63	44
Insula, Orbitofrontal Cortex	-32	28	-2	3.89	45
Precuneus, Posterior Cingulate	10	-40	44	3.64	39
Reward > Punishment
No significant activations	-	-	-	-	-
Punishment > Reward
No significant activations	-	-	-	-	-

x, y, z = MNI coordinates; Z = z-score; k = cluster size

3.4 fMRI: Distributed encoding of surprise in rewarding versus punishing contexts

To identify brain regions modulated by the presence of surprising events independent of motivation orientation, we compared brain activations in response to surprise versus control events for all subjects. Across both groups, the presence of surprise resulted in greater activation throughout the fronto-parietal network and ventral visual stream (p < 0.05, whole-brain corrected; Table 3).

Table 3. Significant activations from the contrast of surprise compared to control events (p< 0.05, whole-brain corrected).

Region	x	y	z	Z	k
Conjunction (Reward, Punishment)
Lateral Occipital Cortex, Middle Temporal Gyrus	6.06	54	-66	0	7407
Fusiform Gyrus	5.98	-50	-72	-2	3145
Inferior Frontal Gyrus, Pre-Central Gyrus	5.31	48	12	18	1322
	3.94	-46	6	24	178
Supramarginal Gyrus, Superior Parietal Lobule	4.92	-42	-50	42	1216
Cerebellum	4.57	-16	-76	-48	115
	3.85	-16	-72	-30	44
Reward > Punishment
Middle Occipital Gyrus	46	-52	-10	4.55	363
	36	-84	-2	3.27	47
Middle Temporal Gyrus	56	-56	0	3.99	53
	-44	-64	2	3.69	54
	44	-64	18	3.49	54
Parahippocampal Gyrus	-38	-46	-16	3.85	194
Inferior Frontal Gyrus	34	6	16	3.58	53
Inferior/Superior Parietal Cortex	58	-54	30	3.57	379
	30	-48	50	3.02	36
	46	-48	46	3.39	171
Superior Temporal Gyrus	48	-38	4	3.3	96
Inferior Occipital Gyrus	-16	-92	-14	3.27	44
Punishment > Reward
No significant activations	-	-	-	-	-

x, y, z = MNI coordinates; Z = z-score; k = cluster size

A direct comparison of surprise processing across groups revealed that, compared to the punishment group, the reward group showed greater sensitivity to surprising events in the ventral visual stream (middle occipital gyrus, inferior occipital gyrus, superior temporal cortex, middle temporal gyrus, and parahippocampal gyrus), and inferior frontal gyrus, inferior parietal cortex, and superior parietal cortex (p < 0.05, whole-brain corrected, Table 3). The reverse contrast (punishment > reward) did not yield any significant activation.

3.5 fMRI: Incentive level differentially influences MTL encoding of surprise in reward versus punishment

To characterize the influence of incentive level on surprise processing, we compared the processing of surprising events (surprise > control) encountered in the context of high versus low incentive levels for each group separately. In the reward group, this analysis yielded only one significant cluster: the left hippocampus was more sensitive to surprising events in the context of high compared to low incentives (p < 0.05, MTL small-volume corrected, Table 4). These results were previously reported in Murty and Adcock (2013). In the punishment group, several clusters showed more sensitivity to surprising events in the context of high compared to low incentive levels (p < 0.05, whole-brain corrected, Table 4) including the parahippocampal cortex (PHC), middle temporal gyrus, cingulate cortex, middle frontal gyrus, and globus pallidus. Interestingly, an analysis of the MTL clusters revealed a significant interaction of motivated surprise processing across regions [F(47) = 11.894, p = 0.001, Figure 3a). In the hippocampus (HPC), incentive enhancement of surprise processing was present in the reward (t(25) = 3.95, p = 0.001) but not punishment group (t(22) = 0.96, p = 0.35), whereas the opposite pattern was found in the PHC, with significant modulation in the punishment (t(22) = 4.49, p < 0.001) but not reward (t(25) = 0.76, p = 0.46) group. Figure 3b visualizes the relative responses of both HPC and PHC in the Reward and Punishment groups, thus revealing the double dissociation.

Figure 3.

A) Reward motivation enhanced hippocampal sensitivity to surprise (left). Conversely, punishment motivation enhanced parahippocampal cortex sensitivity to surprise (right; p < 0.05, small-volume MTL corrected). B) Extracted beta-parameters from the MTL clusters reveal a double dissociation between surprise processing in the MTL across reward and punishment motivated groups.

3.6 fMRI: Network connectivity of MTL regions across reward and punishment

We next investigated whether the MTL regions identified above engaged broader networks of coordinated activity as a function of motivational context by performing a ‘background connectivity’ analysis across reward and punishment groups. This connectivity analysis identifies regions that show greater functional coupling with seed regions of interest when modeling out neural responses reflecting processing of cues, targets, and surprise events (and their corresponding control events). While this analysis removes significant portions of task-related activity, it may also reflect portions of the task that were either unmodeled or not perfectly modeled by our GLM analyses (described above). These types of analyses are thought to reflect context-dependent shifts in intrinsic coordinated activity across regions ⁴⁶. Using this technique, we found that MTL regions showed differential coupling with PFC as a function of the motivational context of the individual. Specifically, the HPC showed greater functional coupling with two clusters in ventromedial prefrontal cortex (vmPFC) in the reward versus punishment group (cluster 1: [x,y,z] = [-4,50,6], z-score = 4.75, cluster size = 97 voxels; cluster 2: [x,y,z] = [2,62,2], z-score = 4.03, cluster size = 35 voxels), whereas, in the punishment versus reward groups, the PHC showed greater functional coupling with both OFC ([x,y,z] = [16,22,-16], z-score = 4.42, cluster size = 58 voxels) and anterior temporal lobe (([x,y,z] = [36,10,16), z-score = 4.12, cluster size = 58 voxels; p<0.05, whole-brain corrected, Figure 4).

Figure 4.

Across-group comparisons of background connectivity revealed that rewarding contexts facilitate coupling between the hippocampus and ventromedial prefrontal cortex compared to punishment (Top, p < 0.05, whole-brain corrected). Conversely, punishing contexts facilitate coupling between the posterior parahippocampal cortex and orbitofrontal cortex (Bottom, p < 0.05, whole-brain corrected).

4.0 Discussion

The current study investigated how motivation by reward versus punishment incentives influences neural sensitivity to and memory for surprising events. Behaviorally, reward motivation increased memory selectivity for surprising events associated with high versus low rewards, whereas punishment motivation had no effect on memory selectivity. Both reward and punishment motivation engaged a network of regions consistent with the mesolimbic dopamine system. Participants in the reward group, however, showed additional engagement of ventral visual and parietal regions compared to the punishment group. Notably, sensitivity to surprise was seen in different MTL regions across groups: in the hippocampus during reward motivation versus the posterior parahippocampal cortex during punishment motivation. These findings of enhanced sensitivity were accompanied by differences in MTL network connectivity across groups, such that in the context of reward, the hippocampus showed greater connectivity with the vmPFC, whereas in the context of punishment the parahippocampal cortex showed greater connectivity with OFC. Together these findings suggest that reward and punishment motivation differentially influence the encoding of surprising events and imply distinct consequences for how individuals encode mnemonic representations of the environment.

Previous research has demonstrated memory selectivity for unexpected events compared to expected events ^1,3,4 however, here we demonstrate that this phenomenon may be specific to certain motivational contexts. The selectivity of memory enhancements for incidental surprises in rewarding, but not punishing, contexts is supported by prior findings of motivational influences on declarative memory. Compared to punishment, rewards enhance incidental memory encoding ⁴⁷ as well as memory for the spatial contexts in which incentivized information was encoded ²⁰. Conversely, punishment motivation enhances memory for target items ³⁵, yet disrupts memory for the locations of target items within punishing contexts ²⁰. Of note, in our study, we did not see a disruption of memory for irrelevant events in punishing contexts, rather an absence of memory selectivity for such events, perhaps implicating a non-specific generalization effect. Facilitation of memory encoding for threatening items, which recruits similar neural circuitry as punishment motivation ⁴², tends to be at the expense of incidental encoding for surrounding fear-irrelevant information ^48-51. Similarly, in our paradigm, surprising events were not incentivized and thus may have been perceived as irrelevant to the individuals' goal of avoiding punishment. Thus, while punishment motivation may only support memory for targets (i.e., threatening items), reward motivation supports memory not just for targets (incentivized information) but also salient information encountered in the environment during goal-pursuit. If these conve rgent findings represent a general principle of adaptive behavior, they suggest it may be beneficial to dedicate resources solely to avoiding potential harm, whereas in rewarding contexts it is beneficial to divide resources to encode broader details of the environment in the service of identifying potential antecedents of obtaining reward.

Despite the observed difference in memory selectivity for surprising events across groups, there were no differences in level of motivation across groups. Punishment and reward cues produced equal facilitation of reaction time for button presses to target images. This similarity in behavior across groups indicates that degree of motivational engagement per se cannot account for the differences in encoding that accompanied differences in motivational orientation. Similar engagement of the mesolimbic dopamine system was also seen across groups. Specifically, high motivation in both contexts was associated with increased activation of the dorsal striatum, ventral striatum, and OFC. These findings fall in line with models of reinforcement-learning in which mesolimbic systems equally contribute to approach and avoidance behaviors ^28,52. Yet, despite similar mesolimbic engagement, we found differences in long-term memory for surprises incidental to goal pursuit across rewarding and punishing contexts. Memory for these events reveals a more nuanced pattern of memory encoding as a function of how the specific motivational state impacts the MTL and its connectivity with other cortical brain regions.

The patterns we report of differential activation within the MTL as a function of motivational orientation may be crucial for a mechanistic understanding of the behavioral memory effects reviewed above. The hippocampus is thought to support representations of items in the broader context in which the items were encoded as well as to monitor and record deviations between internal and external representations of the environment ⁴⁰. Thus, the facilitation of hippocampal encoding by reward may uniquely result in facilitating detailed memory for goal-irrelevant, salient items. By contrast, punish ment motivation only facilitated encoding in posterior parahippocampal cortex, which is theorized to represent global representations of context in isolation, without detailed information about the items appearing in said context ^53,54. Thus, the facilitation of parahippocampal cortex activation to surprises seen under punishment motivation is consistent with a shift to greater sensitivity to detection or localization of surprises in punishing contexts, at the expense of the fine details of how the surprising event deviated from retrieved expectations.

In line with our current findings, divergent patterns of MTL engagement across reward and punishment motivation have been demonstrated during explicit memory encoding tasks. For instance, reward-motivated declarative encoding has been shown to engage the hippocampus in isolation or in addition to the PHC ^17,18,22,23. Conversely, the successful encoding of information associated with threat or punishment has been shown to target the parahippocampal cortex as opposed to the hippocampus in contexts using salient aversive stimuli like electrical shocks ^35-37. Paralleling the previous literature, the current study suggests that motivational states engage similar MTL targets independent of whether encoding is instructed or incidental. If motivation primes how the MTL intrinsically encodes the environment, this influence would have downstream consequences for how the environment is represented in long-term memory. However, it should be emphasized that the current study departs from and builds on prior work as follows: we show that a shift from reward to punishment incentives changes the physiology of MTL encoding and impacts memory outcomes even when task demands and memoranda are identical.

Our results also revealed differences in the network connectivity of these same MTL subregions as a function of motivational context. First, we found that reward motivation preferentially facilitated hippocampal coupling with vmPFC. Previous research has implicated the vmPFC in assigning value to novel items ^55,56. Furthermore, interactions between hippocampus and vmPFC have been associated with both memory stability as well as the formation of schemas to support broader episodic learning ¹⁶. Together these literatures suggest an intriguing possibility that the enhanced coupling between vmPFC and hippocampus in our reward group represents incorporation of potential predictors of valuable outcomes into the broader representation of rewarding environments. In line with this interpretation, vmPFC activation has been related to episodic memory for outcomes that consistently predict reward ⁵⁷. Second, we found that in punishing contexts, we found preferential coupling of posterior parahippocampal cortex with right lateral OFC. Previous work has implicated lateral OFC in the suppression of goal-irrelevant responses ⁵⁸. Thus, in our task, enhanced engagement of OFC might reflect suppression of the resources potentially dedicated to cortical MTL- dependent encoding of surprise signaling. This suppression of goal-irrelevant information would explain why individuals did not preferentially encode surprising events into long-term memory in punishing contexts. Other literature has implicated right lateral OFC in the valuation of aversive incentives ^59-62; this literature would suggest that enhanced coupling between posterior parahippocampal cortex and lateral OFC may actually reflect the valuation and/or disambiguation of surprising stimuli as threats when they occur in punishing contexts.

Beyond the MTL, prior work has shown differential cue-evoked engagement of the VTA and amygdala during reward- and punishment-motivated memory encoding, respectively ^17,35. In these studies, however, differences only emerged as a function of subsequent memory, not as a main effect of motivation. The current study was not designed to look at subsequent memory effects, and due to too few trials per condition, we did not have enough power to look at VTA and amygdala engagement as a function of later memory success. Future studies will need to utilize a similar design with more trials to specifically look at engagement of the VTA and amygdala in supporting subsequent memory for surprising events in rewarding and punishing contexts.

We found other activation differences in neocortical regions that may inform future work on motivational orientation. Compared to the punishment group, the reward group showed greater sensitivity to surprising events throughout the ventral visual stream and parietal cortices, possibly implicating greater attentional resources being attributed to the processing of visual features of the environments ⁶³. These findings are in line with other literature suggesting that reward motivation is associated with the broadening of attention to global features of the environment ³¹, as well as environmental exploration ³². Enhancements in neural sensitivity, however, were not modulated by incentive level of the motivational cues, suggesting that increased sensitivity was in response to the general context of reward as opposed to the specific transient motivational state.

To conclude, our findings show that, given identical task demands and stimuli, the valence of motivational orientation changes how surprise is encoded in MTL networks. Behaviorally, we show that motivation by reward, but not punishment, enhances hippocampal activation and also memory selectivity for goal-irrelevant, yet salient, events in the environment. These findings support a model of motivated memory encoding in which biologically distinct learning states evoked by reward and punishment incentives cause individuals to construct qualitatively different mnemonic representations of the surrounding environment. Thus, while punishment incentives may uniquely facilitate memory in service of a singular, imperative goal of threat avoidance, reward incentives facilitate an inclusive map of complex environments that selectively enhances memory for potential antecedents of future rewards. Thus, the specifics of current motivational state may bias not only the content but also the form of memory, to the advantage or detriment of future behavior.

Highlights.

• -Reward results in better memory selectivity for surprising events than punishment.
• -Reward and punishment engage distinct medial temporal lobe regions to encode surprise
• -Medial temporal lobe connectivity shifts in rewarding versus punishing contexts.