Rewarding images do not invoke the reward positivity: They inflate it

Abstract

Increasing evidence suggests that the reward positivity conforms to an axiomatic reward prediction error – that is, it closely follows the rule-like encoding of surprising reinforcers. However, a major limitation in these EEG studies is the over-reliance on a single class of secondary rewards like points or money, constraining dimensionality and limiting generalizability. In the current suite of studies we address this limitation by leveraging different classes of rewards outcomes, specifically emotionally pleasant pictures. Over a series of three experiments, participants were able to choose idiosyncratically preferred pictures as rewards. During the first two experiments, participants were rewarded with either high or low points or high or low preferred pictures. The reward positivity was modulated by points, but not by pictures (regardless of preference), which instead evoked enhanced N2 amplitudes. In a third study that paired high/low points and preferred/non-preferred pictures, the point-induced reward positivity was inflated by the presence of a preferred picture. In line with past research stating the reward positivity is primarily sensitive to positive reward prediction error, this report finds that it is also influenced by a liking dimension, which possibly acts as an affective state to frame the motivational aspect of extrinsic rewards.

1. Introduction

The ability to learn to seek rewards spans levels of complexity. While the field of reinforcement learning has quantified how rewards can bind values to actions, this process occurs in tandem with complex motivations and affective biases. The present report was designed to test whether neural markers of reward evaluation are also sensitive to different classes of motivated reinforcement varied by dimensions of preference and liking.

Previous studies have detailed how midbrain dopamine projections appear to encode reinforcement prediction errors (RPE), an error signal related to the deviation between the expected and actual outcome to an action (Bayer and Glimcher, 2005; Schultz et al., 1997). The information content of RPEs represents a special case of surprise: by describing whether the event was better (+RPE) or worse (−RPE) than expected, an agent can learn to approach or avoid it in the future. The majority of RPE studies in animal models used primary rewards to elicit +RPEs; such as fruit juice or food reward (Bayer and Glimcher, 2005; Hart et al., 2014; Hollerman and Schultz, 1998; Roesch et al., 2007; Wheeler et al., 2011) or pain to elicit −RPEs (Abercrombie et al., 1989; Sorg and Kalivas, 1991; Ungless et al., 2004). These consequences can be thought of as primary outcomes, in that they promote or challenge homeostasis (Schultz, 2015).

Similar RPEs have also been quantified in humans, both within midbrain regions (Zaghloul et al., 2009), striatum (Cromwell et al., 2005; Delgado et al., 2000; Izuma et al., 2008; McClure et al., 2004; O’Doherty et al., 2004; Schultz et al., 2000), and cortex (Gehring and Willoughby, 2002; Rogers et al., 2004; Smith et al., 2009; Xue et al., 2009). Literature using functional magnetic resonance imaging (fMRI) in humans has also successfully elicited neural signatures of RPEs using primary rewards like juice and food smells (Berns et al., 2001; Howard and Kahnt, 2017; O’Doherty et al., 2004; Schultz et al., 2015). One study in particular (Sabatinelli et al., 2007) found that neurological reward processing centers were activated for positively valenced pictures but not for aversive pictures. In contrast to these fMRI studies, investigations of RPE signals with primary rewards have been, to our knowledge, completely absent in the electroencephalography (EEG) community. Instead, specific extrinsic classes of secondary rewarding conditional feedback (i.e. money or points) have been leveraged in order to modulate behavior and neural reward response (Delgado et al., 2003; Foti and Hajcak, 2009; Hajcak et al., 2007; Yeung and Sanfey, 2004). Needless to say, this over reliance on a single class of reward greatly limits the understanding of the boundary conditions and generalizability of the content that evokes the Rew-P. The current study aims to investigate EEG signatures of reward for different classes of extrinsic rewards.

As stated above, there is a great amount of evidence demonstrating RPE learning signals with human participants, and recently, EEG research has attempted to investigate relationships between ongoing neural activity and RPEs (Cavanagh et al., 2010; Holroyd et al., 2003; Holroyd et al., 2004; Pfabigan et al., 2011; Sambrook and Goslin, 2015). However, there have been ongoing debates about whether these EEG signals actually reflect pure axiomatic RPEs. An axiom is a rule, and rule-based responding is a requirement for a neural signal that reflects an explicit computation, like +RPE. While early theoretical models (Holroyd and Coles, 2002) suggested that the frontocentral negative deflection from about 200–300 ms post-punishment (variously termed the FRN, fERN, or N2) reflected a − RPE, this signal has been shown to be modulated by general surprise (Ferrari et al., 2010; Folstein and Van Petten, 2008) and even reward (Baker and Holroyd, 2011; Cavanagh et al., 2012; Oliveira et al., 2007), demonstrating that it reflects a more generic computation of simple surprise, particularly when behavior needs to be changed (as is common after punishment). A more recent revision of feedback-related phenomena suggests that brain responses to reward in this time range instead reflect a +RPE: the reward positivity (Rew-P; Holroyd et al., 2008). The tendency for people to quantify and even label EEG activities in this time range based on the reward-minus-punishment difference wave exacerbates this confusion: hereafter we refer to the reward condition-specific positive deflection as the Rew-P. This is a critical distinction, as the negative deflection to punishment does not reflect an axiomatic −RPE, yet the Rew-P appears to reflect an axiomatic +RPE (Cavanagh, 2015; Fromer et al., 2016; Meadows et al., 2016; Weinberg et al., 2014). To reiterate: attempts to define this difference wave as an axiomatic binary RPE are ill-conceived, since the information content of the negative deflection is not specific (axiomatic) to punishment. However, there have been few studies investigating the unipolar axiomatic variation of the Rew-P.

As stated earlier, our understanding of the Rew-P is based on only one type of reward class. The current study was designed to expand on the growing understanding of this potential +RPE signal by comparing Rew-P modulation to differing classes of extrinsic rewards (Fiorillo, 2013; Lak et al., 2014; Tobler et al., 2005). Here, we leverage idiosyncratically chosen pleasant pictures as affective rewards (Sabatinelli et al., 2007). In his review of reward processes, Schultz (2015) posits a distinction between varying classes of non-primary rewards. These rewards can take the shape of physical rewards (i.e. money, luxury cars, etc.), or take the shape of some sort of subjective esthetic reward (i.e. experiencing a beautiful sunset). In line with this distinction we leverage what he calls “subjective esthetic reward” by presenting the participants of the current study with self-selected pleasant pictures. For the sake of parsimony, we will refer to these subjective esthetic rewards as affective rewards and point rewards as conditioned rewards.

As evidenced by the amassment of support of the Rew-P as a signal of +RPE, first we hypothesize that reward magnitude will influence Rew-P amplitudes, whereby the neural response for +10 points will be greater than for +1 point. By the same logic, we hypothesized that unexpected affective rewards (pictures) will evoke similar neural responses as conditioned rewards (points), and that Rew-P amplitudes will be greater for more preferred pictures than to less preferred pictures. Furthermore, we investigated the P2 component to early predictors of pending incentive outcome. P2 has been shown to relate to salience and task motivation (Carretie et al., 2001; Parvaz et al., 2012); here we examined if it was sensitive to early predictors of future reward, a phenomenon central to reinforcement learning (Schultz et al., 1997). To presage the outcomes of the following three studies, the central hypothesis of affective rewards was not supported: rewarding pictures did not independently evoke a Rew-P, but they did augment the Rew-P magnitudes to conditioned rewards in the expected direction. These findings are discussed in the context of affective modulation of axiomatic +RPE signaling by the Rew-P.

2. Method

2.1. Affective reward preference task

In order to choose idiosyncratically preferred images, all three experiments utilized a reward preference task prior to the experimental tasks. In this short two alternative forced choice task, participants were presented two images drawn from 1 of 5 affective reward categories: male models, nude women, puppies, babies, or nature scenes. Images classes were based on images that were rated as highly pleasurable from the International Affective Picture System (IAPS; Lang et al., 2008). Standardized ratings of valence (1 = negative to 9 = positive) from the IAPS technical manual were compared by gender. The reward classes used for the current study were chosen from the most occurring image themes. All images were selected from Internet searches (i.e. “Hi-Definition Puppy Images”). All images were inspected in task-presentation conditions in order to assess no image appeared blurry and that sizing was equal for all images. During the task, an image from one of the affective reward categories was presented on the left side of the screen while another image from a different affective reward category was presented on the right side. Participants were instructed to choose with a left or right button press which image of the two they preferred. There were 16 images in each category; each category was paired against each of the others categories a total of four times (total of 40 trials). The participant’s top choice and 4th choice was selected for the experimental tasks (see Fig. 1). The 4th choice was used to avoid the lowest ranked category, which may be considered aversive. A final reward preference check was conducted after each experimental task, where participants viewed 16 original pictures from each of the rewarding image classes as well as 16 neutral images (e.g. lined paper, a light bulb, a door knob, etc.) for a total of 96 trials. Each image was presented for 1 s, and afterwards participants were instructed to rate how pleasant they found each image (1–9). All participants rated their 1st and 4th reward categories above the neutral pictures, verifying that all stimuli were positively valenced.

Fig. 1.

Affective reward image preference task and post experimental pleasantness rating results. Percentages displayed in each of the pie charts relate to the proportion of participants who preferred that type of reward. Mean ratings of pleasantness for each of the participant’s choices (1st-5th) can be viewed on the right side of the figure. These bar plots reveal ratings of pleasantness decrease in tandem with the choice rank from the affective reward image preference task. Interestingly, participants rate their 5th choice as less pleasant as our neutral image class (“0”), supporting our use of the 1st and 4th image class choice.

3. Experiment 1

3.1. Participants

Experiment 1 consisted of 16 participants (7 females) with a mean age of 22.19 (SD_age = 4.79). In all experiments, participants were recruited from the University of New Mexico subject pool. Students received class credits for participation. Participants were excluded from participation if they had a history of head injury that resulted in loss of consciousness for more than 5 min, had a history of epilepsy, had a history of any psychiatric or neurological disorder, or was currently on any psychiatric or neurological drugs. The Institutional Review Board of the University of New Mexico approved the study protocol.

3.2. Materials

The stimulus material consisted of 80 hi-definition original images from the participant’s top choice and their 4th choice. Images presented during the affective reward preference task were not included for main experiment presentation. The tasks were programmed in Matlab using Psychtoolbox (Brainard, 1997). Participants used left and right index finger trigger button presses on a Logitech F310 game controller.

3.3. Data acquisition and preprocessing

Electrophysiological data were collected with a 64Ag-AgCl electrodes embedded in a stretch-lycra cap with a sampling rate of 500 Hz with low and high cutoffs at 0.01–0.100 Hz. CPz served as the reference electrode and FPz as the ground electrode. Data was recorded with a Brain Vision system (Brain Products GmbH, Munich, Germany). Vertical electrooculogram (VEOG) activity generated by blinks was recorded by two auxiliary electrodes placed superior and inferior to the left pupil.

All EEG processing was conducted in EEGlab (Delorme and Makeig, 2004). First, CPz was re-created via computation of the average reference (EEGlab function pop_reref.m). Very ventral electrodes (FT9, FT10, TP9, and TP10) were then removed, as they tended to be unreliable. The average reference was then recomputed for the remaining 60 electrodes. EEG data was filtered between 0.01 and 20 Hz. In order to capture all trial events, data was epoched around reward incentive screen onset (−1000 to 12,000ms). Using statistical deviations from the mean for each EEG channel, FASTER (Nolan et al., 2010) identified artifacts in each epoch for later rejection. Eye blink activities were removed following ICA (runica; Makeig et al., 1996).

Epochs were then baseline corrected (−200 to 0 ms before feedback onset) and averaged to calculate event related potentials (ERP). The Rew-P was quantified at electrode site Cz and was measured between conditions within a 200–350 ms window post feedback onset. The P2 was also quantified at Cz from 150 to 200 ms following presentation of the incentive screen indicating which reward was available on that trial.

3.4. Procedure

On each trial, an incentive screen was followed by three sequential flanker stimuli that were then followed by the appropriate feedback. Before each trial a shape (a star or a moon) was presented for 1000 ms to the participant informing them of the type of reward (points or pictures) they will win on the following trial (i.e. stars = pictures, moon = points), but not the magnitude of the reward (high or low). Participants were instructed that the magnitude of the reward was based upon a “complex in-game algorithm” that would compare their current trial performance with past trial performance. In order to keep the participant engaged each participant performed 3 standard 5-arrow flanker sequences per trial (Eriksen and Eriksen, 1974). Participants indicated with an appropriate button press the direction, of a central arrow that was flanked by two arrows on either side of the target that were either congruent or incongruent with the target stimulus. Following the flankers, participants were presented with either a high or low point reward (10 points or 1 point) or a high or low picture reward (their 1st or 4th image category choice). Feedback was presented for 1000 ms.

Participants were instructed to make accurate but quick (< 500 ms) controller button responses in order to win rewards, however, they were informed the task would not keep track of their earned points, and that these points were essentially useless. However, unbeknownst to the participant, the magnitude of the reward being presented was independent of task performance. There were 160 trials total, where each of the four feedback conditions (Top-Picture, Bottom-Picture, +10, and + 1) was presented to the participant 40 times. The inter-trial interval for this task was 1000 ms.

3.5. Statistical analyses

Greenhouse-Geisser adjusted ANOVAs and planned comparison decompositions were used for data analyses. Reports of effect size for ANOVA are partial-η2, while planned comparison effect sized are reported as d.

4. Results

4.1. Behavioral results

Table 1 displays means and standard deviations for RT, accuracy and ERP amplitudes. In order to assess if the incentivized feedback modulated performance, two independent sample t-tests (Picture vs. Point) were conducted for reaction time and accuracy; there were no significant differences in performance due to incentive (RT: t (15) = 0.566, p = 0.586, d = 0.166; accuracy: t(15) = 0.142, p = 0.889, d = 0.037).

Table 1.

Means and standard deviation for Experiment 1 and Experiment 2.

	Experiment 1	Experiment 2
Picture accuracy	0.94 (0.061)	0.86 (0.081)
Point accuracy	0.94 (0.064)	0.85 (0.099)
Picture reaction time (seconds)	0.37 (0.044)	0.25 (0.028)
Point reaction time (seconds)	0.37 (0.051)	0.25 (0.027)
(N2) Top picture feedback	−2.40 (3.33)	−1.66 (3.82)
(Rew-P) Top point feedback	0.66 (3.02)	1.83 (3.15)
(N2) Bottom picture feedback	−2.92 (1.97)	−1.62 (3.53)
(Rew-P) Bottom point feedback	0.91 (3.04)	1.62 (2.72)
(Rew-P) Null picture feedback	N/A	0.51 (2.65)
(Rew-P) Null point feedback	N/A	0.03 (2.46)
(P2) Picture incentive	2.47 (2.61)	0.42 (1.86)
(P2) Point incentive	1.10 (2.61)	0.30 (2.29)

4.2. ERP results

Fig. 2 depicts the grand average waveforms for stimulus-locked ERPs. The 2 (reward type: picture vs. point) × 2 (reward magnitude: high vs. low) ANOVA on the Rew-P revealed a significant main effect for reward type, where points had larger Rew-P than pictures (F (1, 15) = 23.490, p < 0.001, η² = 0.610). However, this analysis did not reveal a main effect for reward magnitude (F(1, 15) = 0.102, p = 0.754, η² = 0.007) or an interaction (F(1, 15) = 0.808, p = 0.383, η² = 0.051).

Fig. 2.

Experiment 1: ERPs time-locked to trial feedback (a). Bar plots (b) represent mean ERP potentials from the Rew-P temporal window (grey box). Shaded regions represent standard error of the mean. Point reward appears to instantiate a Rew-P while picture rewards do not. However, this lack of Rew-P for picture reward may be related to the inflated P2 evoked during the reward cue incentive screen (c), in which the signal of salience may migrate to the first instance of prediction.

In order to assess P2 modulation for reward incentive cue (prior to the flankers), an independent samples t-test was conducted. This analysis revealed significant P2 modulation differences between the two reward types (t(15) = 3.373, p = 0.004, d = 0.842), whereby P2 amplitude was greater when the participant was incentivized by picture reward then when they were incentivized by point reward (see Fig. 2c).

The significant differentiation between the P2 evoked for reward-type cues led us to conduct two exploratory analyses comparing the relationships between reward type specific P2 amplitudes with Rew-P amplitudes. Picture incentive evoked P2 was not correlated with the ERP response evoked from affective reward presentation (r = 0.271, p = 0.310) however point reward incentive was positively correlated with point Rew-P (r = 0.544, p = 0.029).

5. Experiment 1 discussion

In Experiment 1 we found that Rew-P was sensitive to a specific class of feedback, notably points and not pictures, in contrast to our hypothesis. These null effects are inconclusive tests of axiomatic interpretation of +RPE, as we describe below. Compared to the characteristic morphology of the point-induced Rew-P, the morphology for picture feedback did not show a broad sloping positive voltage potential between the P2 and P3. Instead, pictures appeared to elicit an N2 component.

Our initial hypothesis that higher reward magnitudes would increase Rew-P magnitude was not supported. We suspect this effect was caused by the participants being instructed that the earned points were useless, and thus, these points were encoded essentially as similar wins and not as rewards of importantly differing magnitudes. For picture rewards, it is difficult to assess whether the difference for reward processing between a participant top and 4th choice reward is equivalent to the difference between receiving ten points or one point. One could imagine that the value difference between a picture of a puppy and a picture of a nature scene may be rather small, albeit different from zero. This is a critical point: we presumed that there was uncertainty in the reward in this task. Even though participants knew they would see a picture, we hypothesized that the difference in liking between the top and 4th picture was meaningful. If not, and these were identically reinforcing, it is possible that the known property of fully predictable stimuli would fail to elicit a +RPE (see Fig. 1 in Schultz et al., 1997), explaining the absence of the Rew-P to pictures. A test comparing mean pleasantness ratings for top and 4th picture choice did however reveal a significant top choice > 4th choice effect (t = 5.606, p < 0.001). This finding suggests that although the dimension of liking between the top and 4th choice pictures was meaningful for each participant, yet this difference in liking did not affect Rew-P amplitudes.

Interestingly, P2 amplitude to incentive cues differed between reward types (Picture > Points). P2 is most associated with non-specific stimulus salience (Luck and Hillyard, 1994), suggesting that the signal of reward may migrate to the earliest instance of expectation. Again, this is commensurate with a fully predictable reward - where the earliest deterministic predictor of reward elicits the +RPE. Although reward anticipation enhanced the P2, it did not necessarily evoke the characteristic morphology of the Rew-P (a similar phenomenon was previously observed by Holroyd et al., 2011). Surprisingly, we found a significant correlation between P2 amplitudes for point reward cue and Rew-P for point feedback, but no correlation for ERP’s for picture cue and feedback, suggesting variability in the neural signatures for the anticipation and acquisition of conditioned rewards, but not for affective rewards.

6. Experiment 2

Experiment 1 revealed Rew-P is evoked by points but not pleasant images. However, P2 modulations for picture incentive > point incentive appears to suggest that this signal of salience may migrate to the first instance of prediction. This unresponsiveness to similar reward but evocation due to a fully predictive cue is in line with an axiomatic + RPE if the reinforcing properties of the pictures are equivalent. Furthermore, owing to the sensory complexity of the picture rewards, it is possible that other ERP features mask the Rew-Ps for affective picture rewards. In order to further disentangle these phenomena, we conducted a second experiment where we altered reward expectation by including a no win condition. First, we hypothesized that the increased uncertainty of feedback expectation will equivocate P2 amplitudes for the incentive screen. Second, we hypothesized that both conditioned and affective rewards will evoke a Rew-P compared to the null reward condition, again also modulated by magnitude (+ 10 > +1; top picture > 4th picture).

7. Method

Experiment 2 was the same as Experiment 1, but it consisted of 6 possible feedback situations (top choice, 4th choice, no-picture, +10, + 1, no-point) where each feedback was presented 30 times (total trial count was 180). The no-picture and no-point outcomes were identical yellow bars. Twenty participants (11 females) were recruited for Experiment 2 (M_age = 20.9, SD_age = 4.8). All experimental protocols and data processing procedures were similar between Experiment 2 and Experiment 1.

8. Results

8.1. Behavioral results

Similar to Experiment 1, independent sample t-tests results comparing pleasantness ratings for participants’ 1st and 4th picture choice revealed a significant difference (t = 7.282, p < 0.001) whereby the participants top choice was rated as significantly more pleasant than their forth choice. In order to assess if performance (reaction time and accuracy) differed between reward incentive conditions, two independent sample t-tests were conducted. Similar to Experiment 1, no reaction time differences (t(19) = 1.023, p = 0.319, d = 0.221) or accuracy differences (t(19) = 1.066, p = 0.300, d = 0.263) were revealed between the types of reward incentive.

8.2. ERP results

In order to investigate whether changing outcome expectation had an effect on P2 amplitudes following reward class incentive, an independent samples t-test was conducted comparing P2 amplitudes (150–200 ms) for picture and point reward incentives (see Fig. 3d). This analysis revealed the ERP amplitudes for the incentives were essentially the same (t(19) = 0.423, p = 0.677, d = 0.099), consistent with the hypothesis that uncertainty removed the information from the incentive screen as the earliest faithful predictor of future reward.

Fig. 3.

Experiment 2: ERPs time-locked to trial feedback. Similar to Experiment 1, point feedback invoked a Rew-P (a₁), but picture feedback evoked an N2 instead (a₂). Bar plots (b) display mean amplitudes during the feedback time window (grey box). Comparison ERP plots for points versus pictures for high (c₁) and low (c₂) rewarding feedback clearly reveals that reward points evoke a Rew-P while rewarding images evoke an enhanced N2. ERPs for point and picture lose conditions were not different (c₃). P2 modulations for incentivized reward cues (d) did not differ between reward type incentives.

The 2 (reward type: picture vs. point) × 3 (reward magnitude: high vs. low vs. null) ANOVA also revealed a significant (point > picture) main effect for reward type (F(1, 15) = 25.978, p < 0.001, η² = 0.578). Similar to Experiment 1, this analysis failed to reveal a main effect for reward magnitude (F(1.532, 29.113) = 0.196, p = 0.765, η² = 0.010), however, there was a significant interaction (F (1.917, 36.416) = 18.203, p < 0.001, η² = 0.489).

In order to unpack this interaction, we conducted two ANOVAs for the different reward types. The first ANOVA comparing reward magnitude for point (Fig. 3a₁) outcomes revealed a significant difference between levels of magnitude (F(1.774, 33.700) = 7.837, p = 0.002, η² = 0.292). A series of t-tests failed to reveal any significant difference between the top and bottom points (t(19) = 0.475, p = 0.640, d = 0.109), however, these analyses did reveal a significant top point > null point effect (t(19) = 3.109, p = 0.006, d = 0.601) and a bottom point > null point effect (t(19) = 3.467, p = 0.003, d = 0.781) for Rew-P modulation. Taken together, these effects adhere to the literature pertaining to Rew-P in that rewarding stimuli evoked a Rew-P while not winning a reward led to diminished amplitudes across this time range.

The second ANOVA for picture rewards (Fig. 3a₂) also revealed a significant difference between levels of Reward Magnitude (F(1.708, 32.456) = 7.293, p = 0.004, η² = 0.277). Again, t-test failed to reveal any significant difference between the top and bottom pictures (t (19) = 0.069, p = 0.945, d = 0.017). Surprisingly, this analysis did reveal that ERP amplitudes differed between null trial feedback and top picture (t(19) = 2.877, p = 0.010, d = 0.673) and bottom picture (t(19) = 3.236, p = 0.004, d = 0.748), however these effects were inverse to the point reward outcomes: the amplitude for null feedback was greater than that of both rewarding picture feedbacks. Based on the morphology of these reward generated ERPs it appears that point rewards evoked a Rew-P while rewarding pictures again evoked a N2. Null outcomes were nearly identical between conditions (Fig. 3c₃). Similar to the results seen in Experiment 1, there was no significant correlation between P2 and Rew-P amplitudes for affective rewards (r = 0.404, p = 0.077) but there was a significant relationship for conditioned rewards (r = 0.485, p = 0.030).

8.3. Experiment 2 discussion

Experiment 2 replicated the results from Experiment 1 whereby points, but not pictures, evoked a Rew-P. Against our predictions, the magnitude of reward did once again fail to modulate the Rew-P amplitude, which is still surprising given the known sensitivity of the Rew-P to +RPE (Cavanagh, 2015). Most surprising was that ERP amplitudes for pleasant pictures failed to evoke a Rew-P in the context of uncertain reward, but instead a significantly larger N2 to that of the presentation of no-win feedback condition. Finally, in support of our hypothesis, the P2 modulation for the incentive screen was equivocated; suggesting that obviating the ability to reliably predict feedback diminishes the anticipatory value of this predictive cue. We again observed a significant correlation between P2 and Rew-P amplitudes for point rewards but not for picture rewards. This replicated effect is surprising given that cue evoked P2s showed no difference between incentivized rewards. Taken together, these findings suggest an important boundary condition of the Rew-P: rewarding pictures were always rated more positively than neutral pictures, yet the morphology of the ERP to rewarding pictures was considerably different (a generic N2 component) than the neutral condition (which was the same between point and picture no-rewards). These commonalities between Experiments 1 and 2 held even when we diminished the predictability of reward.

The findings obtained for pleasant image presentation from Experiments 1 and 2 suggest that the Rew-P is not sensitive to all forms of reward. As stated above, rewards can take the form of tangible objects (money and points) or have pleasurable sensory properties (a gorgeous sunset; Schultz, 2015) and yet the lack of any rewarding signal for these pleasurable pictures suggests that these types of rewards were not “better than expected” by participants. However, these affective pictures may still influence reward in an indirect manner. Experiment 3 tested if the Rew-P elicited for conditioned point rewards could be affected when awarded in the context of pleasant pictures.

9. Experiment 3

Experiments 1 and 2 revealed that Rew-P is sensitive to point feedback but not to picture feedback. In both studies, the participants were informed that the points were meaningless, and yet, we consistently show that Rew-P modulation is evoked to these meaningless points, but not to affective rewarding images. If pleasant images themselves are not encoded as rewards, then what are they? One hypothesis is that pleasant images act as an “emotional framework”, or an emotional state, for reward acquisition processing. Animal, human, and neuroimaging studies have demonstrated many shared neural systems between affect and reward systems (Chiew and Braver, 2011; Pessoa, 2008). In fact, one study in particular (Wheeler et al., 2011) has demonstrated dampened dopamine release in rodents during unpleasantly tasting liquid predicting drug cues. This suggests the emotional environment or state framing reward acquisition may modulate these reward signals. In order to test this emotion-reward modulation, we designed a third experiment where by task differing magnitudes of extrinsic rewards (one or ten points) was paired with differing magnitudes of affective rewarding images (a participant’s top choice from the preference task or a neutral picture). We hypothesize that the Rew-P for point rewards will be modulated by pleasant pictures but not for neutral picture-point pairs.

10. Method

Twenty-five participants (18 females) with a mean age of 21.2 (SD_age = 2.8) participated in Experiment 3. Participation criteria, data processing, and EEG processing were the same as they were for Experiment 1 and 2. Similar to Experiments 1 and 2, participants completed the reward preference task, however, for Experiment 3 only the participant’s top choice was used as a rewarding stimulus. To investigate whether pleasant pictures modulate the Rew-P, a neutral image condition (e.g. lined paper, a light bulb, a door knob, etc.) was used to replace the 4th choice condition that was used in Experiments 1 and 2.

In Experiment 3, participant performed a modified time estimation task (Holroyd and Krigolson, 2007) where they were asked to estimate 3.2 s. This task offers a novel opportunity to quantify the effect of reinforcement on behavior, as reward has been associated with faster and more accurate time estimation (Manohar et al., 2015). Before the start of the task participants were informed that the points they earned would allow them the chance to win actual money in a lottery with other participants ($40 USD). Indeed, during Experiment 3, participants were made aware that the point they earned mattered, unlike Experiments 1 and 2 where they were explicitly told that the points were useless.

Prior to each trial participants were presented an incentive screen for 1000 ms where they were informed that they had a chance to win ten points or one point and that these points would be paired with either their most preferred picture or a neutral object. Following the incentive screen, participants were presented a white fixation cross. Participants were instructed that the trial would not start until they pressed the left trigger button, whereupon the fixation cross would turn red, indicating the clock was running. Once the participant believed 3.2 s had elapsed they would press the right trigger button. If the participants’ estimation of 3.2 s was within the correct time window they were presented with the incentivized rewarding image in the center of the screen surrounded on all four borders with the number of rewarded points. Extensive pilot testing confirmed that the entire feedback display was interpretable without needing to resort to saccades. If their estimation was outside of the correct time window the rewarding image was still presented in the center of the screen, but instead of points, yellow bars were displayed around the borders of the image. Participants were instructed that these yellow bars would earn them no points and thus would mean they were not correct in their time estimation.

The window for correct time estimation was 3.2 ms ± 0.8 ms. An in-task algorithm computed a running accuracy percentage over the previous five trials completed by the participant. If the participants accuracy was above 50% the upper and lower ranges around the target time decreased in logarithmic increments. Once the participant’s running accuracy fell below 50% the upper and lower time ranges increased. The task had a total trial count of 160 trials where participants had a chance to win each of the incentivized reward 40 times each. The inter-trial interval was 1000 ms.

11. Results

11.1. Behavioral results

Means and standard deviations for accuracy, target time deviations and ERPs are displayed in Table 2. In order to assess if incentivized rewards modulated behavior, we compared accuracy and absolute target time deviations (|3.2 - trial RT|) across experimental conditions. The 2 (picture type: top vs. bottom) × 2 (point type: top vs. bottom) ANOVA comparing task accuracy failed to reveal any significant main effects for picture or point rewards (F(1, 24) = 0.148, p = 0.704, η² = 0.006; F(1, 24) = 0.492, p = 0.490, η² = 0.020, respectively) or an interaction (F(1, 24) = 1.019, p = 0.323, η² = 0.041). The ANOVA comparing target time deviations failed to reveal a significant picture type or point type main effect (F(1, 24) = 0.118, p = 0.734, η² = 0.005; F(1, 24) = 1.104, p = 0.304, η² = 0.044). This ANOVA also revealed non-significant interaction (F(1, 24) = 2.982, p = 0.097, η² = 0.111); this crossover trend did not appear to offer any evidence about the hypothesis that better rewards boost performance. These results suggests that task performance was independent of the incentivized reward, however, the accuracy scores (mean ranging from 0.52–0.54) suggests that our running accuracy correction was successful whereby our participants did not deviate far from 50% accuracy.

Table 2.

Means and standard deviation for Experiment 3.

	+ 10 points	+ 1 point
Top picture accuracy	0.52 (0.079)	0.52 (0.088)
Bottom picture accuracy	0.54 (0.071)	0.51 (0.074)
Top picture target time deviations (seconds)	0.23 (0.068)	0.29 (0.401)
Bottom picture target time deviations (seconds)	0.20 (0.066)	0.31 (0.453)
(ERP) Top picture win	3.60 (3.99)	2.75 (4.25)
(ERP) Top picture lose	1.81 (4.42)	1.58 (3.72)
(ERP) Bottom picture win	2.76 (3.56)	1.77 (3.28)
(ERP) Bottom picture lose	1.12 (3.99)	0.53 (3.24)
(ERP) Top picture incentive	1.22 (1.59)	1.38 (1.41)
(ERP) Bottom picture incentive	1.07 (1.79)	1.36 (1.83)

11.2. ERP results

In order to test our hypothesis that Rew-Ps for conditioned rewards will be modulated when paired with affective rewards, we compared Rew-P amplitudes using a 2 (picture reward: top vs. bottom) × 2 (point reward: top vs. bottom) ANOVA for win conditions. This 2 * 2 ANOVA revealed a significant main effect of magnitude for point reward (F(1, 24) = 8.481, p = 0.008, η² = 0.261), as well as significant main effect of magnitude picture reward (F(1, 24) = 4.908, p = 0.036, η² = 0.170), but not an interaction (F(1, 24) = 0.068, p = 0.797, η² = 0.003). Because these results revealed significant main effects but no interaction, we can infer that the pairing of conditioned and affective rewards did not have modulatory effect on Rew-P amplitude, but they did have an additive effect.

Although not included in the a priori hypotheses, we conducted an exploratory analysis comparing ERP amplitudes for lose conditions. This analysis failed to reveal a significant main effect of magnitude for point reward (F(1, 24) = 1.458, p = 0.239, η² = 0.057), demonstrating that the lose conditions were not sensitive to the degree that the outcome was “worse than expected” in the context of unachieved point magnitudes. There was a significant main effect of magnitude for picture in the no-win condition (F(1, 24) = 6.393, p = 0.018, η² = 0.252) whereby ERP amplitudes were significantly different for neutral images than for preferred picture when no point reward was given; visual inspection of Fig. 4 suggests that there was a diminished N2 for preferred compared to neutral pictures. Finally there was no interaction (F(1, 24) = 0.617, p = 0.440, η² = 0.025). Taken together with the findings of Experiments 1 and 2, these results suggest that Rew-P is inflated, but not invoked, by positive affective images.

Fig. 4.

Experiment 3: ERPs time-locked to trial feedback. ERPs by condition for rewarding feedback for wins and their corresponding loses (a). As expected Rew-Ps for wins were larger for each of the reward conditions (all t’s > 2.2, all p’s < 0.03). Topographic maps for rewards (scaled: ± 1.25 μV) reveal activation related to emotional visual stimuli (see Foti et al., 2009) at 250 ms. Surprisingly, these plots reveal scalp level activation seen repeated for feedback processing. This activation appears to visually scale with the magnitude of point reward. The bar plot (b) reveals a boost in Rew-P expression when point feedback was presented in the context of a preferred image. Black boxes inlaid within the colored bars represent the mean ERP responses for no-win conditions for each respective condition.

12. General discussion

In the current study, we compared conditioned (points) and affective (idiosyncratically chosen pleasant pictures) types of extrinsic rewards and how they modulate the Rew-P. In Experiment 1 our findings indicated that points, but not pictures, evoked a Rew-P. These findings were replicated in Experiment 2 where we increased uncertainty about feedback outcome. Experiment 1 also showed an enhanced P2 during the rewarding picture incentive compared to point incentive, and we were able to abolish this effect in Experiment 2 with the manipulations of expectation. Taken together, these effects suggest that Rew-P is not reliably evoked to all types of rewarding feedback, although future studies could probe this further with different manipulations of uncertainty. In Experiment 3 we tested whether affective rewards have any modulating effect on Rew-P when paired with conditioned rewards. This experiment revealed that Rew-P was boosted by the magnitude of both point and picture reward. Taken together, affective types of reward do not seem to evoke a Rew-P but instead seem to boost its activity when it is already elicited.

One surprising effect in the current study was the enhanced N2 component for picture rewards over that of null rewards revealed in Experiment 2. The frontal midline N2 is an ERP feature sensitive to unsigned surprise; oftentimes indicating a need for cognitive control (Folstein and Van Petten, 2008; Cavanagh and Frank, 2014). However, Experiment 3 revealed a significant increase in Rew-P amplitudes for high vs. low point and picture reward pairs. This result suggests that emotional forms of rewards might act as an environmental frame during reward expression thus modulating brain signals during reward acquisition. This effect may not seem too surprising considering the evidence implicating similarities between emotion and reward processing centers in the brain (Aron et al., 2005; Baxter and Murray, 2002; Blood and Zatorre, 2001; Murray, 2007). From these outcomes, we can infer that the state in which rewards are acquired change the neural expression above and beyond the response to that reward by itself. These emotional states in which reward sensitivity is affected might be what Sutton and Barto (1998) refer to a reinforcement learning state, whereby the knowledge relating to the environment affects learning rates and, by the same token, reward signals. In line with this direction, a future study may be designed to test if Rew-P and learning rates to point rewards paired with pictures of different valences (positive neutral, and negative) adhere to axiomatic models of RPE.

Prior literature has shown the Rew-P is modulated by a variety of phenomena. One study (Angus et al., 2015) demonstrated that affective state did not directly modulate the Rew-P, but that individual liking of the received rewards correlated with Rew-P amplitude, suggesting that motivational intensity augments the Rew-P. Threadgill and Gable (2016) showed similar results, whereby Rew-P was affected by approach motivation states. We suspect that the effects seen in Experiment 3 adhere to this view of Rew-P modulation. We believe the significant main effects for points and picture that was absent in Experiments 1 and 2 was due to the instruction that the points mattered for a future lottery, and that the images presented to them were either highly valenced positive pictures or neutral pictures. We believe that both of these changes between experiments increased participant motivation and thus neural activity to the performance feedback (Deci et al., 1999). It remains unknown if these affective modulations of the Rew-P simply alter the size of the signal, or if they cause a more fundamental alteration of the information content represented in the signal (i.e. positive affect enhances +RPEs).

The findings here don’t address whether there was a Rew-P to pictures that was simply delayed due to the visual complexities of the stimuli. However, we find that to be unlikely for a variety of reasons. While ERP components are defined by an informal mixture of eliciting event, topography, polarity, and latency, this latter feature can vary depending on incidental circumstance (Luck, 2005) like visual complexity. Yet in Experiment 2, rewarding pictures were immediately discernable from the neutral yellow bar (yet did not elicit a Rew-P), and in Experiment 3 rewarding pictures enhanced the Rew-P. Together, the findings suggest that visual complexity was not prohibitive to the timing of the Rew-P. Moreover, the computation reflected by the Rew-P is likely defined by the bandwidth available in the early stimulus processing time frame. To suggest this same computation may occur later with a faithfully unique ERP signature is unlikely: for example, P3-like processes are complex mixtures of multiple pieces of information and any realization of +RPE at late time points would likely be integrated with a variety of other processes. Thus, the information content in any later signal would not be best defined as an axiomatic +RPE.

12.1. Limitations

One limitation in Experiments 1 and 2 was that we did not ascertain whether the participants actually believed that their responses had no impact on the feedback they received, and this might have affected their motivation to perform the task optimally. However, accuracy performance results for all conditions in both Experiments 1 and 2 were above 85%. These results suggest that the participants were motivated to perform well. A second limitation is in the task differences between the first two experiments and Experiment 3. Although the type of task employed may change the expression of reward processing we don’t believe this experimental task switch was meaningful, particularly since the feedback was not explicitly and directly related in a 1-to-1 manner with the performance.

12.2. Future directions

A surprising effect revealed in Experiment 1 that was later replicated in Experiment 2 was a lack of reward signal (Rew-P) for the picture rewards. Instead, this class of reward evoked an enhanced N2. These effects are startling and might infer (1) a separate neural mechanism of reward processing for sensory complex rewarding stimuli (2) the stimuli used in the experiments were not processed as rewarding by the participant, or (3) the inclusion of novel picture stimuli evoked a signal of surprise that superseded the reward signal. We propose that the latter is the most likely explanation. The N2 has been widely shown in the ERP literature as a signal sensitive to features of novelty, general surprise, and a need for cognitive control (Cavanagh and Frank, 2014; Ferrari et al., 2010; Folstein and Van Petten, 2008). It is very likely that this novelty evoked N2 activity at frontal midline sites for rewarding images superseded the +RPE signal, yet in Experiment 3 we found evidence that aspects of these signals may co-exist when novelty is controlled across reward types. If this hypothesis is accurate, it suggests a fundamental limitation in the use of EEG imaging techniques for complex ecologically valid rewards. We are currently testing this hypothesis with the aim of designing a technique to account for the inherent novelty of complex reward types.

13. Conclusion

Increasing evidence has suggested that the Rew-P reflects an axiomatic +RPE. In the current study, we investigated Rew-P modulation from different types of extrinsic feedback and found that the brain does not encode these different classes of rewards in the same way. Future studies may benefit from investigating if differences exist in the expression of reward processes for primary and secondary reward types.