Single-Trial EEG Dissociates Motivation and Conflict Processes During Decision-Making Under Risk

Abstract

In making decisions under risk (i.e., choosing whether to gamble when the outcome probabilities are known), two aspects of decision are of particular concern. The first, if gambling, is how large are potential gains compared to losses? The subjectively larger, the more rewarding to gamble. Thus, this aspect of decision-making, quantified through expected utility (EU), is motivation-related. The second concern is how easy is it to reach the decision? When subjective desirability between gambling and not-gambling is clearly different from each other (regardless of the direction), it is easier to decide. This aspect, quantified through utility distance (UD), is conflict-related. It is unclear how the brain simultaneously processes these two aspects of decision-making. Forty-five participants decided whether to gamble during electroencephalogram (EEG) recording. To compute trial-by-trial variability in EU and UD, we fit participants’ choices to models inspired by Expected-Utility and Prospect theories using hierarchical-Bayesian modeling. To examine unique influences of EU and UD, we conducted model-based single-trial EEG analyses with EU and UD as simultaneous regressors. While both EU and UD were positively associated with P3-like activity and delta-band power, the contribution of EU was around 200 ms earlier. Thus, during decision-making under risk, people may allocate their attention to motivation-related aspects before conflict-related aspects. Next, following learning the options and before reporting their decision, higher EU was associated with stronger alpha and beta suppression, while higher UD was associated with a stronger contingent-negativity-variation-like activity. This suggests distinct roles of EU and UD on anticipation-related processes. Overall, we identified time and frequency characteristics of EEG signals that differentially traced motivation-related and conflict-related information during decision-making under risk.

1. Introduction

Should one gamble when the chance of winning and losing is 50–50? Neoclassical economists consider this scenario as decision-making under risk (Von Neumann & Morgenstern, 1944). According to their Expected Utility theory, people decompose each choice into 1) utility (subjective un/desirability) and 2) probability (likelihoods of occurrence) of its possible outcomes. Utility and probability form expected utility (EU or predicted subjective desirability). Individuals then are assumed to maximize EU by accepting the gamble only when gambling has higher EU than not gambling. Cognitive psychologists, Tversky and Kahneman (1991), later discovered that when making decisions under risk in situations involving both potential gains and losses, people typically weigh potential losses higher, a phenomenon called loss-aversion. Accordingly, to accommodate loss-aversion in their Prospect theory, Tversky and Kahneman (1971) further modified neoclassical economists’ concept of EU using a function to weight the subjective undesirability involving potential losses. Inspired by Expected Utility and Prospect theories, several fMRI studies have examined the neural correlates of EU of risky choices using various gambling tasks (Glimcher & Fehr, 2014). These fMRI studies typically show positive relationships between EU of risky choices and BOLD activity in regions, such as the middle-frontal gyrus and striatum (Levy, Snell, Nelson, Rustichini, & Glimcher, 2010; Sokol-Hessner, Camerer, & Phelps, 2013), suggesting the existence of a reward-evaluation network that tracts predicted desirability of risky choices.

Yet, focusing only on EU of risky choices and BOLD activity may not provide a complete picture of neural-cognitive processes underlying decision-making under risk. For instance, in the above gambling scenario when the probability of potential gains and losses is the same (i.e., 50–50), EU can be inferred from the relative difference in utility between gambling and not gambling choices. Higher EU for gambling (compared to not gambling) indicates that predicted subjective desirability for potential gains/rewards offsets predicted subjective undesirability for potential losses/punishments, and therefore predicts higher tendencies to gamble. Thus, the EU of risky choices reflects motivation-related (i.e., accepting vs. rejecting the gamble) and valence-specific (i.e., reward vs. punishment) information (Bartra, McGuire, & Kable, 2013). However, each decision also varies in the ease of making it. For instance, in the same gambling scenario, it should be easier to make decisions when the potential losses are much larger than the potential gains (likely resulting in a decision to not gamble) and when the potential gains are much larger than the potential losses (likely resulting in a decision to gamble). This ease in making a decision is conflict-related (i.e., easy vs. difficult) and is reflected in the absolute (i.e., unsigned) difference in utility between gambling and not gambling choices. We referred to the ease of making decisions here as the utility distance (UD). Specifically, higher UD reflects higher ease of decision-making (lower choice conflict), but does not predict which choice to choose, making it valence-non-specific (i.e., UD could be higher when either potential reward or punishment is much larger than the other).

It is unknown how the brain simultaneously processes EU and UD during decision-making under risk. Due to the slowness of BOLD, fMRI may not be an ideal method to disentangle the processing of EU and UD information over time. To capture temporally precise neural dynamics associated with EU and UD, a method with higher temporal resolution like electroencephalogram (EEG) is more appropriate. The rich timing information afforded by EEG may inform us about the speed at which the brain processes EU and UD information. Furthermore, EEG provides multidimensional data (in terms of time, frequency and topography) that may capture distinct neural-cognitive processes involved in EU and UD processing (Buzsáki, 2006; Cohen, 2014; Glazer, Kelley, Pornpattananangkul, Mittal, & Nusslock, 2018). For instance, studies often link the centro-parietal P3 event-related-potential (ERP) and its related delta-band event-related-spectral-perturbation (ERSP) to stimulus evaluation and attentional resource allocation (Kissler, Herbert, Winkler, & Junghofer, 2009; Polich, 2007; Schupp, Flaisch, Stockburger, & Junghöfer, 2006). Accordingly, investigating the timing of P3 and delta-power may help assess whether the brain allocates attentional resources to motivation-related and valence-specific information (EU) sooner or later than to conflict-related and valence-non-specific information (UD) when evaluating choices under risk. Additionally, researchers associate anticipatory processes with different EEG patterns, such as the alpha-band suppression (Hughes, Mathan, & Yeung, 2013; Pornpattananangkul & Nusslock, 2016), beta-band suppression (Doñamayor, Schoenfeld, & Münte, 2012; Meyniel & Pessiglione, 2013) and slow-wave contingent-negativity variation (CNV) (Brunia, van Boxtel, & Böcker, 2011; Walter, Cooper, Aldridge, McCallum, & Winter, 1964). Crucially, recent studies suggest that these anticipatory EEG patterns seem to trace different types of information. For instance, while alpha-band and beta-band suppression are thought to be relevant to motivation-related information, the slow-wave CNV links more closely to instruction-specific information (Grent-’t-Jong & Woldorff, 2007; van den Berg, Appelbaum, Clark, Lorist, & Woldorff, 2016). Thus, the frequency of EEG may also inform us about different anticipatory processes that are associated with the processing of EU and UD.

To this end, this study employed model-based single-trial analyses on EEG activity elicited during decision-making under risk that involved potential gains and losses. Specifically, we fit participants’ choices to computational models inspired by Expected-Utility and Prospect theories (Tversky & Kahneman, 1992; Von Neumann & Morgenstern, 1944) using hierarchical-Bayesian parameter estimation (Gelman, 2013; Kruschke, 2014). This method allowed us to compute trial-by-trial variability in EU and UD, and to have both EU and UD as simultaneous regressors in the same model that predicted trial-by-trial EEG activity. Having EU and UD in the same model enabled us to examine unique influences of EU and UD on EEG during decision making under risk. To control for collinearity of EU and UD, we designed the task and computational models such that EU (the relative difference in utility between the two choices) and UD (the absolute difference in utility between the choices) are orthogonal to each other.

We expected several EEG patterns with distinct characteristics in topography, time and frequency to be associated with EU and UD. First, consistent with their roles in stimulus evaluation and attentional resource allocation (Kissler et al., 2009; Polich, 2007; Schupp et al., 2006), the P3 and delta-band power are usually enhanced by motivational, reward-related anticipatory cues (Goldstein et al., 2006; Pornpattananangkul & Nusslock, 2015, 2016) and modulated by valence of emotional pictures (Cano, Class, & Polich, 2009). Moreover, when engaging in perceptual decision-making, people usually have a stronger P3 for stimuli that are easier to perceive, and therefore, can be judged with ease (O’Connell, Dockree, & Kelly, 2012). Thus, we expected both motivation-related, valence-specific EU and conflict-related, valence-non-specific UD to influence the P3 and delta-band power. More specifically, we expected both higher EU (higher motivation to gamble) and higher UD (higher ease of making decisions) to be associated with higher P3 and delta-band power. The timing at which EU and UD modulated the P3 and delta-band power would reveal whether the brain allocates attentional resources to EU and UD at a similar speed. Next, studies show alpha-band (Hughes et al., 2013; Pornpattananangkul & Nusslock, 2016) and beta-band (Doñamayor et al., 2012; Meyniel & Pessiglione, 2013) suppression when people anticipate seeing and/or prepare actions for reward-related stimuli. Thus, we expected motivation-related EU, but not conflict-related UD, to enhance alpha and beta suppression prior to responding whether to gamble. This pattern would reveal distinct roles of EU and UD on anticipation-related processes during decision-making under risk as indexed by EEG at different frequency.

2. Material and Methods

2.1. Participants

Fifty-six right-handed (< 18, Chapman Handedness Scale; Chapman and Chapman (1987) native English-speaking undergraduates received partial course credit for their participation. Participants also earned an additional monetary bonus based on the outcome of the tasks (see below). Before submitting participants’ behavioral data to the hierarchical Bayesian analysis, we discarded data from eight participants due to poor model-fit indexes (see below). We then discarded data from three additional participants due to excessive EEG artifact. This left 45 participants (24 females; age M = 18.70 years, SD = .88) in the final EEG analysis. Participants had no neurological history of head injury and were not taking psychotropic medications. All participants gave informed consent before the study. The study was approved by the IRB at Northwestern University and was conducted in accordance with the Declaration of Helsinki.

2.2. Experimental Design

The primary task for this study was a modified Mixed-Gamble Loss-Aversion task. Before starting this task, however, participants completed a separate guessing task (Dunning & Hajcak, 2007). We used this guessing task to provide every participant with an identical financial fund for the following Mixed-Gamble Loss-Aversion task and to provide participants with a sense of ownership over their initial fund. In this guessing task, participants had to select between two different choices during each of the 120 trials in this guessing task, and for each trial they could either win 80¢, lose 40¢ or not gain/lose any amount. Unbeknownst to them, we fixed the total accumulative earning at $12 for every participant. However, we fully randomized the order of the outcome across participants.

We modified the Mixed-Gamble Loss-Aversion task used earlier in fMRI and patient studies (Brown et al., 2013; De Martino, Camerer, & Adolphs, 2010; Tom, Fox, Trepel, & Poldrack, 2007) (see Figure 1). In each trial, participants had to decide whether to accept or reject a gamble. If participants accepted the gamble, they would have a 50% chance to gain a specified amount (ranging from $1.5 to $9 in $.50 increment) and a 50% chance to lose a specified amount (ranging from $.75 to $4.5 in $.25 increment). If participants decided to reject the gamble, then they would not gain or lose any amount in this trial. In total, there were 256 trials, covering all of the unique combinations of gain (16 possibilities) and loss (16 possibilities) amounts. Following previous research (Tom et al., 2007), we did not show the outcome of each gamble. This is because descriptive-based decision-making may be altered by the presentation of the outcomes (Hertwig, Barron, Weber, & Erev, 2004), and an outcome of the preceding trial may influence the delta-band power locked to the decision stimulus in the current trial (Opitz, Cavanagh, & Urry, 2015). Accordingly, we were able to focus on decision-making processes that were not confounded by outcome-induced processes. We told participants to treat every decision as equally important since we would randomly pick one trial at the end of the experiment for which they will receive reimbursement.

Figure 1. Schematic representation of the Mixed-Gamble Loss-Aversion task.

[Gain] represents one of the possible 16 gain amounts, from $1.5 to $9 in $.50 increment. Likewise, [Loss] represents one of the possible 16 loss amounts, from $.75 to $4.5 in $.25 increment. “RT (> 1s)” represents the fact that these screens were shown for 1 s if participants pressed earlier than 1 s. If they pressed later than 1 s, these screens would be terminated with a button-press. Thus, these screens presented for at least 1 s. ITI = inter-trial interval; ISI = inter-stimulus interval.

Each trial in the Mixed-Gamble Loss-Aversion task began with a 1-s inter-trial interval, during which a fixation cross appeared. Then, participants saw the Gain screen where we presented a potential gain amount in green at the center of the screen. We asked participants to press a middle key on a button box whenever they thought they could memorize the gain amount for that trial. Next, another fixation cross appeared for 2 s, followed by the Loss screen where we presented a potential loss amount in red. Same as before, participants had to press the middle key to proceed to the next screen when they thought they could memorize the loss amount for that trial. Subsequently, another fixation cross appeared for 2 s. Then, participants saw a Response screen. Here, participants had to press either a left or right key to indicate whether they accepted or rejected the gamble, respectively. The Gain, Loss and Response screens were terminated with a button-press if participants pressed the key after 1 s. If they pressed the key earlier than 1 s, the screen would be shown for 1 s. We asked participants to press with their right index finger, regardless of the key, throughout the task.

Unlike previous fMRI and behavioral studies (Brown et al., 2013; De Martino et al., 2010; Tom et al., 2007) that showed the gain and loss amounts and asked for a response all on one screen, we separately showed Gain, Loss and Response screens. This allowed us to isolate motor movement from the perception of choices as well as to have a better control of EEG artifact (e.g., eye-movements). We used the onset of the Loss screen as the time-locking event for EEG analyses because this screen was when participants had all of the necessary information for making a decision. Furthermore, our computational models (see below) required information that were available only after participants saw both the Gain and Loss screens. Participants would not have enough information to compute either EU (reflecting the relative difference in utility see equation #6) or UD (reflecting the absolute difference in utility see equation #16) after seeing the Gain screen alone. Thus, time-locking our analyses to the Loss screen onset allowed us to test the speed at which the brain processed EU and UD information using the same locking event (i.e., the Loss screen).

2.3. Computational modeling of choice data

We implemented similar participant exclusion procedures used previously (Brown et al., 2013) in the Mixed-Gamble Loss-Aversion task to ensure that every participant took both gain and loss information into consideration when deciding whether to gamble. First, for each participant, we fit his/her trial-by-trial choices to a logistic regression with the potential gain and loss on that trial as regressors, using a ‘nlmefit’ command in Matlab. We then excluded participants whose regression coefficient for either the potential gain or loss did not significantly differ from zero. We also excluded participants whose model fit index (Cox-Snell R²) (Cox & Snell, 1989) was exceptionally low (< .1). This resulted in us excluding eight out of 56 participants, which is consistent with previous research (Brown et al., 2013; Sokol-Hessner et al., 2013).

We next employed the hierarchical Bayesian analysis (HBA) approach (Gelman, 2013; Kruschke, 2014) to fit trial-by-trial choices of all participants to two models inspired by Expected Utility and Prospect theories (Tversky & Kahneman, 1992; Von Neumann & Morgenstern, 1944). The first “linear-value” model assumes a linear value function (Brown et al., 2013; De Martino et al., 2010; Tom et al., 2007):

$$u(x)=\{\begin{array}{cc}\left|x\right|& ifx\ge 0\\ -\lambda *\left|x\right|& ifx<0\end{array}$$ (1)

“u” is the utility of the potential monetary outcome. “x” is the objective value (i.e., the amount shown on the screen) of the potential outcome. “λ” (lambda) is a relative multiplicative weighting of loss to gain amounts. As a participant-specific free parameter, the lambda value indicates individual differences in loss aversion: 1= loss/gain-neutral, <1= loss-averse, >1= gain-seeking. The second “curvilinear-value” model assumes a curvilinear value function (Sokol-Hessner et al., 2013; Sokol-Hessner et al., 2009; Sokol-Hessner, Raio, Gottesman, Lackovic, & Phelps, 2016):

$$u(x)=\{\begin{array}{cc}|x{|}^{\rho }& ifx\ge 0\\ -\lambda *|x{|}^{\rho }& ifx<0\end{array}$$ (2)

That is, the “curvilinear-value” model adds another participant-specific “ρ” (rho) parameter to the “linear-value” model. Rho captures the curvature of the utility function across gain and loss amounts, and its value reflects individual differences in risk attitude: 1= risk-neutral, <1= risk-averse for gains and risk-seeking for losses, >1= risk-seeking for gains and risk-averse for losses.

For both models, to calculate the expected utility for each choice¹, we additionally assumed that our participants linearly combined utilities and probabilities (Mosteller & Nogee, 1951):

$$e{u}_c={\sum }_i{p}_i*u\left(x_i\right)$$ (3)

“eu_c” is the expected utility of each choice. “c” is a member of the set of choices. “p_i” is the probability of obtaining an outcome “x_i”. In our experiment, the choice options included 1) accepting and 2) rejecting the gamble:

$$e{u}_{accepting}=p_{gain}*u\left(x_{gain}\right)+p_{loss}*u\left(x_{loss}\right)$$ (4)

$$e{u}_{rejecting}=p_{sure}*u\left(x_{sure}\right)=1*0=0$$ (5)

We then calculated the overall expected utility for a particular trial as:

$$EU=e{u}_{accepting}-e{u}_{rejecting}=p_{gain}*u\left(x_{gain}\right)+p_{loss}*u\left(x_{loss}\right)-0=.5*u\left(x_{gain}\right)+.5*u\left(x_{loss}\right)$$ (6)

A more positive EU value indicates a higher expected utility to accept, than to reject, the gamble for that trial. To assess how individuals transformed EU into actual choices, we then entered EU calculated by both models to a softmax (logit) function (Luce, 1959). This function predicts a probability (P) of accepting the gamble based on EU:

$$P(accepting)={\left(1+e^{-\tau *EU}\right)}^{-1}$$ (7)

The softmax function has another participant-specific free-parameter, the inverse temperature “τ” (tao), that reflects individual differences in behavioral consistency. Higher values represent greater consistency across trials.

To estimate free parameters, we followed the HBA framework used in the hBayesDM R package (Ahn, Haines, & Zhang, 2017) and previous Loss-Aversion research (Sokol-Hessner et al., 2016). HBA allows estimations of full posterior distributions of parameter values and also enables the whole group tendencies to inform each participant’s parameter values. Several studies have shown that HBA produces a more accurate parameter recovery than a conventional, non-heirarchical Maximum Likelihood Estimation (MLE) method (Ahn, Krawitz, Kim, Busemeyer, & Brown, 2011; Katahira, 2016; Lee, 2011). To implement HBA, we used the Hamiltonian Monte Carlo (HMC) algorithm to run Markov chain Monte Carlo (MCMC) sampling in Stan 2.16 (Carpenter et al., 2017) via R 3.3.3 (R Core Team, 2017). Specifically, each participant’s parameters were assumed to be drawn from group-level distributions. We employed standard normal and half-Cauchy prior distributions for group-level means (μ) and standard deviations (σ), respectively (Gelman, 2006). Following previous recommendation (Ahn et al., 2017), we bounded the lambda (λ) and rho (ρ) parameters between 0 and 5 and between 0 and 2, respectively using an inverse-Probit transformation. Additionally, we bounded the tao (τ) parameter to be a positive value, using an exponential transformation:

$${\mu }_{{\lambda }^{\prime }},{\mu }_{{\rho }^{\prime }}{\mu }_{{\tau }^{\prime }}~\text{ Normal }(0,1)$$ (8)

$${\sigma }_{{\lambda }^{\prime }},{\sigma }_{{\rho }^{\prime }},{\sigma }_{{\tau }^{\prime }}~\text{ half }-\text{ Cauchy }(0,5)$$ (9)

$$\lambda \prime ~\text{ Normal }\left({\mu }_{{\lambda }^{\prime }},{\sigma }_{{\lambda }^{\prime }}\right)$$ (10)

$$\lambda ={\text{ Probit }}^{-1}\left(\lambda \prime \right)*5$$ (11)

$$\rho \prime ~\text{ Normal }\left({\mu }_{{\rho }^{\prime }},{\sigma }_{{\rho }^{\prime }}\right)$$ (12)

$$\rho ={\text{Probit}}^{-1}\left(\rho \prime \right)*2$$ (13)

$$\tau \prime ~\text{ Normal }\left({\mu }_{{\tau }^{\prime }},{\sigma }_{{\tau }^{\prime }}\right)$$ (14)

$$\tau =\text{Exp}\left(\tau \prime \right)$$ (15)

We used four MCMC chains. For each chain, we randomized its initial value and drew 3,000 samples including 1000 burn-in samples. This left a total of 8,000 samples across chains. To evaluate the convergence of the MCMC chains, we visually evaluated the trace plots of the group-level (hyper) parameters, as well as checked the $\widehat{\text{R}}$ statistic computed from the Gelman-Rubin test (Gelman & Rubin, 1992). To examine whether the linear-value or curvilinear-value model explained the choices better, we computed the Leave-One-Out Information Criterion (LOOIC) (Vehtari, Gelman, & Gabry, 2017). Lower values of LOOIC indicates a better fit of the model to the data.

For our main analyses, we focused on the trial-by-trial changes in EEG activity as a function of the overall EU computed using parameters from the better-fit model between the linear-value and curvilinear-value models. As shown earlier, higher EU reflects a higher propensity to accept, than to reject, the gamble in that trial (see equation #6). Because 1) rejecting the gamble in our experiment always means not gaining or losing any amount and 2) the probability of obtaining the potential gain and loss amount is the same, EU becomes the relative difference in the utility between gain and loss amounts in the gambling option. Accordingly, a higher EU reflects a higher utility to the gain, compared to the loss, amount in the gambling option.

In addition to computing EU as a trial-by-trial parameter related to motivational processes (i.e., propensity to accept the gamble), we also computed utility distance (UD) as a trial-by-trial parameter related to conflict processes. We defined UD as the absolute (unsigned) difference in the utility between accepting and rejecting the gamble. UD reflects the ease of making a decision and does not predict whether people would accept or reject the gamble. Lower UD values mean that the utility between the two options are close to each other, making the decision more difficult (i.e., high choice conflict). Higher UD values mean that the utility between the two options are far from each other, making it easier to either accept or reject the gamble (i.e., low choice conflict). Note that the distribution of EU across trials for each participant may not be centered at 0. This depends on how loss-aversive the participants were: higher loss-aversive participants had a more negative EU than lower loss-aversive participants. This may bias UD. To correct for this bias, we applied participant-mean centering to UD:

$$\begin{array}{l}U{D}_{(trial,participant)}=|E{U}_{(trial,participant)}-\overline{E{U}_{(.,participant)}}|\hfill \\ =|e{u}_{accepting}{}_{(trial,participant)}-e{u}_{rejecting}{}_{(trial,participant)}-\overline{E{U}_{(.,participant)}}|\hfill \\ =|p_{gain}\ast u{\left(x_{gain}\right)}_{trial,participant}-p_{loss}\ast u{\left(x_{\text{loss}}\right)}_{trial,participant}-0-\overline{EU{(accepting)}_{.,participant}}|\hfill \\ =|.5\ast u{\left(x_{gain}\right)}_{trial,participant}-.5\ast u{\left(x_{loss}\right)}_{trial,participant}-\overline{EU{(accepting)}_{.,participant}}|\hfill \end{array}$$ (16)

Based on these definitions of EU (the relative difference in utility between the two choices) and UD (the absolute difference in utility between the choices), EU and UD were orthogonal to each other by design. This allowed us to avoid a collinearity problem when having EU and UD as simultaneous regressors in our regression models that seek to explain trial-by-trial variability in EEG activity (see below).

It is important to compare our approach of computing subjective, trial-by-trial parameters (i.e., EU and UD) using participant-specific free-parameters (i.e., λ, ρ and τ) that reflect individual differences in the choice pattern to an approach of deriving objective, trial-by-trial parameters without using participant-specific free-parameters. The objective, trial-by-trial parameter that is similar to EU is Pascal and Fermat’s expected value (EV) (Bernoulli, 1954; Trepel, Fox, & Poldrack, 2005; Von Neumann & Morgenstern, 1944):

$$e{v}_c={\sum }_i{p}_i\ast x_i$$ (17)

“ev_c” is the expected value of each choice. “c” is a member of the set of choices. “p_i” is the probability of obtaining an outcome “x_i”. That is, compared to eu_c (see equation #3), the potential outcome “x_i” in ev_c is not transformed into utility but multiplies directly to the probability of obtaining the outcome. For our experiment, the overall expected value (EV) for a particular trial can be calculated as:

$$\begin{array}{l}EV=e{v}_{accepting}-e{v}_{rejecting}=p_{gain}\ast x_{gain}+p_{loss}\ast x_{loss}-0\hfill \\ =.5\ast x_{gain}+.5\ast x_{loss}\hfill \end{array}$$ (18)

Thus, a higher EV value indicates that accepting the gamble is a choice with a better objective value in the long run. Accordingly, EV should capture a propensity to accept the gamble, similar to EU. Yet, unlike EU, EV does not take individual differences in loss aversion (λ) and risk attitude (ρ) into account. Rather, EV assumes a) that every individual should behave the same, b) that they weigh loss and gain amounts similarly, and c) that they linearly change their propensity to accept the gamble as a function of the magnitude of the potential outcome.² Similarly, we can also calculate the objective trial-by-trial parameter that is similar to UD using EV. We call this value distance (VD).

$$V{D}_{(trial,participant)}=|E{V}_{(trial,participant)}-\overline{E{V}_{(.,participant)}}|=|.5\ast x_{gai{n}_{trial,participant}}-.5\ast x_{los{s}_{trial,participant}}-\overline{EV{(accepting)}_{.,participant}}|$$ (19)

Accordingly, VD reflects how far apart the objective value is between the two choices. Thus, VD should also reflect the ease of making a decision. Lower VD values mean that the objective values between the two options are close to each other, making the decision difficult. Figure 2 shows plots of subjective trial-by-trial parameters (i.e., EU and UD), objective trial-by-trial parameters (i.e., EV and VD) and behavioral measures (i.e., choice and decision RT) at each level of potential gain and loss. These plots can visually show how close the pattern of EU and EV are to the pattern of choices, as well as how close the pattern of UD and VD are to the pattern of decision RT.

Figure 2.

Heatmaps showing averaged objective (Expected Value, Value Distance) and subjective (Expected Utility, Utility Distance) trial-by-trial parameters and behavioral measures (percentage of participants accepting the gamble, decision RT) at each level of potential gain and loss.

2.4. Relationship between Utility Distance and Decision Reaction Time

Here we sought to validate that UD reflects the ease of making a decision, as conceptualized above using decision reaction time (RT). If higher UD reflects a greater ease in making a decision, participants should take less time to respond (i.e., faster RT). First, we transformed RT to the Response screen (see Figure 1) using natural logarithm (ln) to suppress the influence of extremely slow trials. We then removed trials that were faster than 1.5 inter-quartile ranges (IQRs) from the first quartile (4.92 ln(ms) or 137.51 ms). We employed these pre-processing steps to ensure normality of the RT data. Thereafter, we employed a linear mixed-effects (LME) regression (Barr, Levy, Scheepers, & Tily, 2013) using the lme4 package (Bates, Mächler, Bolker, & Walker, 2015) in R 3.5.1 (R Core Team, 2017) to test the effects of EU and UD on ln(RT) at each trial. Specifically, we entered EU and UD as fixed effects. For random effects, we entered random intercepts for participants and by-participant random slopes for both EU and UD, following the recommendation to include the maximal random effects structure justified by the design (Barr et al., 2013) with the following syntax formula:

$$\text{lme}4::\text{lmer}\left(\text{ln}\left(\text{RT}\right)~\text{EU}+\text{UD}+\left(1+\text{EU}+\text{UD|participant}\right),\text{data}=\text{data}\right)$$

To access the statistical significance of each parameter, we computed 95% confidence intervals (CI_95%) using the bootstrap percentile method (1,000 iterations) via the confint.merMod command.

2.5. EEG recording and preprocessing

We collected continuous EEG data with a sampling rate of 500 Hz (DC to 100 Hz on-line, Neuroscan SynAmps RT) from inside an electro-magnetic shielded booth. We used sixty-four Ag/AgCl electrodes placed according to the extended 10–20 system (Electro-Cap International Inc.) and recorded HEOG and VEOG with four eye electrodes. We employed a left mastoid for an on-line reference and kept the reference impedance below 5 kΩ and the scalp and eye electrodes below10 kΩ. We analyzed EEG data using Matlab 2015b (MathWorks Inc.) with EEGLAB 14.1.0b (Delorme & Makeig, 2004) and LIMO 2 (Pernet, Chauveau, Gaspar, & Rousselet, 2011) toolboxes. We first downsampled the data to 250 Hz, re-referenced to linked mastoids and applied high-pass filtering of .01 Hz. Note that for all of the off-line filters, we set the filter order and transition band width according to the heuristic default-estimation algorithm in the ‘eegfiltnew’ command (Widmann, Schröger, & Maess, 2015). We then manually removed large non-stereotypical artifacts and bad channels from the continuous data. Subsequently, we corrected for stereotypical artifacts using ICA with the “binica” command, based on similar procedures recommended earlier (Bigdely-Shamlo, Mullen, Kothe, Su, & Robbins, 2015). To identify artifact components, we employed the IC_MARC classifier (Frølich, Andersen, & Mørup, 2015) and visually inspected and reject artifact components.

2.6. Statistical analysis of EEG data

We separately epoched EEG data for wide-band event-related potential (ERP) and for frequency-specific event-related spectral perturbation (ERSP). For wide-band ERP, we epoched the data from −200 to 3000 ms relative to the Loss-screen onset. To reduce slow-wave artifacts, we applied a linear detrend algorithm on a wider (−2000 to 5000 ms) window. We also implemented a low-pass filter of 30 Hz and a baseline correction using −200 to 0 ms time-window. We then rejected the preprocessed epochs that contained EEG activity ±75 μV, which left 216.44 epochs on average (SD = 28.48) for each participant.

For frequency-specific ERSP, we epoched the data from −2000 to 4678 ms. Similar to ERP, we applied a linear detrend algorithm and a low-pass filter of 31 Hz. We then rejected the epochs that contained EEG activity ±75 μV, which left 221.04 epochs on average (SD = 25.33) for each participant. We then employed band-pass filters and Hilbert transformation to separate EEG power into five frequency bands: delta (.01–3 Hz), theta (4–7 Hz), alpha (8–13 Hz), low beta (14–20 Hz) and high beta (21–30 Hz). Note that for the delta band, we only applied low-pass filtering of 3 Hz given that we already applied high-pass filtering of .01 Hz on the continuous, preprocessed EEG data. To reduce potential edge effects, we then shortened the Hilbert-transformed ERSP epochs to −332 to 3000 ms relative to the Loss-screen onset. Subsequently, we applied the welding baseline method (Ciuparu, Mureşan, & Michel, 2016) to normalize single-trial EEG power. Specifically, for each frequency band and participant, we concatenated EEG power in the baseline period (between −332 to −132 ms) across all trials, and computed means (MB_{freq,participant}) and standard deviations (SDB_{freq,participant}) of this concatenated baseline. We then normalized EEG power at each time point in a particular trial for each frequency band and participant (X_{time-point,trial,freq,participant}) using a pseudo z-scoring procedure, giving rise to normalized values in a z-score unit (Z(X _{time-point,trial,freq,participant})):

$$\text{Z}\left({\text{X}}_{\text{time}-\text{point,trial,freq,participant}}\right)=\frac{{\text{X}}_{\text{time}-\text{point,trial,freq,participant}}-M{\text{B}}_{\text{freq,participant}}}{SD{\text{B}}_{\text{freq,participant}}}$$ (20)

Recent research shows that this method provides a better control for biases compared to other single-trial normalization procedures (Ciuparu et al., 2016).

To examine the extent to which EU and UD were uniquely related to single-trial EEG activity, we employed mass-univariate, hierarchical, generalized linear-models (GLMs) as implemented in the LIMO 2 toolbox (Pernet et al., 2011). For the first-level (i.e., trial-level) analysis, we simultaneously entered trial-by-trial EU and UD as predictors for each participant’s single-trial EEG activity on each electrode and time-point, separately for wide-band ERP and each frequency-specific ERSP. Thus, the parameter values of EU and UD represent the unique variance of EEG activity accounted for by each predictor. For the second-level (i.e., subject-level) analysis, we used 1,000 bootstrapping iterations to conduct statistical tests on beta coefficients of EU and UD computed from each participant’s data (Wilcox, 2016). Specifically, we employed bootstrapped one-sample t-tests to examine if the effect of EU and UD on EEG activity at each time point and electrode was consistent across participants (Pernet et al., 2011). To control for multiple comparisons of tests done across time points and electrodes, we used the spatial-temporal clustering approach (Maris & Oostenveld, 2007) and set the family-wise error (FWE) rate at α = .05. Research shows that using bootstrapping as a robust, non-parametric approach with iterations higher than 800 along with the spatial-temporal clustering correction can sufficiently control for FWE at .05 level (Pernet, Latinus, Nichols, & Rousselet, 2015).

To visualize how EU and UD modulated EEG and ERSP waveforms, we also conducted bin-like analyses. First, we separated 256 trials into eight bins based either on EU or UD values, and selected the bins that had highest and lowest values of EU and UD³. Subsequently, we conducted a permutation test with 1,000 randomizations and α = .05 at each time point (Delorme & Makeig, 2004) to find the differences in EEG activity between bins of high and low values of EU and UD. We used the false-discovery rate (FDR) (Benjamini & Hochberg, 1995) to control for multiple comparisons. We then plotted significant time points along with the mean and standard error of EEG activity from each bins using “std_plotcurve” command (Delorme & Makeig, 2004).

To help further visualize the effect of EU on single-trial EEG, we investigated the association between a gambling choice and EEG activity. Here we separated trials into those in which participants accepted the gamble (Accepting) and those in which they rejected the gamble (Rejecting). We then applied the same permutation analysis with the bin-like analysis above. Given our conceptualization of EU as reflecting motivation-related processes, the differences in EEG activity between Accepting and Rejecting trials should bear resemblance to the effect of EU. Accordingly, we focused this analysis only on frequency bands that that showed the effect of EU (but see Supplementary Figure 3 for the analyses at other indices).

Similarly, to help visualize the effect of UD on single-trial EEG, we investigated the association between decision RT to the Response screen and EEG activity. First, we used natural log to transform RT, resulting in ln(RT), and excluded the trials where participants responded faster than 1.5 IQRs from the first quartile, same as above. Given that ln(RT) is a continuous variable, we employed the same mass-univariate, hierarchical GLM approach as our main single-trial analysis (Pernet et al., 2011), except that we entered ln(RT) (as opposed to EU and UD) as a predictor in the first-level analysis. A higher ln(RT) value (i.e., slower response) should reflect an uneasiness in making a decision, in a manner that is similar to lower UD (i.e., higher choice conflict). Thus, we expected to see the association between ln(RT) and EEG activity to be at a similar time-point, frequency and topography as the effect for UD, but in the opposite direction. Accordingly, we focused our analysis of ln(RT) only on frequency bands that showed the effect of UD.

3. Results

3.1. Computational modeling of choice data

On average, each participant decided to gamble on 40.19 % of all trials (SD = 21.62). LOOIC of the curvilinear-value model (6349.31) was lower than that of the linear-value (6390.35) model, suggesting that the curvilinear-value model provided a better fit it to the choice data. Thus, we used parameters from the curvilinear-value model for analyses. The trace plots in our data confirmed excellent mixing of MCMC samples for the parameters from the curvilinear-value model. Moreover, the range of $\widehat{\text{R}}$ values from all parameters were from 1 to 1.01, suggesting that our MCMC chains converged well. The group-level parameters were estimated as follows: loss-aversion lambda (λ) mean recovered = 2.33, SD = .19; risk-attitude rho (ρ) mean recovered = .85, SD = .06; inverse-temperature tau (τ) mean recovered = 2.95, SD = .48.

Figure 2 presents a heatmap of the percentage of participants accepting the gamble at each level of potential gain and loss. Participants were more likely to accept the gamble when the potential gain was high and the potential loss was low. Figure 2 also presents heatmaps of Expected Value (EV) and Expected Utility (EU). Unlike EV, the middle values of EU were at the diagonal of the heatmap, similar to that of the percentage of participants accepting the gamble. This suggests that having participant-specific free-parameters (i.e., λ, ρ and τ) in the models to compute a subjective trial-by-trial parameter (EU) allowed us to trace propensity to accept the gamble more closely than using an objective trial-by-trial parameter (EV).

2.4. Relationship between Utility Distance and Decision Reaction Time

The average decision RT was 585 ms (SD = 165). Figure 2 presents the heatmap of decision RT at each level of potential gain and loss. Participants were faster both a) when the potential gain was high and the potential loss was low and b) when the potential gain was low and the potential loss was high. Figure 2 also shows the heatmaps of Value Distance (VD) and Utility Distance (UD). Unlike VD, the lowest values of UD were at the diagonal of the heatmap, where participants were also slowest. This pattern is consistent with the idea that low values of UD reflect an uneasiness in making a decision (high choice conflict). Accordingly, similar to the conclusion drawn from the EU heatmap, the UD heatmap suggests that having participant-specific free-parameters (i.e., λ, ρ and τ) in the models allowed us to better trace the ease of making a decision using a subjective trial-by-trial-parameter UD.

Table 1 and Figure 3 show the effects of EU and UD on ln(RT) based on the LME regression. The fixed effects show that ln(RT) had a statistically significant (i.e., CI_95% did not include zero) negative relationship with UD, but not EU. Higher UD was associated with lower (faster) ln(RT). This suggests that participants were faster in making a decision when the utility between the two options were far away from each other, which confirms our conceptualization of using UD to reflect ease in making a decision. The random effects show that the correlations between the slope and intercept were not significant for both EU and UD. Note that, despite being statistically non-significant, these correlations were kept in the model because assuming the correlations between the slope and intercept fit the data significantly better than assuming no correlations (χ²(3) = 9.32, p =.025).

Table 1.

The effects of Expected Utility (EU) and Utility Distance (UD) on decision reaction time (ln(RT)) based on linear mixed-effects regression.

Fixed Effects
	ln(RT) in ln(ms)
Predictors	Estimates	CI95%
(Intercept)	6.17	6.10–6.23
EU	0.00	−0.01–0.01
UD	−0.08	−0.10–−0.05
Random Effects
Residual (σ2)	0.37 (SD = 0.61)
Within-participant variance (τ00)	0.05 (SD = 0.22)
Between-participant variance (τ11) participant.EU	0.0009 (SD = 0.0293)
Between-participant variance (τ11) participant.UD	0.0048 (SD = 0.0694)
Random-slope-intercept correlation (σ01) participant.EU	−0.08(CI95%= −0.56 – 0.37)
Random-slope-intercept correlation (σ01) participant.UD	−0.35 (CI95% = −0.64 – 0.06)
Intra-class correlation coefficient participant	0.11
Observations	11367
Marginal R2 / Conditional R2	0.009 / 0.118

Figure 3. The effects of Expected Utility (EU) and Utility Distance (UD) on decision reaction time (ln(RT)) based on linear mixed-effects regression.

The shaded areas indicate 95% confidence intervals.

3.2. EEG results

Figure 4 shows ERP and ERSP time-frequency plots and their topographies at different time windows collapsing across all levels of EU and UD. These plots display overall changes in ERP amplitude following the stimulus onset: 1) enhancement in central-parietal positive activity at around 250–750 ms followed by 2) enhancement in sustained-frontal negative activity starting around 1250 ms. They also show overall changes in ERSP power following the stimulus onset: 1) enhancement in central-parietal delta-band (.01–3 Hz) power at around 500 ms, 2) enhancement in frontal-midline theta-band (4–7 Hz) power at around 250 ms, 3) suppression in alpha-band (8–13 Hz) power starting at occipital sites around 250 ms and later at motor areas, and 4) suppression in low (14–20 Hz) and high (21–30 Hz) beta-band power mainly at motor areas starting around 250 ms. Figures 4–8 further demonstrate the extent to which EU and UD captured the changes in wide-band single-trial ERP amplitude and specific-band, single-trial ERSP power.

Figure 4. ERP (Figure 4a) and ERSP Time-Frequency (Figure 4b) plots and their topographical maps (Figure 4c), collapsed across trials (i.e., all levels of Expected Utility and Utility Distance).

The shaded areas in Figure 4a indicate across-participant 95% confidence intervals of the ERP waveform. To visually inspect ERSP at different frequencies, collapsed across all levels of EU and UD, we created time-frequency plots (Figure 4b) using a modified complex sinusoidal wavelet that increased from 3 cycles at 1 Hz to 10 cycles at 30 Hz. We used a linear space for both frequency (at every 1 Hz) and time points (2 ms). We removed half of the sliding window of the lowest frequency at the edges (i.e., the beginning and the end) of each epoch to reduce the potential edge effects, leaving an epoch between −332 and 3000 ms after edge removal. We then divided EEG power by the average power during the baseline period (between −332 and −132 ms) across trials at each frequency. The topographical maps (Figure 4c) were based on Hilbert transformed data that was also used for statistical analyses.

Figure 8. Alpha-band (8 – 13 Hz) ERSP as a function of Expected Utility (EU), Utility Distance (UD) and Accepting and Rejecting trials.

Figure 8a,c: The effects of EU (Figure 8a) and UD (Figure 8c). The left plots show parameter values at each electrode and time point. Here, we only plotted parameter values that passed spatial-temporal clustering (α = .05). See caption in Figure 5 for the arrangement of electrodes in these left plots. The middle plots show parameter values at a specific electrode. The shaded areas indicate bootstrap 95% confidence interval. The red line at the bottom indicates significant time points that passed spatial-temporal clustering (α = .05). The right plots show topographical maps of uncorrected parameter values at a specific time window. Figure 8b,d: Alpha-band averaged ERSP waveforms, comparing between bins with high and low EU (Figure 8b), between Accepting and Rejecting trials (Figure 8b) and between bins with high and low UD (Figure 8d). The shaded areas indicate between-condition standard error. The red line at the bottom indicates significant time points that passed FDR correction (q = .05). The blue line at the bottom indicates significant time points that had permutation p-value < .05, but did not pass FDR correction. See Supplementary Figure 3 for the analyses at other midline electrodes.

Figure 5 shows the effect of EU (Figure 5a,b) and UD (Figure 5c,d) on wide-band single-trial ERP. Higher EU was associated with a more positive ERP activity in the central-parietal area at around 350–550 ms. Similar to EU, higher UD was associated with more positive ERP activity in the central-parietal area, but at a later time window at around 500–700 ms. More specifically, these effects of EU and UD on positive ERP activities occurred at the time window and topography of the P3 component (Polich, 2007). In addition to this positive ERP activity, higher UD was also related to sustained, negative ERP activity at around 1200–3000 ms. Within this 1200–3000 ms window, the effect of UD on negative ERP activity was propagated from frontal sites to areas throughout the scalp. The bin-like analyses (Figure 5b and 5d) confirm the patterns observed using the hierarchical GLMs approach (Figures 5a and 5c). We also found more positive wide-band ERP activity in Accepting trials, compared to Rejecting trials, around 200–1000 ms at PZ (Figure 5b).

Figure 5. Wide-band single-trial ERP as a function of Expected Utility (EU), Utility Distance (UD) and Accepting and Rejecting trials.

Figure 5a,c: The effect of EU (Figure 5a) and UD (Figure 5c). The left plots show parameter values at each electrode (Y axis) and time point (X axis). The order of the electrodes is arranged based on their location in two steps. First, eight groups of electrodes are created based on the anterior-posterior dimension, so that anterior and posterior electrodes are at the top and bottom of the Y axis, respectively. Each group of electrodes is separated from each other by dotted lines. Second, within each group, the electrodes are further arranged based on the left-right dimension, so that left, midline and right electrodes are at the top, middle and bottom of the Y axis, respectively. Only the names of the midline electrodes are shown here, and only parameter values that passed spatial-temporal clustering (α = .05) are plotted. The middle plots show parameter values at a specific electrode. The shaded areas indicate bootstrap 95% confidence interval. The red line at the bottom indicates significant time points that passed spatial-temporal clustering (α = .05). The right plots show topographical maps of uncorrected parameter values at a specific time window. Figure 5b,d: Wide-band averaged ERP waveforms, comparing between bins with high and low EU (Figure 5b), between Accepting and Rejecting trials (Figure 5b) and between bins with high and low UD (Figure 5d). The shaded areas indicate between-condition standard error. The red line at the bottom indicates significant time points that passed FDR correction (q = .05). The blue line at the bottom indicates significant time points that had a permutation p-value < .05, but did not pass FDR correction. See Supplementary Figure 3 for the analyses at other midline electrodes.

Figure 6 shows that both EU (Figure 6a) and UD (Figure 6b) were related to changes in delta-band power. These relationships bare a similar pattern with the relationships EU and UD had with the positive wide-band ERP (i.e., the P3-like component) in terms of both time windows and electrode locations. Specifically, higher EU was related to stronger delta-band power and more positive wideband ERP activity, both in the central-parietal area at around 400 ms. Similarly, higher UD was associated with stronger delta-band power and more positive wide-band ERP activity, both in the central-parietal area at around 600 ms. The bin-like analyses (Figure 6c) confirm the patterns using the hierarchical GLMs approach (Figure 6a,b). We also found stronger delta-band power in Accepting trials, compared to Rejecting trials, around 200–700 ms at PZ (Figure 6c).

Figure 6. Delta-band (.01 – 3 Hz) ERSP as a function of Expected Utility (EU), Utility Distance (UD) and Accepting and Rejecting trials.

Figure 6a,b: The effect of EU (Figure 6a) and UD (Figure 6b). The left plots show parameter values at each electrode and time point. Here, we only plotted parameter values that passed spatial-temporal clustering (α = .05). See caption in Figure 5 for the arrangement of electrodes in these left plots. The middle plots show parameter values at a specific electrode. The shaded areas indicate bootstrap 95% confidence interval. The red line at the bottom indicates significant time points that passed spatial-temporal clustering (α = .05). The right plots show topographical maps of uncorrected parameter values at a specific time window. Figure 6c: Delta-band averaged ERSP waveforms, comparing between bins with high and low EU, between Accepting and Rejecting trials, and between bins with high and low UD. The shaded areas indicate between-condition standard error. The red line at the bottom indicates significant time points that passed FDR correction (q = .05). The blue line at the bottom indicates significant time points that had permutation p-value < .05, but did not pass FDR correction. See Supplementary Figure 3 for the analyses at other midline electrodes.

Unlike delta-band power, the relationship between EU and theta-band power at the early time window was not statistically significant (Figure 7 a,b). This is despite the overall enhancement in frontal-midline theta-band power across all trials around 250 ms following stimulus onset (Figure 4, 7b). Rather, from the hierarchical GLMs approach (Figure 7a), there was a significant relationship between higher EU and suppression of theta-band power at around 2500–2750 ms. However, bin-like analyses (Figure 7b) did not confirm the relationship at this window (Figure 7b); thus we limited the interpretation of this relationship. There was no statistically significant relationship between UD and theta-band power.

Figure 7. Theta-band (4 – 7 Hz) ERSP as a function of Expected Utility (EU), Utility Distance (UD) and Accepting vs. Rejecting trials.

Figure 7a: The effect of EU. The left plot shows parameter values at each electrode and time point. Here, we only plotted parameter values that passed spatial-temporal clustering (α = .05). See caption in Figure 5 for the arrangement of electrodes in the left plot. The middle plot shows parameter values at a specific electrode. The shaded areas indicate bootstrap 95% confidence interval. The red line at the bottom indicates significant time points that passed spatial-temporal clustering (α = .05). The right plot shows a topographical map of uncorrected parameter values at a specific time window. Figure 7b: Theta-band averaged ERSP waveforms, comparing between bins with high and low EU, between Accepting and Rejecting trials and between bins with high and low UD. The shaded areas indicate between-condition standard error. The red line at the bottom indicates significant time points that passed FDR correction (q = .05). The blue line at the bottom indicates significant time points that had permutation p-value < .05, but did not pass FDR correction. See Supplementary Figure 3 for the analyses at other midline electrodes.

As for alpha-band power, higher EU was associated with sustained alpha suppression at around 1000–3000 ms (Figure 8a). The effect was more widespread at the posterior sites around 1000–2000 ms then at the frontal sites afterward. Moreover, we found stronger alpha suppression in Accepting trials, compared to Rejecting trials, at the similar time window (Figure 8b). Conversely, there was a small window around 600–800 ms where higher UD was significantly associated with alpha suppression (Figure 8b). The bin-like analyses (Figure 8b,d) confirm the patterns using the hierarchical GLMs approach for EU (Figure 8a), but not for UD (Figure 8c). Accordingly, the effects of EU (but not UD) appear to be robust across the two methods of analyses.

In addition to alpha-band power, higher EU was associated with sustained low-beta suppression at around 800–2000 ms at the frontal area (Figure 9a). Similarly, we also found stronger low-beta suppression in Accepting trials, compared to Rejecting trials, at the similar time window with the effect of EU (Figure 9b). There was no statistically significant relationship between UD and low-beta-band power. The bin-like analyses (Figure 9b) confirm the patterns using the hierarchical GLMs approach (Figure 9a). Both EU and UD did not significantly explain the changes in high-beta-band power (see Supplementary).

Figure 9. Low-beta-band (14 – 20 Hz) ERSP as a function of Expected Utility (EU), Utility Distance (UD) and Accepting and Rejecting trials.

Figure 9a: The effect of EU. The left plot shows parameter values at each electrode and time point. Here, we only plotted parameter values that passed spatial-temporal clustering (α = .05). See caption in Figure 5 for the arrangement of electrodes in these left plots. The middle plot shows parameter values at a specific electrode. The shaded areas indicate bootstrap 95% confidence interval. The red line at the bottom indicates significant time points that passed spatial-temporal clustering (α = .05). The right plot shows a topographical map of uncorrected parameter values at a specific time window. Figure 9b: Beta-band averaged ERSP waveforms, comparing between bins with high and low EU, between Accepting and Rejecting trials and between bins with high and low UD. The shaded areas indicate between-condition standard error. The red line at the bottom indicates significant time points that passed FDR correction (q = .05). The blue line at the bottom indicates significant time points that had permutation p-value < .05 but did not pass FDR correction. See Supplementary Figure 3 for the analyses at other midline electrodes.

Figure 10 shows the association between ln(RT) and EEG activity. Here, we only focused on the wide-band ERP and delta power that showed the widespread effects of UD. This is because of our conceptualization of slower ln(RT) as reflecting the uneasiness in making a decision, in a similar manner to lower UD (i.e., higher choice conflict). We found statistically significant associations between ln(RT) and both wide-band ERP (Figure 10a) and delta power (Figure 10b) at similar time-points and topography as the effect of UD (but in the opposite direction). Slower ln(RT) was associated with more negative wide-band ERP activity and reduction of delta-power at 500–700 ms in the posterior sites. This time window of the wide-band ERP activity is similar to the effect of UD on the P3-like component, but it is later than the effect of EU. Moreover, slower ln(RT) was associated with more frontal sustained positive ERP activity at similar time windows and topography as the effect of UD. This association with frontal sustained ERP activity, while having a p-value lower than .05, did not survive multiple comparison correction with spatial-temporal clustering.

Figure 10. Ln(RT) predicting wide-band single-trial ERP (Figure 10a) and delta-band (.01–3 Hz) ERSP (Figure 10b).

The left plot shows parameter values at each electrode and time point. Here, we plotted both corrected parameter values that passed spatial-temporal clustering (α = .05) and uncorrected parameter values that had a bootstrap p-value < .05, but did not passed spatial-temporal clustering. See caption in Figure 5 for the arrangement of electrodes in these left plots. The middle plot shows parameter values at a specific electrode. The shaded areas indicate bootstrap 95% confidence interval. The red line at the bottom indicates significant time points that passed spatial-temporal clustering (α = .05). The right plot shows a topographical map of uncorrected parameter values at a specific time window.

4. Discussion

When making decisions under risk, neoclassical economists argue that individuals should only gamble when it maximizes their predicted subjective desirability, reflected by EU (Von Neumann & Morgenstern, 1944). However, EU of risky choices may only reflect the motivation-related, valence-specific part of the information processed during decision-making (Bartra et al., 2013). Focusing solely on EU, one misses the conflict-related, valence-non-specific part of the decision-making process, which is reflected in utility distance (UD). In fact, our LME regression analysis shows that UD, but not EU, was related to how fast our participants made decisions, suggesting that UD is an integral part of the decision-making processes. In processing choices, people have to attend to EU and UD relevant information almost instantaneously. Model-based single-trial EEG analyses allowed us to simultaneously enter EU and UD as predictors in the hierarchical GLM analysis, thereby teasing apart the unique influences of EU from UD in time, frequency and topographical space. This, in turn, enabled us to begin to unpack how these two complementary parts of decision making are processed under risk. Overall, we observed a distinct influence of EU and UD on several EEG patterns: higher EU was associated with the enhancement of P3-like activity, delta-band power, alpha suppression and beta suppression, while higher UD was associated with the enhancement of P3-like activity, delta-band power and sustained-frontal negativity. Moreover, we found that the differences in EEG activity between Accepting vs. Rejecting trials appear to resemble the effect of EU. Similarly, we also found that the association between decision RT and EEG resembles the effect of UD. This strengthens our claim that EU and UD respectively represent the motivation-related (reflected by propensity to accept the gamble) and conflict-related (reflected by length of time taken to decide) aspects of decision-making processes.

As predicted, both EU and UD were positively associated with wide-band single-trial positive ERP activity that resembled the P3 component in both time (i.e., around 300 – 600 ms) and topography (i.e., at central-parietal sites) (Polich, 2007). Since EU reflects motivational, valence-specific information, this positive relationship between EU and P3 is consistent with an enhanced P3 to (1) motivational, reward-related anticipatory cues (Broyd et al., 2012; Carrillo-de-la-Peña & Cadaveira, 2000; Goldstein et al., 2006; Novak & Foti, 2015; Pornpattananangkul & Nusslock, 2015; Ramsey & Finn, 1997; Santesso et al., 2012; Zhang, Li, Wang, Liu, & Zheng, 2017) and (2) positive-valence pictures (Cano et al., 2009; Olofsson, Nordin, Sequeira, & Polich, 2008). Since UD reflects the ease in which a decision is made, the positive relationship between UD and P3 is consistent with perceptual decision-making research showing a stronger P3 for stimuli that can be judged more easily (Kelly & O’Connell, 2013; O’Connell et al., 2012; Twomey, Murphy, Kelly, & O’Connell, 2015). Importantly, while both EU and UD modulated the P3, we showed that the unique contribution of motivational, valence-specific EU on the P3 was around 200 ms earlier than that of the conflict-related, valence-non-specific UD. Given the P3’s role in stimulus evaluation and attentional-resource allocation (Kissler et al., 2009; Polich, 2007; Schupp et al., 2006), this may suggest that, during decision-making processes, participants evaluated and attended to the motivational aspect of the choices (e.g., how rewarding of the potential outcome is) before the conflicted-related part (e.g., how easy to choose one choice over the other).

EU and UD had similar effects on delta-band power as they had on P3 in both time and topographical space. This similarity suggests that the influence of EU and UD on wide-band single-trial ERP activity that constituted P3-like activity was likely driven by EEG in the delta band. This finding is consistent with research showing that motivational, reward-related anticipatory cues enhance delta-band power in a similar manner to the P3 (Pornpattananangkul & Nusslock, 2015, 2016). Note that while several studies have shown a relationship between P3 ERP and delta-band ERSP (Cavanagh, 2015; Demiralp, Ademoglu, Istefanopulos, Başar-Eroglu, & Başar, 2001; Demiralp, Ademoglu, Schürmann, Basar-Eroglu, & Basar, 1999; Harper, Malone, & Bernat, 2014), this has not always been the case. For instance, in some situations, alpha-band EEG may also influence the P3 (Intriligator & Polich, 1994; Mazaheri & Jensen, 2008). Because the effects of EU and UD on EEG activity at other frequency bands in our study were not at a similar time window as the P3, delta-band power may account for activity underlying the P3 in the context of decision-making under risk.

As hypothesized, higher EU was associated with both alpha-band and low-beta-band suppression. Unlike UD’s effects on P3 and delta-band power, UD had a weaker effect (i.e., a smaller time window) on alpha-band power and did not have a statistically significant effect on beta-band power. Given that EU reflects motivation-related information, the effect that EU had on alpha-band power is consistent with recent work showing stronger alpha suppression during reward-anticipation processes (Hughes et al., 2013; Pornpattananangkul & Nusslock, 2016; van den Berg, Krebs, Lorist, & Woldorff, 2014). For instance, in one study (van den Berg et al., 2014), participants were able to obtain monetary rewards during a Stroop task on reward trials, but they could not obtain reward regardless of their performance during no-reward trials. The authors reported that cues that signified a reward trial elicited stronger alpha suppression at both frontal and occipital sites, and this change in alpha power predicted improvement in participants’ Stroop performance. Similarly, the effect of EU on beta suppression is consistent with research showing a role of beta power in facilitating action initiation toward expected rewards (Doñamayor et al., 2012; Gable, Threadgill, & Adams, 2016; Meyniel & Pessiglione, 2013). Meyniel and Pessiglione (2013), for instance, employed monetary incentives to motivate participants to take a shorter break in a physical-effort task. They showed that stronger beta suppression in motor areas during the break was associated with higher incentive levels and predicted decisions to have shorter breaks. Accordingly, EU in our study may capture reward-anticipation processes implicated in the alpha-band and low-beta-band suppression.

It is important to highlight the delayed effect of EU on alpha (after 1000 ms) and low-beta (after 800 ms) suppression in the current study. These time windows were close to the ISI preceding the Response screen at which participants had to report their decision whether or not to gamble using their right index finger. During these windows, alpha and beta suppression across trials were topographically distributed above the motor areas (see Figure 3c). Importantly, this suppression was distributed more predominantly to the left motor area, which was contralateral to the upcoming right-hand movement. Thus, higher EU (which by definition reflects a higher reward value of the gamble and, thus, a higher tendency to gamble) may facilitate motor preparation to gamble. Nonetheless, while motor preparation might be one of the underlying processes of EU on alpha and low-beta suppression, there might also be involvement of other higher-cognition processes related to reward-anticipation. We make this conjecture because of the topographical distribution of EU effects. First, the effect of EU on alpha-band power was quite widespread throughout the scalp, not just motor areas. Second, the effect of EU on low-beta-band power was distributed more predominantly at the frontal sites, than at the motor areas. Consistent with this conjecture is previous work (Pornpattananangkul & Nusslock, 2016; van den Berg et al., 2014) showing stronger alpha suppression at the frontal and occipital sites, even during durations that require no motor movement (e.g., while waiting to see feedback).

We also demonstrated an association between higher UD and a stronger sustained-frontal negativity. While we did not hypothesize this pattern, this association was robust, spreading across a large time window (at around 1200–3000 ms) and several electrodes (starting from frontal sites to areas throughout the scalp). Similar to the effect of EU on alpha and low-beta suppression, the effect of UD on this sustained-frontal negativity was during the ISI preceding the Response screen. This sustained-frontal negativity bears some resemblance to the contingent-negativity variation (CNV) in terms of timing (i.e., between two stimuli and prior to a response to the later stimulus) and topography (i.e., frontally distributed) (Brunia et al., 2011; Kononowicz & Penney, 2016; Walter et al., 1964). The frontal-dominant and earlier portion of the CNV is traditionally thought to trace the uncertainty of the upcoming response, while the central-dominant and later portion of the CNV is linked to response preparation (Järvilehto & Fruhstorfer, 1970). When the upcoming response is more certain (e.g., when having attention-directing cues to direct the response), the CNV is usually more negative at the frontal sites (Bayer et al., 2017; Grent-’t-Jong & Woldorff, 2007; Talsma, Kok, Slagter, & Cipriani, 2008; van Boxtel & Brunia, 1994; van den Berg et al., 2016). Moreover, a slower upcoming response was associated with a less negative CNV (Brunia et al., 2011; Rohrbaugh, Syndulko, & Lindsley, 1976). This is consistent with the association between a slower ln(RT) and more frontal sustained positive (i.e., less negative) ERP activity observed in the present study. It is important to note, however, that while both UD and ln(RT) were related to frontal sustained wide-band ERP activity (reflected by bootstrap p < .05), only the effect of UD survived the multiple comparison correction with spatial-temporal clustering (Maris & Oostenveld, 2007). This may reflect the noisiness of RT data, as compared to UD (see Figure 2).

Given that higher UD reflects a greater ease in making an upcoming motor response (as shown in its association with faster ln(RT) in the LME regression analysis), our wide-band ERP finding is consistent with recent research investigating the CNV in the cuing paradigm (Grent-’t-Jong & Woldorff, 2007). In this paradigm, the experimenters presented attention-directing cues to grab participants’ attention to locations where upcoming stimuli may show up, making it easier to respond to these stimuli. This led to a more negative CNV, peaking at around 1200 ms following the cue and 800 ms preceding the response (Grent-’t-Jong & Woldorff, 2007), similar to what we found in the current study. Using this cuing paradigm, van den Berg and colleagues (2016) argued that alpha suppression and the CNV may reflect different anticipation-related processes: while alpha suppression is related to motivation, the CNV is more closely related to instruction-specific attention. Accordingly, in our study, the fact that the CNV was more negative when UD was high might be due to the fact that participants were more certain about which upcoming choice to make. The differential effects of a) motivational-related EU on alpha-band power and b) choice-certainty-related UD on the CNV support the different roles of the two EEG indices in anticipation-related processes.

We found an overall enhancement in theta-band power at the frontal-central sites across all trials approximately 250 ms following stimulus onset. The characteristics of this activity (in terms of frequency, time and topography) are similar to those of the frontal-midline theta (FMT) implicated in conflict monitoring, especially during outcome evaluation (Cavanagh & Frank, 2014; Cavanagh & Shackman, 2015). FMT, for instance, was stronger when people obtained an outcome indicating a loss or incorrect performance or when they made an incorrect response (Luft, 2014). However, unlike other frequency bands, we did not find statistically significant effects of either EU or UD on the FMT. Most reward-processing studies that have investigated FMT focus on the outcome phase, but not the cue or decision phase (for review, see Glazer et al., 2018). Two studies failed to show that rewards modulated the cue-locked FMT (Cavanagh, 2015; Pornpattananangkul & Nusslock, 2016). In fact, we previously showed that while rewards influence both cue-locked and outcome-locked delta power, rewards only modulate the outcome-locked FMT, but not cue-locked FMT (Pornpattananangkul & Nusslock, 2016). Given that the FMT in the current study occurred during the decision phase and that reward cues should have similar psychological processes with the motivation-related EU, the lack of an EU effect on the FMT is consistent with previous (rather limited) research. Nonetheless, the absence of the UD effect on the FMT is more perplexing, given that we conceptualize UD as being related to conflict in making decision. It is possible that conflict from obtaining a loss or incorrect performance outcome that usually enhances outcome-locked FMT (Luft, 2014) is processed differently from conflict during decision-making. For instance, it has been argued that the mechanism underlying the N2 ERP component, which is closely related to the FMT, reflects a template mismatch (Folstein & Van Petten, 2008). At the outcome phase, a loss or incorrect performance outcome is inconsistent (i.e., a mismatch) with one’s expectation, thereby creating conflict and enhancing the N2/FMT. However, during the decision-making phase, this template mismatch does not occur. Future research that manipulates expectation of the upcoming decisions is needed to test the possibility of template mismatch as a mechanism for differential effects on FMT during decision vs. during outcome.

It is important for us to compare and contrast our results to what has been reported in previous EEG research using parametric models during reward-processing and decision-making. This is challenging because a) most EEG studies investigating reward-processing and decision-making typically focus on the outcome phase, and ignore the decision phase (for review, see Glazer et al., 2018) and b) because most EEG studies still rely on averaging signals into components (for review, see Bridwell et al., 2018). There is one reinforcement-learning study (Fischer & Ullsperger, 2013) that investigated single-trial wide-band ERP (but not ERSP) locked to cues in which participants needed to decide whether-or-not to gamble. In this study, participants had to learn the value of each cue via trial-and-error, and, accordingly, the value of different cues changed from trial by trial depending on previous outcomes associated with them. The authors of this study used a reinforcement Q-learning algorithm (Sutton & Barto, 1998) to computationally model a) Q, or the subjective desirability value associated with each cue based on trial-and-error learning, and b) subjective decision certainty (SDC), or the certainty in preferring one option over the other. Conceptually Q and SDC are similar to EU and UD in that they are related to motivation and conflict, respectively, even though the models and tasks used are derived from different theories (i.e., Reinforcement-Learning Theory vs. Expected Utility and Prospect Theory). Fischer and Ullsperger found Q and SDC were associated with wideband ERP at similar time-windows and polarity as our EU and UD effects on the P3-like activity. Although their effect of Q appears to propogate more to frontal sites than our EU effects, which may reflect the differences in tasks and modeling. Overall, while using different approachs, we found patterns consistent with Fischer and Ullsperger’s (2013) findings, but also extend our understanding of model-based single-trial EEG during decision-making to different situations (decision under risk without learning), models (involving loss aversion and risk attitude, but not learning rate), and EEG-frequency (from delta to beta bands).

It is also crucial that we discuss the dissociable effects of EU and UD on the P3-like activity during decision-making in relation to previous research investigating outcome-locked averaged ERP. Previous ERP research using reward-related tasks (Bellebaum, Polezzi, & Daum, 2010; Gu et al., 2011; Meadows, Gable, Lohse, & Miller, 2016) has often shown independent modulations of valence (i.e., positive or negative) and magnitude (i.e., high or low) of an outcome on the outcome-locked P3 (although see Sato et al. (2005) for a contradictory finding). In these studies, outcomes with either a) positive valence regardless of magnitude (e.g., gaining rewards of any amount) or b) high magnitude regardless of valence (e.g., gaining or losing 5 dollars compared to 5 cents) usually lead to higher P3 amplitude. Accordingly, our EU effects on the P3-like activity appear to be consistent with the effect of positive valence, given that higher EU trials in our study correspond to subjectively higher value of potential gains compared to potential losses. However, it seems too early to conclude that our UD effects are similar to the effects of magnitude on P3, simply because trials with higher UD in our study had higher magnitude. We interpret the effects of UD as related to conflict. We base this interpretation on a) the fact that higher UD was related to slower decision RT, and b) that decision RT was associated with the EEG activity in a similar time, frequency and topography as UD (albeit in an opposite direction). However, it is difficult to use conflict as a process underlying the effect of magnitude on outcome-locked P3 reported in previous studies. Future studies with both decision-making and outcome phases that vary in UD are needed to address this question more formally.

Our study is not without its limitation. We presented the Gain and Loss screens in a fixed order and time-locked EEG activity to the later Loss screen (Figure 1). We decided to keep the fixed order to simplify the experiment, similar to previous studies investigating decision-making under risk that typically fix the gain and loss information to the same location throughout the experiment (Brown et al., 2013; De Martino et al., 2010; Sokol-Hessner et al., 2013; Sokol-Hessner et al., 2009; Sokol-Hessner et al., 2016; Tom et al., 2007). Moreover, time-locking EEG to the Loss screen was our attempt to follow the so-called Hillyard principle in EEG study design (Luck, 2014). This principle states that researchers should keep the physical qualities of stimuli as similar to each other as possible across trials while modulating the psychological meaning of these stimuli. Here we changed the psychological meanings of each of 256 Loss screens using trial-by-trial parameters: EU and UD. Nonetheless, we may have inadvertently drawn too much attention to the Loss screen, even though our effects of interests (EU and UD) included both gain and loss information in their calculation. Thus, having a tight control in our design may have limited the generalizability of our findings to situations where gain information is presented before loss information. Perhaps, if loss information is presented before gain information, the gain information might be more salient. Accordingly, one important future direction is to investigate the order of gain and loss presentation and examine if the order interacts with the effects of EU and UD on EEG activity.

5. Conclusions

EU indicates choice values in the reward-punishment continuum while UD is more relevant to the ease in decision-making. Using model-based single-trial EEG analyses, we were able to discern differential influences of EU and UD in time, frequency and topographical space. First, the influences of EU on the P3-like activity and delta power were earlier than those of UD. This finding seems to suggest that, when evaluating choices during decision-making under risk, participants allocated attentional resources to motivation-related information sooner than to conflict-related information. Next, prior to making a response, EU and UD influenced EEG activity at different frequencies. This finding suggests the differential involvement of anticipatory processes associated with EU and UD. While motivation-related EU modulated reward-related anticipation (reflected by alpha-band and beta-band power), conflict-related UD modulated attention-related anticipation (reflected by CNV-like activity). Altogether, for the first time, we showed that EEG signals that traced the motivation-related information during decision-making under risk was dissociable from those that traced the conflict-related information in time, frequency and topographical characteristics.

Highlights.

• Model-based EEG teased apart unique influences of motivation & conflict processes.

• During risky decisions, both were associated with P3-like and delta-band activity.

• The P3-like/delta activity of motivation processes was around 200 ms earlier.

• Motivation processes involved stronger alpha and beta suppression.

• Conflict processes involved stronger CNV-like activity.

Abbreviations

BOLD

blood-oxygen-level dependent imaging

CNV

contingent negativity variation

EEG

electroencephalogram

ERP

event-related-potential

ERSP

event-related-spectral-perturbation

EU

expected utility

EV

expected value

FMT

frontal-midline theta power

HBA

hierarchical Bayesian analysis

HMC

Hamiltonian Monte Carlo

ISI

inter-stimulus interval

ITI

inter-trial interval

LME

Linear-mixed effect

LOOIC

Leave-One-Out Information Criterion

MCMC

Markov Chain Monte Carlo

MLE

maximum likelihood estimation

UD

utility distance

VD

value distance