Withholding a reward-driven action: studies of the rise and fall of motor activation and the effect of cognitive depletion

Abstract

Controlling an inappropriate response tendency in the face of a reward-predicting stimulus likely depends on the strength of the reward-driven activation, the strength of a putative top-down control process, and their relative timing. We developed a rewarded Go/NoGo paradigm to investigate such dynamics. Participants made rapid responses (on Go trials) to high versus low reward-predicting stimuli and sometimes had to withhold responding (on NoGo trials) in the face of the same stimuli. Behaviorally, for high versus low reward stimuli, responses were faster on Go trials and there were more errors of commission on NoGo trials. We used single-pulse Transcranial Magnetic Stimulation to map out the corticospinal excitability dynamics, especially on NoGo trials where control is needed. For successful NoGo trials, there was an early rise in motor activation that was then sharply reduced beneath baseline. This activation-reduction pattern was more pronounced for high versus low reward trials and in individuals with greater motivational drive for reward. A follow-on experiment showed that when participants were fatigued by an effortful task, they made more errors on NoGo trials for high versus low reward stimuli. Together, these studies show that when a response is inappropriate, reward-predicting stimuli induce early motor activation, followed by a top-down effortful control process (which we interpret as response suppression) that depends on the strength of the preceding activation. Our findings provide novel information about the activation-suppression dynamics during control over reward-driven actions, and they illustrate how fatigue or depletion leads to control failures in the face of reward.

INTRODUCTION

One way that inappropriate action tendencies are controlled is via response suppression. In the laboratory, action tendencies are typically induced by creating or capitalizing on a strong relationship between a stimulus and a particular response (e.g., an arrow pointing right signals a right hand response in the stop-signal task, Logan, 1994). While these tasks have yielded insights into how inappropriate action tendencies are controlled, including neural circuits, motor dynamics, and factors that influence control, their relevance to daily life is limited (for reviews on response suppression, see Aron, 2007; Bari & Robbins, 2013; Ridderinkhof, Forstmann, Wylie, Burle, & van den Wildenberg, 2011; Stinear, Coxon, & Byblow, 2009). This is because, unlike the action provocations in these “cold” cognitive psychology tasks, real-world provocations are often driven by the reward-predicting properties of a stimulus (e.g., a tasty food).

In an effort to extend response suppression research to more real-world situations, we recently developed a behavioral paradigm that requires control in the face of motivationally-driven provocations (Freeman, Alvernaz, Tonnesen, Linderman, & Aron, 2015; Freeman, Razhas, & Aron, 2014). In this paradigm, participants were either permitted to respond for a small juice reward (Go trials) or not permitted to respond (NoGo trials), both in the face of a task-irrelevant stimulus that was earlier associated with juice via Pavlovian conditioning. This led participants, on Go trials, to respond more quickly, and, on NoGo trials, to make more errors of inappropriate responding. For the same task, we used single-pulse Transcranial Magnetic Stimulation (spTMS) over primary motor cortex to measure motor system activity. We showed that, on Go trials, the stimulus associated with juice (relative to a stimulus that was not associated with juice) increased motor excitability at 250 ms; whereas, on successful NoGo trials, there was a beneath-baseline reduction at the same time point. We interpreted this reduction as evidence for a response suppression process that helped mitigate the motivationally-triggered activation, yielding an activation-suppression dynamic. However, since motor excitability was only measured at a single time point, those studies did not reveal the finer-grained dynamics of the predicted motor activation and motor reduction processes—how fast the activation appears, how high it reaches, when the control kicks in, and how long it lasts. Moreover, without a picture of the dynamics, those studies could not firmly show that the motor reduction was due to a control process that relates to the strength of the preceding activation. In those studies, TMS was limited to a single time point because of the waning influence of the Pavlovian stimulus from satiation over time (thus limiting trial numbers). Here we sought to capture the putative activation-suppression dynamics using a new paradigm.

Now, rather than using Pavlovian-conditioned stimuli associated with juice, we used stimuli that predicted potential monetary rewards on a given trial. On Go trials, participants made instrumental responses to obtain a reward in points (later converted to money). The number of potential points on a given trial was indicated by a colored rectangle (one color high reward, the other low reward) that was placed behind the Go/NoGo cue. On NoGo trials, participants were required to withhold their pressing, despite the potential provocation induced by the reward stimulus. Because the reward stimulus was now task-relevant and because it entailed monetary reward, there were no restrictions on trial numbers. Accordingly, TMS pulses were delivered at 100, 150, 200, and 250 ms after stimulus onset on different trials. We tested the hypothesis that the reward-predicting stimuli would evoke an early rise in motor excitability on both Go and NoGo trials, and that this reward-driven activation would be immediately followed by a sharp reduction in motor excitability on NoGo trials. We also hypothesized that, on NoGo trials, the high reward stimulus would show a steeper early activation, and we were interested to examine if this sharper rise would be accompanied by a steeper subsequent reduction—which could reflect a more effortful control process that helps mitigate the increased activation. An alternative possibility is that a similar amount of control would be exerted for high and low reward trials, which predicts parallel reduction slopes following a greater initial activation for high reward trials. In a second study, we tested the idea that the reduction phase reflects top-down control over the activation. We did this by first engaging the participants in an effortful task, which should ‘deplete’ top-down resources; we then examined their ability to withhold responding on high versus low reward NoGo trials.

EXPERIMENT 1

METHODS

Participants

There were thirty participants (16 female; mean age = 20.73, SD = 2.7; all right-handed). Two were excluded for having oversaturated motor evoked potentials (MEPs) (i.e. MEPs > 2 mV), and two were excluded due to technical malfunctions with the TMS equipment. Thus, all analyses were run on twenty-six participants, who provided informed consent and passed TMS safety screening.

Task and Procedure

Each participant sat in front of an iMac (Apple Inc., Cupertino, CA) with a 20-inch monitor (60 Hz refresh rate). On each trial, participants saw either a black triangle or a black square in the center of the screen for 1.75 seconds (s) (Figure 1A). Participants were instructed to respond to one of the shapes (Go cue) and to withhold responding to the other shape (NoGo cue). Go and NoGo cues were equiprobable (i.e. 50/50) and the shapes were counterbalanced across participants.

Figure 1. Rewarded Go/NoGo task and behavior.

On Go trials (left panel), continuous presses were made in effort to receive points (later converted to money). If enough presses were made on a given trial, the earned points were displayed to the participant; otherwise, no points were displayed and the next trial began. The amount of potential points to be earned on a given trial was indicated by the background color. One color (shown as yellow here) was associated with substantially higher point rewards than the other color (shown as blue here). On NoGo trials (right panel), responding was to be withheld; else, a red error message appeared. Go and NoGo trials were equiprobable. (B) On Go trials, the first press reaction time was significantly faster for high compared to low reward trials. (C) On Go trials, more presses were made for high compared to low reward trials. (D) On NoGo trials, the error rate was significantly greater for high compared to low reward trials. This indicates that the high reward background stimulus provoked responding on both Go and NoGo trials. Error bars represent SEM across participants. ***p < 0.001.

Upon presentation of the Go cue, participants could continuously press a button with their right index finger to obtain points, which they were told would translate into money at the end of the experiment. Presses were to be made only during the 1.75 s duration of the Go trial and participants were instructed to stop pressing once the Go cue disappeared. Points were delivered on a variable ratio reward schedule. For the first block (designated as the “learning block”), the required number of presses ranged from 4 to 9 presses based on a uniform distribution. After the learning block, the range was adjusted based on the participant’s mean number of presses (rounded to the nearest integer) during the learning block. In order to maximize motivational drive for reward, the range was adjusted such that the participant could obtain a reward on a majority (i.e. > 50%), but not all of Go trials. Thus, a mean press rate of 5 yielded a range of 2–7; a mean press rate of 6 yielded a range of 3–8, and so on. The average proportion of rewarded Go trials was 0.64 (SD = 0.13) across participants. Information regarding the number of presses required for reward was not disclosed to the participants, though they were informed that the required number of presses would vary across trials. If the button was pressed enough times on a given trial, the amount of points earned was displayed at the center of the screen (e.g., “+50”) (Fig. 1A). If the button was not pressed enough times, a fixation cross appeared and the ITI period began.

The number of possible points to be earned on a given trial was indicated by a large, colored (blue or yellow) rectangle that surrounded the Go cue and was presented simultaneously. If enough presses were made on a high reward Go trial, participants received between 50 and 100 points (in increments of 10, chosen randomly). For low reward Go trials, participants received between 1 and 5 points (in increments of 1, chosen randomly). Participants were informed before the experiment that approximately 1000 points yielded $1. High and low reward colors were counterbalanced across participants.

Upon presentation of the NoGo cue, participants were required to withhold responding. If a press was mistakenly made on a NoGo trial, a red error message reading, “Do Not Press the Button!” was flashed for 1 s. The NoGo cue was also surrounded by a blue or yellow rectangle (which signals reward on Go trials), thereby manipulating participants’ motivational drive even while they were required to withhold a response (Fig. 1A).

All trials were separated by a fixation cross for a variable inter-trial-interval (ITI) of 1.75–3 s (in increments of 0.25 s, chosen randomly). Go and NoGo cues were presented pseudo-randomly such that no more than four Go or NoGo cues could occur in succession. There were 14 total blocks with 52 trials in each block, yielding 728 total trials. At the end of each block, the number of cumulative points the participant had earned appeared at the top of the screen. At the end of the experiment, the total number of points earned was divided by 1000 and then converted to a rounded dollar amount. The mode for the total money earned across participants was $9. After the experiment concluded, all participants completed the Barratt Impulsivity Scale (BIS-11) questionnaire (Patton, 1995).

TMS

TMS was delivered using a MagStim 200–2 system (MagStim, Whitland, UK) and a 70 mm figure-of-eight coil. Surface EMG was recorded from the first dorsal interosseous (FDI, corresponding to the task-relevant index finger) muscle of the right hand via 10-mm-diameter Ag-AgCl hydrogel electrodes (Medical Supplies Inc., Newbury Park, CA).

The coil was placed 5 cm lateral and 2 cm anterior to the vertex and repositioned while delivering a TMS stimulus to locate the position where the largest MEPs were observed consistently. The angle of the coil was approximately 45 degrees from the central sulcus. We measured resting motor threshold (RMT), defined as the minimum stimulation intensity required to induce 0.1 mV peak-to-peak amplitude MEP in 5 out of 10 consecutive stimulations (Rossini et al., 1994). Next, starting at RMT, the maximum MEP size was determined by increasing stimulus intensity in 3–4% increments until the MEP amplitude no longer increased. Finally, the TMS stimulus intensity was adjusted to produce a MEP that was approximately half of the maximum MEP amplitude while the participant was performing the task in a practice session. This ensured that the test stimulus intensity was on the ascending limb of the individual’s stimulus–response curve, so that both increases and decreases in corticospinal excitability could be detected (Devanne, Lavoie, & Capaday, 1997). This was the intensity used during the experiment proper (mean intensity across participants was 44.7% stimulator output, SD = 8.16). To measure the dynamics of corticospinal excitability across time, there were four pulse times after stimulus onset (100, 150, 200, and 250 ms), yielding 42 trials per condition at each time point. There was also one pulse time 500 ms before stimulus onset to provide a baseline measure (56 trials). To optimize EMG over the FDI muscle, the right index finger moved inward to press a vertical key.

Behavioral Analysis

We compared high and low reward trials using three dependent measures: (1) the median reaction time to the first press on Go trials (henceforth called first press RT), (2) the mean number of presses during the 1.75 s response interval, and (3) the percentage of commission errors on NoGo trials. As the first block was considered a “learning block” (where participants learned the color-reward associations), these trials were excluded from all analyses. Trials were also excluded if first press RT was less than 100 ms or if a response was not made on a Go trial. Differences between high and low reward conditions were evaluated using two-tailed, paired t-tests.

EMG Analysis

Pre-processing and normalization

An EMG sweep started 200 ms before stimulation. MEPs were identified from the EMG using in-house software developed in Matlab (Mathworks, Natick, MA). Trials were excluded if the root mean square EMG in the 100 ms before the TMS pulse was greater than 0.01 mV or if the MEP was less than 0.05 mV. We also excluded trials if the amplitude maxed out at +1 mV or −1 mV, since we used a CED MICRO 1401 system that has a cut-off at 2 mV (range of +1 to −1). Thus, we could not be sure of the true MEP amplitude when it exceeded 2 mV (e.g., 2.1 mV, 4 mV, etc.). For this reason, we elected to exclude such MEPs that “maxed out”, as we feel that this provides the most accurate version of the MEP dataset. Median peak-to-peak amplitudes of MEPs were calculated for all conditions at each time point. Then, the median MEP for each condition was divided (i.e. normalized) by the median MEP of the Baseline trials (i.e. the time point at 500 ms before stimulus onset). An examination of the normalized root mean square (RMS) values for the 100 ms time window before the TMS pulse showed no significant main effects or interactions (all Ps > 0.05), demonstrating that the MEP patterns described below were not contaminated by differences in the pre-TMS period.

Go and NoGo Dynamics

To provide a detailed picture of the dynamics, we conducted several analyses. First, we separately evaluated Go and NoGo trials using ANOVAs with Reward (high, low) and Pulse Time (100, 150, 200, 250 ms) as factors. For all analyses, we excluded NoGo trials where a press was made (commission error). All Go trials were analyzed, regardless of whether enough presses were made to earn points on the trial. Planned comparisons for high versus low reward were made at each of the four time points using paired t-tests with an alpha value set at 0.05. Due to the strong prediction of larger MEPs for high versus low reward on Go trials, one-tailed t-tests were used for this analysis, while two-tailed t-tests were used for the NoGo analysis (since the timing could not be predicted a priori). Unless otherwise specified, all reported p-values were corrected for four comparisons using the Holm-Bonferroni procedure.

Percent-change across time points on NoGo trials

To better capture the change across time for NoGo trials, we calculated the percent-change of the “activation phase” (i.e. where MEPs were predicted to increase across time, reflecting response prepotency) and the “reduction phase” (i.e. where MEPs were predicted to decrease across time, likely reflecting response suppression). We entered the percent-change values into a repeated-measures ANOVA with Reward (high, low) and Phase (activation, reduction) as factors. We then used two-tailed, one-sample t-tests to examine differences between each condition and a value of zero (representing no change). Pairwise comparisons across conditions were then made using two-tailed, paired t-tests. Unless otherwise specified, all reported p-values in this analysis were corrected for eight comparisons using the Holm-Bonferroni procedure.

Relationship between reward-based activation and reward-based reduction on NoGo trials

We were interested in examining the relationship between motor excitability during the (predicted) activation and reduction phases, particularly as a function of the reward value. Thus, in each participant, we calculated “reward-based” (high minus low reward) difference scores for each phase. Specifically, we subtracted the percent-change for high reward from the percent-change for low reward in the activation and reduction phases. A Pearson’s correlation was then used to test the relationship between participants’ reward-based activation and their reward-based reduction.

Relationship between NoGo dynamics and error rates

We postulated that the motor dynamics on NoGo trials would relate to participants’ self-control failures. We therefore examined how the activation, the reduction, and the activation-reduction processes together related to participants’ overall error rates on NoGo trials (including high and low reward trials). To quantify participants’ activation and reduction levels, we computed an average score of percent-change for the activation and reduction phases separately and correlated these measures with participants’ overall NoGo error rates using Pearson’s correlations. We also correlated their NoGo error rates with a composite measure of motor activity in both the activation and reduction phases—henceforth called the “activation-reduction index”. To calculate the activation-reduction index, we first summed the activation and reduction phases for high and low reward trials separately. We then took the average of these two scores to generate an index that reflects both phases and reward values. In essence, this measure provides an index of the strength of the reduction process when taking into account the preceding activation. A Pearson’s correlation was then used to test the relationship between participants’ overall error rates and the activation-reduction index.

Go and NoGo dynamics for fast and slow RT groups

In addition to characterizing the overall dynamics, we reasoned that the motor dynamics in a reward task might depend on participants’ basic motivational drive for reward. To examine this, we conducted a median split on the twenty-six subjects based on their RTs, which we used as a behavioral index of motivational drive for reward (faster RT corresponds to higher motivation for reward; Avila & Lin, 2014; Clithero, Reeck, Carter, Smith, & Huettel, 2011). Specifically, we computed the average of the median high reward RT and the median low reward RT and took this average as the behavioral index of motivational drive for reward. We then conducted the same dynamics analyses as above for both fast and slow RT groups. Unless otherwise specified, all reported p-values were corrected using the Holm-Bonferroni procedure.

Relationship between trait impulsivity and reward-based MEP differences

We acquired answers to a single questionnaire—the Barratt Impulsivity Scale (BIS-11)—to explore a possible relationship between trait impulsivity and sensitivity to reward. We correlated participants’ overall BIS-11 scores with their reward-based (high minus low difference score) RT, NoGo errors, NoGo activation phase (percent-change from 100–150 ms), and the peak activation point on NoGo trials (at 150 ms).

RESULTS

Behavior

On Go trials, first press RTs were significantly faster for high reward (M = 480.8 ms, SD = 51.4 ms) versus low reward (M = 525.5 ms, SD = 55.4 ms), t₂₅ = 7.3, p < 0.001 (Fig. 1B), showing that the action was invigorated. Participants also made more presses on high reward (M=7.55, SD = 2) versus low reward (M = 6.99, SD =1.8) trials during the 1.75 s interval, t₂₅ = 6.2, p < 0.001 (Fig. 1C). On NoGo trials, there was a higher percentage of commission errors for high reward (M = 2.2%, SD = 2.7%) versus low reward (M = 0.93%, SD = 1.3%), t₂₅ = 3.6, p = 0.001 (Fig. 1D), suggesting that the action was also invigorated on NoGo trials, which might make it more difficult to withhold. It is worth noting that while there was a differential increase in NoGo error rates for high versus low reward trials, NoGo error rates for both trial types were low. Thus, our MEP analysis focused solely on successful NoGo trials, as there were insufficient trial numbers to analyze unsuccessful NoGo trials.

Motor Evoked Potentials (MEPs)

This study aimed to examine the dynamics at 100, 150, 200 and 250 ms after high and low reward stimuli on Go and NoGo trials separately. We were particularly interested in examining the putative activation-reduction dynamics on NoGo trials and how this was different for high versus low reward stimuli.

Go and NoGo Dynamics

For Go trials, there was a significant main effect of Reward (F_1,25 = 5.87, p = 0.023) with MEPs for high reward greater than low reward. There was also a significant Reward × Pulse Time interaction (F_3,75 = 2.97, p = 0.037). For high reward Go trials, there was a significant linear increase in motor excitability across the four time points (r₃ = 0.95, p = 0.026); whereas, for low reward Go trials, motor excitability decreased from 100–200 ms, followed by an increase from 200–250 ms (Fig. 2A). Follow-up t-tests showed significantly elevated MEPs for the high reward stimulus at 150 ms (t₂₅ = 2.41, p = 0.047) and marginally elevated MEPs at 200 ms (t₂₅ = 2.16, p = 0.06) (Fig. 2A). This shows that very early motor activity is influenced by the value of a reward-predicting stimulus, which is consistent with several previous studies (Klein, Olivier, & Duque, 2012; Klein-Flügge & Bestmann, 2012; Mooshagian, Keisler, Zimmermann, Schweickert, & Wassermann, 2015; Suzuki et al., 2014)¹.

Figure 2. TMS dynamics in Experiment 1.

(A) High reward Go trials showed a linear increase in motor excitability across the four time points. Low reward Go trials showed an initial decrease in motor excitability (from 100–200 ms), followed by an increase from 200–250 ms. (B) On NoGo trials, high and low reward showed an initial increase in motor excitability from 100–150 ms (activation phase), followed by a decrease from 150–250 ms (reduction phase). (C) On NoGo trials, the percent-change for the reduction phase was twice as strong (measured via effect size) for high versus low reward trials and only high reward trials showed a significant difference between percent-change in the activation and reduction phases. (D) On NoGo trials, greater reward-based (i.e. high minus low reward) activation was related to greater reward-based suppression. Error bars represent SEM across participants. ^p < 0.06, *p < 0.05, ***p < 0.001. All p-values are adjusted according to a Holm-Bonferroni correction.

For NoGo trials, there was a main effect of Pulse Time (F_3,75 = 16.09, p < 0.001), where an initial increase in MEPs was followed by a sharp decrease (Fig. 2B). The high reward trials evidenced a greater early elevation in MEPs (at the 150 ms time point) compared to low reward trials (1.11 mV versus 1.03 mV), though the difference was not significant (t₂₅ = 1.1, n.s.) (Fig. 2B). Following the initial activation, there was a steep, beneath-baseline reduction in motor excitability for both high and low reward trials (250 ms time point versus baseline: p < 0.001 for high and low reward). Thus, as predicted, NoGo trials exhibited a pattern where an initial activation was followed by a steep reduction in motor excitability. We now explore the activation and reduction dynamics in more detail.

Percent-change across time points on NoGo trials

The activation phase on NoGo trials evidently occurred from 100–150 ms after stimulus onset, while the reduction phase occurred from 150–250 ms (Fig. 2B). To quantify the MEP change across time, we calculated the percent-change from 100–150 ms (constituting the activation phase), as well as the percent-change from 150–250 ms (constituting the reduction phase) for the high and low reward stimuli. In the activation phase, there was some evidence for an early increase in motor excitability for the high (t₂₅ = 2.3, p = 0.03, uncorrected, d = 0.45), but not the low reward stimulus (low: t₂₅ < 1, n.s, uncorrected, d = 0.15). In the reduction phase, both the high and low reward stimuli showed significant decreases in motor excitability (t₂₅ = 6.31, p < 0.001 and t₂₅ = 3.18, p = 0.03, respectively); however, the effect size was more than twice as large for the high reward stimulus (high reward: d = 2.5, low reward: d = 1.2) (Fig. 2C). Moreover, only the high reward stimulus showed a difference in the percent-change values between the activation and reduction phases (high reward: t₂₅ = 4.96, p < 0.001; low reward: t₂₅ = 2.18, n.s.) (Fig. 2C). Taken together, these results support the hypothesized activation-reduction dynamics, and also suggest that a larger initial increase in motor excitability (induced by the high reward stimulus) influences the dynamics of the reduction phase. This is in contrast to the possibility that the activation and reduction processes are independent on one another, which would result in similar reduction slopes regardless of differences in initial activation.

Relationship between reward-based activation and reward-based reduction on NoGo trials

We next asked if, across participants, the reward-based activation (percent-change in MEPs for high minus low reward from 100–150 ms) correlated with the reward-based reduction (percent-change in MEPs for high minus low reward from 150–250 ms). There was a strong negative correlation (r₂₅ = −0.74, p < 0.001), such that participants who showed stronger reward-based activation also showed a stronger reward-based reduction (Fig. 2D). For exploratory purposes, we also tested the relationship between the activation and reduction phases for the low and high reward NoGo trials separately. For low reward trials, there was a significant correlation between the activation and reduction processes (r₂₅ = −0.52, p = 0.006). For high reward trials, the relationship between the activation and reduction processes did not reach significance (r₂₅ = −0.32, p = 0.11). However, as the results were strongly influence by one significant outlier (Mahalanobis distance > 3), the relationship was significant with a non-parametric Spearman’s test (ρ = −0.49, p = 0.01), and when the outlier was removed from the analysis (r₂₄ = −0.43, p = 0.03). Together, these results show that the degree of reduction on NoGo trials is influenced by the strength of the preceding activation. This could be explained by mere passive decay (what rises higher has further to fall), or by a top-down control process. This distinction is tested in Experiment 2.

Relationship between NoGo dynamics and error rates

We next examined how the activation, the reduction, and the activation-reduction processes together (reflected in the activation-reduction index) related to participants’ self-control failures. Results showed that neither the activation nor reduction processes significantly correlated with participants’ overall NoGo error rates (activation phase: r₂₅ = 0.34, p = 0.09; reduction phase: r₂₅ = 0.26, p = 0.21). However, there was a significant correlation between the activation-reduction index and NoGo error rates (r₂₅ = 0.44, p = 0.02) (Figs. 3A–C). Specifically, those people who showed a relatively larger increase in the activation phase compared to the decrease in the reduction phase made more errors on NoGo trials.

Figure 3. Relationship between the NoGo error rate and the different phases of NoGo trials.

(A) Greater activation from 100–150 ms was marginally positively correlated with the NoGo error rate. (B) Greater reduction from 150–250 ms was positively (but non-significantly) correlated with the NoGo error rate. (C) The activation-reduction index on NoGo trials (a composite measure of the activation and reduction phases) showed a significant positively correlation with the NoGo error rate.

Go and NoGo dynamics for fast and slow RT groups

The strength of response activation in a reward task such as this likely depends on the participant’s basic level of motivational drive for reward. We therefore split participants into two groups based on RT on Go trials (fast RT vs. slow RT, ostensibly reflecting high and low motivation, respectively). For Go trials, a mixed ANOVA with the within-subject factors Reward (high, low) and Pulse Time (100, 150, 200, 250 ms) and Group as a between-subject factor (fast RT, slow RT) revealed a significant main effect of Reward (F_1,24 = 5.68, p = 0.026), with greater MEPs for high versus low reward trials. There was also a significant Reward × Pulse Time interaction (F_3.72 = 2.86, p = 0.043) (Figs. 4A), as was the case in the main analysis with all participants. There was no main effect of group.

Figure 4. TMS dynamics based on fast RT and slow RT groups in Experiment 1.

(A) Go trial dynamics. High reward Go trials showed an increase in motor excitability across all four time points, while low reward Go trials showed a decrease from 100–200 ms, followed by an increase from 200–250 ms. High and low reward Go trials in the slow RT group largely resembled that of the fast RT group. (B) High reward NoGo trials in the fast RT group showed an initial steep increase (from 100–150 ms), followed by a sharp decrease (from 150–250 ms). This pattern was markedly different than low reward NoGo trials, which did not show the initial increase and also a less steep decrease. In contrast to the fast RT group, the slow RT group showed no differences in motor excitability between high and low reward NoGo trials during the activation and reduction phases. Follow-up analyses showed that group differences in the reward motor dynamics were only in the 150 ms time point. Error bars represent SEM across participants. ## = p < 0.05 for Reward × Pulse Time × Group interaction, # = p < 0.05 for Reward × Group interaction.

For NoGo trials, there was a significant main effect of Pulse Time (F_3,72 = 15.83, p < 0.001), in which MEPs began to decrease at 200 ms after stimulus onset. The Reward × Pulse Time × Group interaction was also significant (F_3,72 = 2.75, p = 0.049) (Figs. 4B). We investigated the triple interaction with separate Reward × Group ANOVAs for each of the four time points, as this would help reveal the specific time points that showed group differences in the reward motor dynamics. We found a significant Reward × Group interaction at only the 150 ms time point (F_1,24 = 7.46, p = 0.01), in which the fast RT group showed a larger MEP difference between high and low reward trials than the slow RT group. This impression was confirmed with t-tests that showed a significant high vs. low reward difference in the fast RT group (t₂₆ = 2.61, p = 0.02), but no difference in the slow RT group (t < 1, n.s.). It should also be noted that the difference in overall NoGo dynamics between the two groups cannot be readily explained by the group difference in overall RT, as this would predict similar patterns of activity, but at different latencies (which was not seen here). Thus, the group that responded more quickly overall on Go trials (more putative motivational drive) showed greater sensitivity to the high reward stimulus on NoGo trials, particularly at the 150 ms time point.

Percent-change across time points on NoGo trials for fast and slow RT groups

We now examined the percent-change (percent-change) from 100–150 ms and 150–250 ms for the activation and reduction phases on NoGo trials, for the two groups. An ANOVA with Reward (high, low) and Phase (activation, reduction) and Group (fast RT, slow RT) revealed a significant main effect of Phase (F_1,24 = 34.01, p < 0.001), as well as a Reward × Phase × Group interaction (F_1,24 = 6.57, p = 0.017). Follow-up ANOVAs for the activation and reduction phases separately showed significant Reward × Group interactions for both phases (activation: F_1,24 = 5.62, p = 0.026; reduction: F_1,24 = 5.41, p = 0.029). This is in line with the result above that pinpointed the 150 ms time point as the locus for differential reward motor dynamics across the groups, as it is the only time point that contributes to the percent change in both the activation and reduction phases.

Relationship between reward-based activation and reward-based reduction on NoGo trials for fast and slow RT groups

A Pearson’s correlation for the fast RT group showed a strong negative correlation between the reward-based activation and reward-based reduction across individuals (r₁₂ = −0.84, p < 0.001) (Fig. 5C). This correlation was not present for the slow RT group (r₁₂ = −0.39, p = 0.18) (Fig. 5D). A direct comparison of the two correlation coefficients using a Fisher r-to-z transformation showed that the correlation for the fast RT group was significantly stronger than that of the slow RT group (Z = 1.82, p = 0.03, one-tailed). This again indicates that the reduction phase depends on the strength of preceding reward-based activation.

Figure 5. Percent-change across time points for fast and slow RT groups on NoGo trials Experiment 1.

(A) During the activation phase, the fast and slow RT groups showed different reward motor dynamics, resulting in a significant Group × Reward interaction. Specifically, the fast RT group showed greater sensitivity to the high versus low reward stimulus. (B) During the reduction phase, there was also a significant Group × Reward interaction, where the fast RT group again showed greater sensitivity to the high versus low reward stimulus. (C) The fast RT group showed a significant correlation between the degree of reward-based (i.e. high minus low reward) activation and reward-based reduction. (D) The correlation for the slow RT group did not reach significance; further, a Fisher’s r-to-z transform test revealed that the correlation for the fast RT group was significantly greater than the slow RT group. Error bars represent SEM across participants. #p < 0.05 for Reward × Group interaction.

Relationship between trait impulsivity and reward-based MEP differences

Trait impulsivity was only significantly correlated with the peak activation point on NoGo trials (r₂₅ = 0.39, p = 0.047, uncorrected for four comparisons), such that higher impulsivity was related to greater sensitivity to the high versus the low reward stimulus at the peak point of activation. This suggests that trait impulsivity is related to the reward value in the activation process.

DISCUSSION

TMS was delivered at 100, 150, 200, or 250 ms after a high or low reward stimulus on Go and NoGo trials in order to map the dynamics of putative response activation and control. On Go trials, the high reward stimulus produced an early motor activation (within 150 ms) that preceded the average RT by almost 350 ms. In contrast, the low reward stimulus showed an initial decrease in motor activation (from 100–200 ms), followed by an increase (from 200–250 ms)², resulting in a significant high versus low reward difference at 150 ms.

On NoGo trials, both reward stimuli (especially the high reward stimulus) induced a brief increase in motor excitability (from 100–150 ms), followed by a sharp reduction (from 150–250 ms) that reached levels far beneath the pre-stimulus baseline. Notably, those participants with greater activation also showed greater reduction on NoGo trials. This suggests that the dynamics of the reduction phase depend on the strength of early reward-driven activation and also suggests that both processes are important when evaluating one’s ability to withhold a reward-driven action. In support of this, we found that only a measure that takes into account both the activation and reduction processes together was predictive of participants’ overall errors rates on NoGo trials. This indicates that higher levels of reward-driven activation are detrimental if a proportionately larger reduction process does not follow. Furthermore, this result highlights the importance of using self-control paradigms that can capture both the provocation and control processes with high temporal resolution.

We also found that individuals with higher motivational drive showed greater sensitivity (i.e. stronger activation and reduction processes) to the high versus low reward stimulus on NoGo trials. This indicates that the influence of reward value on the activation-reduction dynamics was highly dependent upon participants’ motivational drive for reward and also provides further support for the close activation-reduction relationship. Finally, individual differences analyses across all participants revealed that at the peak point of activation (150 ms), trait impulsivity was positively correlated with the degree of reward-based activation. This indicates that trait impulsivity is related to the reward value in the activation process and, in accordance with our other findings, suggests that greater recruitment of control mechanisms may be required when impulsive individuals view a high reward stimulus. However, these results should be interpreted with caution, as this analysis was not corrected for multiple comparisons.

We interpret the sharp, beneath-baseline reduction in motor excitability on NoGo trials as a top-down suppression process that depends on the strength of preceding activation. However, other accounts exist. For example, it is possible that a similar degree of control is instantiated on high and low reward NoGo trials, and that the steeper reduction on high reward trials is simply a side-effect of there being higher initial activation (i.e. further to ‘fall’). In this case, the reduction phase would reflect a control process that does not necessarily depend on the strength of preceding activation. It is also possible that the reduction phase merely reflects a passive withdrawal of voluntary drive, which could also manifest in reduced motor excitability. In the next experiment, we test these competing accounts using a well-established finding that failures in self-control tend to increase when immediately following a very demanding task (Baumeister & Heatherton, 1996; Heatherton & Wagner, 2012). We reasoned that if there is top-down response suppression and its strength depends on the preceding activation, then there should be more effortful control recruited on high compared to low reward NoGo trials. This then predicts that depleting top-down resources with a demanding earlier task will increase the NoGo error rate more for high versus low reward trials in our rewarded Go/NoGo paradigm.

EXPERIMENT 2

Two groups of participants performed the rewarded Go/NoGo paradigm before and after an extended working memory (WM) task (one group easy; one group difficult). We chose a WM manipulation because it allowed us to differentially tax top-down control brain regions, including lateral prefrontal and parietal cortices (Braver et al., 2001; Owen, McMillan, Laird, & Bullmore, 2005; Zanto, Rubens, Thangavel, & Gazzaley, 2011) that would ostensibly be important for controlling the rapid activation on high reward NoGo trials we observed in Experiment 1.

Based on our hypothesis that top-down response suppression was engaged more on high versus low reward NoGo trials, we made two specific predictions. First, we predicted that when top-down resources are depleted (i.e. the High load group), the change in error rate for high reward NoGo trials would be greater than for low reward NoGo trials. Second, we predicted that when top-down resources are not depleted (i.e. the Low load group), there would be no difference between high and low reward trials.

METHODS

Participants

Forty-two (10 male) participants were tested (mean age = 21.36, SD = 5.5; all right-handed). Two participants were excluded due to technical malfunctions. One participant in the Low load group was excluded for having a high reward error rate of 50%, which was more than five standard deviations from the group mean. Thus, all analyses were run on thirty-nine participants, with nineteen participants in the Low load group (age = 21.5, SD = 7.7) and twenty participants in the High load group (age = 21.15, SD 1.9). All participants provided IRB consent.

Task and Procedure

There were three parts to the procedure (see Fig. 6A). In Part 1, all participants completed a task identical in design to the rewarded Go/NoGo task in Experiment 1 (Fig. 1), with the only difference being that there were now 4 blocks of 48 trials, yielding 208 total trials. As in Experiment 1, the first block was considered a “learning block” and was not included in the analysis. The data from this rewarded Go/NoGo task served as a baseline measure for each participant in order to determine the change in the NoGo error rate following the WM manipulation.

Figure 6. Experiment 2 task design and results.

(A) There were 3 parts in the task. In part 1, participants completed the rewarded Go/NoGo task (as in Experiment 1). This provided a baseline measurement for each participant’s NoGo error rate on high and low reward trials. In part 2, participants either completed a cognitively demanding three-back working memory task (High load) or a less demanding zero-back task (Low load). For the zero-back task, they were required to press a button (indicated by a red outline) whenever they saw the letter “P”. For the three-back task, they were required to press a button whenever the current letter matched the letter presented three letters before. In part 3, participants completed another rewarded Go/NoGo task to examine the change in error rates on high and low reward NoGo trials following the WM manipulation. (B) A proportion correct measurement shows that the high load (three-back) task was significantly more difficult than the low load (zero-back) task. (C) Only high reward trials in the High load group showed a significant increase in the error rate following the WM manipulation. This increase was significantly greater than the low reward trials in the High load group. The Low load group showed no difference between high and low reward NoGo trials and no pre-post changes in error rates. Error bars represent SEM across participants. *p < 0.05, ***p < 0.001.

In Part 2, participants were assigned to either the Low or High load WM group. For both groups, consonant letters appeared one at a time on the screen (letter duration: 0.75 s; ITI duration: 1.75 s). Participants in the Low group were instructed to make a response (using their left index finger to make the tasks as orthogonal as possible) as quickly as possible every time the letter “P” appeared on the screen (on trial n). For all other letters, no response was to be made. Participants in the High group were instructed to make a response (using their left index finger) as quickly as possible every time they saw the same letter as presented three letters before (on trial n-3). For all other letters, no response was to be made. Participants in this group were told that in order to complete the three-back task, they had to hold three letters at a time in working memory and continuously update the three letters with every new letter presentation. For both groups, there were 6 blocks of 100 analyzable trials (for the three-back task, the first three trials of each block were excluded), yielding a total of 600 analyzable trials. Participants were given a 20 s break between each of the 6 blocks. All participants completed a practice session of 30 trials. In total, Part 2 took approximately 30 minutes for both groups.

In Part 3, participants again completed the rewarded Go/NoGo task. However, there were now 3 blocks of 48 trials (yielding 144 total trials), but with no learning block (since the color-reward relationships had already been learned in Part 1). Thus, the number of analyzable trials was identical for Parts 1 and 3.

Data analysis

The main dependent measure was the change in error rate on NoGo trials from before the WM manipulation to after (pre-to-post). We therefore calculated post minus pre difference scores in the error rate for high and low reward in both load groups. To test for pre-to-post changes in error rate, we used one-sample t-tests to compare the conditions against a value of zero (representing no pre-to-post change). We also directly compared the pre-to-post change for high versus low reward with paired t-tests. Based on the strong directional predictions, one-tailed tests were used.

RESULTS

We first verified that the WM manipulation was successful. A two-sample t-test showed that performance in the Low load condition was significantly better than in the High load condition (t₃₈ = 8.58, p < 0.001) (Fig. 6B). We then verified that, prior to the WM manipulation, there were no group differences in high minus low reward RT or total errors (all Ps > 0.1; see Table 1 for behavioral measures). Lastly, we verified that there were no pre-to-post changes or group differences for overall RT or number of presses (t < 1, n.s.), suggesting that cue processing speed and motivational drive were not affected by the WM manipulation.

Table 1.

Behavioral measures for Experiment 2

	Load	Overall RT (ms)	High RT (ms)	Low RT (ms)	Overall Error Rate	High Error Rate	Low Error Rate
Pre	Low	419.3 (62)	406.1 (64)	434.7 (67)	4.86 (4.67)	3.65 (3.47)	1.46 (2.68)
	High	418.4 (40)	408.2 (48)	431.1 (43)	3.19 (4.44)	1.81 (2.89)	1.39 (2.63)
Post	Low	410.6 (63)	393.4 (60)	425.7 (64)	6.26 (5.32)	4.39 (3.38)	2.19 (3.28)
	High	412.2 (43)	398.3 (43)	430.9 (48)	3.89 (6.40)	3.06 (3.92)	0.83 (3.13)

Our main analysis showed that, in the High load group, there was a significant pre-to-post increase in error rate for the high (t₁₉ = 2.13, p = 0.02), but not the low reward stimulus (t₁₉ < 1), and a significant difference between the high and low reward stimuli (t₁₈ = 2.29, p = 0.02) (Fig. 6C). For the Low load group, there were no significant pre-to-post changes for either the high or low reward stimulus, nor was there a difference between the two conditions (all P’s > 0.2) (Fig. 6C). A direct comparison between the groups using a mixed ANOVA with Load (High, Low) as a between-subject factor and Reward (high, low) as a within-subject factor showed a trending interaction (F_1,37 = 2.1, p = 0.08).

DISCUSSION

One group of participants underwent the rewarded Go/NoGo task, then an easy (low load) WM task, then the rewarded Go/NoGo task again; whereas, another group of participants did the same sequence but with a difficult (high load) WM task. The high load version was an effortful three-back WM paradigm that putatively “depletes” cognitive resources commonly involved in top-down control (Chmielewski, Mückschel, Stock, & Beste, 2015; Mitchell, Macrae, & Gilchrist, 2002). In the high load group, we observed a significant pre-to-post increase in the NoGo error rate only for high reward trials, which was significantly greater than the pre-to-post change for low reward trials. This was not the case for the low load group, most likely because top-down resources were not depleted. Ideally, there would also be a significant difference between groups, which was only present here at a trend level. However, the comparison was not between high WM and no intervening task, but between high and low load WM (which would also be depleting to some extent). Notwithstanding, the increased error rate for high versus low reward NoGo trials in the high WM group suggests that greater top-down control is needed on high reward NoGo trials. This, along with the evidence for stable levels of motivation, argues against the possibility that the reduction phase reflects a withdrawal of voluntary drive and is in line with the hypothesis that it reflects a top-down suppression process that is related to the strength of preceding activation.

GENERAL DISCUSSION

Recent studies suggested that motivationally-driven action tendencies can be countered by a response suppression mechanism (Freeman et al., 2014; 2015), but they did not reveal the putative activation-suppression dynamics. It was therefore unclear if the motor reduction previously observed on NoGo trials was preceded by an early rise in reward-driven motor activation, and whether or not the reduction was directly related to the putative early activation. Here, we employed a rewarded Go/NoGo paradigm with better characteristics for mapping out the corticospinal dynamics. In Experiment 1, we found that NoGo trials showed an initial activation phase (within 150 ms), followed by a sharp reduction phase (within 200 ms) that fell beneath pre-stimulus baseline levels by 250 ms post-stimulus onset. The activation-reduction pattern was more pronounced (i.e. showed a greater magnitude in the slope change) for high versus low reward NoGo trials, indicating that the reduction phase was related to the degree of preceding activation. In support of the activation-reduction link, there was a strong correlation between the amount of activation and reduction across individuals, suggesting that both processes are important when evaluating one’s ability to withhold a reward-driven action. In line with this, we found that a measure that takes into account both the activation and reduction processes together was predictive of participants’ overall errors rates on NoGo trials. Moreover, sub-group analyses revealed the importance of taking into account individuals’ basic motivational drive for reward, as individuals with higher drive showed greater sensitivity (i.e. response activation) towards a high-reward stimulus, followed by a steeper reduction slope. We hypothesized that the steeper reduction slope reflects greater top-down suppression. If so, then depleted cognitive resources should increase the NoGo error rate more for high versus low reward trials, which is what we found in Experiment 2. Thus, our results show that when it is inappropriate to respond, reward-predicting stimuli still induce an early rise in motor activation that is subsequently controlled. This leads to a reduction of motor excitability well beneath baseline, and the strength of this reduction appears to depend on the strength of the preceding activation. Together, these results suggest that controlling reward-driven responses may critically depend on the tight relationship between the activation and reduction phases, and that a weakened reduction process can lead to failures in self-control.

What is the top-down control process that apparently “kicks in” on high reward NoGo trials? One possibility is that it is response suppression, as we have previously postulated. This is consistent with many response control studies that have demonstrated the recruitment of an active suppression mechanism that countermands an action tendency from a prepotent or an already-initiated response, for example in the stop signal paradigm (Aron & Poldrack, 2006; Aron, 2007; Schmidt, Leventhal, Mallet, Chen, & Berke, 2013). It is also consistent with several stop signal and Go/NoGo studies that have used spTMS to characterize the timing of the putative response suppression process on stop or NoGo trials. In particular, those studies have typically observed response suppression at 140–200 ms after stimulus onset (Coxon, Stinear, & Byblow, 2006; Hoshiyama et al., 1997; Stinear et al., 2009; van den Wildenberg et al., 2010; Yamanaka et al., 2002), which closely mirrors the timing in the current study.

An alternative top-down control process could be attentional modulation. This could direct resources away from the reward stimulus and/or towards the NoGo cue (Giesbrecht, Woldorff, Song, & Mangun, 2003; Harris, Hare, & Rangel, 2013; Hickey & Peelen, 2015; Hopfinger, Buonocore, & Mangun, 2000). For example, the study by Harris, Hare, and Rangel (2013) found evidence for an early attentional filtering mechanism during the exercise of self-control in the face of appetitive food items. Notably, the attentional filtering mechanism was instantiated 150–200 ms after stimulus onset, which also closely resembles the timing of the reduction phase in the current study. It is therefore possible that, here, the reduction phase actually reflects a reduction in motivational drive following reduced processing (via attentional control) of the reward stimulus. Notwithstanding this possibility, this attention explanation has difficulty accounting for the beneath-baseline reduction we observed. Instead, an attentional filtering of the reward stimulus would more likely cause a reduction in motor excitability to pre-stimulus onset levels where no stimulus is displayed.

Future studies could more definitively disentangle such mechanisms with functional neuroimaging. For example, a response suppression account predicts the involvement of regions implicated in stopping action, such as the right inferior frontal gyrus, pre-supplementary motor area, and the subthalamic nucleus (Aron & Poldrack, 2006; Aron, 2007; Chambers, Garavan, & Bellgrove, 2009; Garavan, Hester, Murphy, Fassbender, & Kelly, 2006). Alternatively, the attentional control account predicts the involvement of regions implicated in down-modulating task-irrelevant distractors, such as the superior frontal cortex, inferior frontal junction, and parietal cortex (Giesbrecht et al., 2003; Hopfinger et al., 2000; Zanto et al., 2011). Ultimately, clarifying the underlying mechanism could help inform when and how control is implemented over reward-driven provocations. In turn, this information could be useful in determining optimal tasks that may be used to train individuals over extended periods of time in an effort to reduce failures in self-control.

Here, we show that one condition that leads to increased failures in self-control is when a strong activation must be withheld (i.e. on high reward NoGo trials) after top-down resources have been heavily taxed. This result fits with a large literature that consistently finds increased failures in self-control immediately after cognitive resources have been “depleted” in a separate effortful task (often referred to as “ego depletion”) (Baumeister & Heatherton, 1996; Hagger, Wood, & Stiff, 2010). To account for these findings, it is thought that self-control draws from a somewhat global, limited resource and that exhausting it reduces the amount (or allocation) of available self-control resources to be deployed in the near future (Baumeister & Heatherton, 1996; Baumeister, 2014; Gailliot et al., 2007). An alternative theory explains the decrement in self-control as a decrease in participants’ motivational state during the second task (Inzlicht, Schmeichel, & Macrae, 2014). Our results argue against the motivational account, as we found no pre-to-post changes in participants’ motivational drive for reward (measured via RT and number of presses). Moreover, it is unclear why the motivational change would only occur for high reward NoGo trials, as the low reward NoGo trials showed no pre-to-post change. Instead, our results suggest that the reduction phase reflects a top-down control process, and that the implementation of top-down control is affected by a demanding WM task.

The current approach has greater ecological validity than typical studies of response control, as we have studied the control over a reward-driven response tendency, rather than merely a response tendency that is pre-established or automatic (as in the Simon or Flanker tasks). Yet, our approach is still limited by the fact that the NoGo cue is an external signal³. In many real-world situations of self-control, there is no cue or signal instructing individuals to withhold an action. Instead, people must often generate the control process in an endogenous manner (there are, however, some real-world situations that are analogous to the current case; for example, the NoGo trials in the current study are perhaps analogous to the scenario in which a smoker views a pack of cigarettes that has a large warning message on the front). While paradigms have been designed to investigate endogenous control (for review, see Filevich, Kühn, & Haggard, 2012; Ridderinkhof, van den Wildenberg, & Brass, 2014), studying endogenous control poses several challenges. For one thing, withholding a response endogenously is a subjective, decision-based process, which makes it difficult to measure a response inhibition failure. For another, the timing of the activation and control processes are more variable, which could limit the use of techniques such as spTMS to map the dynamics. Future studies will therefore benefit from discovering neural markers that signify both the activation and control processes within a single task, as this will allow a “readout” of the their timing and relative strength during endogenous recruitment. A second limitation of the current study is that, from Experiment 2, we could only infer that the increase in errors was due to a change in the reduction phase dynamics. A future study could more definitively establish that this is the case using spTMS with the rewarded Go/NoGo task following a depletion manipulation. We predict that, whereas the activation phase would show a similar pattern as we observed here, the reduction phase on high reward trials would show a less steep decrease in motor activity, thereby eroding its relationship with the preceding activation.

In conclusion, we show that when a reward-driven action was withheld, there was an initial rise in motor activation that was modulated by the value of the reward-predicting stimulus and the individual’s motivational drive for reward. Further, the initial activation phase was followed by a steep reduction in motor excitability, with the degree of the reduction corresponding to the strength of preceding activation. This pattern of dynamics, along with the observation that an effortful task apparently depletes the ability to withhold a response in the face of the high reward stimulus, suggests that the control process involved top-down response suppression. Future studies could validate this, which would highlight the importance of using response suppression to control provocations driven by the motivational content of a stimulus. More generally, these dynamics suggest that failures in controlling reward-driven actions may be due to insufficient or depleted response suppression mechanisms that follow a quick rise of reward-driven activation. This may explain why self-control is more difficult and fails more often when following demanding tasks (Baumeister, Bratslavsky, Muraven, & Tice, 1998; van der Linden, Frese, & Meijman, 2003) or consumption of substances (e.g., alcohol; Kähkönen, Wilenius, Nikulin, Ollikainen, & Ilmoniemi, 2003; Moselhy, Georgiou, & Kahn, 2001) that reduce functioning in brain regions involved in top-down control. Specifically, our findings suggest that reduced functioning in top-down control may lead to a weakened suppression process, contributing to failures in self-control.