Easy to Learn, Hard to Suppress: The Impact of Learned Stimulus-Outcome Associations on Subsequent Action Control

Abstract

The inhibition of impulsive response tendencies that conflict with goal-directed action is a key component of executive control. An emerging literature reveals that the proficiency of inhibitory control is modulated by expected or unexpected opportunities to earn reward or avoid punishment. However, less is known about how inhibitory control is impacted by the processing of task-irrelevant stimulus information that has been associated previously with particular outcomes (reward or punishment) or response tendencies (action or inaction). We hypothesized that stimulus features associated with particular action-valence tendencies, even though task irrelevant, would modulate inhibitory control processes. Participants first learned associations between stimulus features (color), actions, and outcomes using an action-valence learning task that orthogonalizes action (action, inaction) and valence (reward, punishment). Next, these stimulus features were embedded in a Simon task as a task-irrelevant stimulus attribute. We analyzed the effects of action-valence associations on the Simon task by means of distributional analysis to reveal the temporal dynamics. Learning patterns replicated previously reported biases; inherent, Pavlovian-like mappings (action-reward, inaction-punishment avoidance) were easier to learn than mappings conflicting with these biases (action-punishment avoidance, inaction-reward). More importantly, results from two experiments demonstrated that the easier to learn, Pavlovian-like action-valence associations interfered with the proficiency of inhibiting impulsive actions in the Simon task. Processing conflicting associations led to more proficient inhibitory control of impulsive actions, similar to Simon trials without any association. Fast impulsive errors were reduced for trials associated with punishment in comparison to reward trials or trials without any valence association. These findings provide insight into the temporal dynamics of task irrelevant information associated with action and valence modulating cognitive control. We discuss putative mechanisms that might explain these interactions.

1. Introduction

The brain's spontaneous processing of irrelevant information can directly affect performance, even to the point of leading behavior astray or interfering drastically with efficient completion of goal-directed actions. This is perhaps best illustrated by so-called conflict tasks, such as Simon (Simon, 1969) or Flanker (Eriksen & Eriksen, 1974) tasks, in which the ineludible processing of irrelevant information in a visual display activates a response tendency that directly conflicts with goal-directed action. These tasks not only permit investigation of the brain's susceptibility to processing irrelevant information and preparing incorrect responses, but also how cognitive control is engaged reactively to inhibit this processing and suppress interference from inappropriate actions that are triggered by irrelevant stimuli.

While these cognitively or perceptually-driven forms of stimulus-response associations are undoubtedly an influential source of conflict in daily life, the processing of other forms of irrelevant information may also contribute to conflict and directly influence cognitive control processes. In particular, stimuli, relevant or irrelevant to behavioral goals, that have been associated with reward and its acquisition (and potentially punishment and its avoidance) are potent modulators of our attention and directly engage relevant circuitries involved in reward processing (Anderson, Laurent & Yantis, 2011a,b; Della Libera & Chelazzi, 2011; Della Libera, Pelato & Chelazzi, 2009; O'Connor, Upton, Moore & Hester, 2015;Raymond & O'Brien, 2009; for a review see Chelazzi, Perlato, Santandrea, & Della Libera, 2013).

The resolution of conflict in Simon and related response conflict tasks involves cognitive control circuitries engaging prefrontal and motor areas of the frontal cortex and the basal ganglia (Botvinick, Braver, Barch, Carter & Cohen, 2001; Forstmann, van den Wildenberg & Ridderinkhof, 2008; Ridderinkhof et al., 2011). A central component of cognitive control in times of conflict is the inhibition of conflicting response alternatives, which has been linked to frontal projections to basal ganglia that engage the indirect and hyperdirect basal ganglia pathways to brake actions selectively (Aron et al., 2007; Jahfari et al., 2011; Mink, and Thach, 1993). Disorders associated with basal ganglia dysfunction produce pronounced deficits in conflict resolution and inhibitory control, and pharmacological (e.g., dopamine) and deep brain stimulation manipulations of basal ganglia function modulate these processes directly (Gillan et al., 2011; Holl et al., 2013; Worbe et al., 2011; Wylie et al., 2009a, 2009b, 2010, 2012, 2013).

These same cortical-basal ganglia circuitries are implicated in reinforcement learning, reward expectation, and the formation of stimulus-action-outcome associations (Alexander, DeLong & Strick, 1986; Bogacz & Guerney, 2007, Aron et al., 2007; Schultz, 2002; McClure, Berns & Montague. 2003). In fact, recent theories postulate roles for long-term potentiation and depression in direct and indirect basal ganglia pathways as a potential mechanism for associating action and inaction to reward acquisition and punishment avoidance (Frank and Fossella, 2011; Kravitz et al., 2012). While action control and action-outcome processes are generally studied in isolation, emerging ideas suggest a potential interface in basal ganglia circuitries (modulated by dopamine) that integrates action control and valences of action outcomes.

1.1 Current study

The central aim of the current investigation was to determine how response conflict in a Simon task and inhibitory control processes involved to resolve this conflict are influenced by the simultaneous processing of irrelevant information that has been associated previously with reward acquisition or punishment avoidance. Encountering irrelevant stimulus information associated with a particular valence may activate reward (or punishment) processing circuits of the brain directly (i.e. the fronto-striatal connections that are activated when receiving actual reward outcomes, D'Ardenne, McClure, Nystrom, & Cohen, 2008; McClure, Berns, & Montague, 2003; O'Doherty, Dayan, Friston, Critchley, & Dolan, 2003; Pagnoni, Zink, Montague, & Berns, 2002), which may in turn impact one's susceptibility to acting on strong motor impulses or interact directly with the control processes engaged to inhibit impulsive response tendencies.

Our general approach was to embed stimulus information associated previously with reward acquisition or punishment avoidance as irrelevant attributes of the visual display in a conventional Simon conflict task. In many reinforcement learning paradigms, only overt actions are associated to reward or punishment outcomes (i.e., instrumental learning) (Frank, Seeberger & O'Reilly, 2004; O'Doherty, 2004; van Wouwe, Ridderinkhof, Band, van den Wildenberg & Wylie, 2012). However, in many situations, refraining from action is necessary for reward acquisition and punishment avoidance. Moreover, learning that only involves selection among overt action alternatives conflicts with inherent biases evoked by punishment (bias to refrain from action) (Cavanagh, Eisenberg, Guitart-Masip, Huys, & Frank, 2013;Everitt, Dickinson & Robbins, 2001; Freeman, Razhas & Aron, 2014; Freeman, Alvernaz, Tonnesen, Linderman & Aron, 2015; LeDoux, 1996; Gray & Naughton, 2000). To accommodate these issues, we adapted a probabilistic learning task to require either action or inaction to obtain reward or avoid punishment. That is, we orthogonalized valence (reward acquisition, punishment avoidance) and action choice (action, inaction) factors during the learning task so that participants learned each of four color stimuli representing a unique combination of these factors (see design pioneered by Guitart-Masip et al., 2012a,b).

We tested two alternative predictions based on prior work linking patterns of action-valence learning to specific neural effects. On the one hand, prior work shows that stimulus-action-outcome valences can energize or de-energize motor cortical activity; that is, stimuli associated with reward activate motor cortex, whereas stimuli associated with punishing outcomes decrease motor cortex activity, even before action selection takes place (motivation-to-action ‘spillover’ account; Chiu, Cools, & Aron, 2014). Based on this pattern, encountering irrelevant stimulus features associated with reward, and particularly reward and action, would be expected to activate motor cortex, which in the context of the Simon task, would potentiate impulsive actions and interfere with inhibitory control. Stimuli associated with punishment avoidance, and particularly punishment avoidance and inaction, would reduce motor cortex activity and produce opposite effects on behavior, thus reducing impulsive errors and making it easier to inhibit impulsive response tendencies triggered in the Simon task.

Alternatively, Guitart-Masip and colleagues (2012a,b) have demonstrated inherent biases during the learning of these action-valence associations that are accompanied by distinct effects on conflict signaling in the brain. Two conditions reflect natural biases between valence and action (i.e., action with reward, inaction with punishment avoidance), whereas two conditions conflict with these natural biases (i.e., action with punishment avoidance, inaction with reward). Learning and implementing the conflicting action-valence conditions are accompanied by medial prefrontal oscillatory activity commonly associated with conflict detection or conflict-induced control signals. These signals are absent or substantially reduced when processing the two conditions reflecting natural biases between valence and action (Cavanagh et al., 2013). Since the conflict control system is also engaged by the response conflict produced in the Simon task, encountering inherently conflicting valence-action associations should similarly activate the conflict control system, which would then be expected to either facilitate or, at a minimum, have little impact on the proficiency of conflict control required to resolve the motor conflict in the Simon task. In contrast, encountering inherently natural action-valence associations may interfere with the conflict control system (e.g., take it offline), the effect of which would be a disruption in the engagement of cognitive control to resolve the conflicting motor responses in the Simon task.

Both accounts predict that action-reward associations will likely reduce inhibitory control whereas the accounts differ with respect to predictions for inaction-punishment associations; according to the action-valence conflict account these natural associations might interfere with conflict control. According to the motivation-to-motor ‘spillover’ account on the other hand, irrelevant information associated with inaction or punishment, and particularly both inaction and punishment, might be expected to induce a bias toward action restraint and facilitate the proficiency of inhibition.

2. Method

2.1 Participants

Fifty-six subjects, with a mean age of 21.79 (STD =4.16), a slight female bias (33 female: 23 male) and mostly right handed (53 right handed: 3 left handed), participated in this study. Participants enrolled through Vanderbilt University's web-based research recruitment system. Exclusionary criteria included history of neurological condition, unstable mood disorders, bipolar affective disorder, schizophrenia, or other psychiatric or medical conditions known to compromise executive cognitive functioning. Participation in the study was voluntary, and informed consent, compliant with the standards of ethical conduct in human research as governed by the Vanderbilt University human investigation committees, was obtained from all participants.

2.2 Design and Procedure

The basic task structure combined two experimental paradigms, a probabilistic action-valence learning task and the Simon task, which are described below and shown in Figure 1. Both tasks were run on computers using Eprime 2.1 software (Psychology Software Tools, Pittsburgh, PA). All stimuli were presented against a grey background on a 17-inch screen located approximately one meter from the participant and positioned so that the stimuli appeared at eye-level. Responses were made with the left and right thumbs using a handheld button box placed comfortably in the participant's lap.

Figure 1.

A) Two example trials of the action-valence learning paradigm. Subjects were instructed to maximize their winnings by either action or inaction. With each color subjects chose to act (press both buttons) or withhold action (do not press any button) and presented with feedback. On ‘an act to obtain reward’ trial, subjects were rewarded for acting. On ‘an act-to avoid punishment’ trial, subjects were not punished (acting yielded nothing) when acting. On a ‘withhold to obtain reward’ trials subjects were rewarded for inaction. On a ‘withhold to avoid punishment’ trial, subjects were not punished if they decided not to act. B) An example trial of the Simon arrow task with colors from the learning paradigm embedded. Subjects respond based on the direction of the arrow, irrespective of the location and color of the stimulus.

Subjects first completed practice sessions on both tasks (20 trials of the action-valence learning task plus 30 trials of the Simon task). Practice trials were similar to the actual experimental trials with the exception that the colors in the practice were not presented again during the actual learning and the Simon arrow trials were white instead of colored. After the practice, subjects performed an extensive session (3 blocks of 40 trials) of the action-valence learning task to associate combinations of action (action, inaction) and valence (reward acquisition, punishment avoidance) to a specific stimulus color. After forming these associations, subjects performed a first session of the Simon task (6 blocks of 64 trials) that required left or right hand button presses to left- or right-pointing arrow stimuli that appeared either to the left or to the right of fixation. The arrow stimuli were also colored using the same colors learned in the action-valence learning task. Thus, the color of the arrows represented a task-irrelevant feature of the Simon stimuli (i.e., responses were based on the direction of the arrows) that had been associated with acting or withholding action in order to gain reward or avoid punishment in the action-valence learning task.

After completing the initial session of the Simon task, subjects performed a shorter version of the action-valence learning task (1 block of 40 trials) to maintain the previously learned associations. This was followed by another session of the Simon task (6 blocks of 64 trials). This pattern was repeated once more. In total, subjects completed an extended plus two shortened sessions of the action-valence learning task with 3 interposed sessions of the Simon task. We next describe each task in detail.

2.2.1 Probabilistic Reward Learning Task

Subjects were instructed that the goal of this task was to learn to act or withhold action to each of 4 color patches in order to maximize monetary earnings by gaining rewards and avoiding punishments. Specifically, subjects viewed a series of color patches that were presented one at a time in the center of a computer screen. A trial began with the presentation of a centered fixation point for 750 ms. The fixation point was then replaced by the appearance of one of 4 colored patches at fixation that remained on the screen for 3500 ms (see Figure 1a). Upon the presentation of a color patch, subjects were instructed that they had 1500 ms to either act (i.e., make a two-handed button press) or to withhold action. After the 1500 ms window expired, feedback was displayed for 2000 ms in the center of the color patch indicating that the action decision led to monetary reward (+15 cents), monetary punishment (−15 cents), or no monetary outcome (0 cents). The feedback and color patch were then extinguished, and the next trial began. A running total of earnings was presented in the upper center of the screen throughout the task. The 4 color patches appeared in random order and with equal probability across three blocks of 40 learning trials that comprised the initial learning session. Thus, each color appeared 10 times within a block of trials for a total of 30 exposures across the three initial learning blocks. Each block of trials took 8.5 minutes to complete, with a brief 1 minute break between blocks.

Unbeknownst to the subject, two of the color patches provided outcomes that were either rewarded or unrewarded, and the remaining two colors provided outcomes that were either punished or unpunished. Thus, the former colors were associated with reward learning, whereas the latter colors were associated with punishment avoidance learning. Also unknown to the subject, 1 color from each set produced the optimal outcome (i.e., either gain reward or avoid punishment) by acting, but the other color from each set produced the optimal outcome by withholding action. This design completed the 2 × 2 factor design that orthogonalized both valence and action.

To make the learning challenging, we designed the task so that feedback was partly probabilistic rather than fully deterministic (i.e., rewards or punishments didn't occur with 100% certainty for a particular action choice). This semi-probabilistic design was applied to each color patch as outlined below:

• (1)Stimulus A: Learning to act to gain reward. Selecting action to this stimulus is rewarded 80% of the time (unrewarded 20%), but withholding action to it is unrewarded 100% of the time; only action yields reward.
• (2)Stimulus B: Learning to suppress action to gain reward. Suppressing (withholding) action to this stimulus is rewarded 80% of the time (unrewarded 20%), but selecting action to it is unrewarded 100% of the time; only withholding action yields reward.
• (3)Stimulus C: Learning to act to avoid punishment. Selecting action to this stimulus avoids punishment 80% of time (punished 20%), but withholding action to is punished 100% of the time; only acting yields punishment avoidance.
• (4)Stimulus D: Learning to suppress action to avoid punishment. Suppressing (withholding) action to this stimulus avoids punishment 80% of time (punished 20%), but selecting action to it is punished 100% of time; only withholding action yields punishment avoidance.

Data Analysis for Action-Valence Learning Task

As noted above, subjects performed 3 experimental sessions (1 extended, and 2 shortened) of the action-valence learning task. Across session, subjects completed 200 learning trials, including 50 trials for each color patch. Because our primary interest was in the effects of learning on conflict resolution and inhibitory control in the Simon task, we only included subjects (n=41) who demonstrated and maintained > 75% accuracy in learning the associations for each color patch after the initial learning session and subsequent shortened sessions (see Guitart-Masip et al., 2012a,b, for similar criterion). Accuracy was defined by the percentage of trials in which the subject selected the optimal response. We analyzed learning by first analyzing accuracy across the initial three blocks using within-subject analysis of variance techniques to distinguish effects related to the within-subject factors of Action (Action, Inaction), Valence (Reward Acquisition, Punishment Avoidance), and Block (1,2,3). Next, we confirmed that learned associations persisted by comparing learning in the final block of the initial session to learning in each block of the shortened sessions. Thus, the analysis was the same, but the Block factor now included only blocks 3, 4, and 5, each of which included 10 exposures to each color patch.

2.2.2 Simon Task¹

This task required participants to respond to the direction of an arrow that appeared either to the left or to the right of a centrally located, square-shaped fixation mark. The fixation mark appeared at the outset of a block of trials and remained on the screen until the block ended. A trial began with a variable and randomly selected foreperiod duration ranging from 700-1000 ms (in 50 ms increments) in which only the fixation mark was visible. This was followed by the onset of a left or right pointing arrow that remained on the screen for 400 ms (see figure 1B). The arrows subtended a 1.15 horizontal visual angle and an edge-to-edge separation of 0.8 cm from the fixation mark. Arrows appeared in one of the four colors that were used in the action-valence learning task. Each color was selected semi-randomly (i.e. color repetition on a trial by trial basis was excluded) and occurred with equal probability across a block of trials. Additionally, each color occurred with equal frequency for left and right pointing arrows and for corresponding and non-corresponding trial types (see description below). Subjects had 1200 ms to respond to the direction of the arrow (e.g., left pointing arrow=left-hand button press; right pointing arrow=right-hand button press), after which no response was recorded and the next trial commenced. The end of a block of trials was signaled by the offset of the fixation mark and an instruction screen.

Trials were defined as either corresponding (C) or non-corresponding (NC) depending on the correspondence between the response signaled by the spatial location of the arrow in the left or right visual hemifields and the response indicated by the direction of the arrow. For Corresponding trials, the spatial mapping and arrow direction signaled the same response (i.e., the arrow appeared to the side that the arrow pointed; e.g., a left pointing arrow appeared in the left visual half-field). Conversely, on Non-corresponding trials, the spatial mapping and arrow direction signaled conflicting responses (i.e., the arrow appeared to the side opposite the direction indicated by the arrow; e.g. a left pointing arrow appeared in the right visual half-field).

Subjects received feedback about their RT and accuracy at the end of each block to help ensure that they complied with the instructions.

Data Analysis for Simon Task

Data collected from the Simon task is conventionally analyzed using mean RT and accuracy rates to determine interference effects (i.e., RT slowing and reduced accuracy) produced on NC trials compared to C trials. Our task was designed to examine the effects of two additional within-subjects factors, Color Action (Action, Inaction), representing the action tendency associated previously with each color stimulus, and Color Valence (Reward Acquisition, Punishment Avoidance), representing the valence of the outcome associated previously with each color. After removing anticipatory reactions (RT < 150 ms) and outlying slow reactions (RT > 2 standard deviations slower than the mean) in each correspondence condition separately, which accounted for fewer than 1% of all trials, we analyzed mean RT and accuracy rates (square-root transformed to adjust for non-normal distribution) using repeated-measures analysis of variance techniques to determine the main and interactive effects of Correspondence (C, NC), Color Action (Action, Inaction), and Color Valence (Reward Acquisition, Punishment Avoidance).

Our primary approach to analyzing and interpreting action-valence effects on inhibitory control in the Simon task was guided by the Dual-Process Activation Suppression (DPAS) Model (Ridderinkhof, 2002; Wylie et al., 2009a,b; for a review see van den Wildenberg et al., 2010). The DPAS model of conflict effects provides a powerful framework for distinguishing the dynamics of incorrect response activation followed by its suppression (as an act of cognitive control). The strength of incorrect response activation, also referred to as response capture, is inferred by the percentage of fast, impulsive errors on conflict trials that are revealed by plotting accuracy rates as a function of RT (i.e., conditional accuracy functions, or CAFs). Higher error rates at the fastest RTs indicate greater susceptibility to incorrect response capture (i.e.. stronger incorrect response activation). To calculate CAFs, the single-trial RT data, inclusive of both correct and incorrect responses, were first divided by color and by level of correspondence. RTs were then rank-ordered from fastest to slowest and divided into 7 equal-sized bins. Mean accuracy rates were calculated for each bin and plotted as a function of the mean RT for each bin separately for each color and level of correspondence. The percentage of fast, impulsive errors for the fastest RT bin was used as the primary dependent measure to infer the strength of incorrect response activation, with higher error rates associated with stronger incorrect response activation (Kornblum et al., 1990; see Ridderinkhof, 2002).

According to the DPAS model, the activation of incorrect response impulses in the Simon task triggers reactive cognitive control processes to suppress the incorrect response activation. This is revealed by plotting interference effects (RT difference between conflict and no-conflict Simon trials) as a function of RT (i.e. a delta plot). In Simon tasks delta plots show a characteristic form in which interference effects increase across early to intermediate response latencies, but transition to a reduction of interference toward the slow end of the RT distribution. The DPAS model posits that this transition from a positive-going slope function to a negative-going slope function reflects the engagement of inhibitory control that suppresses the interference from the conflicting response. Without any interference suppression, the DPAS model predicts a linear increase between RT and the Simon Effect (which would result in a positive going slope). Thus, the proficiency of inhibitory control is inferred by the steepness of the negative-going slope associated with the suppression of response interference (i.e., the suppression slope). The DPAS model has received broad empirical support from behavioral, electromyographic, and imaging studies (e.g., Forstmann et al. 2008; Forstmann, van den Wildenberg, & Ridderinkhof, 2008; see Ridderinkhof, Forstmann, Wylie, Burle, & van den Wildenberg, 2011, for a review). Using the DPAS framework, we investigated how the processing of irrelevant information associated with action-valence tendencies impacted the strength of incorrect response capture versus the proficiency of inhibitory control engaged to resolve response conflict.

For the delta-plots, the same binning procedure was used, but only using correct response data. Next, the Simon effect (RT_NC – RT_C) was calculated for each bin and plotted as a function of the mean RT for each bin separately by color. The slope connecting the two slowest RT bins has been found to be the most sensitive measure of the proficiency of response suppression and, accordingly, our analysis focused on comparing this suppression slope from the final segment of the delta plot across color conditions (Wijnen & Ridderinkhof, 2007; Burle, van den Wildenberg, & Ridderinkhof, 2005; Burle, Possamai, Vidal, Bonnet, & Hasbroucq, 2002; van den Wildenberg et al., 2010).

While our primary predictions centered on action-valence effects on inhibitory control (i.e., the suppression slope of the delta plot), the DPAS framework allows investigation of how these action-valence associations affect susceptibility to acting on fast, impulsive response impulses. Based on findings that visual distractors associated with reward more strongly capture visual attention, it is possible that the processing of action-valence associations may directly increase susceptibility to acting on the spatially-driven response. In particular, the processing of color associated with reward or action tendencies might enhance capture by the spatially-driven response, whereas processing of color associated with punishment avoidance or withholding action might have the opposite effect by reducing response capture. Thus, a secondary focus of our analyses was to test whether action-valence associations intervene on early response activation processes or on later inhibitory control processes.

3. Results

3.1 Final Sample

It was critical to our experiment that participants learned the four action-valance conditions so that these influences could be evaluated in the Simon task. Of the 56 participants enrolled in the study, 15 participants showed insufficient learning in at least 1 or more of the conditions and were excluded from further analysis. It is worth noting that the specific patterns of learning across the four action-valence associations remained unchanged whether or not this group was included or removed from the learning analyses.

Two additional participants showed RT and error rate performances on the Simon task that were more than 2 standard deviations above the group mean and were also excluded. A final sample of 39 participants showed intact learning of all 4 action-valence conditions across the initial and maintenance learning blocks as well as interpretable performance in the Simon task.

3.2 Probabilistic Action-Valence Learning

Figure 2a shows the average learning accuracy for each action-valence condition across the three blocks of learning trials comprising the initial, extended learning session and across blocks 4 and 5 representing learning sessions interposed later in the experiment. Learning accuracy improved across all three blocks of trials, (Block, F(1,38)=140.36, p<.001), with accuracy in block 3 higher than in each of the preceding blocks (all ps<.001). Learning rate for action (84%) was higher than for inaction (78%), (Color Action, F(1,38)=15.98 , p<.001), but learning to gain reward (82%) and avoid punishment (80%) showed similar rates, (Color Valence, F(1,38)= 2.06, p=.16).

Figure 2.

Learning accuracy as a function of trial block (10 trials per block) for each action-valence combination (a) and average accuracy at the end of the initial learning phase (b) .

Learning varied as a function of combined action-valence associations, (Color Valence × Color Action, F(1, 38)= 67.98 , p<.001). As easily seen in Figure 2a (initial learning session), the ordering of learning accuracy across the 4 color conditions remained relatively invariant across learning blocks, but the magnitude of learning rate differences between conditions reduced across the blocks (Color Action × Color Valence × Block, F(2,76)= 4.88 , p<.05). To visualize these learning biases more clearly, the learning rates for each condition from the final block of learning (block 3) are plotted in Figure 2b. Replicating Guitart-Masip et al. (2012a,b), participants more easily associated action to reward than inaction to reward (t(38)=4.63, p<.001), and more easily associated inaction to punishment avoidance than action to punishment avoidance (t(38)=2.22, p<.05). These patterns confirmed inherent biases that map action to reward and inaction to punishment avoidance.

Comparing the final block 3 of the initial learning session with the refresher learning blocks 4 and 5 confirmed that learning was maintained from the initial session and, in fact, steadily increased across blocks of trials (block 3 = 92%, block 4 = 94%, block 5 = 96%)(Block, F(2, 38)= 7.23 , p<.01). All remaining patterns of effects remained similar to patterns reported above across blocks 1-3. These analyses confirm that learning remained stable across interposed learning sessions.

3.3 Simon

3.3.1 Mean Effects

Table 1 shows mean RT and accuracy rates separately for each of the four action-valence conditions. Across conditions, a robust Simon effect was produced; mean RTs slowed and accuracy rates were reduced on NC (RT: 397 ms; Accuracy: 90%) compared to C trials (RT: 363 ms; Accuracy: 97%) trials, (Correspondence, RT: F(1,38)=248.38, p<.001; Accuracy, F(1,38)=69.87, p<.001). Neither the action nor the valence associated with a color nor their shared variation influenced mean RT or accuracy rates, (Color Action, RT: F(1,38)=1.07, p=.31, Accuracy: F(1,38)=2.27, p=.14; Color Valence, RT: F(1,38) =.18, p=.68, Accuracy: F(1,38)=2.12, p=.15; Valence × Action, RT: F(1,38)=2.38, p=.13, Accuracy, F(1,38)=.002, p=.97). Thus, previously learned action-valence associations did not influence overall mean RTs or accuracy rates.

Table 1.

Experiment 1 Simon correspondence effects (reaction time and accuracy) separated by action and valence conditions.

	Action		Inaction
	Reward acquisition	Punishment avoidance	Reward acquisition	Punishment avoidance
RT (ms)
Corresponding	362 (5.2)	364 (5.0)	365 (5.0)	363 (5.2)
Non-Corresponding	396 (5.7)	396 (5.6)	397 (5.5)	397 (5.5)
Accuracy (% correct)
Corresponding	97.17 (0.3)	96.77 (0.4)	97.54 (0.4)	97.14 (0.3)
Non-Corresponding	89.85 (1.0)	89.55 (1.1)	90.23 (1.1)	89.95 (1.1)

The cost of non-correspondence (i.e., Simon effect) on both mean RT and accuracy rates was neither influenced by color processing associated previously with reward acquisition or punishment avoidance, (Correspondence × Valence, F(1, 38): RT, F=.10, p=.75; Acc, F =.02, p=.88), nor by color processing associated with action versus inaction, (Correspondence × Action, F(1, 38): RT, F=.04, p=.85; Acc, F<.01, p=.99). Moreover, the combined influence of Color Valence and Color Action processing did not affect mean correspondence effects, (Correspondence × Action × Valence, F(1,38): RT, F=1.66, p=.21; Acc, F<.01 , p=.98).

3.3.2 Response Capture

The CAFs for each of the four action-valence color stimuli are plotted in Figures 3A and 3B, respectively, for C and NC trials. Accuracy rates were uniformly high on C trials across the entire RT distribution. In contrast, a pattern of pronounced fast errors with high accuracy at intermediate and slow response latencies was observed for NC trials. The magnitude of fast errors (1^st bin) was significantly higher on NC (29% errors) compared to C trials (1% errors), (Correspondence, F(1,38)=366.47, p<.001).

Figure 3.

Conditional accuracy functions (accuracy plotted against mean bin RT, for the fastest (bin 1) to the slowest (bin 7) bins) for each action-outcome combination separate for corresponding (A) and non-corresponding responses (B).

Fewer impulsive, fast errors were made to colors associated with inaction than to colors associated with action (Inaction: 9%; Action: 11%), (Action, F(1,38)=5.14, p<.05), although this did not vary across correspondence conditions, (Action × Correspondence, F(1,38)=.77, p=.39). Colors associated with reward acquisition and punishment avoidance produced similar patterns of fast errors, (Valence, F(1,38)=.06, p=.81). None of the higher order interactions were significant (all ps > .20), indicating that combined action-valence associations did not produce differential effects on fast, impulsive errors (i.e., the strength of incorrect response capture). In other words, the impulsive response errors generated by processing of the spatial location of the stimulus (i.e., the typical pattern in the Simon task) were neither enhanced nor reduced by the concurrent processing of irrelevant information linked to previously learned action-valence associations.

3.3.3 Response Suppression

The delta plots for each of the four color-stimulus conditions are plotted in Figure 4. As predicted by the DPAS model, delta plot functions reveal an increase in the interference effect across fast and intermediate response latencies that transitions into a less positive-going or a negative-going slope function toward the slow end of the RT distribution. The late reduction of interference is attributed to the buildup of inhibitory control to suppress the interfering response tendency. The slope connecting the final RT bins provides the most sensitive measure of inhibitory control, with more negative-going slopes associated with more proficient suppression (Ridderinkhof et al. 2002, 2011).

Figure 4.

Delta's plots (Simon interference effect plotted as a function of mean bin RT, for the fastest (bin 1) to the slowest (bin 7) bins) for each action-valence combination.

Guided by the DPAS model, we focused our analyses on the suppression slope (i.e., the slope between the final two RT bins). Our theoretical approach was also statistically confirmed, see Supplementary Material. Colors associated with action versus inaction did not differentially influence the suppression slope (Action, F(1, 38)=.74, p=.40). Colors associated with reward acquisition versus punishment avoidance had no differential effect on the suppression slope (Valence, F(1, 38)=.05, p=.82). However, combining these factors produced a very clear interaction on the suppression slope, (Valence × Action, F(1, 38)=9.78, p=.003). As visible in Figure 4, colors mapped to less natural action-valence associations, ‘inaction with reward’ and ‘action with punishment avoidance’, showed the typical negative-going suppression slope reflecting the proficient inhibition of the impulsive action. In contrast, colors mapped to more natural action-valence associations, ‘action with reward’ and ‘inaction with punishment avoidance’, disrupted the typical suppression slope pattern, instead producing a positive-going slope consistent with poor inhibition of action impulses.

The suppression slope was more negative for the color associating ‘action with punishment avoidance’ (an unnatural association) compared to colors associating ‘action with reward’, (t(38)=2.44, p<.05), and ‘inaction with punishment avoidance’, (t(38)=2.33, p<.05), the two natural association conditions. The suppression slope linked to the color associating ‘inaction with reward’ (the other unnatural association) also tended to be more negative-going than slopes linked to the colors associating ‘inaction with punishment avoidance’, (t(38)=1.88, p=.07), but was similar to the suppression slope produced by the color associating ‘action with reward’, t(38)=1.46, p=.15). To test the conflict hypothesis directly, we compared the suppression slope for colors linked to natural associations versus unnatural associations. This confirmed a more negative suppression slope (m=−.06) produced by colors linked to unnatural action-valence associations compared to natural action-valence associations, which produced a positive slope (m=.05), (t(38)=3.13, p=.003), and inferentially, poorer inhibitory control.

3.4 Conclusion Experiment 1

The results of Experiment 1 support three main findings: (1) the learning task replicated inherent biases during the formation of action-valence associations (i.e., reward acquisition paired most effectively with action, punishment avoidance paired most effectively with inaction), (2) the modified version of the Simon conflict task replicated patterns of fast, impulsive errors on conflict trials and interference suppression as RT slowed, both of which are consistent with the DPAS model, and (3) in the Simon task, the presence of an irrelevant stimulus feature (i.e., color) associated previously with certain action-valence tendencies directly influenced the proficiency of inhibiting impulsive motor actions. Encountering stimulus information associated previously with natural action-valence combinations lead to less proficient inhibition on the Simon task compared to encountering conflicting action-valence associations.

The second experiment replicates and extends the findings from the first experiment by comparing conflicting and natural action-valence associations with performance on (action-valence) bias free neutral Simon trials. Additionally, the learning phase of the second experiment was made less difficult to ascertain equal learning between the action-valence conditions. Thus, the effect of action-valence associations on inhibitory control cannot not be explained in terms of learning rate differences between the conditions.

Experiment 2

4.1 Participants

Thirty one subjects, with a mean age of 21.52 (STD =2.68), 13 male and mostly right handed (30 right handed), participated in this study. Recruitment, participation procedures, consenting processes, and exclusionary criteria were identical to Experiment 1.

4.2 Design and Procedure

The experimental approach was similar to Experiment 1 with some exceptions. First, the learning demands on the probabilistic action-valence task were simplified to ensure high and equivalent rates of learning each of the action-valence conditions. This was done by separating the reward acquisition and punishment avoidance learning conditions into separate learning blocks and by changing the probabilities to 90% (reward/avoid punishment)-10 %(unrewarded/punished). Subjects were instructed at the start of each learning block whether they would have a chance to be rewarded or whether they had to minimize punishment (reward and punishment learning were separated). Since the task was easier, monetary rewards were reduced to 5 cents. Second, we introduced blocks of Simon task trials containing a neutral color that had no previous action-valence association. Ideally, the neutral color stimuli would be embedded in the Simon task interspersed with the learned action-valence colors. However, the neutral color trials were intentionally presented in a separate block in the current study. Introducing neutral stimuli within the context of already seen color stimuli might make the neutral stimuli novel and deviant. Novel or deviant stimuli have been shown to induce arousal, post stimulus slowing, and directly modulate the dopamine and noradrenergic systems (Wessel et al., 2014; Schomaker et al., 2015). We elected to avoid these possible (uncontrolled) side effects.

Subjects first completed a practice session on the Simon task (40 trials of the Simon task). Subjects then performed a reward acquisition learning block or a punishment avoidance learning block of the action-valence learning task (20 practice and 6 blocks of 25 trials) to associate either valence with action or inaction. The order of learning blocks was randomized over participants. Because each learning session was now reduced to two action-valence trial types (action or inaction), a third color condition was added to each learning block to serve as a filler (5 trials within a block of 25 trials) to keep the task from being binary. The filler color had a 50% chance of reward or punishment (depending on the block) with either action or inaction responses. The filler color was not used in the Simon task.

After forming the critical action-valence associations in reward and punishment blocks, subjects performed the Simon task (10 blocks of 40 trials) in which the imperative stimulus appeared in the colors used during learning.

After completing the Simon task, subjects performed the action-valence task again (20 practice and 6 blocks of 25 trials); if they started with reward acquisition they would turn to punishment avoidance or vice versa. This was followed by another session of the Simon task (10 blocks of 40 trials). In total, subjects completed two sessions of the action-valence learning task with 2 interposed sessions of the Simon task. At the end, subjects performed one more session of the Simon task (5 blocks of 40 trials) with neutral (white) arrows.

Data Analysis for Action-Valence Learning Task

Subjects performed 2 experimental sessions (reward acquisition and punishment avoidance) of the action-valence learning task. Across sessions, subjects completed 150 learning trials, including 60 trials for each color patch of interest (guess color patches did not return in the Simon task). Again, we only included subjects (n=25) who demonstrated and maintained > 75% accuracy in learning the associations for each color patch in the last two bins of the learning block. Accuracy was defined by the percentage of trials in which the subject selected the optimal response. Since the learning across 6 blocks was completed before onset of the Simon task (no refresher blocks in between the Simon task like in experiment 1), we applied one analysis to the learning. We analyzed learning by analyzing accuracy across the 6 blocks using within-subject analysis of variance techniques to distinguish effects related to the within-subject factors of Action (Action, Inaction), Valence (Reward Acquisition, Punishment Avoidance), and Block (1,2,3,4,5,6). Each block included 10 trials of each color patch.

4.2.1. Simon Task

The Simon task remained identical to Experiment 1, with a few changes in stimuli and the number of trials. Arrows appeared in one of the two colors that were used in the action-valence learning task (except for the filler color), which were either colors associated with reward or colors associated with punishment, depending on the preceding learning task. At the outset of the experiment (before the learning task), each subject completed a 40 trial practice session with white colored arrows. Subsequent to each action-valence learning task (reward/punishment), a session of Simon trials was presented, each partitioned into 10 blocks of 40 trials. In total, subjects performed 20 blocks of 40 colored Simon task trials, which resulted in 800 experimental trials across the experiment. This provided 200 trials for each arrow color that were equally divided among C and NC trial types. The session at the end of the experiment with neutral (white) arrows also provided us with 200 trials across C and NC trial types.

Data Analysis for Simon Task

To replicate the results of Experiment 1, we analyzed mean RT and accuracy rates (square-root transformed to adjust for non-normal distribution) using repeated-measures analysis of variance techniques to determine the main and interactive effects of Correspondence (C, NC) and Color Action (Action, Inaction), and Color Valence (Reward Acquisition, Punishment Avoidance). In addition to that we analyzed action-valence effects on inhibitory control (i.e., the suppression slope of the delta plot) and early response activation (fast impulsive errors). To calculate CAFs and delta's, RTs were then rank-ordered from fastest to slowest and divided into 5 equal-sized bins. Note that we reduced the number of bins in this experiment in order to maintain a similar number of trials within each bin as in the first experiment (around 20 with optimal performance). The Simon task in experiment 2 was shorter in comparison to experiment 1, so we had the option to use the same number of bins, but with reduced trial counts per bin, or reduce the number of bins and preserve the trial count per bin. We have reported previously that distributional plot solutions based on varying bin numbers do not alter the statistical patterns of effects (Wylie et al., 2010), thus we preserved the number of trials per bin and use fewer bins.

To extend the results of Experiment 1, we averaged mean RT, accuracy rates, CAFs and delta's for conflicting and non-conflicting Action-Valence and contrasted them to performance on the trials without any Action-Valence association (neutral). Repeated measures ANOVA's were used to test main and interaction effects of Correspondence (C, NC) and Color Association (Conflict, Natural, Neutral) on mean RT, accuracy rates and the effect of Color Association (Conflict, Natural, Neutral) on suppression slopes and response activation. Specific contrasts were used to test whether conflict or non-conflicting Action-Valence associations were different from neutral trials.

5. Results

5.1 Final Sample

It was critical to our experiment that participants learned the four action-valance conditions so that these influences could be evaluated in the Simon task. Of the 31 participants enrolled in the study, 3 participants showed insufficient learning in at least 1 or more of the conditions and were excluded from further analysis. Three additional participants showed RT and error rate performances on the Simon task that were more than 2 standard deviations above the group mean and were also excluded. A final sample of 25 participants showed intact learning of all 4 action-valence conditions across the initial and maintenance learning blocks as well as interpretable performance in the Simon task.

5.2 Probabilistic Action-Valence Learning

Figure 5a shows the average learning accuracy for each action-valence condition across the six blocks of learning trials. Learning accuracy improved in all conditions across blocks, (Block, F(5,120)=40.01, p<.001). Specific contrasts indicated that learning accuracy in block 6 (98%) was higher than learning accuracy in blocks 1 (85%) and 2 (95%) (all ps<.001), but not compared to block 3 (97%), 4 (96%) and 5 (98%) (all ps>.1), indicating that learning stabilized after the 3^rd block. Accuracy was also higher when learning to associate color with action (96%) compared to inaction (93%), (Color Action, F(1,24)=6.46 , p<.05). Overall, participants showed similar rates of learning to gain reward (94%) and to avoid punishment (96%), (Color Valence, F(1,24)= 2.90, p=.10).

Figure 5.

Learning accuracy as a function of trial block (10 trials per block) for each action-valence combination (a) and average accuracy at the end of learning (b) .

Across blocks, learning also varied as a function of combined action-valence associations, (Color Valence × Color Action, F(1, 24)= 4.49 , p<.05). Participants more easily learned to associate inaction with punishment (95%) compared to inaction with reward (91%) (t(24)=2.19, p<.05), whereas action was equally well associated with both reward and punishment (both 96%), (t(24)=.06, p=.95).

The magnitude of learning rate between the conditions was similar across the blocks (Color Action × Color Valence × Block, F(5,120)= .78 , p=.57). The learning rates for each condition in the final block of learning (block 6) are plotted in Figure 5b. When comparing learning rates within the final learning block, the interaction between Color Valence and Color Action was gone, (Color Valence × Color Action, F(1, 24)= 2.8 , p=.11). Participants learned to associate reward with action (98%) as well as inaction (95%) equally well, t(24)=1.37, p=.18. Similarly, at the end of learning, punishment was associated with both action (98%) and inaction (98%), t(24)=.70, p=.49.

5.3 Simon

5.3.1 Mean Effects

Table 2 shows mean RT and accuracy rates separately for each of the four action-valence conditions and the neutral condition. Similar to Experiment 1, a Simon effect was produced across all conditions; NC trials showed slowed mean RTs and reduced accuracy rates (RT: 397 ms; Accuracy: 90%) compared to C trials (RT: 360 ms; Accuracy: 98%), (Correspondence, RT:F(1,24)=122.81, p<.001; Accuracy, F(1,24)=57.51, p<.001).

Table 2.

Experiment 2 Simon correspondence effects (reaction time and accuracy) separated by action and valence conditions and an additional neutral condition.

	Action		Inaction
	Reward acquisition	Punishment avoidance	Reward acquisition	Punishment avoidance	Neutral
RT (ms)
Corresponding	356 (7.6)	361 (6.9)	360 (7.1)	361 (5.8)	357 (5.8)
Non-Corresponding	395 (9.0)	399 (7.4)	398 (9.3)	396 (7.2)	393 (8.0)
Accuracy (% correct)
Corresponding	98.51 (0.3)	98.22 (0.4)	98.20 (0.3)	98.20 (0.4)	97.85 (0.3)
Non-Corresponding	89.53 (1.1)	90.93 (1.1)	90.44 (1.2)	91.03 (1.0)	88.62 (1.1)

Action or valence associated with the colors (and their interaction) did not affect mean RT or accuracy rates, (Action, RT: F(1,24)=.70, p=.41, Accuracy: F(1,24)=.29, p=.60; Valence, RT: F(1,24) =.06, p=.81, Accuracy: F(1,24)=.97, p=.34; Valence × Action, RT: F(1,24)=3.56, p=.07, Accuracy, F(1,24)=1.3, p=.72). Thus, previously learned action-valence associations did not influence mean performance in the Simon task.

The cost of non-correspondence (i.e., Simon effect) on both mean RT and accuracy rates was neither influenced by color processing associated previously with reward acquisition or punishment avoidance, (Correspondence × Valence, F(1, 24): RT, F=.90, p=.35; Acc, F =2.10, p=.16), nor by color processing associated with action versus inaction, (Correspondence × Action, F(1, 24): RT, F=1.79, p=.19; Acc, F=1.41, p=.25). Moreover, the combined influence of Color Valence and Color Action processing did not affect mean correspondence effects, (Correspondence × Action × Valence, F(1,24 ): RT, F=.32, p=.57; Acc, F=.51 , p=.48).

When comparing reaction times and accuracy rates between conflict, natural and neutral Color Associations, the Simon effect was produced again across all conditions (Correspondence, RT:F(1,24)=101.31, p<.001; Accuracy, F(1,24)=72.39, p<.001). No Color Association effect or an interaction with Correspondence was found on the RTs (Fs<.9, ps>.4). Participants were significantly more accurate on conflicting (94%) and natural (94%) Color Associations compared to the neutral (93%) Simon trials (Color Association F(1,24): conflict-neutral, F=5.86, p<.05; natural-neutral, F=6.72, p<.05.) No interaction between Color Association and Correspondence was present on Accuracy, F(2,48)=2.26, p=.12.

Like in Experiment 1, we used distributional methods to separate inhibitory control and activation of incorrect motor impulses and determine how previously learned action-valence associations impact these dynamic phases of control relative to trials without any action-valence association.

5.3.2 Response Capture

The CAFs for each of the four action-valence color stimuli and the neutral condition are plotted in Figures 6A and 6B, respectively, for C and NC trials. A significant increase in impulsive errors on NC (29% errors) compared to C trials (1% errors) was found at the fastest response bin, (Correspondence, F(1,24)=66.10, p<.001).

Figure 6.

Conditional accuracy functions (accuracy plotted against mean bin RT, for the fastest (bin 1) to the slowest (bin 5) bins) for each action-valence combination (upper panel) and averaged for natural and conflicting action-outcome associations versus neutral stimuli (lower panel), separate for corresponding (A, C) and non-corresponding (B, D) responses.

Processing stimulus colors previously associated with punishment tended to produce fewer overall impulsive errors than colors associated with reward (Errors Punishment: 15%; Reward: 18%), (Reward, F(1,24)=3.35, p=.08), and this varied across correspondence conditions, (Reward × Correspondence, F(1,24)=4.56, p<.05). Fewer fast, impulsive errors were made on NC trials in the presence of punishment avoidance associations than in the presence of reward acquisition associations, (t(24)=2.14, p<.05; Simon Effect in % Errors, Punishment: 27%, Reward: 33%).

Color stimuli associated with action and inaction did not produce differential effects on overall patterns of fast errors, (Action, F(1,24)=2.66, p=.12). Fast errors on NC trials tended to be smaller when colors were associated with Inaction (29% errors) compared to Action (32% errors), whereas fast errors to congruent trials were similar for these associations (Action × Correspondence, F(1,24)=3.80, p=.06). None of the higher order interactions was significant (all ps > .64), indicating that combined action-valence associations did not produce differential effects on the strength of incorrect response capture.

The pattern of fast impulsive errors on NC compared to C trials was consistent across conflict, natural and neutral Color Associations (Correspondence, F(1,24)=88.11, p<.001). However, fewer fast errors were made to colors associated with conflicting (16 %) and natural (17%) action-valence association compared to the neutral (21%) color (Color Association F(1,24): conflict-neutral, F=8.42, p<.05; natural-neutral, F=6.24, p<.05). A trending interaction between Color Association and Correspondence was present on fast impulsive errors, F(2,48)=3.18, p=.05. The Simon Effect was larger on neutral trials (37%) than on conflict (30%) and natural trials (30%), Color Associations Color Association F(1,24): conflict-neutral, F=4.41, p<.05; natural-neutral, F=3.97, p=.06 . When contrasting each condition's fast impulsive errors on the non-corresponding condition separately against the neutral condition with paired sampled t-tests (corrected for multiple comparisons, p=0.013), ‘inaction to avoid punishment’ (t(24)= 3.36, p =.003) and ‘action to avoid punishment’ (t(24)= 2.72, p=.012) show significantly fewer fast impulsive errors, whereas the other conditions were not significantly different from neutral; ‘inaction to gain reward’ (t(24)= 2.4, p= .02) and ‘action to gain reward’ (t(24)=1.02, p=.32).

5.3.3 Response Suppression

The delta plots for each of the four color-stimulus conditions and the neutral condition are plotted in Figure 7a. Figure 7b shows the delta plots for neutral, natural and conflicting associations. Similar to Experiment 1, we focused our analyses on the final delta slopes. The final delta slopes were not differentially influenced by the processing of colors associated with reward acquisition versus punishment avoidance (Valence, F(1, 24)=0.35, p=.56). However, colors previously with inaction showed more negative going slopes (m=−.03) than colors associated with action (m=.08) (Action, F(1, 24)=4.77, p<.05). Also, combining these factors produced an interaction on the final delta slopes, (Valence × Action, F(1, 24)=4.31, p<.05)

Figure 7.

Delta's plots (Simon interference effect plotted as a function of mean bin RT, for the fastest (bin 1) to the slowest (bin 5) bins) for each action-valence combination (A) and averaged for natural and conflicting action-outcome associations versus neutral stimuli (B).

Pairwise comparisons confirmed that the slope linked to the color associating ‘inaction with reward’ (m=−.1) was significantly different from ‘action with reward’ (m=.13, t(24)=2.76, p<.01) and tended to be more negative-going than slopes linked to the colors associating ‘inaction with punishment avoidance’, (m=.05, t(24)=1.90, p=.07). The final delta slope for the color associating ‘action with punishment avoidance’ (m=.04) was not significantly different from the colors associating ‘action with reward’, (t(24)=1.33, p=.23), and ‘inaction with punishment avoidance’, (t(24)=.18, p=.86).

Again, we combined colors according to their inherent or conflicting action-valence biases and confirmed that colors associated with inherent action-valence biases produced a more positive final delta slope (m=.09) compared to colors associated with conflicting action-valence mappings, which produced a negative final slope (m=−.03), (t(24)=2.07, p=.049). Similar to Experiment 1, inhibitory action control was differentially affected by natural action-valence associations (less proficient inhibition) versus conflicting action-valence associations (more proficient inhibition).

The comparison between conflict, natural and neutral Color Associations indicated a Color Association effect, F(1, 24)=6.69, p<.05. Specific contrasts pointed out that the natural action-valence associations were significantly different from neutral (F(1,24)=16.62, p<.001), whereas the conflicting action-valence associations were not (F(1, 24)=2.27, p=.15). When looking at Figure 7a, the negative going slope in the conflict conditions seems to be most prominent in the ‘inaction to gain reward’ condition and the positive going slope seems to be most pronounced in the ‘act to gain reward’ condition. Thus, we contrasted each condition separately against the neutral condition with paired sampled t-tests (corrected for multiple comparisons, p=.013). The final delta slopes of ‘inaction to avoid punishment’ (t(24)= 3.45, p =.002) and ‘action to gain reward’ (t(24)=3.56,p= .002) were significantly different from neutral, whereas ‘action to avoid punishment’ (t(24)= 2.26, p=.03) and ‘inaction to gain reward’ were not (t(24)=.50, p= 62).

5.4 Conclusion Experiment 2

The reductions in the learning demands for Experiment 2 produced higher overall learning rates across conditions compared to Experiment 1 as well as equal learning across all action-valence conditions (performance across action-valence conditions was equal by the end of learning). This minimized subsequent effects on the Simon task would could be attributable to differences in learning rates across conditions. Similar to Experiment 1, the modified version of the Simon conflict task replicated patterns consistent with the DPAS model (fast, impulsive errors on conflict trials and interference suppression as RT slowed).

Also, the presence of an irrelevant stimulus feature (i.e., color) previously associated previously with certain action-valence tendencies influenced the proficiency of inhibiting impulsive motor actions. Thus, Experiment 2 replicates and extends the findings of Experiment 1 by showing that in comparison to performance on neutral Simon trials, encountering stimulus information paired with natural action-valence associations disrupts inhibitory control of motor impulses on the Simon task. In contrast, encountering stimuli paired with conflicting action-valence associations had no affect on inhibitory control relative to a neutral stimulus in the Simon task.

Additionally, colors associated with punishment led to a reduction in fast impulsive errors on non-corresponding trials compared to neutral or reward-associated colors.

4. Discussion

The current investigation integrated two cognitive paradigms, a learning task that links stimulus features to specific actions (action vs. inaction) and outcomes (reward acquisition, punishment avoidance), and a response conflict task that measures the spontaneous activation and subsequent suppression of incorrect action impulses. The goal of this approach was to embed the learned stimulus-action-valence associations from the former paradigm as an irrelevant stimulus feature in the response conflict task to directly assess how spontaneous processing of these irrelevant action-valence associations impact ongoing efforts to suppress impulsive action tendencies. In other words, we investigated whether cognitive control processes can be altered unintentionally by the brain's spontaneous processing of irrelevant information that is associated with specific action tendencies and valence outcomes.

Both experiments support the conclusion that encountering stimulus information associated previously with directional action tendencies and outcome valences can intervene directly on cognitive control processes.

4.1 Biases in Learning Action-Valence Associations

In Experiment 1, the formation of specific action-valence associations replicates learning biases reported recently by Guitart-Masip and colleagues (Guitart-Masip et al. 2012a,b; Cavanagh et al. 2013, Freeman et al, 2014). Specifically, participants more easily associated action with reward acquisition and inaction with punishment avoidance. These action-reward and inaction-punishment associations are thought to reflect inherent learning biases that resemble Pavlovian-like tendencies (Chiu, Cools & Aron, 2014; Everitt, Dickinson and Robbins, 2001; LeDoux, 1996; Gray & Naughton, 2000). Comparatively, associations that conflicted with these biases, i.e., associating inaction with reward acquisition and action with punishment avoidance, were acquired less proficiently. While learning rates for all action-valence conditions were high, these inherent learning biases proved important for understanding effects on inhibitory control in the Simon task. In Experiment 2, we simplified our learning task to ensure high, equivalent learning of all action-valence associations prior to embedding in the Simon task.

4.2 Action-Valence Associations Modulate Cognitive Control

In both Experiments, the Simon task (as modified for the current study) interpreted within the DPAS framework, reproduced two essential patterns: (1) fast, impulsive reaction errors on conflict trials, and (2) delta plots showing the expected pattern of increasing interference effects across early to intermediate response latencies that transitioned into interference reduction (i.e., negative-going slope) at the slowest response latencies. Having reproduced these two key patterns, we could directly test how the presence of an irrelevant stimulus feature associated with specific action-valence tendencies impacted the susceptibility to committing fast, impulsive errors and the proficiency of suppressing incorrect action impulses.

4.2.1 Action-Valence Effects on Susceptibility to Impulsive Action Errors

Overall, interactions between action and outcome valence factors did not influence patterns of fast, impulsive errors (i.e., response capture). The two experiments did, however, yield partially overlapping but also distinct main effects of these factors on response capture effects. In Experiment 1, where the action-valence conditions were mixed during learning and during Simon task performance, colors associated previously with inaction reduced fast errors on Simon conflict trials compared to colors associated with action. Thus, stimuli associated with inaction had a general effect on reducing susceptibility to making fast errors. In Experiment 2, fast error rates, especially on Simon conflict (NC) trials, tended to be reduced for colors previously associated with inaction. However, fast errors were significantly reduced for colors associated with punishment avoidance compared to colors associated with reward. Thus, findings from both experiments point to an early modulatory effect of inaction learning on the strength of response capture, but only the second experiment pointed to an effect of punishment avoidance learning on response capture. An important difference between the designs of the two tasks was the mixing (Experiment 1) versus blocking (Experiment 2) of the levels of the outcome valence (reward acquisition, punishment avoidance) factor across learning and Simon tasks. The isolation of valence outcomes in the blocked design of Experiment 2 may have altered the relative strength or effects of outcome valence compared to the mixed design. Deconstructing these experimental design effects on response capture awaits further experimentation.

Future work will be important to determine not only the conditions under which action and outcome valence associations modulate response capture, but also to discover whether these associations directly alter the visual saliency of the irrelevant spatial information or modulate motor system responsiveness. Notably, other studies have demonstrated the effects of reward on early visual processes that in turn influence behavioral performance. For example, target and distractor stimuli associated previously with reward enhance attentional capture in visual search tasks (Anderson et al., 2011a,b; Della-Libera & Chelazzi, 2009; Padmala &Pessoa 2011). Recent imaging studies in Stroop conflict (Krebs et al., 2013) and perception (Doallo, Patai & Nobre, 2013) tasks also revealed effects of reward expectation on early visual processing components and on pre-SMA (Krebs et al.,2011). Alternatively, Chiu et al. (2014) showed that affective and aversive cues increase or decrease, respectively, activation patterns in the motor cortex before action selection. ERP studies would be particularly useful to determine if these learning associations modulate early visual potentials versus motor activation signals that emerge in the Simon task.

4.2.2 Action-Valence Effects on Inhibitory Control

Our primary analyses demonstrated a direct effect of learned stimulus-action-outcome associations on the proficiency of inhibitory control. We tested two accounts of how action-valence associations would impact inhibitory control. The motivation-to-motor ‘spillover’ account predicted that processing an irrelevant stimulus feature associated with action and reward (i.e., an approach association) would be maximally disruptive to inhibitory control, whereas an irrelevant stimulus feature associated with inaction and escaping punishment (i.e., an avoidance association) would maximally facilitate inhibitory control. Our findings across both experiments showed that, contrary to this account, both of these natural action-valence associations were disruptive to inhibitory control, whereas the two conflicting action-valence associations (action with punishment avoidance, inaction with reward) were not disruptive to inhibitory control proficiency. Instead, this pattern seems more consistent with the action-valence conflict account, which argues that cognitive control systems are unnecessary for learning natural action-valance associations, so they are disengaged when natural associations are encountered. Conversely, the formation of conflicting action-valence associations depends on the engagement of cognitive control systems to override natural action-valence tendencies, thus cognitive control is promoted when conflicting action-valence associations are encountered.

The inclusion of a neutral Simon task condition in Experiment 2 provided direct evidence that the deviant inhibitory control patterns were associated with the natural action-valence conditions rather than the conflicting action-valence conditions. That is, conflicting action-valence associations did not make inhibitory control more effective, but natural action-valence associations clearly reduced inhibitory control. What might explain the differential effects of natural versus conflicting action-valence associations on inhibitory action control? One possibility is that activation of action-valence associations has a direct effect on the engagement or disengagement of inhibitory control circuits. Guitart-Masip and colleagues (2012b) found that healthy individuals who learned conflicting associations most efficiently also showed an accompanying enhancement in inferior frontal cortex and subthalamic nucleus, regions implicated in inhibitory control circuits. This pattern was not observed during learning of inherent action-valence associations. Similarly, Freeman et al. (2015) showed that conflicting action-valence associations (inaction to a cue previously associated with action and reward) reduce the tendency to act and reduce activity in the motor system on subsequent trials. Both studies suggest that action-valence associations may operate directly on inhibitory control circuits.

Alternatively, the effect of action-valence associations on inhibitory control may be indirect, arising from the influence of these associations on processes upstream from inhibitory control. Cavanagh and colleagues (2013), using a similar learning paradigm and simultaneous scalp electroencephalographic (EEG) recordings, reported a specific increase in midfrontal theta activity specific to trials involving learning of the conflicting action-valence conditions; this activity was reduced for the natural action-valence associations. These midline activities are typically associated with conflict detection or the mobilization of control processes following detected conflict (Cavanagh, Zambrano-Vazquez, & Allen, 2012; Cohen, Elger & Fell, 2009; Kerns, 2006; Yeung et al., 2004). Thus, encountering natural action-valence associations may relax or interfere with conflict detection processes that are critical for triggering reactive inhibitory control mechanisms.

Another possible upstream mechanism is worth mentioning. Previous work has suggested that the engagement of conflict detection and control systems depends on alerting mechanisms signaled upstream by the salience network (Harsay, Spaan, Wijnen & Ridderinkhof, 2012; Ullsperger et al., 2010). According to this view, the salience network, comprised of the anterior insular cortex (AIC) and regions in the anterior cingulate cortex (ACC), detects motivationally salient stimuli, thereby alerting control systems to prepare for controlled processing. By this account, the conflicting, as opposed to the natural, action-valence associations may have acquired enhanced motivational salience during learning because they predicted increased need for control. Thus, the differential effects on inhibitory control in the Simon task may have resulted from upstream engagement of the salience network, which either put the control system on alert (i.e., in the case of conflicting action-valence associations) or on standby (i.e., as in the case of natural action-valence associations). Resolution of the roles of these proposed mechanisms awaits further investigation.

Our results add to the growing literature investigating interactions between reward (and punishment) processing and cognitive control. The relationship is clearly complex and depends on a number of relevant factors. For example, the expectation of reward or punishment as an incentive may produce very different effects on cognitive control compared to the unexpected delivery of reward or punishment during performance. Many studies report that the expectation of reward enhances cognitive control during conflict task performance (Dreisbach and Goschke, 2004; Dreisbach & Fischer, 2012;Boehler et al., 2012; Braem, Verguts, Roggeman, & Notebaert, 2012; Kanske & Kotz, 2011a,b; Krebs et al., 2013; Krebs, Boehler, & Woldorff, 2010; Locke & Braver, 2008, Stürmer, Nigbur, Schacht, & Sommer, 2011). In contrast to the facilitation of control with reward expectation, van Steenbergen and colleagues (2009, 2010) have provided evidence that the unexpected occurrence of reward diminish the normal increase in cognitive control following conflict trials (i.e., so called conflict adaptation effects).

Our findings expand previous work by showing that cognitive control processes can be modulated by task irrelevant stimulus information that does not directly lead to reward or punishment experience, but nonetheless has been associated previously with action or inaction tendencies that lead to reward acquisition or punishment avoidance. Valence associations affected the initial response threshold towards action or inaction (as shown by the valence effects on the fast impulsive errors), whereas the combined features of action and valence modulated the response specific interference control that builds up over time.

4.3 Limitations and Extant Issues

The current study demonstrated that response activation and suppression are affected by different aspects of action-valence associations (either valence alone or the interaction between action and valence). An outstanding issue remains how, and through which neural mechanism, different features of action-valence associations affect the distinct control processes as defined by DPAS.

One potential limitation of the current study is that the learning paradigm was not fully probabilistic, but combined probabilistic and deterministic outcomes. This was intended to minimize ambiguity in the formation of specific action-valence associations. As a result, the task design created the potential for learning to be easier for avoiding punishment compared to acquiring reward, irrespective of action choice. Two data patterns challenge this possibility. First, overall learning accuracy was similar for both types of learning in both experiments (i.e., there was no main effect of valence). Second, the data patterns from Experiment 1 across all four action-valence learning conditions replicated patterns reported by Guitart-Masip (2012ab). Thus, the adaptation of the design did not appear to influence the learning patterns and was successful at preventing any ambiguity in action-valance associations.

Our design investigated the influence of general action tendencies (established in the learning task) on specific action selection processes in the Simon task. A more direct effect of specific action-valence associations could be measured if specific responses were mapped to the action choice during learning.

In the current study, we have not directly measured whether particular action-valence associations lead to conflict during the learning phase, although previous imaging studies have already demonstrated that natural versus conflicting action-valence associations lead to distinct neural activation patterns (Cavanagh et al.,2013; Freeman, Razhas & Aron, 2014; Freeman et al., 2015).

4.4 Future Directions and Conclusions

An important future direction is uncovering the neural mechanisms that integrate cognitive control functions and reward/punishment outcomes. Several lines of research implicate important roles of dopamine and the direct and indirect basal ganglia pathways in theories of reinforcement learning and in theories of action selection and inhibition (Alexander, DeLong & Strick, 1986; Bogacz & Guerney, 2007, Aron et al., 2007; Schultz, 2002; McClure, Berns & Montague. 2003). Moreover, recent ideas outline putative basal ganglia mechanisms, including long-term potentiation and depression of direct and indirect pathways, for associating action and inaction to reward and punishment avoidance (Frank and Fossella, 2011; Kravitz et al., 2012). The shift toward a more integrative study of reinforcement learning and cognitive control functions of the frontal-basal ganglia circuitries will be an important focus of future research.

The role of action-valence learning and its effects on cognitive control processes will undoubtedly be important to the study of certain clinical populations, particularly those whose pathology is linked with basal ganglia disruption. Changes in reinforcement learning and in action control (e.g., inhibitory motor control) are reported in many of the most common psychiatric and neurologic basal ganglia disorders, including Obsessive-Compulsive Disorder (Gillan et al., 2011; Marsh et al., 2015), Tourette's Syndrome (Worbe et al., 2011; Wylie et al., 2013), Parkinson's Disease (Wylie et al., 2009a,b; Frank, 2005), and Huntington's Disease (Lawrence et al., 1999; Holl et al., 2013), but have been studied in isolation. A first question will be to determine if different forms of basal ganglia dysfunction alter the action-valence learning biases in dissociable ways. Second, how these formed associations affect cognitive control, such as inhibiting impulsive behavioral tendencies, may differ among basal ganglia disorders. This could shed new insights on the unique ways in which learning and control interact to create or sustain adaptive behaviors, and on their disruption in disease. How the processing of irrelevant information associated with reward or punishment influence inhibitory control in times of conflict has been relatively unexamined in patient groups. This knowledge could be helpful in developing treatment plans by enabling patients to steer away from their inherent biases for example by increasing the proficiency of inhibition or by recreating stimulus-action-valence associations.

In conclusion, the current study demonstrates that the brain's spontaneous processing of task-irrelevant information that has been associated previously with specific action (action versus inaction) and valence (reward acquisition versus punishment avoidance) tendencies can directly modulate cognitive control processes. The presumed activation of these associations can directly interfere or facilitate inhibitory control processes. The direction of interference appears to be governed, respectively, by whether the associations activate natural or conflicting action-valence tendencies.

Highlights.

• Recent literature shows that inhibitory control is affected by motivation.

• We studied how inhibitory control is modulated by task-irrelevant action-valence associations.

• Pavlovian-like associations (action-reward, inaction-punishment) reduced inhibitory control.

• Conflicting associations (action-punishment, inaction-reward) enhanced inhibitory control.