A Computational Perspective on the Roles of Affect in Cognitive Control

Abstract

Previous work has demonstrated that cognitive control can be influenced by affect, both when it is tied to the anticipated outcomes for cognitive performance (integral affect) and when affect is induced independently of performance (incidental affect). However, the mechanisms through which such interactions occur remain debated, in part because they have yet to be formalized in a way that allows experimenters to test quantitative predictions of a putative mechanism. To generate such predictions, we leveraged a recent model that determines cognitive control allocation by weighing potential costs and benefits in order to determine the overall Expected Value of Control (EVC). We simulated potential accounts of how integral and incidental affect might influence this valuation process, including whether incidental positive affect influences how difficult one perceives a task to be, how effortful it feels to exert control, and/or the marginal utility of succeeding at the task. We find that each of these accounts makes dissociable predictions regarding affect’s influence on control allocation and measures of task performance (e.g., conflict adaptation, switch costs). We discuss these findings in light of the existing empirical findings and theoretical models. Collectively, this work grounds existing theories regarding affect-control interactions, and provides a method by which specific predictions of such accounts can be confirmed or refuted based on empirical data.

Introduction

Many of our everyday behaviors, including making coffee, turning on our computer, and opening a news website, are well-served by relying on automatic or habitual forms of processing. However, many situations require us to engage cognitive control in order to override these default processes and better achieve our goals (Botvinick & Cohen, 2014; Diamond, 2012; Friedman & Miyake, 2017; Posner & Snyder, 1975; Shiffrin & Schneider, 1977). When we decide to stop reading the news and start working, we will need to inhibit any distractions and flexibly shift our attention between multiple tasks. A longstanding question centers on how we determine when control is needed, and how much. Over the last few decades, this question has been addressed by a variety of normative theories which postulate that the amount of control allocated varies based on changes in the task environment (e.g., the amount of conflict between competing response tendencies, or the likelihood of making an error) (Alexander & Brown, 2011; Botvinick, Braver, Barch, Carter, & Cohen, 2001; Brown & Braver, 2005; Verguts & Notebaert, 2008; Wessel, Danielmeier, Morton, & Ullsperger, 2012). More recent theories have focused on the role of motivation in cognitive control (e.g., variations in the incentives for and cognitive demands of the task) (Brown & Alexander, 2017; Holroyd & McClure, 2015; Lieder, Shenhav, Musslick, & Griffiths, 2018; Shenhav, Botvinick, & Cohen, 2013; Silvetti, Alexander, Verguts, & Brown, 2014). This work has been successful in accounting for how control allocation varies with explicit incentives (e.g., monetary rewards) but, with few exceptions (Dreisbach & Fröber, 2018; Inzlicht, Bartholow, & Hirsh, 2015; Pessoa, 2009), it has largely overlooked a major source of variability in control: affect.

A person’s affective state can have a substantial influence on how they allocate control. For instance, affect can determine the degree to which a person is motivated to reach a particular goal state (e.g., one that increases positive affect or reduces negative affect) in the moment. Affect can also determine how a person perceives their task environment. For instance, being in a positive or negative mood may alter what a person believes the requirements and payoffs of a task to be (e.g., answering email can seem easier when we are in a good mood). Research has demonstrated both forms of affective influence in the lab, showing that cognitive control varies as a function of affective experiences evoked by the incentives for performance – those integral to performance evaluation (i.e., performance-contingent rewards; e.g., Krebs et al., 2010; Locke & Braver, 2008; Padmala & Pessoa, 2011; for reviews see: Botvinick & Braver, 2015; Parro et al., 2018) – and as a function of affective experiences evoked by factors unrelated (incidental) to task performance, for instance those that induce a particular mood state (i.e., positive mood induction or performance non-contingent rewards; e.g., Dreisbach & Goschke, 2004; van Steenbergen, Band, Hommel, Rombouts, &Nieuwenhuis, 2015; for reviews see: Inzlicht et al., 2015; Pessoa, 2008; Dreisbach & Fröber, 2018). While a number of such influences of affect on control allocation have been documented (see Table 1 for a non-exhaustive overview of the empirical findings), the mechanisms by which these influences occur remain mysterious. Here, we seek to leverage a recent integrative account of control allocation to help resolve this mystery by enumerating several possible mechanisms underlying affect-control interactions.

Table 1.

Effects of affect on two classic measures of cognitive control performance, conflict adaptation and task-switching behaviour (See main text for examples of affect’s influence on other measures of performance.)

Integral affect	Process	Performance-contingent rewards
	Conflict adaptation	Increased conflict adaptation (Braem et al., 2012).
	Task-switching	Performance contingent rewards increase switch costs (cf. Müller et al., 2007).
Incidental affect		Mood induction	Performance non-contingent rewards
	Conflict adaptation	Reduced conflict adaptation (Kuhbandner & Zehetleitner, 2011; van Steenbergen et al., 2010; van Steenbergen et al., 2015)	Reduced conflict adaptation (van Steenbergen et al., 2009)
	Task-switching	Reduced switch costs (Dreisbach & Goschke, 2004)

The Expected Value of Control (EVC) theory offers a normative account of cognitive control allocation, suggesting that such allocation is determined by weighing relevant costs and benefits (Shenhav et al., 2013, 2017). The theory assumes that this cost-benefit decision determines the type(s) of control to allocate (control signal identities; e.g. pay attention to the ink color in a Stroop task) and the intensity with which to engage these control signals (e.g. the amount of attention paid to the ink color in a Stroop task). Building on past theories of motivation (cf. Atkinson, 1957; Brehm and Self, 1989; Vroom, 1964; Wabba and House, 1974), the theory assumes that this decision-making process weighs the utility and cost of allocating control in order to specify a control signal with the highest expected value of control (Figure 1). At the neural level, the theory proposes that this decision-making process occurs in the dorsal anterior cingulate cortex (dACC) which then projects the output of this decision (a particular allocation of control) to downstream regions that execute this control (Shenhav et al., 2013; Shenhav, Cohen, & Botvinick, 2016). Recent work has implemented the EVC theory within an explicit computational framework (Lieder et al., 2018; Musslick, Cohen, & Shenhav, in press; Musslick, Shenhav, Botvinick, & Cohen, 2015; Musslick, Cohen, & Shenhav, 2018), and used this model to simulate an agent’s behavioral performance across a variety of tasks. These simulations have not only reproduced a number of key phenomena in the cognitive control literature – including performance costs related to response conflict (congruency effects), the influence of such congruency on subsequent control adjustments (congruency sequence effects), and performance costs resulting from switching versus repeating task sets (switch costs) – they have also demonstrated how these phenomena are influenced by changes in task demands, performance incentives and individual differences in decision-making parameters (e.g., how sensitive a given person is to reward, and how effortful they perceive control to be).

Figure 1.

According to the EVC theory, people select the type and intensity that maximizes the overall expected value of control (blue curves), which is calculated as the difference between the expected payoff for a given control allocation (e.g., the reward for giving a correct response; green curves) minus the associated controls costs (red curve). The peak of the EVC curve identifies the optimal control allocation (vertical black arrows). A) Increases in task difficulty result in a rightward shift in the payoff curve (reflecting the fact that more control is needed to attain a given level of accuracy), in this example resulting in a rightward shift in the peak EVC (recommending an increase in control). B) An increase in the payoff for a correct response results in a heightening of the payoff curve, and attendant increases in the EVC-maximizing control intensity. Adapted from Shenhav et al. (2013).

While the EVC model does not explicitly address the role of affect in influencing cognitive control, it does constrain the possible routes through which these influences may occur. From the perspective of this model, the overall value of control (and therefore the ultimate allocation of control) is determined by expected outcomes, perceived task difficulty, and the subjective cost of exerting mental effort. Each of these can be influenced either directly or indirectly by one’s affective state. For instance, expected outcomes (e.g., the rewards expected for task performance) will scale with their affective salience (Knutson & Greer, 2008; Slovic, Finucane, Peters, & MacGregor, 2007; Tversky & Kahneman, 1991; Wilson & Gilbert, 2005), which can in turn vary in relation to a person’s current mood state (Clore, Gasper, & Garvin, 2001; Eldar & Niv, 2015; Eldar, Rutledge, Dolan, & Niv, 2016; Isen, Nygren, & Ashby, 1988). Affective states can also alter the perceived difficulty of a task, making it seem like more or less effort is required to achieve one’s goal (Efklides & Petkaki, 2005; Gendolla, 2000; Gendolla, Abele, & Krüsken, 2001). Finally, variability in one’s affective state can also change how effortful it feels to exert control. For example, several studies have shown that individuals with depression, characterized by prolonged negative affect, experience a task as more effortful compared to healthy controls (Cléry-Melin et al., 2011; Brinkmann & Gendolla, 2007, 2008). Critically, each of these hypothesized mechanisms (which are not mutually exclusive) have implications for how affect should influence control evaluation (Figures 2–5).

Figure 2. Effects of anticipated reward on the expected value of control and behavior.

A) According to the EVC theory, anticipation of higher rewards is predicted to increase control intensity. B) Higher anticipatory affect (anticipation of higher rewards) increases the congruency sequence effect in reaction times and in error rates. C) Higher anticipatory affect (anticipation of higher rewards) increases the switch cost in reaction times and in error rates.

Figure 5. Effects of control costs on the expected value of control and behavior.

A) According to the EVC theory, higher costs are predicted to decrease control intensity. B) Positive mood (low cost) increases the congruency sequence effect in reaction times and in error rates. C) Positive mood (high utility discounting) increases the switch cost in reaction times and in error rates.

In order to elaborate on these mechanisms, here we use the EVC model to investigate which components of one’s valuation of control may be influenced by affect. We simulate multiple possible accounts of these affect-control interactions, specifically whether affect influences one’s reward sensitivity, utility discounting, expected task difficulty, and cost of control. We examine the specific predictions each of these accounts makes for control allocation and resulting measures of task performance, showing that they produce dissociable influences on measures of conflict adaptation and task-switching. We discuss these results in light of existing empirical findings and theoretical frameworks. By enumerating specific accounts of affect-control interactions and their predictions for behavior, our work provides a path toward identifying the mechanisms that best account for such interactions, and to intervene on these mechanisms when they are maladaptive for an individual.

Methods

The computational implementation of the EVC theory allows for the simulations of behavior across different cognitive tasks. Here, we generate behavior from a computational model of EVC theory that has been previously used to simulate a variety of different control phenomena (Musslick et al., 2015; Musslick, Cohen & Shenhav, 2018; Musslick, Cohen & Shenhav, 2019). Simulated EVC agents solve a task by specifying the control signal on every trial. The control signal is chosen optimally based on an internal model of the next trial (inferred state Ŝ). This signal is then used to interact with the environment, for example to commit one of the two possible responses in the task (actual state S). After each trial the agent updates the internal model based on its observation of the current trial.

In order to generate reaction times and responses on each trial, we use the drift diffusion model (DDM; Bogacz et al., 2006; Ratcliff, 1978). Within the DDM framework, a response on the task can be conceptualized as a result of the noisy accumulation of evidence toward one of the two possible responses (e.g. responding based on the ink color in a Stroop task). Here we assume that the rate of evidence accumulation toward one of the two responses is governed by a controlled and an automatic component.

$$d=d_{control}+d_{automatic}$$

The automatic component reflects the automatic processing of the ink color and word content of the stimulus when no control is engaged:

$$d_{automatic}=a_{color}+a_{word}$$

The magnitude of the color-response (a_color) and the word-response (a_word) association depends on the strength of the association between the stimulus and the response. The sign of the association depends on the response (e.g. a_color < 0 when the response is associated one button, a_color > 0 if the response is associated with the other button). It follows that on incongruent trials the a_color and a_word have the opposite sign, while they have the same sign on congruent trials.

The controlled component of the drift rate is the sum of stimulus response-associations, a_color and a_word, each weighted by the intensity of the corresponding control signal – one for processing the color (u_color) and one for processing the word content (u_word):

$$d_{control}=a_{color}·u_{color}·\epsilon +a_{word}·u_{word}$$

Each of the two control signals (u_color and u_word) bias the processing toward one of the two dimensions of the stimulus. In the case of the Stroop task, higher control signal for processing the ink color dimension improves the performance on the task. Reaction times and probabilities of each of the two responses are derived from an analytical solution to the DDM (Navarro & Fuss, 2009).

The optimal set of control signals U = {u_color, u_word} for each trial is determined on the basis of the internal model of the trial Ŝ = {â_color, â_word} so that the expected value of control is maximized. The expected value of control, for a set of control signals and for an inferred state, is calculated based on the expected rewards and costs associated with an outcome,

$$EVC(U,\widehat{S})=P(U,\widehat{S})·V(R)–Cost(U)$$

where P(U, Ŝ) represents the probability of reaching the decision threshold of a correct response and V(R) represents the value of committing a correct response (cf. Figure 1). To simulate the discounting of utility with increases in anticipated reward (increases in subjective value are assumed to diminish as a function of anticipated reward), subjective value and is calculated as V(R) = 25 · log_e (v · R + 1) where R represents the anticipatory amount of reward offered for a correct response in the task¹, which is discounted by the agent’s responsivity to reward v, henceforth referred to as reward sensitivity. The cost term Cost(U) = Cost_impl(U) + Cost_reconf(U) represents the total cost of cognitive control (cost) and is composed of an implementation cost that increases exponentially with the amount of control being allocated,

$$Cos{t}_{impl}(U)=e^{c_{impl}\cdot u_{color}}+e^{c_{impl}\cdot u_{word}}$$

as well as a reconfiguration cost that scales exponentially with the degree to which control signals need to be changed relative to their previous state

$$Cos{t}_{reconf}(U)=e^{c_{reconf}\sqrt{{\left(u_{color,t}-u_{color,t-1}\right)}^2+{\left(u_{word,t}-u_{word,t-1}\right)}^2}}$$

where the implementation cost is scaled by parameter c_impl and the reconfiguration cost is scaled by parameter c_reconf The two cost terms influence behavior in different ways. A higher implementation cost leads the model to allocate control with a lower intensity, leading to overall poorer performance on a task. A higher reconfiguration cost prevents the model from adjusting its control signal when task demands change. The latter may result in performance costs associated with task switches. The model then selects a set of control signals U which maximize² the EVC within the inferred next trial Ŝ:

$$U^{*}\leftarrow argma{x}_i[EVC(U,\widehat{S})]$$

The reaction time and the response in the actual state S are then determined by the influence of the chosen signals on the rate of the accumulation of evidence toward a decision bound (drift rate). After observing the actual state, the agent updates its inferred state for each stimulus-response association as follows

$${\widehat{a}}_{color,t}={\widehat{a}}_{color,t-1}+\alpha \left({\widehat{a}}_{color,t-1}-a_{color,t}\right)$$

where α is the learning rate. Finally, agent then re-evaluates the optimal control policy for the next trial based on its revised model of the task environment.

We simulated³ the effects of incidental and integral affect in the classic Stroop experiment, as well as a task switching experiment. In the Stroop paradigm, the agent is presented with a two-dimensional stimulus, one dimension representing an ink color and another dimension representing a color word. On each trial, the EVC model is required to indicate the response associated with the ink color. In congruent trials, the word feature of the stimulus is associated with the same response as the ink color whereas in incongruent trials, the color and word features are associated with different responses. The experiment sequence encompassed 101 trials, and was fully balanced (excluding the first trial) with respect to congruent and incongruent stimuli, as well as with respect to all four transitions between the two trial types (congruent-congruent, congruent-incongruent, incongruent-congruent, incongruent-incongruent). To simulate congruent trials, we set a_color = 0.38, a_word = 0.40 such that both stimuli dimensions promote the same response. On incongruent trials, we set a_word = −0.40 such that the word dimension is associated with a different response than the color dimension. Note that the absolute magnitude of a_word is higher than a_color, reflecting the assumption that word reading is a more automatic process than color naming (Cohen, Dunbar, & McClelland, 1990). We assessed the congruency sequence effect as an interactive effect between the congruency of the current trial and the congruency of the previous trial on performance.

In the task switching paradigm, the agent had to switch between two different tasks. Each tasks required the agent to indicate the response associated with a target stimulus while ignoring the response associated with a distractor stimulus. Similar to the Stroop task, trials in each of the two tasks could either congruent, with a_target = 0.38, a_distractor = 0.40 or incongruent, with a_target = 0.38, a_distractor = −0.40⁴. The trial sequence encompassed 100 trials that were randomly sampled with respect to stimulus congruency (congruent, incongruent), the currently relevant task and the task transition with respect to the previous trial (task switch, task repetition). We assessed the switch costs in terms of the difference in RTs and error rates between task switch trials and task repetition trials. In both paradigms, the model allocated control between the two control signals (u_color, u_word in the Stroop task, u_target, u_distractor in each of the tasks in the task switching environment) using the same range of control intensities as described in the Stroop task. All parameters were selected such that EVC agents achieved an overall accuracy of at least 70% for each of the affect manipulations. We varied the range of control signal intensities from 0 to 10 in steps of 0.2 for both control signals and set the anticipated reward received for a correct response to R = 70. DDM parameters were set as follows: starting point x₀ = 0.0, noise coefficient c = 0.7, non-decision time T₀ = 0.2s and threshold z = 0.4. Note that the noise parameter can c be used as a proxy for task difficulty, whereas the noise parameter of the inferred state ĉ can be taken as a proxy for the expected task difficulty. For each experiment, we simulated neutral affect using the following default values: reward sensitivity v = 1, implementation cost c_impl = 3, reconfiguration cost c_reconf = 1.5, and learning rate α = 0.4.

We simulated effects of integral affect by increasing the anticipatory amount of reward received for accurate performance to R = 300. We simulated the effects of positive incidental affect, by either decreasing an agent’s reward sensitivity to v = 0.1 (high utility discounting) or by decreasing its implementation cost to c_impl = 1. We also considered a decrease in expected task difficulty as a proxy for positive incidental affect, either for a low range of expected task difficulties (0.5 < ĉ < 1), or for a high range of expected task difficulties (1 < ĉ < 2). Note that we varied only one parameter at a time while holding the other parameters constant. For each parameter setting, we simulated 20 agents in both paradigms to assess congruency sequence effect, as well as performance costs associated with task switches.

Results

To examine potential mechanisms for affective influences on control, we focus on two cognitive control phenomena that have been found to be susceptible to manipulations of affective state (Table 1): (1) performance improvements (faster and more accurate responding) when an incongruent trial (e.g., in a Stroop or Eriksen Flanker Task) is preceded by another incongruent trial, referred to as a congruency sequence or conflict adaptation effect (Gratton, Coles, & Donchin, 1992); and (2) performance decrements (slower and less accurate responding) when the current task differs from the task performed on the previous trial (e.g., categorizing the parity rather than the magnitude of a numeral) referred to as switch costs (Allport, Styles, & Hsieh, 1994; Rogers & Monsell, 1995). As shown in Figure S1, the EVC model is able to reproduce these classic observations, as well as the more basic observation that performance worsens (slower and less accurate responding) on incongruent relative to congruent trials (Musslick et al., 2015; Musslick, Cohen, et al., in press; Musslick, Shenhav, & Cohen, in prep). We next consider how differences in integral and incidental affect could influence how these agents allocate control, and the implications this would have for observed behaviors. In accordance with findings in the literature, we focus our analysis on affect modulations of the congruency sequence effect, as well as task switch costs (for a depiction of congruency effects and overall control signal intensity, see Figures S2 and S3 in Supplementary materials).

Integral affect

People vary in the degree of positive affect they experience upon receiving a reward, and in the degree of positive affect they anticipate experiencing when deciding how strongly to weigh that reward when making a decision (Berridge & Kringelbach, 2015; Cloninger, 1987; Corr, 2004; Gray, 1970; Knutson & Greer, 2008; Pizzagalli, 2014; Zald & Treadway, 2017). We simulated this variability in anticipatory affect by varying the amount of expected reward across simulated agents; agents which anticipated higher rewards assigned a higher hedonic utility to a given performance-contingent reward (e.g., money or positive social feedback for completing a task) than agents which anticipated lower rewards (Figure 2A). Consistent with analogous simulations reported in previous work (Lieder et al., 2018; Musslick et al., 2015), we found that increasing anticipatory affect predicts increased control allocation for equivalent rewards (Table 2). As a result, compared to agents which anticipated low rewards, agents which anticipated high rewards perform better overall (are faster and more accurate; Table 2) and demonstrate lower congruency effects and higher congruency sequence effects (Figure 2B). At the same time, these agents also exhibit higher switch costs (Figure 2C). While counterintuitive on their face, these higher switch costs reflect a well-known tradeoff whereby increasing focus on a particular task (in this case, resulting from increasing reward expected from that task) means having to pay a higher cost to disengage and switch to another task (Dreisbach & Goschke, 2004; Goschke, 2000; Musslick, Shenhav, Jang, Shvartsman, & Cohen, 2018; Ueltzhöffer, Armbruster-Genç, & Fiebach, 2015, but see also: Kleinsorge & Rinkenauer, 2012, Umemoto & Holroyd, 2015).

Table 2.

Results of the simulations

	Integral Affect	Incidental Affect (positive vs. neutral)
	Enhanced anticipated rewards	Decreasing marginal utility	Decreasing expected difficulty (lower range)	Decreasing expected difficulty (upper range)	Decreasing cost of control
Control intensity	↑	↓	↓	↑	↑
Overall performance	↑	↓	↓	↑	↑
Congruency sequence effect	↑	↓	↓	↑	↑
Switch costs	↑	↓	↓	↑	↑

Note. The arrows pointing up indicate an increase, and the arrows pointing down a decrease in the effect. Performance improvements are marked in green, and performance decrements in red.

Importantly, these findings express variability predicted both at the trait and state level – the different performance profiles we observe for agents high versus low in anticipated affect apply equally to states in which a given agent expects more or less performance-contingent reward, whether as a result of actual or perceived changes in available incentives. These state-based predictions are consistent with observed changes in performance with increasing performance-contingent rewards (Table 1). In sum, these results suggest that increased integral positive affect, resulting from receiving performance-contingent rewards, produces increases in control allocation.

Incidental affect

Positive affect can influence the subjective value of outcomes even when it is not tied to performance on a task (Clore et al., 2001; Eldar et al., 2016; Isen et al., 1988), for instance when an individual is induced to feel good by a performance-noncontingent reward (Eldar & Niv, 2015; van Steenbergen, Band, &Hommel, 2009) or a mood induction procedure (Dreisbach, 2006; van Steenbergen et al., 2015). Here we explore several possible mechanisms by which such changes in incidental affect (i.e., increases in positive mood) might influence decisions about control allocation.

First, it has been proposed that the subjective utility of rewards increases logarithmically, such that rewards have decreasing marginal returns beyond some level (Bernoulli, 1738/1954; Coombs & Avrunin, 1977; Kahneman & Tversky, 1984; Tversky & Kahneman, 1991). Under this assumption, it is possible that a given performance-contingent reward has less utility to someone (i.e., utility is discounted) in a very positive mood compared to someone in less positive mood (Figure 3A). We simulated agents that exhibited such utility discounting, under conditions where they were already in an elevated baseline reward state (equivalent to greater positive mood) – and therefore cared less about potential task rewards – and compared these to conditions where those agents were in the equivalent of a neutral mood. In these simulations, positive mood led to decreased control allocation (because a given reward was viewed as having lower utility than when in a neutral mood; Table 2), resulting in smaller congruency sequence effects (Figure 3B) and smaller switch costs (Figure 3C). These effects were evident both in response times and error rates.

Figure 3. Effects of utility discounting on the expected value of control and behavior.

A) According to the EVC theory, discounted utility is predicted to decrease control intensity. B) Positive mood (high utility discounting) reduces the congruency sequence effect in reaction times and in error rates. C) Positive mood (high utility discounting) reduces the switch cost in reaction times and in error rates.

A second possible mechanism by which incidental affect could influence control allocation is via perceptions of task difficulty. It has been proposed that positive states lead people to perceive tasks as less difficult, that is, as requiring less effort to achieve a given outcome (Efklides&Petkaki, 2005; Gendolla, 2000; Gendolla et al., 2001; Gendolla & Krüsken, 2001). We simulated such influences of mood on expected task difficulty (Figure 4A), and found that under these conditions positive mood exerts a nonlinear influence on control allocation. When tasks are perceived as low to moderate in difficulty, positive mood leads to a smaller control allocation than neutral mood because the agent (Table 2). Within this range of perceived difficulty, both types of agents perceive the task as manageable, with the agent in a positive mood perceiving it as less demanding of control. As a result, positive mood leads to smaller congruency sequence effects and smaller switch costs than neutral mood (Figure 4B–E). Conversely, when the task is perceived as especially difficult, an agent in a neutral mood is apt to divest their control allocation (and/or quit the task entirely) whereas an agent in a positive mood would be more likely to “persevere,” investing a higher level of control to meet the challenges of the task. As a result, in this upper range of perceived difficulties, the findings in Fig. 4 (B and C) reverse, with positive mood resulting in larger congruency sequence effects and larger switch costs (Figure 4D–E).

Figure 4. Effects of perceived task difficulty on the expected value of control and behavior.

A) According to the EVC theory, higher perceived difficulty is predicted to decrease control intensity. B-C) In the lower range of expected task difficulty, positive mood (low expected difficulty) reduces the congruency sequence effect and switch cost in reaction times and in error rates. D-E) In the upper range of expected task difficulty, positive mood (low expected difficulty) increases congruency sequence effect and switch cost in reaction times and in error rates.

Finally, it is possible that, rather than incidental affect influencing the perceived utility of or demands for control, it instead influences how people experience the control being allocated. Specifically, it is possible that exerting control feels less effortful when one is in a positive rather than neutral mood (cf. Cléry-Melin et al., 2011). We simulated control allocation based on this account, allowing positive mood to decrease the expected cost of control (i.e., how aversive a given allotment of control is; Figure 5A). Under these conditions, agents in a positive mood were overall willing to invest more control in a task (Table 2), resulting in larger congruency sequence effects and larger switch costs (Figure 5B–E). Note that while the magnitude of the presented results is a function of the chosen parameter range, we focused our analysis on the qualitative direction of the effects⁵.

Discussion

Affect has a pervasive influence on various cognitive processes such as perception and attention (Pourtois, Schettino, & Vuilleumier, 2013), cognitive control (Pessoa, 2008, 2009), and judgment and decision-making (Blanchette & Richards, 2010; Lerner, Li, Valdesolo, & Kassam, 2015; Slovic et al., 2007). While empirical studies have demonstrated the importance of affect in directing information processing (Dreisbach & Fischer, 2012; Dreisbach & Fröber, 2018; Inzlicht et al., 2015), normative theories of cognitive control have largely overlooked affect’s role in control allocation. In this study, we leveraged a computational implementation of the EVC theory (Lieder et al., 2018; Musslick et al., 2015; Shenhav et al., 2013) to simulate several candidate mechanisms through which cognitive control can be influenced by integral affect (e.g., performance-contingent rewards) and incidental affect (e.g., positive mood induced in a performance-noncontingent manner). In addition to capturing behavioral effects commonly found in conflict and task-switching paradigms (congruency sequence effects and switch costs), these simulations demonstrated how such effects would vary based on several putative accounts of affect-control interactions (including whether incidental positive affect modulates discounted utility, expected task difficulty, or the cost of control). These findings provide quantitative and testable predictions that can be compared directly with existing and future empirical findings.

People differ in the amount of positive affect they experience when anticipating potential rewards (Berridge & Kringelbach, 2015; Cloninger, 1987; Corr, 2004; Gray, 1970; Knutson & Greer, 2008; Pizzagalli, 2014; Zald & Treadway, 2017). We tested how variability in the (integral) positive affect one anticipates for successful completion of a task (i.e., performance-contingent reward) would influence their control allocation and performance on such tasks. Our results show that increases in anticipated rewards lead to increased allocation of control. This result is in agreement with empirical (Botvinick & Braver, 2015) and computational (Lieder et al., 2018; Musslick et al., 2015) work demonstrating that, holding the strength of anticipatory affect constant, increases in incentives lead to greater control allocation (Figure 1A). Our computational model successfully captures findings showing that conflict adaptation effects increase with increasing performance-contingent reward (Braem, Verguts, Roggeman, & Notebaert, 2012). At the same time, our findings also predict that larger performance-contingent rewards come at the expense of higher switch costs, reflecting a tradeoff between cognitive stability in the face of distraction (achieved by allocating high amounts of control to a single task) versus cognitive flexibility (achieved by allocating low amounts of control to a previously executed task, making it easier to reconfigure to a new task when a switch occurs) (Musslick et al., 2018). Large performance-contingent rewards increased the amount of control allocated to a single task (Lieder et al., 2018; Musslick et al., 2015), and therefore require overcoming higher reconfiguration costs. Finally, the results predict that traits that result in enhanced anticipatory affect (e.g., Carver & White, 1994), should result in both increased conflict adaptation and higher switch costs.

Positive affect can be induced by factors incidental to the task at hand, and can influence several components crucial for deciding how to allocate control. First, incidental affect can change the subjective value of outcomes in the task (Clore et al., 2001; Eldar et al., 2016; Isen et al., 1988). The subjective utility of performance-contingent rewards is known to increase logarithmically (Kahneman & Tversky, 1984), thus having diminishing returns. Positive mood could increase the baseline expectation of rewards, thus resulting in discounted subjective utility for people in positive compared to those in neutral mood. Second, incidental positive affect can influence the expectations about task difficulty. Positive mood orthogonal to the task at hand can reduce the expected difficulty of the task (Efklides & Petkaki, 2005; Gendolla, 2000; Gendolla et al., 2001). Third, it is possible that affect modulates the subjective experience of the effort exerted in a task. In this way, positive affect could reduce the cost of control allocation (cf. Cléry-Melin et al., 2011). We simulated each of these accounts, and showed that they make divergent predictions, that can be validated against existing findings. For instance, a number of studies have shown that incidental positive affect reduces the conflict adaptation effect (Kuhbandner & Zehetleitner, 2011; van Steenbergen et al., 2009; van Steenbergen, Band, & Hommel, 2010; van Steenbergen et al., 2015) and decreases switch costs (for a recent review see: Dreisbach & Fröber, 2018). Our results demonstrate that this pattern of findings can be reproduced by an account where incidental affect influences the marginal utility of reward but not the cost of control. A perceived difficulty account can explain such findings under some conditions but not others (see below). Thus, our model not only generates quantitative predictions regarding different underlying mechanisms of affect-control interactions, it also constrains possible accounts of prior findings.

Of the three proposed mechanisms for control’s interactions with incidental affect, modulation of the expected task difficulty was the only one which produced nonmonotonic changes in control intensity. From the perspective of this account, when a task is expected to be moderately difficult at “baseline” (under a neutral mood), positive mood will make it seem easier and will lead to a relaxation of control. However, when the baseline expectation is that a task is very difficult, positive mood can lead a person to increase control rather than give up. Thus, the influence of mood on control will crucially depend on the difficulty of the task(s) at hand. This result provides a clear set of predictions that can be tested in future studies.

Our current work focuses on potential influences of affect on the evaluation of control. Other theoretical frameworks have considered alternate roles for affect, including whether increases in control are driven by aversive experiences (e.g., anxiety) that are triggered by response conflict, in order to help regulate such affective experiences (Dreisbach & Fischer, 2012; Inzlicht et al., 2015; van Steenbergen, 2015). These aversive experiences thereby induce increases in, for instance, conflict adaptation. These theories share our model’s prediction that control will tend to increase with increasing conflict. However, unlike our model, they do not predict (in any obvious way) that control should decrease once conflict/difficulty exceeds a particular threshold. These theories and our own identify potential roles for affect in the selection/allocation of control, but there is an important gap between the determination and execution of control (for early work see: Ach, 1935; Gollwitzer, 1993) and other theories have proposed that affect/emotion could directly influence the way in which control is executed. For instance, it has been proposed that positive affect may increase cognitive flexibility (e.g., task-switching) by influencing the gating of information into and/or out of working memory (Ashby, Isen, & Turken, 1999; Dreisbach & Fröber, 2018). There is reason to believe that positive affect may influence both the selection and execution of control, through associated increases in dopamine (Westbrook & Braver, 2016). At the same time, recent work also shows that these same mechanisms produce significant individual variability in the encoding of incentives, showing that individual differences in baseline dopamine modulate the influence of incentives on control (Aarts et al., 2010, 2011, 2014; Froböse et al., 2018), producing nonlinear (U-shaped) effects on performance and decision-making analogous to those we find when varying perceived difficulty.

Other frameworks have focused on the effects of positive affect on cognitive flexibility (e.g., task-switching). Ashby and colleagues have proposed that the increases in flexibility due to positive affect are mediated via the influence of positive affect on dopamine (Ashby, Isen, & Turken, 1999). More recently, it has been proposed that positive affect can lower the updating threshold of working memory and thus increase flexibility (Dreisbach & Fröber, 2018). These mechanisms are assumed to be mediated by dopamine, a neurotransmitter crucial for reward processing and cognitive control (Cools, 2019). While our current work does not examine affect’s influences at each of these levels, it does not preclude the possibility that these function in parallel. Future modelling work should attempt to explicitly include the role of dopamine to better understand the interactions between affect and cognitive control. Importantly, we also provide potential points of divergence from the existing frameworks. For instance, while the aversive conflict account shares our model’s general prediction that control will tend to increase with increasing conflict, our account differs in its prediction (noted earlier) that control should decrease once conflict/difficulty exceeds a particular threshold.

Our computational approach to investigating the role of affect in cognitive control offers several important directions for future research. First, in order to understand the mechanisms by which affect exerts its effects on task performance (e.g., conflict adaptation and task-switching), it will be crucial to further investigate how affect modulates perceived demands and incentives for engaging in cognitively demanding tasks. Recent work provides a promising example of such modeling being applied to understanding how mood dynamically shapes expectations of reward (Eldar et al., 2016), providing a platform for building on (and further constraining) the work we describe here. Second, our approach also reveals that the same measurable outcome (e.g., a reduction in the conflict adaptation effect) can result from multiple mechanisms (e.g., higher utility discounting or decreased cost; cf. Musslick, Cohen, et al., 2018). Determining which of these provide the best account of affect-control interactions will therefore require combining modeling, measures of behavior and neural activity, and, most importantly, task paradigms that are carefully designed to vary the construct of interest (e.g., perceived utility vs. difficulty). By the same token, our work points to additional sources of heterogeneity in empirical findings, arising from individual differences in affect’s influence on control valuation both within and across individuals.

The formal approach used here allows for a more direct comparison between the predictions of different models. In this study we have used the computational implementation of the EVC theory, but several other neurocomputational models of cognitive control (Brown and Alexander, 2017; Holroyd and McClure, 2015; Verguts et al., 2015) and theories of motivation (Brehm & Self, 1989; Manohar et al., 2015; Silvestrini, 2017) include some of the components which we have investigated here and make a number of predictions that qualitatively overlap with the EVC theory. For example, motivational intensity theory (Brehm & Self, 1989) posits that effort investment depends on task difficulty in a non-monotonic fashion: as the difficulty of a task increases, an agent may choose to invest more effort as long as success is possible. However, once the task difficulty is high enough so that success on the task is no longer expected, an agent may choose to disengage from the task. Support for this prediction comes from physiological studies which use the responses of the cardiovascular system as a measure of effort mobilization (Wright, 1996; Silvestrini & Gendolla, 2019). In this way there is a convergence of motivation theory and physiological studies on one side, and the neurocomputational accounts of effort investment (Manohar et al., 2015; Shenhav et al., 2013; Verguts, Vassena & Silvetti, 2015) on the other. Silvestrini (2017) has proposed an integrated framework that aims to bridge the research on effort and cardiovascular reactivity with the cognitive control research with a specific focus on the EVC theory. Future modelling work should explore similarities and differences between the predictions of these different theoretical accounts when it comes to the role of affect in cognitive control.

Divergent predictions of these accounts can be tested with a combination of behavioural measures that index task selection and performance; peripheral physiological measures that index arousal, affect, attention, and effort output (e.g., pupil dilation, corrugator muscle contraction, cardiovascular activity); and neural measures that index the processing of incentives, task demands, motivation, and control (Gendolla et al., 2012; Inzlicht et al., 2015; Shenhav et al., 2017; van der Wel, P., & van Steenbergen, 2018). In particular, several theories predict that dACC sits at the interface of affect, motivation, and cognitive control (Cavanagh & Frank, 2014; Holroyd & Yeung, 2012; Inzlicht et al., 2015; Shackman et al., 2011), including the EVC theory, which proposes that dACC integrates EVC-relevant information to calculate EVC and determine (and subsequently motivate) the optimal allocation of control (Shenhav et al., 2013, 2016). These theories would thus predict that the influence of affect on control should be observable in dACC activity and associated EEG indices of performance and feedback monitoring, consistent with extant findings (Cavanagh & Shackman, 2015; Hajack et al., 2004; Proudfit, 2015; Shackman et al., 2011; Ullsperger et al, 2014).

Formalizing the relationship between affect and cognitive control, as we have here, can also help to inform research on psychopathology. For instance, reward-related anticipatory affect and approach motivation are known to be enhanced in certain disorders (e.g., addiction; Dalley & Robbins, 2017; Koob & Volkow, 2010) and diminished in others (e.g., depression and schizophrenia; Barch, Pagliaccio, & Luking, 2015; Pizzagalli, 2014; Zald & Treadway, 2017). While our current work has focused on factors related to positive affect (like reward anticipation), a similar approach can be used to also inform our understanding of maladaptive influences of negative affect and cognitive control, which have been observed in disorders of mood (Gotlib & Joormann, 2010; Joormann & Vanderlind, 2014) and anxiety (Eysenck & Derakshan, 2011; Eysenck, Derakshan, Santos, & Calvo, 2007). An important next step in this field is to propose and test putative maladaptive mechanisms through which affect interacts with cognitive control and other cognitive processes (cf. Grahek, Shenhav, Musslick, Krebs, & Koster, 2019). While the research on affect and cognitive control in psychopathology has mostly been guided by qualitative models (Grahek, Everaert, Krebs, & Koster, 2018), further computational work could lead to formalized models that can be studied within the framework of computational psychiatry (Huys, Maia, & Frank, 2016; Montague, Dolan, Friston, & Dayan, 2012). We hope that this formal approach can help guide future studies in this direction. One interesting candidate for a maladaptive mechanisms of negative affect is the precision with which control signals are implemented once they are specified. In this work, we investigated how the specification of control signals is affected by different motivational parameters. However, EVC theory distinguishes the specification of a control signal from its implementation. Constraints on the latter may account for variability in one’s capacity to exert cognitive control (see Musslick, Shenhav & Cohen, 2019). A promising avenue for future work is therefore the exploration of computational mechanisms that mimic impaired performance in cognitive control as a result of negative affect.

In conclusion, here we have demonstrated multiple routes through which affect can influence the allocation of cognitive control. While empirical data points to an important role of affect in cognitive control allocation, the normative models of control have largely overlooked the role of affect. Here we have relied on the computational implementation of the EVC theory to simulate the potential mechanisms which can explain the existing empirical data. Our results suggest that affect can influence cognitive control via its influence on perceived task difficulty, the amount of effort needed to complete a cognitive task, and/or the influence of affect on the marginal utility of successfully performing the task. In this way affect plays a crucial role in determining when and how much cognitive control to allocate.

Highlights.

• Currently there is a need for formal models which can account for the interactions between affect and cognitive control

• To examine how integral and incidental affect can influence cognitive control allocation, we leverage the computational implementation of the Expected Value of Control theory

• Affect can influence cognitive control via its influence on perceived task difficulty, the amount of effort needed to complete a cognitive task, or the influence of affect on the marginal utility of successfully performing the task

• We discuss how normative models of cognitive control can be used to advance the theoretical and experimental understanding of the interface between affect and control