Cognitive and Behavioral Training Interentions to Promote Self-Control

Abstract

This review article discusses various cognitive and behavioral interventions that have been developed with the goal of promoting self-controlled responding. Self-control can exert a significant impact on human health and impulsive behaviors are associated with a wide range of diseases and disorders, leading to the suggestion that impulsivity is a trans-disease process. The self-control interventions include effort exposure, reward discrimination, reward bundling, interval schedules of reinforcement, impulse control training, and mindfulness training. Most of the interventions have been consistently shown to increase self-control, except for mindfulness training. Some of the successful interventions are long-lasting, whereas others may be transient. Most interventions are domain-specific, targeting specific cognitive and behavioral processes that relate to self-control rather than targeting overall self-control. For example, effort exposure appears to primarily increase effort tolerance, which in turn can improve self-control. Similarly, interval schedules primarily target interval timing, which promotes self-controlled responses. A diagram outlining a proposed set of intervention effects on self-control is introduced to motivate further research in this area. The diagram suggests that the individual target processes of the interventions may potentially summate to produce general self-control, or perhaps even produce synergistic effects. In addition, it is suggested that developing a self-control profile may be advantageous for aligning specific interventions to mitigate specific deficits. Overall, the results indicate that interventions are a promising avenue for promoting self-control and may help to contribute to changing health outcomes associated with a wide variety of diseases and disorders.

Impulsivity is a multi-faceted construct (Evenden, 1999) that is often broadly categorized into impulsive choice and impulsive action. Impulsive choice is most commonly measured using choice tasks that involve trade-offs in choices between different amounts and delays. Impulsive choice is the preference for a smaller reward that is available sooner (smaller-sooner or SS) over a larger reward that is available later (larger-later or LL), particularly when the larger reward choice is relatively more optimal. Optimality defined here means maximizing obtained reward after factoring in delays to reward and calories expended to earn reward.

The predominant construct that is used to explain choice behavior in these tasks is delay discounting, which refers to the degradation of the subjective value of a reward as a function of the delay to its receipt. The degradation is a hyperbolic function (i.e., steep decline in value across short delays transitioning to a shallower decline in reward values across increasing delay values) that describes the equivalent value of a delayed reward (V_delay) relative to the reward value if the reward was provided promptly: ${\text{V}}_{\text{delay}}=\frac{\text{A}}{1+k\text{D}}$., where the delayed-reward amount is denoted by A, delay is denoted by D, and the decay rate is represented by the discounting rate parameter k (Mazur, 2001). Figure 1 displays the discounting of SS and LL rewards at the time of the choice point and the implications of the discounting rate for choice. As seen in panel A, with a low discounting rate, the LL is more valued than the SS even though it is delayed considerably. However, when the discounting rate is high (panel B), the SS is more valued. As a result, individuals that are more prone to make impulsive choices should have higher k-values.

Figure 1.

Delay discounting functions for smaller-sooner (SS) and larger-later (LL) outcomes. As the time to a reward is nearer in the future, the value increases until it reaches the actual value at the time of reward receipt. When two outcomes are simultaneously available, their discounted rates can be compared at the choice point, which is at the y-intercept on the graph. The discounting rate (k-value) determines the decay rate over time. When there is a low discounting rate (A), the decay rate is shallow and the LL will be preferred, but when there is a high discounting rate (B), the decay rate is steep and the SS will be preferred.

Individual k-values are proposed to serve as a stable trait variable (Odum, 2011a, 2011b) that predict a wide range of diseases and disorders, thus leading to the suggestion that impulsive choice is a trans-disease process (Bickel, Jarmolowicz, Mueller, Koffarnus, & Gatchalian, 2012; Bickel & Mueller, 2009). Additionally, the discounting rate appears to be a stable trait variable in rats (Peterson, Hill, & Kirkpatrick, 2015) and is a predictor of drug self-administration in humans and non-human animal models (de Wit, 2009; Perry & Carroll, 2008). This cross-species correspondence improves confidence that cognitive and behavioral treatments for impulsivity discovered in animal models (which better lends itself to experimental research) may generalize to a clinical population.

In comparison to impulsive choice, impulsive action refers to the inability to restrain response tendencies or stop ongoing responses. Inhibitory processes, and possibly other executive functions such as attentional regulation, are key predictors of impulsive action challenges (Bari & Robbins, 2013). Impulsive choice and impulsive action are proposed to be distinct processes (Broos et al., 2012), but both are related to substance abuse (Diergaarde et al., 2008; Grant & Chamberlain, 2014), suggesting some partial overlap.

Impulsive choice and impulsive action vary along a continuum with the opposing anchor of the continuum being self-control. In the case of impulsive choice, self-control typically relates to the ability to delay gratification and choose the larger, delayed reward. Self-control can also potentially be expressed through choosing the smaller, certain reward, as opposed to being tempted to gamble and select the larger uncertain reward (i.e., probability discounting) and choosing the larger, more effortful reward as opposed to choosing the easier option (i.e., effort discounting). Although these paradigms are not as commonly associated with self-control, there is an element of controlled behavior in being able to choose the more optimal option in the face of temptation. In impulsive action paradigms, self-control involves the ability to delay, suppress, or cease behaviors, thus making more controlled responses. Self-control as a broader concept, which involves various facets including impulsive choice and impulsive action, has a large effect on human health (e.g., Moffitt et al., 2011).

Although self-control is generally valued in our modern society, it is worth noting that unilateral self-control may not always be the optimal outcome. In some situations, such as more volatile environments where the future is less certain, self-control could be disadvantageous. If waiting is risky and unlikely to pay off, then self-control may be less likely to result in positive outcomes compared to choosing a more certain outcome in the here and now (an impulsive choice). This may explain, to an extent, the evolutionary origins of impulsivity, which may have stemmed from an evolutionary history with volatile environments in which resource availability was often uncertain. Even in our relatively more stable modern environments, it is important to consider the behavioral outcomes of self-controlled or impulsive choices in relation to the environmental situation. This relates to the construct of ecological rationality (Stevens & Stephens, 2010), that the rationality of a decision (i.e., self-control versus impulsive) depends on the situation in which the decision occurs.

Self-control can be stimulus- or context-specific, in that it may be exhibited to specific stimuli or situations. For example, gamblers are more self-controlled in a non-gambling context (Dixon, Jacobs, & Sanders, 2006) and smokers show better self-control for money decisions compared to choices involving cigarettes (Bickel, Odum, & Madden, 1999), although both groups generally show poor self-control overall. On the other hand, self-control is often reported to generalize across a variety of stimuli and/or situations. The trait nature of self-control is consistent with context-general processes (Odum, 2011a, 2011b), and self-control is often consistent across commodities (Baker, Johnson, & Bickel, 2003; Odum & Rainaud, 2003). This has important implications for the development of interventions to alter self-control. To the extent that self-control is generalizable across contexts, then specific interventions that improve self-control may generalize across stimuli or situations. Thus, some interventions may be capable of exerting influence on self-control across a variety of contexts. On the other hand, if self-control is context-specific, then specific interventions may result in limited generalization within a restricted set of contexts.

Given the importance of self-control to human health, there have been various attempts to develop cognitive and behavioral training interventions to promote self-control. Rung and Madden (2018) recently published a meta-analysis of techniques designed to reduce delay discounting and impulsive choice. They found that the most robust techniques involved cognitive or behavioral training with a focus on learning specific content or skills. These procedures often involve training that is designed to target specific processes, such as delay exposure training designed to increase delay tolerance, but that may operate through other processes as well (see below for details). Their meta-analysis did not include interventions designed to target impulsive action, nor did they assess self-control training more generally. The current review will focus predominantly on cognitive and behavioral training interventions designed to target specific processes that promote self-control within paradigms that may invoke self-control in choice or controlled response situations. Our focus is specifically on interventions that are designed to engineer changes in individuals rather than environmental changes such as priming or framing. Environmental changes can produce significant changes in behavior but are often transient and dependent on specific environmental conditions (although it is worth noting that individuals could learn to frame events differently through interventions), so they are often context-specific by their nature. Individual engineering may have a stronger potential for producing lasting self-control changes. In addition, we specifically examine whether the interventions are likely producing domain-specific or domain-general effects on self-control where possible. Domain specific processes target specific individual processes that indirectly affect self-control, whereas domain general processes target broad self-control. We offer a preliminary diagram of self-control regulation which suggests that self-control emerges from a collection of domain-specific processes that may nevertheless contribute to overall self-control and can result in generalization across situations.

Effort and Self-Control

Challenges to self-control can occur when the larger reward also requires greater effort on the part of the organism. Time and effort correlate in the real world, and any project that requires work will also often require that the reward not be delivered until after the task has been completed. As a result, more effort often requires more time, but may have a higher payoff compared to the alternatives. While effort detracts from the value of the option requiring it, an organism can adapt and learn to tolerate effort and potentially even learn to value it (Eisenberger, 1992; Inzlicht, Shenhav, & Olivola, 2018).

Effort is generally divided into physical effort (e.g., intensity of a handgrip squeeze; Mitchell, 2004b) or cognitive effort (e.g., difficulty in solving a puzzle; Botvinick, Huffstetler, & McGuire, 2009). Regardless of the effort modality, as the degree of effort required to earn a reward is increased, preference for a lower-effort alternative will increase, known as the law of least work (Hull, 1943). Just as the subjective value of a larger-amount option is a declining function of its delay to receipt, the subjective value of that option is also a declining function of the effort for its receipt – effort discounting (Hartmann, Hager, Tobler, & Kaiser, 2013; Mitchell, 1999a, 2004b; Sugiwaka & Okouchi, 2004). Research is ongoing regarding the form of the effort discounting function and whether such a function has parameters capable of providing theoretical meaning in the same way that the k parameter does in the delay discounting function (see Figure 1). Using a hypothetical choice task involving effortful options, Sugiwaka and Okouchi (2004) showed that effort was discounted hyperbolically in the same manner as delay. Likewise, Mitchell (1999a, 1999b, 2004a, 2004b) required human participants to squeeze a hand dynamometer for 10 s to earn reward (calibrated individually based upon the percentage intensity of the individual’s maximal voluntary contraction) and reported a hyperbolic effort discounting function. However, Klein-Flügge et al. (2015) and Hartmann et al. (2013), used a hand dynamometer to demonstrate that a negative sigmoidal model (a z-shaped function characterized by shallow discounting across early increases in effort intensity followed by steeper discounting) better described the effect of effort on preference. Regardless, effort discounting produces a hyperbolic or hyperbolic-like discounting function that describes the diminished value of an effortful reward.

Neurobiology of effort.

Delay and effort impulsivity on the surface may appear to be mediated through a common underlying psychological construct given that they both involve discounted rewards, but there is evidence that they are mediated through different neural pathways (Prévost, Pessiglione, Météreau, Cléry-Melin, & Dreher, 2010). Prévost et al. (2010) reported that, in humans, the subjective value of delayed rewards (waiting to see erotic stimuli) was associated with fMRI signal strength in the ventral striatum and ventromedial prefrontal cortex, but the subjective value of effortful rewards (dynamometer use to view erotic stimuli) was associated with signal strength in the anterior cingulate cortex (ACC) and anterior insula (AI). In another study dissociating temporal discounting from effort discounting, Mies et al. (2018) showed that while children and teenagers with attention deficit hyperactivity disorder (ADHD) were more likely than control participants to discount delayed rewards, they did not discount effortful rewards more steeply. Thus, delay and effort impulsivity are not necessarily comorbid in the same population of individuals, suggesting separate processes underlie those forms of impulsivity.

In addition to the ACC and AI, another region of interest regarding the neurological processing of effort-based decision-making is the mesolimbic dopaminergic pathway in humans (Treadway et al., 2012) and rats (Salamone, Correa, Yang, Rotolo, & Presby, 2018). Dopamine (DA) antagonists produce a low-effort bias in effort-discounting tasks with rodents across a variety of studies (Bardgett, Depenbrock, Downs, Points, & Green, 2009; Floresco, Tse, & Ghods-Sharifi, 2008; Robles & Johnson, 2017), while not affecting delay tolerance tested with a progressive-interval task (Wakabayashi, Fields, & Nicola, 2004). Dopamine release in the nucleus accumbens (NA) is positively correlated with effort to respond for reward, but not reward consumption (Salamone, Cousins, McCullough, Carriero, & Berkowitz, 1994). Likewise, inhibiting DA functioning will disrupt a rat’s willingness to climb a barrier to obtain reward, but not their willingness to consume an easily available reward (Salamone, Cousins, & Bucher, 1994). Floresco et al. (2008) compared rats’ choices in delay and effort discounting tasks and reported that DA antagonism increased both effort and delay discounting, and that it increased effort discounting above and beyond any contribution from the delay to reward. Walton et al. (2006) showed that both rats and monkeys are more likely to favor the low-effort option in an effort choice task if the DA mesolimbic pathways (i.e., ACC connections to the NA) were disrupted. Consistent with rats and monkeys, Botvinick et al. (2009) reported that NA and ACC activation in humans was subdued under high-demand conditions.

The relationship between mesolimbic dopaminergic pathways and effort tolerance is clinically relevant when considering disorders that involve dysfunctional motivation. Treadway and colleagues (2012) used an effort discounting task and found a greater intolerance of effort (sustained rapid button pressing) in individuals with major depressive disorder (a disorder that may involve dysfunction of dorsal striatum in effort-based decision-making tasks; Yang et al., 2016). Furthermore, the magnitude of the depressive symptoms was positively correlated with effort intolerance (Yang et al., 2014). The negative symptoms of schizophrenia (e.g., blunted affect, amotivation, asociality) are also linked to dysfunctional dopaminergic systems (Fervaha et al., 2013) and abnormal effort-based decision-making (Bismark et al., 2018; Treadway, Peterman, Zald, & Park, 2015). Gold et al. (2013) reported a negative correlation between preference for higher-effort (high-payoff) option and negative symptoms. Finally, the influence of dopaminergic pathways on effort-based decision-making has been observed in individuals with Parkinson’s disease who are less likely to complete an effortful trial (at low reward levels) than control individuals. However, as reward magnitude increases, individuals receiving DA treatment are relatively more likely to work for rewards (Chong et al., 2015) when compared to untreated individuals.

To summarize, dysfunction of the mesolimbic dopaminergic circuits has corresponding behavioral dysfunction in effort tolerance in human and animal research. This dysfunction may underly disorders in human populations, including depression, schizophrenia, and Parkinson’s disease. Interventions designed to improve effort tolerance may function as a treatment for effort-related impulsivity. If those interventions produce associated changes in neuroplasticity, then it may also produce collateral treatment for the associated diseases.

Justification of effort.

It is quite apparent that effort functions as a barrier to making an optimal choice when the path of least resistance is available. However, prolonged experience working through effortful tasks can improve self-controlled choices regarding effort (i.e., increase grit). It appears that exposure to effortful tasks can increase the value of an act or outcome by itself (Inzlicht et al., 2018). A classic example is the justification of effort effect where individuals appraise events more positively after there was an associated cost (Aronson & Mills, 1959; Festinger, 1957). Additionally, individuals place greater value on things that they have built over identical things that they were given (i.e., the IKEA effect, Mochon, Norton, & Ariely, 2012; Norton, Mochon, & Ariely, 2012; Sarstedt, Neubert, & Barth, 2017).

The increased value of an effortful gain does not require a culture emphasizing a good work ethic (Zentall, 2016). Animals have also demonstrated a preference for a stimulus leading to reward that occasioned greater effort (Clement, Feltus, Kaiser, & Zentall, 2000; Zentall, 2010; Zentall & Singer, 2007). Clement et al. (2000) provided pigeons two different discrimination trial types. In a low-effort trial, pigeons had to emit few responses to initiate a discrimination trial (e.g., peck red key for food or yellow key for no reward) and in a high-effort trial, pigeons had to emit many (20x more) responses to initiate a different discrimination trial (e.g., peck green for food or blue for no reward). After the pigeons learned both discriminations, probe trials presented the pigeons the two correct options (a red and green key) from the two different discrimination trials. Despite two keys offering the same amount of reward, the pigeons favored the key that followed the high-effort trial (e.g., green key). Zentall and Singer (2007) argued that environmental cues that reliably signal an improvement in circumstances will acquire reward value (i.e., a within-trial contrast effect), and therefore organisms will favor that stimulus compared to others. The high-effort response requirement was relatively more aversive than the low-effort requirement; therefore, a stimulus signaling food availability following a high-effort trial had relatively greater value.

The within-contrast effort effect has been replicated in starlings (Kacelnik & Marsh, 2002) and humans (Klein, Bhatt, & Zentall, 2005; Tsukamoto & Kohara, 2017; Tsukamoto, Kohara, & Takeuchi, 2017) using comparable tasks. Conceptually, the within-trial contrast effect has also been demonstrated when the contrast involved a stimulus signaling a transition from a long delay (e.g., Digian, Friedrich, & Zentall, 2004) or a transition to food when the organism was in a relatively more extreme food deprived state (e.g., Marsh, Schuck-Paim, & Kacelnik, 2004). Dobryakova, Jessup, and Tricomi (2017) conducted an experiment in humans which suggested that positive feedback in a cognitive effort task produced a stronger fMRI signal in the ventral striatum (correlated with reward feedback value) following a high-effort trial compared to a low-effort trial. These data are consistent with the hypothesis that positive feedback is more valuable following a difficult task. Although there is promising evidence for within-trial contrast, this effect has not been reliably observed (Arantes & Grace, 2008; Vasconcelos, Urcuioli, & Lionello-DeNolf, 2007a, 2007b) and the mixed results suggest that the contrast hypothesis does not entirely explain the data (Aw, Vasconcelos, & Kacelnik, 2011; Fox & Kyonka, 2014; Tsukamoto & Kohara, 2017). Overall, more research is necessary to fully understand the conditions under which organisms will learn to value stimuli correlated with a transition to reward following a high-effort task. Nonetheless, there are a range of experiments demonstrating that some stimulus-driven mechanism promotes preference for an option that leads to a reward requiring relatively more effort.

Effort-based interventions.

Learned industriousness, opposed to within-trial contrast, argues that exposure to effort and its hard-earned rewards can cause effort itself to become a conditioned reward (Eisenberger, 1992). The hypothesis of learned industriousness originates from experiments with humans and animals, and it provides the basis for behavioral interventions to promote self-control. Eisenberger et al. (1979) demonstrated that when rats were given an effortful task in one context, they were more likely to demonstrate greater response output on a different task with a different response in another context relative to rats that had low-effort training or no training. This result showed that training on one effortful task generalized to greater effort tolerance in another task. Expanding on this finding, Eisenberger et al. (1989) demonstrated that effort training in one task produced more self-control in an effort discounting task. They trained two groups of rats to run down a runway for food in a low-effort (one trip) or high-effort (five trips) task. Additionally, rats in control groups received pellets yoked to the high and low effort conditions. Following effort and delay training the rats chose between a larger-effortful option and a smaller-easier option. The rats that required many laps to earn pellets chose the large-effortful option more often than the rats in the low-effort group or the yoked control groups. This demonstrated that effort training produced improvements in effort-based self-control, but that this improvement was limited to effort training, since the control group that received delay training did not display greater preference for the larger-effortful option. This further suggests domain specificity of the effort-based (as opposed to delay-based) training on transfer to effortful tasks.

The effect of effort training on improving self-control is not isolated to rats. Eisenberger, Mitchell, and Masterson (1985) divided children into high-effort, low-effort, or no task groups. The high- and low-effort task groups were given a series of tasks (object counting, picture memory, shape matching) to complete, but the high-effort group had to work harder to successfully complete each task. Following those tasks, all three groups were given an effort-based self-control test that offered 2 cents for passively waiting (low-effort, impulsive) or 3 cents for completing a task of copying nonsense words (high-effort, self-controlled). Children that were previously in the high-effort task group showed many more self-controlled choices in the test. This demonstrated that, like rats, high-effort experience in one task generalized to more effort-based self-controlled choices in other tasks.

Following the previous study, Eisenberger and Adornetto (1986) examined the domain-generality of effort- and delay-based self-control training. They had children engage in a variety of tasks that offered high or low effort and high or low delay to reward. The children that received long delay training showed better self-control in a subsequent delay-based self-control task relative to an effort-based task and vice versa. This study further showed that effort-based self-control and time-based self-control are not entirely governed by the same underlying processes. While effort and delay training generalized within their respective dimensions, they did not generalize between those dimensions, consistent with the previous research in rats (Eisenberger et al., 1989).

To summarize, intolerance to effort-based costs in decision-making can result in a maladaptive preference for suboptimal (low-reward) choices, but there is evidence that effort-based interventions can increase high-effort choices in both humans and animals. Furthermore, the behavioral and neurological data suggest that effort-based and delay-based impulsivity are a function of different underlying processes and any intervention designed to address self-control problems in humans will need to either be tailored to the form of self-control problems encountered (i.e., be it “laziness” or “impatience”) or include elements that separately address both. The within-trial contrast perspective argues for a mechanism where cues signaling a positive transition from effort to reward acquire greater value when the effort level is greater (i.e., higher contrast), but the learned industriousness perspective suggests that exposure to effortful activities increases tolerance against the aversive qualities of high effort. It is not yet known whether these two perspective mechanisms operate independently or reflect a common mechanism. The argument for these two operating under domain-general process is that they both, through the learning process, result in increased value for a high-effort option. The argument for these two operating as domain-specific processes is that the contrast perspective is tied to environmental cues signaling the transition to a better state of affairs, whereas the experiments surrounding learned industriousness has demonstrated that increased effort tolerance in one specific context (e.g., runway activity in rats, object counting in children) generalizes to increased tolerance in another context (e.g., lever pressing in rats, copying nonsense words in children). For this generalization to occur the contrast effect would need to be generalized such that learned cues signaling an improved transition in one context would spontaneously produce an effect for different cues in a different context. It is uncertain if such a generalization occurs, especially in rats.

Reward and Self-Control

The common denominator for all forms of impulsivity is the currency or commodity that is discounted—reward amount or quality. Given that most self-control tasks explicitly involve different reward amounts, it is reasonable to assume that sensitivity to the different reward amounts would also affect self-controlled choices. The reward dimension is separable from the timing dimension in impulsive choice at a neuronal level of analysis (e.g., Ballard & Knutson, 2009; C. T. Smith et al., 2015), and the two dimensions can be captured independently in behavioral models of impulsive choice (Ho, Mobini, Chiang, Bradshaw, & Szabadi, 1999; Locey & Dallery, 2009; Young, 2018).

Neurobiology of reward.

Reward amount sensitivity does appear to be an important contributor to impulsive choice in rats and humans. Ballard and Knutson (2009), using fMRI data, established that self-controlled choices in response to reward amounts were associated with greater activity in the NA, medial prefrontal cortex, and posterior cingulate cortex. On the other hand, self-controlled choices in response to reward delays were associated with greater activity to the dorsolateral prefrontal cortex and posterior parietal cortex. C. T. Smith et al. (2015) reported that, in people, decreased activation (measured with PET scan) in the midbrain was correlated with reduced discounting with greater reward amounts, consistent with the proposal that the midbrain dopaminergic system contributes to processing reward magnitudes (Tobler, Fiorillo, & Schultz, 2005). This implicates the NA as a key region of interest regarding reward amount processing. In experiments using rats, lesions to the NA core have been associated with increasing sub-optimal, impulsive choices (Basar et al., 2010; Galtress & Kirkpatrick, 2010; Steele, Peterson, Marshall, Stuebing, & Kirkpatrick, 2018), this region appears to serve as a domain-general reward integration region (Cardinal & Cheung, 2005; Cardinal & Howes, 2005; Steele et al., 2018).

Individuals with ADHD display greater rates of temporally-based impulsive choice (Barkley, Edwards, Laneri, Fletcher, & Metevia, 2001; Luman, Oosterlaan, & Sergeant, 2005; Mies et al., 2018; Sonuga-Barke, Taylor, Sembi, & Smith, 1992; Wilson, Mitchell, Musser, Schmitt, & Nigg, 2011) along with poor reward processing related to dysfunction in the dopaminergic system (Acheson et al., 2006; Alsop et al., 2016; Luman, Tripp, & Scheres, 2010; Mairin, Boden, Rucklidge, & Farmer, 2017; Tripp & Wickens, 2008). The comorbidity of impulsivity with poor reward discrimination abilities (Alsop et al., 2016) further links dysfunctional dopaminergic processes with impulsivity and poor reward sensitivity.

Reward-based interventions.

In a straightforward method of assessing the relationship between reward sensitivity and impulsivity, Marshall and Kirkpatrick (2016) demonstrated that rats’ self-controlled choices positively correlated with their pellet-amount discrimination accuracy. However, that relationship was not found when the reward magnitude discrimination procedure involved the successive presentation of trials and discrimination ratios were used to measure the effect (Marshall, Smith, & Kirkpatrick, 2014). This insensitivity may have occurred because relative response rates on schedules of reinforcement that vary on reward amount is an insensitive method to assess discrimination of reward amounts (Bonem & Crossman, 1988; Catania, 1963). Overall, their results suggest that reward magnitude discrimination may contribute to impulsive choice, but further research is needed to better understand this relationship.

Presently, there are few studies that have attempted to experimentally manipulate an organism’s exposure to varying amounts in an attempt to determine the effect on impulsive choice. Marshall and Kirkpatrick (2016, Experiment 2) exposed rats to a reward-based intervention task designed to increase the rats’ sensitivity to the relative reward magnitudes between the two options. In the intervention, the number of pellets earned was varied between two levers across three-session phases; however, in the control group the rats earned a constant number of pellets. The intervention increased self-controlled choices compared to the control group. However, the difference between the intervention and control groups diminished over successive sessions, possibly due to a ceiling effect in the intervention group. A follow-up magnitude discrimination task confirmed that the intervention improved the rats’ magnitude discrimination abilities compared to the control group.

The underlying process by which the reward-based intervention improved self-controlled choices is not yet entirely understood. One possibility is that it improved the rats’ abilities to correctly estimate the differential reward magnitudes between the two options, thus allowing for better informed choices. An alternative hypothesis is that the intervention experience facilitated their learning to assign greater value to whatever option offered more pellets. The rats in the intervention group, but not the control group, demonstrated a numerical distance effect in their discrimination performances (Gallistel & Gelman, 2000), where their accuracy was higher under conditions where the ratio of the pellet magnitudes between the two options was higher (e.g., 1 vs 2 pellets was easier to discriminate than 4 vs 5 pellets). This is a hallmark of good numerical cognition and such an effect would be expected if the rats were making better informed choices. This does not preclude the possibility that the reward-based intervention affected self-controlled choices by a value-based learning mechanism, however. Future research is required to dissociate the two hypotheses.

Another method of increasing self-controlled choices in rats is to bundle rewards. Bundling rewards effectively compresses a series of impulsive choice contingencies into a single choice trial. For example, while a typical choice might result in 1 pellet after 5-s and a transition to a new trial, a bundled choice might result in one pellet after 5 s, a second pellet after 10 s, a third pellet after 15 s and then a transition to a new trial. This procedure resembles some real-life situations where a single choice does not represent a single consequence but rather a cascade of consequences stretched across time. The bundling of rewards increases preference for the self-controlled option in humans (Hofmeyr, Ainslie, Charlton, & Ross, 2011; Kirby & Guastello, 2001) and rats (Ainslie & Monterosso, 2003; Stein, Smits, Johnson, Liston, & Madden, 2013). Stein et al. (2013) demonstrated that exposing rats to bundled rewards not only increased preference for the self-controlled option in the bundled trials but also generalized to increasing self-controlled choices in a conventional impulsive choice task (with no reward bundles). Thus, the bundling functioned as a form of intervention to increase self-controlled choices. It is not immediately clear what mechanism allows bundling to increase self-controlled choices. Rats were exposed to both increased delay and magnitude during the bundling trials. The shift in preference for the self-controlled option in the bundling phase suggests that the rats were more sensitive to magnitude (if the rats were predominately attending to the delay aspect of the contingencies, then one might expect them to favor the impulsive option with less of a delay), but the delay exposure may have also altered their choice behavior (see delay tolerance below). It remains to be seen whether this procedure enhances reward discrimination abilities. Given previous intervention studies, the most likely mechanisms are either increased reward discrimination or value-based learning of the outcomes.

Temporal Processes and Impulsivity

There are two temporal processes that might promote impulsive choice: delay-intolerance and timing deficits. Waiting is aversive, so the ability to tolerate a delay should increase LL choices that may pay off more in the long run. Likewise, being able to accurately and precisely time delays should be important for reducing errors in decision making, resulting in increased LL choices when those choices are relatively more optimal. These two processes are potentially separable but may often operate in tandem. We will discuss each in turn and then their potential interrelationship.

Timing deficits and impulsive choice.

Self-control likely requires an organism to accurately weigh the relative value between the SS and LL. Organisms, however, do not compute the value of their options without a degree of error. The subjective value of an option in an impulsive choice task is the composite of an organism’s estimation of delay to reward and expected magnitude of reward. The processes responsible for obtaining those psychological estimates of time are prone to inaccuracies and imprecision. Therefore, one potential source for the expression of impulsive choices might be the error in estimating the delays associated with the two options, and this may result in inaccurate reward value estimates (Galtress, Marshall, & Kirkpatrick, 2012). For instance, in an impulsive organism, an objective LL option offering 5 rewards after 30 s (10 rewards per min) might be subjectively experienced as 5 rewards after 45 s (6.67 rewards per min), thus resulting in an error in the estimate of reward rate. Additionally, timing imprecision (inconsistent timing estimates) might disrupt temporal decision-making and make the immediate option more attractive in its apparent certainty. If inaccurate and imprecise timing underlies some instances of impulsive choice, then that would suggest that interventions that focus on improving timing precision and/or accuracy may reduce impulsive choice.

There is ample evidence that atypical timing processes and impulsive choice are related in humans (Baumann & Odum, 2012; Berlin, Rolls, & Kischka, 2004; Kim & Zauberman, 2009; McGuire & Kable, 2012, 2013; Noreika, Falter, & Rubia, 2013; Reynolds & Schiffbauer, 2004; Rubia, Halari, Christakou, & Taylor, 2009; Wittmann & Paulus, 2008; Zauberman, Kim, Malkoc, & Bettman, 2009) and rodents (Bailey, Peterson, Schnegelsiepen, Stuebing, & Kirkpatrick, 2018; Marshall et al., 2014; Smith, Marshall, & Kirkpatrick, 2015). The perception of timed events can affect impulsive behavior; for example, humans perceiving that an upcoming delay interval is longer will be less persistent in waiting for delayed rewards (McGuire & Kable, 2013). It is reasonable that misperception of temporal events would also bias preference for a sooner option if the delayed option is overestimated, as in the example above. Baumann and Odum (2012) reported that within a population of college-aged individuals, not screened for a diagnosed disorder, overestimating delays was correlated with higher degree of temporal discounting. Just as individuals with ADHD display poor reward processing, they also exhibit disordered timing (Rubia et al., 2009; Toplak, Dockstader, & Tannock, 2006). Methylphenidate (also known as Ritalin), a commonly used drug used to treat ADHD symptoms, has been shown to both reduce temporal discounting for experiential outcomes (Shiels et al., 2009) and to increase precision in a temporal production task (Baldwin et al., 2004; Noreika et al., 2013) in children with ADHD. However, most of the evidence relating these processes is correlational in nature and does not answer the question of whether timing processes directly contribute to impulsive choice.

Neurobiology of timing and impulsivity.

Timing and impulsivity appear to share neural circuitry, suggesting potential overlapping mechanisms. There are cases where brain injury causes timing deficits and increased impulsive choices. Berlin, Rolls, and Kischka (2004) reported that individuals with damage to the orbitofrontal cortex (OFC) tended to both have a higher rate of temporal discounting and mis-estimated time intervals. Experimentally induced frontal traumatic brain injury in rats also produce greater impulsive choice (Vonder Haar et al., 2017) and similar injuries produce impairments in timing precision (Scott & Haar, 2019).

In rats, the dopaminergic system is implicated in timing processes, particularly in the nigrostriatal region (Meck, 2006). Specifically, lesions to the caudate putamen (CPu) and substantia nigra (SN) produced a flat timing function in rats experiencing a peak interval (PI) procedure instead of peaking at the expected time of food delivery. The timing deficit from disabling the SN pathway was recovered when the rats were treated with L-Dopa (a dopamine precursor). Dopamine’s impact on timing in humans parallels what has been reported in rats. C. T. Smith et al. (2015) showed that weak activity in the CPu in humans was correlated with a preference for the SS in an impulsive choice task. Pastor et al. (1992) reported that individuals with Parkinson’s disease, a disorder resulting from a deterioration of functioning DA neurons in the SN, underestimated and overproduced time intervals and that carbidopa/levodopa treatment to increase DA functioning improved the timing estimates. While levodopa may help correct inaccurate timing estimates, it may also enhance impulsive choice (Housden, O’Sullivan, Joyce, Lees, & Roiser, 2010; Pine, Shiner, Seymour, & Dolan, 2010), and it can trigger an impulse control disorder (Voon et al., 2011). This outcome might seem paradoxical since DA agonists have been involved in the treatment of impulsive behavior in individuals with ADHD. However, it appears that other neurotransmitter and neuromodulator systems (e.g., serotonin) may be responsible for the self-control promoting effects (Dalley & Robbins, 2017; Dalley & Roiser, 2012; C. A. Winstanley, Dalley, Theobald, & Robbins, 2003).

Winstanley and colleagues (2003) gave rats amphetamine prior to an impulsive choice task and reported increased self-controlled responding (similar to humans), but the effect of amphetamine was reduced in rats that had their serotonin levels depleted. Serotonin depletion did not affect impulsive choice on its own; instead the interaction with amphetamine was necessary to produce the effect. Thus, the role of DA in impulsive choice is broad (e.g., timing processes, effort-based motivation processes, sensitivity to reward magnitude) and complex (e.g., promoting impulsivity alone, but promoting self-control in interaction with serotonin).

Timing interventions and impulsive choice.

Rodent models have proved useful in illuminating the processes involved in time-based self-control research. Rats with more precise timing are also less impulsive and these individual differences correspond with what has been reported in humans (Marshall et al., 2014; McClure, Podos, & Richardson, 2014). There is mounting evidence suggesting that impulsive choice is related to timing processes, specifically more impulsive rats are less precise in their timing and/or inaccurate in their estimation of timed events. However, the causal relationship between timing capacities and impulsive choice has not been as well-studied. While exposure to tasks that require waiting has improved impulsive choice (Bailey et al., 2018; Fox, Visser, & Nicholson, 2019; Renda & Madden, 2016; Rung & Madden, 2018; A. P. Smith et al., 2015; Stuebing, Marshall, Triplett, & Kirkpatrick, 2018), only a few studies have independently assessed timing processes to determine whether improvements in subjective timing accompany improvements in self-control. Smith et al. (2015) introduced rats to a variety of interventions designed to improve timing abilities. Differential reinforcement of low rate (DRL, schedules required rats to withhold responding for a prescribed interval of time and respond after the time interval has elapsed to earn pellets), fixed-interval (FI, in which the first press after a fixed time interval produced pellets), and variable-interval (VI, in which the first press after a variable time interval produced pellets) interventions were all effective in reducing impulsive choice and increasing precision in timing. All three of these schedules, including the VI schedule, have been shown to induce anticipatory timing of food delivery and are proposed to invoke the timing system (Church, Lacourse, & Crystal, 1998; Pizzo, Kirkpatrick, & Blundell, 2009; Roberts, 1981). Peterson and Kirkpatrick (2016) exposed middle-aged rats to a VI intervention and reported that the intervention improved self-controlled choices. The VI intervention did not improve timing at a group level (i.e., the rats in the VI intervention group did not generally perform better than the rats in the control group), but individual rats that made more self-controlled choices post-intervention also showed greater timing precision in a temporal bisection task, which is used to measure discrimination of time intervals.

Stuebing et al. (2018) extended the FI time-based intervention to female rats and reported that the intervention improved self-controlled choices compared to a no-delay control group. However, the female rats did not show improved timing precision in a temporal bisection task. The female rats in the intervention group did show better timing accuracy in peak interval tests (i.e., increased peak rates on the LL lever) and progressive interval tests (i.e., fewer ineffective presses prior to the target interval duration). Thus, improved self-control choices and better timing performances were related, but not in the expected way. Time-based interventions do not always improve both self-controlled choices and timing performances. Rung, Buhusi, and Madden (2018) demonstrated that a response-initiated 17.5-s delay exposure intervention improved self-controlled choices in an ensuing impulsive choice task; however, there was no improvement in timing accuracy or precision in a temporal bisection assessment at the group level or the individual-subject level. Eckard and Kyonka (2018) exposed mice to several DRL schedules of reinforcement and reported decreased timing precision (in contrast with A. P. Smith et al., 2015) using a peak interval assessment in mice. Fox et al. (2019) showed that DRL interventions improved self-controlled choices and that DRH (differential reinforcement of high response rates; requiring rapid responding) worsened self-controlled choices. In addition, FI 30-s and 60-s interventions improved self-controlled choices, and a no-delay control worsened self-controlled choices in a pre-post intervention comparison. It is worth noting that other no-delay controls have not been previously reported to have this effect (Renda, Rung, Hinnenkamp, Lenzini, & Madden, 2018). Finally, peak interval timing was not significantly improved in the intervention groups, suggesting that the interventions may have improved other temporal processes such as delay tolerance. The inconsistent results linking timing imprecision and impulsive choice in rats require further investigation in terms of the experimental conditions where a timing relationship is evident, or the methods used to assess timing and choice.

Experimental investigations of impulsive choice interventions have the benefit of testing for the longevity of a given intervention on impulsive choice. Interventions with short-lived effects will not be prime candidates for clinical translation. Renda and Madden (2016) demonstrated that a delay-exposure intervention (akin to Rung et al., 2018) increased self-controlled choices immediately following the intervention exposure and that the effects lasted for at least four months following the intervention. However, Renda and Madden (2016) did not assess whether rats’ timing abilities were improved in this assessment. Bailey et al. (2018) exposed rats to FI and VI interventions and demonstrated that both were effective in increasing self-control choices, but only the FI intervention demonstrated long-term intervention effects in a nine-month follow up choice assessment. The advantage of the FI might be due to this schedule invoking specific timing processes; however, this was not directly measured in the Bailey et al. study. Overall, it appears that time-based intervention effects last for a long time and that interventions with temporal predictability (e.g., FI or delay exposure interventions) perform best at producing long-term benefits. The advantage of temporally-predictable interventions suggests that timing is a candidate process for reducing impulsive choices and that correcting a timing dysfunction may lead to better long-term outcomes for interventions promoting self-control. It is also not yet known what aspect of the timing performance, accuracy or precision, is most improved by time-based interventions. However, hyperbolic discounting can be directly derived from scalar variance in the timing system (Cui, 2011; Gibbon, 1977). This relationship suggests that factors that affect timing precision (i.e., increase or decrease scalar variance) such as changes in the variance in clock speed, fluctuating attention to time, and/or changes in temporal decision thresholds (Gibbon & Church, 1984; Lejeune, 1998; Wearden, 2004) would be the top candidates for future study.

Behavioral intervention effects on timing processes are largely limited to nonhuman animal experimental research at this time (Rung & Madden, 2018). It is evident that time-based interventions do not always produce measurable improvements in timing performances. Thus, misperceptions of time are unlikely to entirely explain the time-based intervention effects on improving self-control. Dissociation experiments are necessary to determine the conditions where improved self-control is aided by improvements in timing processes or improving choice through other mechanisms such as delay tolerance (see subsequent section). Given that Rung and Madden (2018) used a delay-exposure intervention that response-independently delivered a pellet after the delay and A. P. Smith et al. (2015) used an FI schedule, it is possible that the FI intervention encouraged rats to time more precisely because of the response-dependent contingency. Future research may wish to directly compare response-dependent and response-independent timing interventions to see whether (1) response-dependent interventions improve timing abilities and (2) improved timing is correlated with better self-controlled choices above and beyond a delay-exposure intervention that may not directly improve timing abilities.

Delay intolerance.

Waiting is aversive, but those who can tolerate delays (i.e., demonstrate “willpower”) from a young age are better suited to get ahead in the world as an adult (Mischel, Shoda, & Rodriguez, 1989). The ability to tolerate a delay should increase LL choices that may pay off more in the long run. Organisms can be conditioned to increase their delay tolerance, which is the ability or willingness to wait for a reward (Schweitzer & Sulzer-Azaroff, 1988; Sonuga-Barke et al., 1992). A related concept is delay gratification, which relates to the ability to see the value in waiting for reward. Delay gratification will be addressed in the upcoming section on mindfulness training, but it is worth noting that delay tolerance training may affect delay gratification. Improving delay tolerance may help individuals discount rewards less because they become less averse to waiting. Delay tolerance can be measured using tasks that gradually increase the delay to reward and can be altered by interventions that expose participants to increasing or decreasing delays, so they potentially learn how long the delays are and/or decrease their aversion to waiting.

Marshall and colleagues (2014) attempted to parse out the relationship between timing processes, delay intolerance and impulsive choice using a series of temporal discrimination tasks followed by an impulsive choice task in rats. Rats that made more self-controlled choices during the impulsive choice task also had greater timing precision and greater delay tolerance in a progressive interval task, where the delay to reward gradually increased (Marshall et al., 2014). These results suggest that timing processes and delay intolerance are both related to impulsive choice behavior and that improving delay tolerance ability could potentially alter impulsive choices.

Behavioral interventions designed to improve delay tolerance have been developed and employed in both animals and humans. Fading (progressively increasing or decreasing) the SS or LL delays within a choice task is one of the most common paradigms used to promote delay tolerance. One of the first delay tolerance interventions was conducted by Mazur and Logue (1978). The pigeons in the intervention condition were exposed to a delay tolerance fading procedure where the SS delay (6 s) was initially the same as the LL delay but gradually became shorter across sessions until reaching 0 s by the end of the study. A control group received a 2-s SS delay throughout training followed by additional tests with 0- and 5.5-s SS delays. The pigeons that received the intervention chose the LL more often than the control group at the 0-s SS delay. It is possible that the pigeons experienced a carryover effect due to a long history of choosing the LL when the SS delays were increased. However, the pigeons did show changes in behavior as the SS delay was reduced and the group differences were maintained when the SS and LL keys were switched, suggesting that the pigeons were primarily under the control of the reinforcement history.

In addition, delay tolerance can be provided as training before an impulsive choice assessment. Stein and colleagues (2013) gave male rats 120 sessions of delay exposure training where the delay to food reward increased throughout the sessions. Rats that completed this training made more LL choices in an impulsive choice task compared to a control group of rats that received immediate reinforcement during training (Stein, Johnson, et al., 2013), but their responding was comparable to a group that received exposure to a constant long delay. Thus, it is not clear whether the gradual delay increases were necessary to produce delay tolerance.

Delay tolerance paradigms produce similar effects in children and adults. Multiple studies have shown that increasing the delay to the LL reward during the intervention phase increased LL choices in subsequent impulsive choice tasks in children with ADHD (Binder, Dixon, & Ghezzi, 2000; Neef, Bicard, & Endo, 2001), pre-school aged typically-developing children (Schweitzer & Sulzer-Azaroff, 1988), children with developmental disabilities (Vessells, Sy, Wilson, & Green, 2018), children and adults with severe behavior disorders (Fisher, Thompson, Hagopian, Bowman, & Krug, 2000), and adults with intellectual disabilities (Dixon et al., 1998; Dixon, Rehfeldt, & Randich, 2003). These improvements were observed after multiple sessions of training. Combined, these results suggest that improving tolerance to long delays can affect choice behavior in a variety of populations, both clinical and non-clinical.

While these results suggest that gradual fading of delays is an effective intervention to reinforce self-controlled behavior, further research with appropriate control groups are needed to confirm the findings. Many studies involving human participants did not have any condition in which participants received the same treatment in all respects except for the key experimental manipulation (sham training) or an alternative training condition that did not involve prolonged timing exposure. Some studies designed with a pre-assessment of impulsive choice included other manipulations of effort or reward magnitude which complicate the assessment of the relationship between delay tolerance and impulsive choice (Neef et al., 2001). The meta-analysis evaluation of delay fading and exposure training by Rung and Madden (2018) excluded papers (Dixon et al., 1998; Dixon et al., 2003; Fisher et al., 2000; Neef et al., 2001) that did not include appropriate control groups or sample sizes large enough to calculate an effect size. After accounting for those limitations, Rung and Madden (2018) found that the delay tolerance behavioral interventions were associated with large effect sizes in promoting self-controlled choices and their analysis suggests that delay tolerance is a robust and reliable method to improve self-control.

Timing deficits and delay intolerance.

Timing deficits and delay intolerance are temporal processes that have been shown to play a potential role in impulsive choice. These two processes are likely related but are theoretically distinct. For example, if an organism’s cognitive capabilities result in poor perception of the intertemporal reward contingencies, then even a delay tolerant animal may opt for the immediate option. If inaccurate and imprecise timing underlies at least some instances of impulsive choice, then that would suggest that interventions that improve timing precision and/or accuracy may reduce impulsive choice by decreasing errors in judging delays regardless of any changes in delay tolerance.

Much of the intervention research has focused on improving delay tolerance, and the research investigating the mechanism for delay tolerance has not teased apart timing processes from the effects of delay exposure on delay intolerance in improving self-controlled choices. Researchers have proposed that timing abilities predict impulsive choice (Marshall et al., 2014; Wittmann & Paulus, 2008) and that exposing subjects to time intervals can improve their ability to time the delays to reward. However, some of the previous timing interventions may have targeted delay tolerance rather than specific timing processes. This ambiguity could be parsed apart using an adjusting choice task or a systematic choice task following a timing intervention. If timing interventions improve timing processes, rats that received the intervention should make more self-controlled choices in the systematic choice task, where delays are altered systematically across trials or sessions. In comparison improvement in timing should have little impact on choices in an adjusting delay task, where delays adjust frequently based on the individual’s choices, as adjusting tasks have been found to be less reliant on timing of specific delays (Cardinal, Daw, Robbins, & Everitt, 2002; Peterson et al., 2015). On the other hand, if timing interventions primarily improve delay tolerance, then rats that received the intervention should make similar amounts of self-controlled choices on both choice tasks.

It is also possible that delay intolerance and timing may interact directly in affecting impulsive choice. For example, errors in temporal perception may lead impulsive individuals to overestimate the duration of experienced delays (Wilson et al., 2011; Wittmann & Paulus, 2008), thus leading to delay intolerance. In contrast, delay intolerance may drive impulsive individuals to avoid longer delays which could impair learning about delays and ultimately lead to increased timing deficits (Galtress, Garcia, & Kirkpatrick, 2012; Kirkpatrick, Marshall, & Smith, 2015; C A Winstanley, Eagle, & Robbins, 2006). Whether delay intolerance leads to temporal processing deficits or vice versa, timing interventions produce robust and lasting increases in self-control that are a potentially effective tool that could translate for potential treatment of impulsivity. Further research should focus on parsing out the causal relationships between timing deficits, delay aversion, and impulsive choice.

Inhibitory Interventions

Impulsive behavior may originate from a decreased ability to inhibit an automatic response to choose the SS and instead of waiting for the LL. A decreased ability to inhibit responding is a common symptom associated with ADHD. Researchers have argued that ADHD is comprised of multiple endophenotypes, two of which are poor response inhibition and excessive impulsive decision-making (Castellanos & Tannock, 2002; Slaats-Willemse, Swaab-Barneveld, De Sonneville, Van Der Meulen, & Buitelaar, 2003). While the causal relationship remains unclear, those who make more impulsive choices also make more errors on response inhibition tasks (Solanto et al., 2001). This section focuses on behavioral inhibition, specifically response inhibition, as interventions in this area are more numerous. Response inhibition encompasses actions like waiting, withholding a response, or stopping a response (Bari & Robbins, 2013). Tasks used to measure behavioral inhibition are easier to reliably measure in both humans and animals compared to cognitive inhibition tasks (Bari & Robbins, 2013).

DRL schedules.

Inhibition is commonly studied in humans using tasks such as the go/no-go paradigm (in which individuals must respond to some stimuli and withhold responses to other stimuli) or the stop-signal task (in which individuals must stop an ongoing response when a signal occurs) that involve testing the ability to inhibit a planned response (Bari & Robbins, 2013). These paradigms in addition to the five-choice serial reaction time task have not typically been used as interventions. Instead, researchers use them to measure impulsive action (e.g., motor impulsivity). Other tasks, such as DRL schedules, have been targets of behavioral interventions. Under DRL contingencies, organisms must inhibit responding until a specific time has passed after which they can respond. As mentioned above, DRL schedules may also be used as time-based interventions to improve self-control (Fox et al., 2019; A. P. Smith et al., 2015). DRL contingencies have also been successfully incorporated into behavioral inhibition interventions with children. For example, three typically developing children completed a DRL intervention to help reduce the number of requests for the teacher’s attention during class (Austin & Bevan, 2011). The DRL intervention significantly decreased the number of attention requests from the children. These results suggest that DRL interventions can decrease problem behavior in humans as well as animals.

Although the literature on DRL schedules to improve self-control is relatively small, the DRL interventions do appear to produce a large effect in increasing self-controlled choices in animals and humans. The exact mechanism by which the DRL schedules operate is still unclear. DRL schedules do not appear to consistently improve timing precision (Eckard & Kyonka, 2018; Fox et al., 2019), but may increase delay tolerance, which results in fewer impulsive choices. Alternatively, DRL schedules may improve other inhibitory processes such as deferred gratification (Bari & Robbins, 2013), which may help suppress the urge to act impulsively.

Inhibitory control training.

While few DRL intervention studies have used human participants, the use of inhibitory control training (ICT) to reduce unhealthy choices has received more attention. ICT typically involves a go/no-go task or a stop signal task where the responses that require inhibition are presented with unhealthy choices. The general approach of ICT training, using healthy and unhealthy foods as an example, is to train participants to respond to pictures of healthy foods or no food and inhibit responses to unhealthy foods. This is often compared to the reverse condition where they make a response to the unhealthy pictures. Successful inhibition of real-world unhealthy food choices has been obtained using the go/no-go task cued with pictures of unhealthy food choices (van Koningsbruggen, Veling, Stroebe, & Aarts, 2014; Veling, Aarts, & Papies, 2011). The go/no-go ICT has also been applied to increasing healthy food choices in children (Porter et al., 2018) and decreasing alcohol consumption in undergraduate students (Bowley et al., 2013; Houben, Nederkoorn, Wiers, & Jansen, 2011). However, these studies did not include a sham control group where all pictures are presented with go or no-go responses equally, so it is unclear whether the unhealthy/go condition made them worse compared to the unhealthy/no-go condition.

A few studies have incorporated a sham control to address these weaknesses and have yielded mixed results. For example, Houben and Jansen (2011) selected college-aged females who craved chocolate to participate in a go/no-go task that served as training to inhibit their responses to chocolate. During training, the go cue was presented with pictures of empty plates or plates of snacks and required a response on a keyboard, while the no-go cue was presented with pictures of chocolate and required no response. The sham training group completed the same task, but all the pictures were presented with a go cue half the time and a no-go cue the other half of the time. Another control group completed the same task, but the go cue was presented with pictures of chocolate and the no-go cue was presented with pictures of snacks or empty plates. After training, the participants were presented with three bowls of chocolate and instructed to eat however much they wanted so they could judge the tastes and textures of each. Compared to the sham training group, the participants who inhibited responding to pictures of chocolate during training ate less chocolate which suggested that ICT strengthened the participants’ ability to inhibit a response to chocolate (Houben & Jansen, 2011). However, the participants who inhibited responding to pictures of chocolate did not differ from those in the condition that associated chocolate with the go cue for the entirety of training. Houben and Jansen (2015) repeated the previous study minus the sham training group and found that participants who inhibited responding to pictures of chocolate ate significantly less chocolate than those who had chocolate associated with the go cue, suggesting that the intervention was effective. Further investigation of these two conditions, chocolate/no-go and chocolate/go, supported the conclusion that the female participants increased their ability to inhibit responding to chocolate (Houben & Jansen, 2015).

Similar work using a stop signal task has been conducted to improve inhibitory control over food and alcohol. Stimuli are presented to the right or left of center on the screen. Participants responded once on two different keys to indicate what position the stimulus occupied on go trials. In the experimental condition, the unhealthy or “bad” stimulus was often presented with the stop cue, so participants must inhibit responding on either key. The control group, on the other hand, typically had to respond twice to the unhealthy stimulus. Control participants responded twice to unhealthy options to equate attentional demand across groups. The stop-signal intervention effectively reduced food consumption during a snacking phase in undergraduates (Lawrence, Verbruggen, Morrison, Adams, & Chambers, 2015) and reduced alcohol consumption during a taste test (Jones & Field, 2013). This suggests that the effect of ICT is not a product of the control group resulting in poorer inhibitory control, but instead appears to be due to the intervention increasing inhibition.

Inhibitory control training can be effective outside a laboratory/clinical setting as well. The ICT interventions can be adapted into smartphone applications for daily training. Improvements to self-control were observed after participants completed a modified Stroop-task for four weeks (Cranwell et al., 2014). These improvements were measured by comparing a baseline assessment of self-control administered before the 4-week training period to a post-assessment. This line of research has the potential to bring effective interventions to a wide range of populations that would benefit from daily inhibitory control practice and should be explored further.

Many studies have concluded that ICT effectively reduced appetitive behaviors, and meta-analyses have shown that ICT significantly improves self-control after training (for a comprehensive review of effect sizes, see Jones et al., 2016). Despite the growing literature in this field, the mechanism responsible for the improvements to self-control are not yet fully understood. Researchers have determined that the association between unhealthy stimuli and response inhibition is essential for the improvements to occur, but the way in which this association formation improves impulsive behavior remains ambiguous (Jones et al., 2016).

One avenue to understanding the mechanisms of action can be through examining the neurobiology associated with inhibitory training. However, the neurobiological mechanisms involved in behavioral inhibition are also unclear. Tasks such as ICT engage multiple neurotransmitters and cortical regions that are dependent on the context. Researchers agree that the prefrontal cortex and pre-motor areas have a major role in response inhibition, but the way in which dopamine, norepinephrine and serotonin interact is widely debated (for a comprehensive review of pharmacological studies, see Bari & Robbins, 2013). In addition, challenges with procedurally separating inhibition from other processes such as attention have complicated the understanding of the roles of each neurotransmitter system. Future studies need to address whether the mechanisms of the interventions are through strengthening of inhibition, or through some other attention-related process such as mindfulness.

Mindfulness training

One potential mechanism for improving self-control is through increasing attention to one’s own actions and the consequences of those actions. Attentional processes have been the targets of mindfulness interventions. The prior sections have conceptualized self-control in terms of distinct forms (e.g., effort-based, time-based, waiting-based, reward-based, and inhibitory-based). However, self-control has also been described as consisting of two interacting systems – hot and cool (Metcalfe & Mischel, 1999; Mischel & Ayduk, 2002). The hot system is based off emotions and motivations leading to reactive impulsive behavior (perhaps related to delay intolerance and poor inhibitory processes). Whereas the cool system is based off executive/cognitive control processes (e.g., reflective logic and planning) leading to self-controlled behavior. Attentional control, the construct of being able to observe or neglect aspects of the environment, is a necessary function of the cool system. Within the cool system, attentional control is needed to ignore emotional reactions and embrace a calm pensive reaction. Deficits in attentional control impair the ability to utilize the cool system and may result in impulsive behavior. The importance of attentional control in promoting self-control has led to the development of mindfulness interventions to promote attention.

Mindfulness is the practice of maintaining attention to the current situation while concurrently acknowledging and accepting any thoughts of feelings that arise (Bishop et al., 2004). Learning processes appear to be involved in establishing and mediating the effects of mindful behavior. That is, individuals can learn that they are susceptible to engaging in impulsive behaviors and modify their attention accordingly. For example, they could modify their attention to focus on the benefits of avoiding the temptation to eat a cookie that will violate their dietary goals. Alternatively, they could modify their behavior by making plans to only have healthy foods in the house to avoid abandoning a diet in a moment of weakness. These actions can avoid decisions that individuals may later regret. Out of all the prior practices for self-control, mindfulness training is the only technique that relies on some degree of awareness or self-monitoring by requiring individuals to become aware of their own tendencies to misbehave. Mindfulness training addresses factors incorporated in attentional control such as sustaining attention and awareness, regulating emotions, and inhibiting intrusion of outside thought (Bishop et al., 2004; Semple, 2010). These factors are essential to utilize the cool system of self-control by allowing individuals to remain calm and attentive. There are multiple methods to accomplish attentional control that fall under the umbrella of mindfulness training. This review will discuss mindfulness training more broadly rather than parsing out by individual practices.

Mindfulness training has been utilized in healthy (Chambers, Lo, & Allen, 2008; Mrazek, Franklin, Phillips, Baird, & Schooler, 2013; Valentine & Sweet, 1999; Van Dillen & Papies, 2015) and clinical (Baer, 2003; Semple, 2010) populations to improve attentional control. Different factors of attention are measured to address the efficacy of mindfulness training. These factors include but are not limited to sustained attention, attentional inhibition, and attention switching (Bishop et al., 2004). During mindfulness practice, sustained attention is needed to maintain focus on the present and attentional inhibition prevents leaving the state of mindfulness. Attention switching is important for being able to acknowledge thoughts or feelings that arise during practice while then switching back to focusing on the present moment.

The importance of attention has been exemplified in school children that were less likely to fail at a delay to gratification task when they could not attend to the SS reward (Mischel & Ebbesen, 1970; Mischel, Ebbesen, & Raskoff Zeiss, 1972). Mischel et al. (1972) presented children with a delay to gratification task and instructed them to think about other things while a lesser preferred reward was placed in front of them tempting them to forfeit the claim to the better delayed reward. The children that were directed to self-distract from grabbing the immediate reward were more likely to wait for the preferred reward, further demonstrating the importance of attention in self-control.

The attentional processes involved in the delay to gratification task are not limited to humans. The diverting of attention leading to delay of gratification is noted in research on collateral behaviors. Collateral behaviors are behaviors that occur during an operant behavioral task (Hemmes, Eckerman, & Rubinsky, 1979) and may function as a method of self-distraction. In pigeons this behavior is commonly expressed as pecking a portion of a screen, response strip, or response key that is not reinforced or punished. This pecking improves performance on a DRL schedule as it occupies the delay interval between reinforced responses (Hemmes et al., 1979; Schwartz & Williams, 1971). For example, Hemmes et al. (1979) trained pigeons to peck a response strip to earn food reinforcement. Once training was completed, a DRL schedule was put in place in which part of the response strip delivered food following the first response after a delay interval. Pecking the portion of the strip without the DRL schedule did not result in reinforcement or punishment but the behavior was recorded. The birds that pecked the neutral portion of the response strip during the delay interval performed better on the DRL schedule and earned more food than the bird that did not. These results suggest that non-human animals may divert attention to improve performance on challenging tasks and this may extend to other species as well (Slonaker & Hothersall, 1972; Van Hest, Van Haaren, & Van de Poll, 1987).

More recently, Evans and Beran (2007) tested chimpanzees in a task where candies would accumulate in a receptacle as long as the chimps did not grab the receptacle. This is variant of a delay to gratification task where self-control through waiting results in more candies earned. In the experimental condition, the chimps waited for candies to accumulate and had access to toy stimuli that they could manipulate. The researchers measured toy manipulation behaviors and how soon the chimps grabbed the candies. In one control condition, the chimps had to wait without toy stimuli, and they measured the time it took to grab the candies. In another control condition, the monkeys had access to toys, but they did not have the opportunity to grab the candies early. Instead, toy manipulation behaviors were measured. The chimps waited longer to maximize candy accumulation when they had toys available than when they did not have toys, and the chimps’ manipulation of the toys was more likely to occur when the candies were available to grab. The chimps did not have multiple trials per day that could potentially allow them to make up for candies that they lost from grabbing the receptacle too soon. Thus, the impulsive behavior was associated with an actual cost of missing candies that they could have received. The results suggest that the chimps instrumentally directed their attention to the toys while the candies were accumulating to avoid making an impulsive choice and increase reward earning. The significance of this finding, and the findings of other types of collateral behaviors, is that it demonstrates that nonhuman animals without verbal reasoning skills can still learn to redirect attention in a way that helps them be more self-controlled. This may involve a rudimentary process implicated in mindfulness training in humans as well.

In terms of specific attentional processes, the results of the mindfulness studies suggest that the largest improvements are in sustained attention, which is the ability to maintain attention on a specific task or stimulus for a longer period. Considering that maintaining attention to the present moment is at the core of mindfulness training, this is not surprising. It can be argued that the ability to sustain attention requires inhibition; individuals need to inhibit thoughts that would remove them from a mindful state. Future work should address the role of attentional inhibition in relation to mindfulness. If mindfulness training is improving attentional inhibition, this may improve the use of the cool system for self-control.

The direct use of mindfulness training as a method of decreasing impulsivity has been explored as well. The efficacy of training is still in question as experiments have produced mixed results (Ashe, Newman, & Wilson, 2015; Cairncross & Miller, 2016; Hendrickson & Rasmussen, 2013, 2017; Morrison, Madden, Odum, Friedel, & Twohig, 2014). A major challenge of using mindfulness training to improve self-control is that it seems to be stimulus-specific. For example, Hendrickson and Rasmussen (2013) had participants complete food and monetary discounting tasks prior to a mindfulness training that was specific to food. Following training, participants completed the discounting tasks again and more self-controlled responses were observed in the food discounting task for the mindfulness training group, but this was not observed in the monetary discounting task. The stimulus specificity of mindfulness training will likely impact the ability of this treatment to generalize to other impulsive situations.

Even when addressing an underlying component of impulsivity such as delay tolerance, selective effects of mindfulness training come into play. Morrison et al. (2014) recruited individuals with steep discounting functions (i.e., high k-values) to participate in mindfulness training to assess the effects on Acceptance and Action Questionnaire-II (AAQ-II), Distress Tolerance Scale (DTS), and monetary discounting. All measures were completed prior to training and participants were assigned to the mindfulness training group or the “wait-list” control group. Those in the mindfulness group completed pre-tests, received training, and then completed post-test assessments. The mindfulness training emphasized overcoming the discomfort experienced when waiting and was tailored to the specific individual as needed. Those in the “wait-list” control group completed the initial pre-test but no treatment while the mindfulness group received their treatment and then completed the post-test. The wait-list group subsequently received mindfulness training prior to completing an additional final post-test. Reductions were shown in delay discounting for the initial training group and when groups were combined (no control group) to analyze pre-and post-treatment. Decreased distress intolerance was only significant when analyzing pre- and post-treatment. The results suggest that mindfulness training may be a useful strategy at improving self-control in a delay discounting task but may not consistently alleviate the negative emotions experienced when waiting. The inability to consistently reduce distress from waiting may impact the ability to maintain use of the cool system.

Taken together these two experiments raise several questions regarding the use of mindfulness as a direct method of improving self-control. It is likely that mindfulness training will not generalize to all aspects of impulsivity, so it may be a better treatment option for specific deficits. In both experiments, the cool system was utilized via mindfulness training, but it is possible that utilizing the cool system is not enough to ward off impulsive behaviors. An individual needs to be able to comprehend what constitutes an impulsive behavior. The inability to determine what is or is not an impulsive behavior may be due to alternative deficits that the cool system cannot address. For example, in a standard delay discounting task an individual using the cool system may be able to inhibit responding promoting a contemplative reaction to the choices at hand. Despite this calm and thoughtful reaction, an impulsive choice can still be made as the individual is unable to decipher which choice is optimal. The inability to comprehend what choice is the impulsive choice may be due to impairments in other processes discussed above such as timing or reward processes. Therefore, mindfulness training in conjunction with other interventions previously discussed may be ideal. Administering mindfulness training prior to other interventions could improve attentional control, allowing other learning processes to be more effective in the behavioral interventions discussed above.

General Discussion

The present review discussed a range of cognitive and behavioral training techniques that have been employed to promote self-control in various ways. The most successful interventions appear to be the learning-based techniques that target specific aspects of self-control such as high-effort training, reward bundling, interval schedules, delay-exposure, and ICT (inhibitory) training. Some of these interventions have been applied strictly in controlled laboratory settings with substantial training, so their application to real-world setting remains to be determined. However, the inhibitory training and mindfulness training techniques have been used to varying degrees of success in applied settings, suggesting that some interventions may be transferrable to real-word problems. In some cases (e.g., fixed interval and delay exposure training), the effects of the interventions were quite long lasting. However, the effects of many of the interventions do appear to be stimulus/context-or domain-specific. For example, high-effort training was found to generalize to effort discounting choices but not delay discounting choices and vice versa (Eisenberger & Adornetto, 1986). And, mindfulness training specific to food was found to generalize to food choices but not to other types of choices (Hendrickson & Rasmussen, 2013). The fixed interval intervention (Bailey et al., 2018) did generalize to both delay and reward choices within an impulsive choice task, suggesting some degree of generalizability. However, it seems unlikely that transfer would be seen to other types of choice tasks (e.g., effort and probability discounting) or to other controlled response tasks, except perhaps to other interval schedules such as the DRL – this remains to be seen.

Mindfulness training, on the other hand, targets general attentional control processes which theoretically could apply to a wide range of situations. This type of training has only received preliminary attention with regard to self-control, and results have been fairly mixed in terms of intervention efficacy (Ashe et al., 2015; Cairncross & Miller, 2016; Hendrickson & Rasmussen, 2013, 2017; Morrison et al., 2014). Rung and Madden (2018) concluded that there was no overarching evidence that mindfulness training decreased delay discounting on its own. The weaker pattern of evidence for mindfulness training may, in fact, stem from its failure to target specific cognitive processes. Repeated activation of neural pathways to target specific cognitive processes may be a key ingredient to robust and lasting intervention effects. Even so, the mechanisms of the specific interventions remain poorly understood and much further study is needed to identify the cognitive and neural mechanisms that are targeted by these interventions. More focus on nonhuman animal experiments in which mechanisms can be more easily targeted through causal methodologies could help to identify key substrates. These can then be corroborated with human research, particularly those that involve neural correlates that can then be linked back to the mechanisms discovered in the animal studies.

Figure 2 displays a diagram of the different training techniques and the proposed processes they may be targeting. Given the paucity of evidence on this topic, this diagram should be viewed with caution and seen as an early attempt to organize current evidence and motivate further research in this area. The different interventions that have been discussed here include effort exposure, reward discrimination, reward bundling, delay exposure, interval schedules (including FI, VI, and DRL schedules), inhibitory control training (e.g., stop-signal and go/no-go tasks), and mindfulness training. We propose that each intervention targets one or more domain-specific processes that most closely relate to the underlying neural systems and learning mechanisms that are invoked by the intervention. Effort exposure is proposed to induce effort tolerance (Eisenberger, 1992; Treadway et al., 2012), but may in some cases also involve some form of within-trial contrast effect (Zentall & Singer, 2007). Reward discrimination and reward bundling techniques are proposed to primarily affect reward differentiation and/or value-based learning mechanisms (Ainslie & Monterosso, 2003; Hofmeyr et al., 2011; Kirby & Guastello, 2001; Marshall & Kirkpatrick, 2016; Stein, Smits, et al., 2013). Interval schedules that do not have explicit inhibitory elements should primarily alter interval timing processes (A. P. Smith et al., 2015), but may also affect delay tolerance (Marshall et al., 2014) and potentially attention to time (Galtress, Garcia, et al., 2012). On the other hand, DRL schedules that involve temporal processing and inhibitory elements may affect inhibitory processes (Eckard & Kyonka, 2018; A. P. Smith et al., 2015), alter interval timing (A. P. Smith et al., 2015), and/or increase delay tolerance (Fox et al., 2019). ICT training should primarily affect inhibitory processes. Mindfulness training is the most challenging to pin down as it may potentially affect different processes depending on the targets of the mindful thinking. Based on the delay discounting literature, mindfulness likely alters attentional control.

Figure 2.

A preliminary proposed diagram relating the intervention types discussed in the above sections with the target processes that may contribute to general self-control. Solid arrows represent better-established relationships, whereas dashed lines represent potential hypothesized relationships.

There are some interesting questions that stem from the diagram in Figure 2. First, because the evidence so far suggests domain specificity of at least some interventions, the diagram suggests that each target process could promote general self-control but given that there are multiple processes feeding into self-control, one would expect to see only partial moderation of choice and/or controlled responses. This prediction does indeed appear to be the case. Although the interventions described here do increase self-control, they do not lead to extreme self-control. For example, Binder et al. (2000) reported that the delay tolerance intervention made two of the children less impulsive, but they did still make some impulsive choices. ICT training results in reduced consumption of unhealthy foods, but participants do not refuse to eat unhealthy foods following the intervention (Houben & Jansen, 2011, 2015; Veling et al., 2011). And, rats that received time-based interventions significantly increased their self-controlled choices but still made a considerable number of impulsive choices (A. P. Smith et al., 2015).

This does raise an interesting question, however, about the goal of these interventions. Although self-control is a virtue and a predictor of human health (e.g., Moffitt et al., 2011), ultimately the goal of interventions should be to engender good decision making. Many of the intervention studies described in this review have found partial moderation of choices and controlled responses so that individuals may no longer be in the highest risk category in terms of impulsive behaviors. Partial moderation may be enough to move individuals towards healthier decisions. Although interventions often hold up self-control as a gold standard, impulsivity can be advantageous in some settings. It would be interesting to assess the effects of the interventions on behaviors under situations where different strategies may be more or less optimal (such as the ecological rationality hypothesis discussed earlier). Impulsivity is often sub-optimal, but not always, and it may be possible to be self-controlled to a fault (see, for example Bailey et al., 2018). This is an interesting possible direction for future research.

A second issue implied by the diagram is that the domain-specific nature of the interventions suggests that they may be most successful if applied to target populations. This suggests that screening individuals to create a self-control profile would be beneficial to determine if they have specific deficits. The intervention tasks may themselves be beneficial here in revealing the deficits that they are also designed to treat by examining early performance on these tasks. If an individual is impulsive, then giving them an array of tasks such as high-effort tasks, timing schedules, reward discrimination tasks, and inhibitory tasks may reveal specific deficits for intervention. In cases where they do show specific deficits, then further training on the interventions could target those deficits, potentially leading to more successful therapeutic approaches. On the other hand, if individuals exhibit consistent self-control deficits across a range of tasks, then they may suffer from generalized self-control problems, in which case a multi-dimensional intervention may be needed.

A further issue raised by the proposed diagram is whether the individual interventions may be additive or synergistic. The preliminary proposal that the individual processes all influence general self-control and given that each process may be at least partially independent it should be possible to produce additive effects by delivering multiple interventions to the same individual. This could be beneficial for individuals with more general self-control challenges if each intervention can target individual processes and sum together to promote additional increases in self-control in terms of the magnitude and/or longevity of the intervention effects. It is additionally possible that at least some of the interventions may be synergistic, particularly if they target some shared processes. For example, mindfulness training could potentially be used in conjunction with interval schedules as a scaffolding technique to produce enhanced delay tolerance. This could be especially beneficial for individuals with serious delay intolerance challenges, such as ADHD patients.

Overall, it appears that there are some promising techniques for increasing self-control by using cognitive and behavioral training methods. Many of these methods are well established in the laboratory but need further testing for real-world applications. The target processes for the interventions are somewhat known, but the mechanisms by which these processes alter self-control requires further research. In addition, further research is needed to determine whether individuals with specific profiles may benefit differently from different interventions and whether the interventions may be used in concert to produce additive or synergistic effects. Although much remains to be studied in this area, there is reason for hope that these interventions may be a foundation for moderating poor self-control.