Reinforcement Pathology and Obesity

Abstract

Obesity is, in part, a result of positive energy balance or energy intake exceeding physiological needs. Excess energy intake is determined by a series of food choices over time. These choices involve both motivational and executive function processes. Problems arise when there is excessive motivation to eat and low impulse control, a situation we have termed reinforcement pathology. Motivational and executive function processes have also been implicated in the development of drug dependence and addiction. In this review we discuss the application of reinforcement pathology to obesity, and implications of this approach for obesity treatment.

INTRODUCTION

Obesity is a result of positive energy balance or energy intake exceeding physiological needs. Differences in energy balance are primarily determined by choices about food and meals that are made repeatedly over time. Food choices that emphasize immediate gratification often result in positive energy balance, while weight loss or weight maintenance are a result of choices that focus on delayed rewards such as long-term health outcomes [1]. Food reinforcement has been implicated in the motivation to eat while impulsivity has been associated with both decision-making and the ability to inhibit responding. Imbalances in the interaction of these two systems may lead to a tendency to engage in unhealthy behaviors that cause weight gain over time. These imbalances can lead to what we have termed reinforcement pathology or an abnormal motivation to eat combined with increased impulsivity [2].

Food is intrinsically motivating and is considered a primary reinforcer. Increased relative reinforcing value of food predicts increased energy intake [3, 4], greater Body Mass Index (BMI) [5] and greater prospective weight gain [6]. Energy intake mediates the relationship between food reinforcement and BMI [7] such that high food reinforcement predicts increased energy intake, which then contributes to an increased BMI. Obese people have higher food reinforcement than their leaner peers [4, 8, 9] corresponding to increased energy intake [8, 9]. Reinforcing value is influenced by state variables such as deprivation [10–12] and environmental conditions [13–15] including the availability of alternative reinforcers [13, 16, 17]. Motivational processes guide behavior through reward centers in the limbic system [18], which direct attention and available energy towards relevant environmental stimuli, such as food and water [19].

The reflective [20] or executive system [21] allows for collective evaluation of short-term and long-term rewards through areas of the prefrontal and frontal cortices. Baddeley [22] divided executive function into three separate but interconnected components including the visuospatial sketchpad, the phonological loop and working memory. The last of these, working memory, integrates and manipulates information from the other components [22, 23]. Individual differences in the ability to compare rewards across time and alternatives are related to measures of impulsivity [24, 25]. There are several aspects of impulsivity [25] and this paper will focus on impulsivity as related to delay discounting and poor inhibitory control or response inhibition. Delay discounting refers to the hypothetical ‘discounting’ of larger rewards presented in the future versus smaller immediate rewards. Humans will discount rewards presented at a delay in a hyperbolic fashion, such that discounting steepens as reward delay increases [26]. Steeper discounting curves indicate increased impulsivity and a preference for immediate rewards. This has been associated with increased BMI [27], tobacco and alcohol use and decreased use of sunscreen [28]. Response inhibition is a second construct of impulsivity [25], generally measured by a stop/go or go/no-go task, in which participants are asked to respond to a stimulus and inhibit their responses when a second stimulus is simultaneously presented. When measuring response inhibition, impulsivity is indicated by a greater number of response errors (responding when cues indicate to withhold response). Steep delay discounting and poor response inhibition has been associated with greater BMI [29], poor obesity treatment outcomes [30] and long-term weight gain [31].

Motivational and executive systems interact to regulate eating behavior and there may be a subset of individuals who find food highly reinforcing and have low impulse control [2, 32], a state we refer to as reinforcement pathology. Individuals with reinforcement pathology have a greater risk for increased energy intake, weight gain and high BMI. The concept of reinforcement pathology has been previously used to understand addictive behaviors [21, 33]. There are many parallels between drug and food reinforcers suggesting reinforcement pathology would be a viable theory for obesity, and may provide new ideas for treatment development. This review will focus on a brief overview of some of the parallels between food and drug reinforcers, review evidence of reinforcement pathology in obesity research and discuss implications for clinical interventions for obesity.

PARALLELS BETWEEN NATURAL AND DRUG REINFORCERS

Primary or natural reinforcers are intrinsically motivating to humans and animals and include both food and drugs. Reward processes are mediated mainly through dopaminergic pathways in the limbic system, including the connections between the ventral tegmental area and the nucleus accumbens, dorsal striatum, hippocampus and amygdala [34]. Dopamine release in these regions includes phasic and tonic release patterns [35]. Natural rewards, such as food, result in phasic dopamine release in the nucleus accumbens and the pharmacological actions of drugs of abuse mimic these bursts [36]. It is hypothesized that phasic dopamine release underlies the reinforcing properties of both food and drugs providing a metric to calculate the value of a reinforcer [37]. Commodities, that increase dopamine release are found to be highly reinforcing, and reinforcing value is an important determinant of consumption.

Drugs initiate a supraphysiological level of neurotransmitter release that results in a euphoric ‘high’ [36]. Repeated exposure to elevated dopamine can cause compensatory mechanisms to down regulate dopamine receptor availability as well as tonic and phasic dopamine release [35]. These neuroplastic changes in turn lead to blunted responses to both drug and natural reinforcers and a need to progressively increase the drug dosage to achieve the previous level of reward. While consumption of food does not result in the same magnitude of dopamine release, repeated overeating could similarly lead to down-regulation of dopamine receptors [38]. Neuroimaging studies show that children at risk for obesity initially have increased brain activation in response to food, corresponding to high value of food rewards [39]. Over time, down-regulation of the dopamine receptors may decrease the rewarding properties of food stimuli and may result in an increase in food intake to compensate [39].

Craving or compulsions for drug and food rewards also involve dopaminergic pathways. Upon repeated exposure to either a drug or food reinforcer, internal or environmental stimuli can gain the capacity to elicit dopamine release, forming conditioned cues that predict reinforcer receipt [40–42]. Exposure to these cues will then elicit dopamine release prior to the consumption of the reinforcer and is thought to be the basis of craving [40, 43]. These cues predict drug or energy intake through dopamine release and is hypothesized to be a main cause of drug relapse [35]. Conditioned cue exposure leads to craving for the specific reinforcer cued and not a general drug or food craving [44, 45].

Drug dependence is characterized by several key behaviors including withdrawal, tolerance and sensitization [33]. Withdrawal symptoms are apparent when the drug is removed or when there is a greatly reduced intake of the drug and are generally somatic opposites of drug effects [46]. Tolerance is developed slowly over time, such that progressively more drug is needed to achieve the same level of euphoria [47]. Sensitization is the increase in physiological responding upon repeated exposure to a drug which can increase in the reinforcing value of the drug leading to more time, effort and risks taken to obtain drugs [48].

As an example of how these processes may apply to food reinforcers, Avena and colleagues [49, 50] argue that animals can become dependent on sugar. In their model of sugar addiction, rats are exposed to daily intermittent access to sugar resulting in tolerance, sensitization and withdrawal symptoms [51]. Rats progressively increase sugar consumption over days with most of their daily sugar being consumed in the first hour of access, similar to binge behavior and implying the formation of tolerance and sensitization [49, 51]. There is also a concurrent increase in lever pressing in these animals and withdrawal symptoms such as teeth chattering and anxiety are apparent when sugar is removed [49]. Treatment with naloxone, an opioid receptor antagonist used to precipitate opioid withdrawal, produced signs of withdrawal in the sugar-dependant animals [52]. It appears that repeated experience with a restricted sugar source is necessary for these behaviors to manifest as control groups exposed to restricted chow, free sugar access and one session of sugar access did not exhibit these behaviors during the test session. These results correlate with differences in dopamine kinetics in the nucleus accumbens, such that the intermittent sugar access prolongs the initial release of dopamine in this area, suggesting a sustained motivation to obtain sugar that is not present in the control groups [51].

Sensitization can also occur to food in humans, as demonstrated in a series of studies by Temple and colleagues [53–55]. After a baseline measure of relative reinforcing value, non-obese participants were asked to either consume prepackaged portions of snack foods daily, or to restrict their intake of these foods. Restriction increased, or sensitized, reinforcing value over the two week period, while daily intake decreased reinforcing value in this population [53]. In another study, relative reinforcing value differences between obese and non-obese were examined after two weeks of daily snack food consumption. Non-obese participants again decreased responding for snack foods after daily intake, while obese participants sensitized as seen by their increased snack food reinforcement [53]. In a third study, this effect was examined with both high-energy dense and low-energy dense snack food consumption. It was found that obese participants sensitized to high-energy dense, but not low-energy dense foods, while lean participants decreased responding to both high and low-energy dense snack foods [55].

Impulsivity also influences intake of both food and drugs. The availability of both drugs [18] and food [56] reinforcers influences their consumption. When reinforcers are easily available impulse control mechanisms must be constantly engaged to control intake. Activity in the prefrontal cortex [18] is associated with to impulse control of both drug [57] and food intake [58] in former addicts and current dieters who are actively restricting consumption of a target reinforcer. Dopamine receptor availability and release also play a role in these processes. The dopamine D2 receptor is hypothesized to be a key component in vulnerability to problems of excess motivation. A decreased availability of D2 receptors in the striatum has been associated with increased risk of both drug addiction and obesity [43]. Availability of dopamine D2 receptors in this area influences activity in frontal cortical brain regions [59] and predicts resistance to familial alcoholism [60]. In a study of morbidly obese subjects, the reduced availability of dopamine D2 receptor in the prefrontal cortex was also associated with reductions in metabolic rate in these regions [43]. Activation in these areas corresponds to the ability to control impulses and decreased dopamine receptor density is associated with reduced impulse control. These findings suggest that dopamine is involved in both reinforcement and impulse control albeit in different brain regions.

EVIDENCE OF REINFORCEMENT PATHOLOGY IN OBESITY

Reinforcement pathology theory predicts that subjects who have an increased reinforcing value of food and are more impulsive should be the most vulnerable to excess energy intake [61]. Both a preference for immediate rewards and an inability to inhibit responding interact with measures of food reinforcement and reward to determine food choices. This suggests that measures of food reinforcement and impulsivity will interact to predict energy intake and BMI. A combination of low food reinforcement and high impulse control should predict low energy intake and BMI, while high food reinforcement and low impulse control should predict high energy intake and BMI.

Research in non-obese and obese samples supports the hypotheses put forward by reinforcement pathology. Rollins [62] measured food reinforcement, delay discounting and energy intake in a sample of non-obese women. While it was found that impulsivity or preference for the immediate reward, did not independently predict energy intake, there was a significant interaction between delay discounting of monetary rewards and relative reinforcing value of food. The combination of high impulsivity and increased food reinforcement predicted the greatest energy intake. Appelhans [63] completed a similar study in obese and overweight women using delayed discounting of monetary rewards, the power of food scale and an eating in the absence of hunger procedure. He also found that increased reward sensitivity only predicted increased energy intake when impulsivity was high, while reward sensitivity did not significantly influence energy intake when impulsivity was low.

Nederkoorn and colleagues [31] studied implicit preference for snack foods, and response inhibition as predictors of BMI change over one year [31]. Neither predictor independently predicted weight change over one year. However, there was a significant interaction such that participants with high implicit snack food preference were significantly influenced by response inhibition levels, with low inhibitory response inhibition predicting greater weight gain and high inhibition predicting weight loss over one year [31].

These studies suggest that both impulsivity and food reinforcement are important regulators of energy intake. It may be a combination of failure to inhibit responding for immediate rewards that interacts with increased food reinforcement to influence energy intake and BMI. Consistent discounting of delayed rewards is a correlate of impulsivity, while a preference reversal for the immediate reward as they choice becomes more proximal in time is a measure of response inhibition or ‘loss of control’. Bickel argues that delay discounting procedures can capture both these constructs [64]. It may be important to measure both constructs to identify at-risk populations and simultaneous manipulation may then significantly influence weight change, above and beyond modifying each construct independently.

CURRENT INTERVENTIONS FOR OBESITY

Reinforcement pathology suggests that food reinforcement and impulse control are fundamental elements of eating behaviors and that their interaction is important in predicting weight change. Food reinforcement is a strong motivator of eating behavior [1, 65], and it is not surprising that those high in food reinforcement consume more food in ad lib taste tests [4, 8, 9] than leaner peers. It is also not surprising that eating behavior is hard to change since most obesity treatment programs involve dieting, potentially resulting in restriction or deprivation of food, which would tend to increase food reinforcement [10, 11, 54, 55]. Since eating is fundamentally a choice between reinforcers, general principles of behavioral economics [66] may be useful to modify eating habits. Research has shown that people will choose healthier foods or non-eating activities when access to food is constrained [13, 16, 17, 67, 68], which is evidence of substitution. Substitutes are commodities, such as coffee and tea, which can replace on another.

One approach to reducing energy intake is to identify behaviors that are substitutes for food reinforcement. These substitutes can be healthier foods that substitute for less healthy foods, or behaviors that substitute for eating. There is very limited research on developing behavioral substitutes for food reinforcement [69]. An important implication from basic research on substitutes is that some constraint on access to food reinforcers may be needed to shift choice, and the degree of constraint may be related to the reinforcing value of the substitutes.

Eating may also occur as a complement to other behaviors. Complements are behaviors that change in the same direction as other behaviors [66]. Food reinforcement is increased through exposure to complementary reinforcers to eating and constraints on these reinforcers may decrease food reinforcement and energy intake. Television watching is a complement to eating [70, 71] and it has been shown that decreasing television access concurrently decreases BMI and energy intake [72]. In addition, placing constraints on behaviors that may increase food reinforcement, such as television or other sedentary activities, may influence energy intake and physical activity. One may frequently eat while watching television and reducing this activity would decrease the energy normally consumed. It may also encourage the substitution physical activities for sedentary ones [73]. Identifying complementary behaviors to eating provides treatment programs with target behaviors that can be modified to reduce energy intake and influence weight.

Stimulus control is a common component of behavioral weight loss interventions. Obese and overweight persons are asked to modify their home environment to encourage physical activity and healthy eating, while discouraging unhealthy eating through the manipulation of environmental cues [74]. This may involve restricting or reducing access to unhealthy foods, such as cookies, by placing them in hard to reach places or out of sight. Conversely, encouraging healthy eating can be initiated by removing barriers to healthy eating such as placing fruit in commonly used areas. Sedentary behaviors can be reduced through increasing the work required to engage in these activities, such as moving televisions and computers to back rooms and keeping them unplugged.

There is a substantial literature suggesting that taste preferences are learned, and it may be possible to use operant or classical conditioning methods to modify preferences [75–78]. It is surprising that there is so little research that has attempted to increase preference for healthy food, or reduce preference for less healthy foods. Consider how easy it would be to lose weight if obese persons learned to really like fruits and vegetables and dislike high fat foods. Current research, however, presents conflicting results, particularly in children, in which experimenters have attempted to increase preference for vegetables [76, 79]. Research has examined the influence of deprivation state [77], energy density and portion size [78] and the addition of sugar on food preferences [76]. In adults, the addition of caffeine to novel beverages increases preference for these beverages [80, 81], but only when participants were also caffeine deprived [82, 83]. These studies suggest ways to potentially increase and decrease the reinforcing value of target foods under specific conditioning procedures.

Conditioned cues may increase craving for palatable foods and signal meal initiation and energy intake [44, 84, 85]. Several studies have found that exposure to food smells, including pizza and cookies, increased energy intake of the respective food and not the non-cued food [44, 86, 87]. Cue exposure is particularly relevant for reinforcement pathology since exposure can activate the motivational system to initiate and increase eating, and energy intake is moderated by impulse control mechanisms. In a study by Tetley et al. [88] cue-reactivity and energy intake was measured in deprived and non-deprived subjects. Impulsive participants were more sensitive to food cues regardless of motivational state, while participants with high reward sensitivity experience greater hunger when exposed to cues in a non-deprived state only. Conditioned food cues may influence reinforcement pathology by eliciting disinhibited eating patterns. Continually engaging impulse control may limit resources available in working memory, reducing the ability to make healthy choices [89]. Decreasing exposure to these cues may prevent temptation to eat and allow executive processes to engage in healthy choices.

Impulsivity is related to working memory [90, 91], and can be influenced by the availability of cognitive [89]. Increasing the resources available in working memory enhances the ability to compare alternative rewards both between different reinforcers and across time [24]. Conversely, insufficient working memory resources lead to a narrowed time perspective where the value of immediate rewards becomes magnified. Working memory has a limited capacity, such that performing two concurrent tasks decreases available working memory and increases preference for immediate rewards [89]. Therefore, increases in available resources in working memory should decrease impulsive decisions. This myopic pattern of decision-making is often associated with increased risk of adverse outcomes in the future.

Working memory training implemented over several weeks has been shown to improve performances on both practiced and non-practiced tasks indicating an overall improvement in working memory [92, 93] and exhibited changes in brain activation through fMRI measures [92]. In addition, these effects were more pronounced for subjects who were initially very impulsive [93]. A computer-based money management system was similarly used in cocaine addicts [94] that required participants to set up long-term budgets. The decreased rates of discounting for monetary rewards extended to drug use, resulting in increased cocaine abstinence. These studies suggest that training individuals to consider long-term goals and planning may reduce impulsivity. For a weight-loss treatment, it is important to learn to set goals and plan both meals and time for physical activity. Working memory training may be useful in reducing impulsive decision-making in overweight subjects. A money management system may be modified to track weekly calories consumed or time engaged in physical activity to help subjects increase their capacity to think about long-term goals related to weight management.

Another approach to reducing impulsivity involves enhancing episodic future thinking, in which a participant is to imagine themselves at the delayed time point to pre-experience the future event [95]. Identifying long-term goals and imagining achievement may help to increase goal saliency. Research has shown attaching future episodic events identified by participants to delay periods in a discounting task reduced discounting compared to a control condition [96]. Imagining long-term goals may help individuals conceptualize the greater reward attached to waiting and thus decrease impulsive, immediate gratification choices.

These interventions to modify food reinforcement and impulsivity may be implemented alone or in combination with traditional child or adult weight control programs [97–100]. It may be that modifying these constructs that are so central to the regulation of eating that by themselves are sufficient to promote and maintain weight change, but it may also be necessary to implement these interventions in the context of traditional weight control programs.

CONCLUSIONS

Research suggests that the toxic combination of low inhibitory control and high food reinforcement predicts greater energy intake [62, 63] and weight gain [31]. Both the motivational and executive systems are driven through dopamine pathways and common brain centers, including the prefrontal cortex and amygdala. Shared brain structures suggest that imbalances in dopamine may influence both systems. Individual differences in dopaminergic activity could be influenced by genetic predisposition, feedback mechanisms that regulate dopamine and receptor availability, and learning or experience. Determining the relationships between food reinforcement and impulsivity may be an important direction for future research.

Reinforcement pathology provides a new theoretical approach to obesity that may lead to new approaches to treatment. In the current obesigenic environment access to unhealthy food is prevalent and effortless [101]. This may influence the interaction between impulsivity and food reinforcement in deleterious ways. The ease of food access and ready-to-eat food items may exaggerate the impact of impulsivity on eating. Meal decisions are frequently between palatable, easily-prepared food now and a healthy meal that will be available at a delay after it is prepared. Both food reinforcement and impulsivity would predict the choice of immediately available energy dense foods over healthy options, amplifying effects of reinforcement pathology. This leads to the idea that targeting impulsivity and food reinforcement should be an important aspect of a behavioral treatment for obesity to avoid binges or relapses to unhealthy behaviors. Research is needed to identify the optimal way to implement changes in food reinforcement and/or impulsivity. Should interventions for both be implemented simultaneously, or individually? Which intervention should come first? Are there simultaneous effects on both processes from interventions designed to change one process? What are the long-term benefits to modifying food reinforcement or impulsivity? Will interventions aimed at these two processes improve both weight loss and maintenance? Addressing these issues, and other issues of implementation, are critical for translating basic science on food reinforcement and impulsivity into evidence-based treatments for obesity.

Future Research Questions.

• Interventions that target impulsivity and working memory, such as working memory training and money management, have been successful in a population of drug users to improve working memory, impulsivity and reduce substance use. Future research should determine the effectiveness of these interventions on reducing energy intake and body weight.

• The idea of reinforcement pathology suggests that targeting both reinforcing value of food and impulsivity in an obese population should lead to greater weight loss and weight maintenance. However, further research must be done to determine the additive effects of reductions in reinforcing value and impulsivity and whether interventions should be implemented simultaneously or individually.

Key Learning Objectives.

• Food reinforcement is a significant predictor of energy intake, body mass index (BMI) and weight change over time and impulsivity moderates these relationships. Individuals with high reinforcing value and high impulsivity consume the most energy and have the highest BMI, a condition we have termed reinforcement pathology.

• Neurobiological responses to food and drugs of abuse have many similarities including the ability to stimulate limbic system and dopaminergic pathways, to condition cues, and the potential to activate and sensitize brain centers controlling impulsive drug and food intake. Dopamine may also be involved in both the reinforcing properties of food and drugs, and controlling impulses in the striatum. The parallels between these reinforcers suggest that similar interventions may be appropriate for both addiction and obesity.

• Energy balance is determined by a variety of factors including deprivation state, palatability and food variety. However, other factors such as higher cognitive functions, such as impulsivity, food reinforcement, working memory and conditioning have been shown to influence energy intake and food choice. It may be important to target these behaviors in a behavioral intervention for obesity to improve long-term weight loss and maintenance.