Food reinforcement, delay discounting and obesity

Abstract

Choice is a central construct in behavioral economics, with choice research divided into choice of concurrent alternative reinforcers, which is conceptualized as relative reinforcing value, or choice of small immediate versus larger delayed rewards, usually of the same commodity, which is conceptualized as delay of gratification and delay discounting. Relative reinforcing value, delay of gratification and delay discounting paradigms can be used to study obesity, which involves strong motivation to obtain and consume food reinforcers. Strong food reinforcement and difficulties in delay of gratification are risk factors for child weight gain, and both are related to individual differences in overweight/obesity. Delay discounting interacts with food reinforcement to predict energy intake. We provide a selective review of research on each of these areas, and argue that the division of choice into reinforcing value versus delay discounting is based on an arbitrary definition based on the temporality of choices. We present a model that integrates reinforcing value and delay discounting approaches. Implications of this theoretical approach to better understand excess energy intake and obesity are discussed.

1. Food reinforcement, delay of gratification and energy intake

Choice is a fundamental part of life, and the choices people make determine in part how they live their life. Most people make hundreds of choices a day, what to wear, when to get up in the morning, what to eat for breakfast, lunch or dinner, how to allocate free time before or after work or on the weekends, and so on. Some of the choices involve concurrent choices, as to eat a muffin or cereal for breakfast, while other choices involve time, such as whether to buy a TV now or save the money, and buy a more expensive TV later. Since choice and decision making is so central to life, there have been many theoretical approaches to developing an understanding of the processes that influence choice. One comprehensive theoretical approach is behavioral choice theory or behavioral economics. This paradigm combines research from economics, learning theory, behavioral and cognitive psychology, and neuroscience to understand how people decide to allocate their time, efforts or resources among available alternatives. Behavioral choice theory has primarily focused on problems of drug abuse [1,2] and gambling [3,4] but has been extended to other health behaviors, such as eating, physical activity and obesity [5,6]. Obesity is increasing in children and adults, and is associated with a substantial burden of morbidity and mortality [7–10]. Obesity results from positive energy balance in which energy intake exceeds energy expenditure, and the choices people make are important to understand energy balance. Behavioral economics provides a theoretical approach to understand factors influencing everyday choices which lead to positive energy balance and obesity [5,6].

A basic principle of behavioral economics is that individuals allocate their limited resources (e.g. money and time) to gain access to goods (commodities or reinforcers) within a system of variable constraint [1,11]. These constraints can be conceptualized as monetary value, cost, effort (e.g., resistance, distance) or behavioral response. Constraints can be manipulated in varying the price or the effort or the amount of work required to gain access to the commodity.

Current theoretical approaches differentiate choice in terms of the timing of the choices (concurrent vs. delayed) and in terms of the alternatives involved in the choice (different quantities of a same commodity vs. different commodities) [2,6,11]. Concurrent choices are often conceptualized in terms of relative reinforcing value, whereas delayed choices can be conceptualized in terms of delay of gratification or delay discounting. Reinforcing value of food refers to the motivation to obtain food, or how hard someone will work or how much time they will allocate to obtain food. Delay of gratification or delay discounting refers to the preference for smaller immediate versus larger delayed rewards. We briefly review the conceptual bases for the reinforcing value of food or delay discounting of food and illustrate with new empirical research that reinforcing value and delay discounting interact to predict energy intake. In the final section of the paper we provide a conceptual framework to integrate relative reinforcing value and delay discounting perspectives for eating, and provide examples of how this integrative approach may lead to new insights into methods for changing eating behavior, modifying energy intake, and treating obesity.

2. Food reinforcement

Reinforcement is a fundamental determinant of choice. People choose to engage in behaviors that are reinforcing, and given a choice among several behaviors that are equally accessible and with equal constraints, most people would choose the most reinforcing alternative. Reinforcers are important for the development of our behavioral repertoires, and to create the texture of life. They shape and strengthen behaviors that they follow, and facilitate learning of simple and complex behaviors. Reinforcers are fundamental to learning and motivation as they shape individual differences. Grades in school, praise from a loved one, winning a medal in an athletic event, providing food to a crying baby or learning music from the feedback the instrument provides are part of the fabric of life. It is hard to imagine not engaging in behaviors in which we seek the opportunity to eat preferred foods, read preferred books, participate in preferred sports or watch a preferred television show. Reinforcers shape the development of many aspects of our behavioral repertoires, including some that we refer as our “lifestyle”. The same principles that apply to how reinforcers are important for normal development are also involved in the development and maintenance of unhealthy behaviors. There is a vast literature on how reinforcers and schedules of reinforcement shape, maintain and increase adaptive as well as maladaptive behavior such as drug self-administration, gambling, overconsumption of less healthy foods and time allocation biased towards a sedentary lifestyle.

Primary reinforcers do not require prior learning and are biologically pre-established to increase behavior, such as food when you are hungry. Food is a powerful reinforcer which can be used to satisfy physiological needs, but which can also serve other functions. Reinforcers can also be secondary or conditioned, gaining their motivational strength by learning, such as money or socialization or grades. In either case the motivation to consume these reinforcers may gain strength over time and people may allocate more and more resources to obtain and consume these reinforcers. While the reinforcers may vary, most people are driven at different times to experience or consume reinforcers. Motivation to eat increases as food deprivation increases; long and dark winter increases the motivation to get outside. If you are really active, you may be motivated to be active after long periods of inactivity, while if you like to read books, you may be motivated to read your favorite book if you have been prevented from reading for some time (12).

Reinforcing efficacy of food describes the relationship between the work to gain access to food and consumption, using the same methods that are used to understand the reinforcing value of drugs. The usual approach is to provide access to a specific food under a progressive ratio schedule of reinforcement. Ratio schedules require a specific amount of work to gain access to the reinforcer, which is specified by the schedule, and in a progressive ratio schedule the amount of work progressively increases. The maximal amount of work performed to gain access to the food, or the break point, provides an index of reinforcing efficacy [11,13]. This can be done using both human and animal models. There have been decades of research in animals using these methods, and this basic paradigm has been important to understanding how reinforcing food is [14–18], and characteristics of food that motivate consumption [19].

Reinforcers are usually not available in isolation, but are available in a concurrent choice situation in which the person has to choose which behavior to engage in. Concurrent choices between foods or between food and alternative commodities are conceptualized in terms of relative reinforcer efficacy or value. In a choice situation food selection and consumption depends on both the costs and the value of the alternative. Thus, the choice of behavior A reflects the value and cost of obtaining A, but also the value and cost of behavior B, or the relative value of A compared to B. The measurement of relative reinforcing value assesses the amount of work that someone is willing to engage in when given a choice of two alternatives, which can be two foods, such as healthy or less healthy food, or a food and an alternative activity, such as sedentary behaviors [20]. For many people consumption of prepared snack foods is more reinforcing than consumption of fruits and vegetables. However, as the behavioral cost to obtain snacks foods increases, participants may shift their choice to healthier options [20].

Choice often depends on the food alternatives available. For example, consider the choice of a hamburger versus liver. For many people the hamburger would be more reinforcing, but consider if the liver option was replaced with a sirloin steak. Many people who chose hamburger when liver was the alternative would now switch to sirloin steak. An important extension of basic research on food reinforcement is to apply behavioral economic principles to responding for food under different schedules of reinforcement and derive demand curves for responding. The slope of the line relating cost to consumption represents the demand function [11].

One interesting possibility in choice studies is the ability to compare the relative reinforcing value of food versus other commodities. This is illustrated in a recent study comparing the relative reinforcing value of food versus different sedentary behaviors in overweight and lean youth. The expected greater reinforcing value of food for the overweight/obese was observed, along with the observation that the reinforcing value of the alternatives differed for the lean and obese, as the lean found the alternatives more reinforcing and therefore shifted responding for nonfood alternatives [21]. There have been several studies on choice of food versus drugs of abuse such as heroin or cocaine under varying access to the alternatives in non-human primates and rat models [17,22]. These studies have shown that as access is reduced, either by reducing the availability of the choices or by increasing the schedules of reinforcement, responding for food was maintained, while responding for drugs was reduced. When schedules were increased, or constraints were placed on access to food and drugs, demand for drugs was more elastic than demand for food, as responding changed more in response to reduced access for drugs than for food [16,17,22].

3. Relationships among behaviors: substitutes, complements and own and cross-price elasticity

Substitutes and complements describe ways in which reinforcers interact. In general, behavioral economic theory suggests there are three ways in which commodities can be related in a choice paradigm, commodities can be substitutes for each other, commodities can be complements, or commodities can be unrelated [23]. A common example of a substitute is tea for coffee. You may drink coffee for the caffeine, but if coffee is not available, or the price of coffee has increased substantially, tea may be a good substitute as a hot drink that provides caffeine. Complements are foods or behaviors that go together. Mustard is a complement to hot dogs, so that if the price of hot dogs increases, purchases and consumption of mustard will decrease even if the price of mustard has not changed. Many foods are unrelated. Using foods from the previous examples, reducing access to coffee should have no relationship to consumption of mustard.

Each of these relationships are tested by varying the behavioral cost or constraints on access to one of the behaviors, and keeping the behavioral cost or constraints on access to the other behaviors constant. For example, if the cost of obtaining the more desirable alternative A increases, but the cost to get the alternative B stays the same, many people will shift choice from A to B. The change in price that is needed to shift choice from A to B represents how substitutable the alternatives are. If there is no preference between alternative A and B, such as between two brands of the same coffee, then they are considered substitutes or near perfect substitutes. Likewise, if someone has no preference between mayonnaise or mustard, then they are considered to be substitutes. If it requires a price increase of 50% for responding for alternative A to shift to alternative B1, versus a price increase of 100% to shift to commodity B2, then commodity B1 is more substitutable for A then B2. For example, tea may be a better substitute for coffee than hot water.

One of the challenges of pediatric obesity treatment is to identify alternatives to compete with highly reinforcing obesogenic behavior and reduce energy balance. We have completed studies on food and non-food substitutes for high energy density, low nutrient density snack foods. In these studies we observed that subjects preferred to work for the high energy density snack foods when the behavioral costs were equal, but the choice shifted to healthier foods or sedentary activities when the cost of the less healthy food increased [20].

A promising behavioral substitute for food relates to social support or increased social interaction. This possibility has implications for pediatric obesity as overweight and obese children often have problematic relationships with the larger peer group [24]. Food is a powerful reinforcer for overweight/obese youth [21] and overweight youth may be more likely to eat then engage in alternative behaviors because they find food more reinforcing, but also because they may have fewer pleasurable alternatives to eating [25,26]. Conceivably, peer relationships and friendships may increase the value of alternatives and promote substitution of alternatives to eating or even provide an alternative to obesogenic activities. However, inadequate social skills, a history of teasing and unhappy experiences in social situations and weight criticism may limit overweight youth’s access to social activities. Overweight youth may substitute powerful and easily accessible reinforcers such as food for social activities and increasing the child’s social network could potentially result in a reduction in eating. Salvy and colleagues provided children with the option to work for food or social interactions with friends or peers. When the cost of food was increased relative to social interactions with unfamiliar peers, the social interactions substituted for food, and in a symmetric fashion, when the cost of social interactions with an unfamiliar peer were increased, food substituted for social interactions. However, when the choice was between social interactions with a friend and food, food did not substitute for social interactions [27]. Another example of behavioral substitution that may help understand the substitutability of food and social interaction is the relationship between smoking and social support [28]. Fisher [29] has argued that situations such as stressful events, personal losses, and performance challenges may instigate choices for either social support of smoking, and that both social support and smoking appear to reduce anxiety, elevate mood and enhance performance. It is possible that the constraints imposed on social interactions for overweight youth (i.e., ostracism, peer victimization and marginalization) increase their demand and time allocation to activities that are really reinforcing and easily performed alone. In other words, removing social activities from children’s repertoire might redistribute time allocation towards eating. Alternatively, building strong friendships for children who encounter peer difficulties could lead to substituting social interactions for eating.

Complements represent the shift in choice of B in the same direction as alternative A. For example, if the price of A is reduced (and the consumption of A increases), and the consumption of B increases, that is evidence for a complementary relationship between the two reinforcers. There are many important complementary relationships that influence behavior. Eating is a complementary behavior to television watching for many people [30], smoking is a complement to peer smoking and other substance use [31], and some drug use complements other drug use [2,32]. Food can also be a complement to other behaviors or activities. We have shown in both field and laboratory studies that energy intake changes in a complementary manner to changes in television watching. Eating is often paired with television watching, and reducing television watching can spontaneously reduce energy intake [30,33]. These relationships need not be symmetric, as reducing eating may have no effect on television watching. Increased eating during television watching may be due to the distractor effects of attending to an alternative activity when eating [34].

Elasticity is used to describe the relationship between changes in constraints such as price or responses required to obtain a product and demand for a product. The degree of elasticity is determined by how sensitive demand or consumption is to changes in price or constraints on behavior. A behavior that is elastic to changes in constraints will show a reduction as constraints increase, but an inelastic behavior will not change in the face of increasing constraints. Own-price elasticity describes the relationship between changes in prices for commodity A and demand for commodity A. In behavioral studies reinforcement schedules provide an analog to price [35]. An elasticity coefficient of −1.0 would mean that a 10% increase in price would lead to a 10% reduction in demand. Substitutes can be described in terms of elasticity, and a negative cross-price elasticity (an inverse relationship between price of A and purchases of B) represents the degree of substitutability. Higher cross-price elasticity represents greater substitution, and probably less initial preference differences between the two commodities. For example, we have shown that in laboratory studies that an increase in price of less healthy foods can lead to cross-price elasticity, or an increase in purchase of healthier foods that have not changed in price [36]. Complements can also be considered in terms of elasticity, and the magnitude of the positive relationship between the price of A and purchases of B represent the strength of the complementary relationships.

Information about relationships between behaviors may be relevant in achieving multiple behavioral changes. In obesity treatment the general goals are to reduce energy intake and increase energy expenditure, which may require multiple behavior changes. In addition, the goals for reducing energy intake may involve reducing intake of some high-energy dense foods, while increasing intake of low energy density foods. Similarly, the goals for increasing energy expenditure may involve increasing physical activity while also targeting a reduction in sedentary behavior that may compete with being physically active.

If multiple behavior change is important for treatment success, and multiple targeted behaviors have complementary relationships with one another, one approach would be to identify behaviors that naturally change together to enhance multiple behavior change. There are many possible examples of this, which are not fully exploited during obesity treatments. For example, reducing television watching may have the added benefit of also reducing energy intake [33], and increasing high nutrient dense but low energy dense foods may increase satiation, and naturally reduce intake of high energy dense foods, thus reducing energy intake [37–40]. Rather than developing separate interventions to target each behavior separately, it would be more efficient to target a behavior that could be a substitute for a less healthy behavior, while at the same time also showing complementary changes in another healthy behavior. For example, it would be very cost efficient in comparison to developing programs for three independent behaviors if reducing sedentary behavior could increase the likelihood for physical activity, while at the same time reducing energy intake since eating is often paired with selected sedentary behaviors.

Elasticity coefficients may provide an idea of which behaviors should be targeted, and in what order. Elasticity coefficients provide an index of value of that commodity or behavior, which may relate to how easy or hard it is to make changes in that behavior [35]. Given two behaviors that vary in their own-price or demand elasticities, the experimenter may want to target the behavior with the larger elasticity coefficient, as that behavior may be more responsive to changes in response requirements. However, in many cases the challenge is to increase behaviors that are resistant to changes in the environmental contingencies supporting these behaviors. One limitation to using elasticity coefficients to develop hierarchies of behavior change is that an elasticity coefficient applies only to a specific behavior, and detailed analyses of a number of behaviors is needed to develop the hierarchy of behavior change. This area of multiple behavior change is new, but may prove to be very important as the simplest and most cost effective approaches to treatment are identified.

There is a growing literature on factors that influence food reinforcement in humans. One very important individual difference variable is obesity. One way to think about the excess energy intake in obese humans is that they eat more because they find food more reinforcing, or they are more motivated to eat. Both animal and human data have shown that elevated food reinforcement is a risk factor for weight gain [41,42]. A series of studies have also shown that obese children and adults find food to be more reinforcing than non-obese peers [21,43]. These data would argue that identifying methods for modifying the reinforcing value of food in obese persons would be very important for successful weight management.

4. Delay of gratification and delay discounting

To function effectively in many situations, individuals must voluntarily postpone immediate gratification and persist in goal directed behavior to achieve later outcomes [44]. The ability to plan and defer outcomes refers to delay of gratification and individuals vary greatly on the continuum of the ability to delay gratification. There are many examples of individual differences in delay of gratification of food, with perhaps the most widely cited being Mischel’s study of four year old’s having the choice of one marshmallow now or two later. This simple task tapped an important individual difference characteristic that predicted behavior more than 10 years later [44]. The basic delay of gratification paradigm has been used to study obese/non-obese differences, with research showing the obese youth were more likely to choose smaller, immediate rewards, and that difficulties in delaying gratification were greater for food than alternatives [45–47]. In addition, new research has shown that the inability to delay gratification in young children predicts weight gain from ages 3–11 [48,49].

The idea that people may prefer smaller immediate rewards over larger but delayed ones is also the construct that is central to delay discounting, where people are asked a series of questions regarding whether they would prefer a smaller, but more immediate reward, versus a larger, but more delayed reward [50,51]. There is a wealth of research showing that people with drug addictions of various types discount the future more than non-drug abusers [52].

Two interesting variants of delay discounting are discounting the past, and probabilistic discounting. The core construct in delay discounting is that people make choices which discount maximization for immediate gratification, rather than better choices that would improve health in the future. People also may differ in their ability to take advantage of their past behaviors, so that they tend to repeat the previous behaviors. For example, dependent smokers have been shown to discount the past more than comparable controls [53]. Obviously, people who do not learn from their past, and are not able to delay gratification for immediate pleasure over later health, are more likely to engage in risky behaviors associated with obesity and poor health.

Delay discounting procedures vary time intervals and reward magnitude to influence choice, but in some situations the issue is not making a choice for a small immediate reward now versus a larger delayed reward, but a small reward that is definite, versus a larger reward that is probable, but uncertain. In other words, do you want to make sure of getting something small now, or take the risk of getting something larger later. This is relevant for many behavior changes since behavior changes do not always appear to lead to improved health, so it is easy to see how some people may make decisions for behaviors associated with a sure, though smaller reward, rather than behaviors associated with greater, though perhaps probabilistic benefits. In fact, gamblers, who are regularly engaged in making choices based on probabilistic outcomes, discount probabilistic rewards less than delayed rewards in comparison to non-gamblers [3].

There are many implications for obesity that can be derived from the relationship between behavioral choice and challenges in delaying gratification. Obese individuals may not be as able to delay gratification for food than leaner individuals which manifests itself through increased food cravings or decreased ability to inhibit food cravings. Today’s food environment contains a larger variety of ‘convenience’ foods than ever. An obese individual’s inability to delay rewards may cause them to choose a more immediate, less nutrient dense ‘convenience’ food rather than taking more time to choose a nutritious meal. Obese individuals may also steeply discount the long term rewards of a nutritious diet and exercise in favor of more pleasurable high fat and high sugar foods and sedentary activities. Investigators have begun to assess delay discounting and obesity, with obese women showing more discounting of future rewards than leaner women [54]. Or course there are costs to the healthy behaviors and hedonic value to the unhealthy behaviors which may need to be taken into account in addition to the delay in consequences of consuming a nutritious meal versus a less healthy, more immediately available snack.

Applehans developed a compelling argument for eating being simultaneously influenced by both food reinforcement and delay of gratification or delay discounting for food [55]. Consider the situation when two people who find food to be very reinforcing are faced with the opportunity to eat, but they differ in how able they are to delay gratification or not discount the future in relationship to food. The person who cannot delay gratification is likely to consume more food, since the person who can delay gratification may possess sufficient self-regulatory resources to inhibit or regulate food consumption. There are people who find food very reinforcing, who like to cook and who are choosy about the foods they consume, but who also can delay gratification, and may not feel the need to eat everything that is in front of them. These “foodys” combine high food reinforcement with the ability to delay immediate gratification to control consumption. Moreover, the combination of high food reinforcement and inability to delay gratification in a delayed discounting task may be the behavioral phenotype that may be most associated with high energy intake.

To test the hypothesis that the interaction of high food reinforcement and the inability to delay gratification predicts that greatest energy intake, we recently completed a study with 24 lean women who were studied in three sessions, an ad libitum eating session, a session to measure food reinforcement and a third session to measure delay discounting for food [56]. Results showed that food reinforcement, but not delay discounting, was independently associated with energy intake, suggesting that food reinforcement may be a more powerful independent predictor of energy intake than delay discounting. However, knowledge of delay discounting increased the prediction of food reinforcement beyond food reinforcement alone, supporting the hypothesis that the combination of high food reinforcement and inability to delay gratification will lead to greater energy intake.

5. Integrating reinforcing value, delay discounting and economic principles as models of behavioral choice

These two different theoretical approaches to choice have one important commonality; both paradigms involve selection between alternatives. They differ in terms of the temporal characteristics of the choices as well as in terms of the characteristics and nature of the alternatives involved. Relative reinforcing value involves concurrent choices, in which the two alternative commodities or reinforcers are available, and behavioral costs are involved in gaining access to the reinforcers. Delay discounting usually involves a choice of different magnitudes of the same reinforcer at different points in time, and the subject need not make any responses to gain access to the reinforcers, but they are available after the delay

One factor that is common to both relative reinforcing value and delay discounting is that both choice paradigms involve making a choice of one alternative versus the other. Choice in both paradigms may in part be a function of the opportunity cost, or cost of something in terms of the cost forgone by not choosing the other alternative. For example, in a relative reinforcing value paradigm someone may choose to be active and go for a run first thing in the morning, rather than have a cinnamon bun, cup of coffee and read the morning paper. If someone regularly makes this choice they will do wonders for their fitness, and perhaps their body weight and stress level, but they may be compromised on discussions about local or world events around the water cooler at work. Similarly, someone may choose to spend some of their salary on a new plasma TV set, rather than save the money and buy a bigger set later with greater resolution and probably at a cheaper price. They get the immediate pleasure of improving their television watching experience, but miss the opportunity to have an even better experience later.

5.1. Unit price as a common metric for relative reinforcing value and delay discounting paradigms

The factors that influence choice of one reinforcer over another reinforcer in a relative reinforcing value paradigm may be influenced by the cost to obtain the reinforcer. In many cases you can shift choice between alternatives by varying the constraint over access (price, effort, resistance). The shift in choice from a highly reinforcing alternative to one that is less reinforcing is possible by increasing the cost of the more reinforcing alternative, or by reducing the cost of the less reinforcing alternative. Delay discounting can also be considered to be related to the cost of the delay. Time is money represents the idea that waiting comes at a cost, and there may be a discrete cost to delaying the consumption or availability of a reinforcer, even if the magnitude is larger.

Perhaps time and cost can be integrated by using the concept of unit price to understand both relative reinforcing value and delay discounting. The fundamental idea of choice theory is that individuals allocate their limited resources to gain access to goods (commodities or reinforcers) within a system of variable constraint. Unit price relates the magnitude of the reinforcer to the cost of the reinforcer, which subsumes cost and benefit factors into a single metric and describes the relationship between constraints and reinforcers. Thus if it takes 60 responses to get a candy bar, or 120 response to get 2 candy bars, they both have the same unit price. Most research on unit price compares choice of the same commodity at different magnitudes and/or response requirements or costs. This research generally shows that unit price is an important variable that influences choice, not the specific cost or reinforcer magnitude [57]. It is also possible to assess the effects of alternative reinforcers on choice when unit price of one reinforcer is varied [58,59].

Perhaps the more interesting situation is the choice of two different reinforcers when both can be chosen at various response costs. For example, consider the choice of spending time watching television or exercising. The conceptual challenge is to derive a common metric for magnitude of the reinforcer. If it is just considered to be time, then the comparison of unit price is simply the magnitude (10 min)/cost (100 responses). If the choice is between 10min of watching television or 10 min of exercising, many people would choose the television, and it would be interesting to assess how much (or little) exercise would be equivalent to 10 min of watching television by varying the cost of exercising until the person was indifferent about the choice, or they chose each 50% of the time. This would provide the very useful metric of how much exercise is worth for a given amount of television watching. The same logic could apply to food, with serving size as the magnitude. If given the choice of a piece of apple pie or an apple, how many servings of apples are needed before the unit price of apple pie and apples are the same, or how much would you have to reduce the cost of getting an apple relative to getting a piece of apple pie before they are indifferent?

The complexity may increase when the choice is between a drug or different commodity and food. It would not make sense to compare the same weight of the two commodities, since a smaller amount of a drug versus food may produce satisfaction. Common serving sizes of food or doses of the drug could be used, and modified to compare how the magnitudes of each are compared when the behavioral cost is the same, or likewise, how much of each are needed to equate the choice when the behavioral costs are different.

The concept of unit price may also be applicable for delay discounting. For example, the delay to obtain the delayed reinforcer can be considered a cost. A common delay that has been studied is the delay that occurs when you purchase a product online. It may cost more to get something you ordered fast than slowly, and if the delay is considered a cost, you may be able to understand how people value obtaining the item faster by varying costs associated with faster or slower delivery. As Hantula and Bryant [60] have shown, how much individuals would be willing to pay for immediate versus later free delivery fit a hyperbolic discounting curve, consistent with considering time to receipt a cost. Research is needed to determine the metric to extend unit price to delay discounting, and how to generalize the ratio of money to time across different combinations of delay and reward magnitude.

The development of a common metric for relative reinforcing value and delay discounting, unit price, provides a way to integrate the intertemporal choices for relative reinforcing value and delay discounting. Relative reinforcing value often involves no or very brief intertemporal choice, while delay discounting can involve large delays between the choices. Such a metric could consider differences in the amount of responding for access to a reinforcer in relative reinforcing value paradigms in terms of delay, since schedules with greater response requirements necessarily will institute a greater delay before reinforcer acquisition. This may account for some of the effects of different reinforcement schedules, and it also provides another commonality between relative reinforcing value and delay discounting tasks.

5.2. Substitutes and complements

Substitutes and complements describe ways in which reinforcers interact. Substitutes and complements are important constructs in behavioral economic analysis of relative reinforcer efficacy, and they may be useful in deriving new methods for behavior change of multiple behaviors. Substitutes and complements can also be considered in terms of delay discounting. With no delay, it is obvious that alternatives that are equally chosen are substitutes for each other. For many people a serving of apple pie is preferred to an apple, but if you increased the cost of the apple pie, that person might shift choice to the apple. Substitutes may be relevant for delay discounting, as someone who may prefer a piece of apple pie now may choose the apple now if they have to wait a week to get the apple pie. If sufficiently hungry, they may choose the apple now rather than the apple pie at the next meal! Thus, the relative value people assign to two or more payoffs at different points in time become destructive to the maximization of the utility. This narrowing in perception of alternatives reduces the ability to seek alternative behaviors or to delay the consumption of the reinforcer.

Complements may also be studied from the perspective of delay discounting. Consider a coffee drinker who prefers milk in their coffee. When offered both simultaneously, the person would always consume coffee with milk. But the degree of the complementary relationship can be tested by implementing a delay in availability of milk. If the complementary relationship is not strong, then the person may choose to have black coffee now rather than wait for coffee plus milk after 5 min. However, if the complementary relationship is strong, then the person would wait a very long time to get coffee plus milk rather than drinking black coffee now.

The importance of understanding substitutes and complements is critical to our model of behavioral choice and obesity. One challenge of behavior change in obesity is the degree to which reinforcers become substitutes or complements of an exponentially increasing array of behaviors and stimuli. For example, for many people socialization is a complement to eating, and providing a choice between socializing and eating would lead to the choice of socialization. However, for someone with severe obesity, eating may become a substitute for many alternatives, and the more severe the obesity, the more food becomes a substitute for alternative reinforcers, and the harder it is to identify substitutes to eating. In fact, we argue that the reinforcing value of eating a particular food may sensitize, or increase with repeated consumption over time, while the reinforcing value of alternatives is reduced, making the choice to reduce eating even harder [61,62]. Food consumption may also become a complement as a result of regular pairing between consumption and other behaviors and stimuli [30,33]. Stimuli associated with complements may gain discriminative function for reinforcer consumption or strengthen the magnitude of the reinforcer, as a given commodity may be more reinforcing if engaged in with the complement than without. For example, if watching television and eating are commonly paired together, then watching television may signal eating, and the combination of watching television and eating may be more reinforcing than watching television alone. If the complementary combination of watching television and eating increases their reinforcing value, or increases the amount of work that someone will do to get access to these complementary reinforcers, then the unit price of the combination increases, making it a more attractive alternative than watching television alone. Similarly, exercising with a friend may provide greater complementary reinforcing effects than either alone and serve discriminative functions for future exercising or interacting with this friend.

In addition to a potential shift in reinforcing value as the degree of obesity increases, and food becomes a substitute for other reinforcers, the inability to delay gratification may increase as time becomes more costly. Many people have the experience of having the opportunity to consume multiple snacks throughout the day, but choose instead to consume one more palatable dessert later in the day with their meal. In addition, most persons would make the choice to delay consumption of really palatable food for the potential larger benefits of good health, a good job and a good family life. However, someone with severe obesity may have problems delaying food consumption for later, or delaying food consumption for the promise of future health, job or family. This challenge in delay discounting to choose the larger, but delayed reward, may become even more difficult if the immediate reinforcer is paired with a complement. Similar to the above example of eating as a complement to watching television, it may be much harder to wait to watch a favorite television show for later if there is the choice of a less favorite show that is paired with popcorn, chips and salsa or any of our favorite snack foods now.

Based on the analogy between waiting for the delay in delay discounting paradigms and response effort in reinforcing value paradigms, it is reasonable to assume that the unit price of obtaining a small amount of a complementary set of reinforcers (watching one television show with access to food) is less for some people versus waiting longer for a larger amount of one of the reinforcers (watching an entire evening of television). This could be accomplished by the use of the indifference point methodologies for both relative reinforcing value and delay discounting paradigms. For relative reinforcing value this might be the point where there is no preference between two alternatives. Using the watching television versus watching television plus eating choice, it may be that people may work twice as hard to gain access to the same amount of television watching if food was available than if just television watching was the alternative. Similarly, the indifference point for watching a short bout of television plus eating versus watching a longer period of television without food would be the amount of television that is not preferred at a given time interval versus immediate access to watching television plus eating.

There are several differences in the choice paradigms that need to be resolved in the process of integrating the two choice paradigms. The first is the fact that relative reinforcing value is always assessed by comparing preference for at least two alternatives, with variations on the magnitude or behavioral cost of the alternatives, while delay discounting is almost always assessed by comparing the amount of time that someone will wait for the same alternative that differs in terms of amounts and delay. The most common procedure is to assess delay discounting using choice of small immediately available amounts of money or larger, but delayed amounts of money. This situation is analogous to the common choice of buying now or saving for something better later. It is also common to determine delay discounting for reinforcers such as food or drugs. In these comparisons the choice is between a small amount of the reinforcer or a larger amount later, for example a pack of cigarettes now or 20 packs of cigarettes later. These procedures may not model all types of choices that are critical to understanding why it is so hard to make healthy choices for later positive benefits. For example, not smoking now for a reduced risk of cancer or heart disease later, or not eating a high energy dense snack to be leaner later. In both of these examples, which are common representations of the difficult choices that people face regarding choosing not to engage in a really reinforcing behavior now for later health benefits, involve the comparison of different alternatives at different time points. We believe that the integrated model must accommodate delayed choices of different magnitudes of alternative reinforcers, as well as delayed choices of different magnitudes of the same reinforcer.

A second issue is the framing of the delay discounting task. The concurrent choice procedure explicitly assesses choice of two alternatives, which is the analog to many choice situations. The delay discounting task assess the choice of a small immediate versus larger delayed reinforcer, with the implicit assumption either that there are no alternatives or that the choice does not depend on the alternatives. It is easy to consider situations in which providing concurrent alternatives would be likely to have a powerful influence on the choice of a small immediate or larger delayed reward, and which provide a good model for choices that may occur outside of the laboratory. For example, consider the choice of watching one television show now versus an entire evening of television that evening. For the person who chooses the immediate alternative the choice may be very different if there was a choice of watching television or socializing with friends. This is an important point, because outside of the laboratory the choice is almost never to engage in a behavior now or later, or consume a small amount of a commodity now or more later. The choice for an obese person attempting to lose weight is not to eat a small amount now or a larger amount later, but rather to not eat high energy density and low nutrient density foods or engage in an alternative behavior now and reduce the risk of disease later.

Of course, as noted earlier, there is no inherent reason why the choice in a delayed discounting task cannot be between two different reinforcers that vary in their availability. That would make the delay discounting task very similar to the concurrent choice task, particularly if one can generalize from time to schedules of reinforcement. For example, consider the choice to eat a piece of apple pie now, or a healthier delayed choice of a crisp red delicious apple and a handful of grapes later. In a delay discounting task the apple pie a la mode would be available immediately, and the weight loss would be available at different times in the future. This choice can also be studied using a relative reinforcing value substitution paradigm by providing the choice between apple pie and weight loss at a schedule that does not require much work to get access to food, with the schedules of reinforcement to obtain the apple pie increasing while the schedule of work to obtain apples is constant.

Another difference between delayed discounting and relative reinforcing value is that in delay discounting the reinforcer is available after a specific time has passed, and no work is required to gain access to the reinforcer. While there may be some situations in which reinforcers are provided automatically after time passes, there may be other situations in which reinforcers are only available after some responding is needed to gain access to the reinforcer occurs. Waiting may be associated with access to a larger reward than if you responded immediately, but responding may also be needed for the delayed option.

This discussion has pointed out similarities between the two constructs, and potential ways to equate reinforcing value and delay discounting, based on the idea that choice at time 0 represents relative reinforcing value and time at choice 0 represents one choice point in delay discounting. While it is reasonable to assume that a time interval in delay discounting is equivalent to response effort in a reinforcing value task, research is needed to validate this hypothesis, just as research is needed to validate the hypothesis that response effort can be equated to money in constructing demand curves [35]. Theoretical and empirical work is needed to determine whether how the concepts of reinforcing value and delay discounting are complementary or how they differ in conceptualizing choice. This provides a rich area of research that may be useful to integrate different aspects of behavioral theories of choice.

6. Summary

The purpose of this review was to provide a background into how relative reinforcing efficacy and delay discounting behavioral phenotypes may be relevant to the study of energy intake and obesity. In addition, we provide preliminary data to support Appelhans [57] thesis that energy intake is influenced by the interactive effects of food reinforcement and delay discounting. But we believe the unique contribution of this review is to begin a conceptual analysis of similarities and differences between reinforcing value and delay discounting as complementary theories of choice. While a considerable amount of research is needed to determine whether theoretical attempts to bring these two approaches together improve our explanatory power to better understand energy balance and obesity, we believe these approaches to choice share many similarities and providing new ways to bridge differences between the conceptual approaches may lead to new insights into obesity.

One way to provide testable hypotheses about advantages of bringing reinforcing efficacy and delay discounting together is to develop quantitative relationships that describe both choice situations. Each of these areas has led to quantitative ways to describe the choices. For example, demand curves are a quantitative way to describe responding for a reinforcer, while hyperbolic curves of indifference points are used to describe delay discounting. The ensuing step is to formulate a quantitative framework to describe both types of choices.

Perhaps the main new theoretical idea we discussed are methods to quantify differences among behaviors to facilitate choice. Two new ideas we have presented here, that demand elasticity, along with understanding substitutes and complements provide new ways to make decisions about which behaviors to modify in situations in which multiple behaviors are targeted. For example, it may be more challenging to modify a behavior that was demand inelastic to changes in effort, had many complementary behaviors that set the occasion for this behavior, and had no substitutes, than a behavior that was demand elastic, had few complementary behaviors and many substitutes.

Understanding factors that alter concurrent choice among alternatives, as well as shifting choice from unhealthy immediate choices to healthier delayed outcomes could be very important in modifying eating and activity behaviors that would lead to reductions in body weight and improved health. We hypothesize that simultaneously considering both aspects of choice may provide new insights into factors relevant for the development and treatment of obesity.

While this review focused on integrating relative reinforcing efficacy and delay discounting as complementary approaches to the study of obesity, we believe obesity is just one problem that can be conceptualized from this framework. There are a number of problems that are driven by the strong motivation to consume these reinforcers, and they shift choice of people from healthier behaviors to less healthy alternatives and limit the horizons of people for healthier future alternatives, leading to discounting of future benefits. These could include some forms of drug abuse, gambling, a sedentary lifestyle, some sexual problems and impulsive shopping. We collectively conceptualize these problems that involve excessive motivation to obtain the commodity coupled with poor ability to delay gratification or discounting the future as reinforcement pathologies. Over time these reinforcers may gain strength and substitute for alternative reinforcers. They may also become complementary commodities as a result of regular pairing of reinforcer consumption in association with adjacent behavior or stimuli which gain discriminative function for reinforcer consumption. In addition, as reinforcement pathology grows, the strength of alternative behaviors to compete with the reinforcement pathology weakens. We argue that there is a narrowing in perception of alternatives, reducing the ability of the person with the reinforcement pathology to seek alternative behaviors or to delay the consumption of the reinforcer. Future research is needed to integrate strengths of reinforcement theory and delay discounting to address obesity and other problems that may share an excessive motivation to engage in a behavior coupled with poor ability to delay gratification.