Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Mar 1.
Published in final edited form as: Appetite. 2017 Sep 19;122:44–50. doi: 10.1016/j.appet.2017.09.014

Progressive ratio (PR) schedules and the sipometer: Do they measure wanting, liking, and/or reward? A tribute to Anthony Sclafani and Karen Ackroff

HR Kissileff 1,*, M Herzog 1
PMCID: PMC5797486  NIHMSID: NIHMS935287  PMID: 28935479

Abstract

This paper honors the contributions made by Anthony (Tony) Sclafani and Karen Ackroff to both the Columbia University Seminar on Appetitive Behavior and to the field of ingestive behavior in general. We review their use of the progressive ratio (PR) licking paradigm, to determine whether the taste of sucrose, independent of its post-ingestive effects, is always positively reinforcing in animals. They demonstrated a monotonic increase in licking as concentration increased, and obtained results identical to those obtained with a lever-pressing paradigm, but licking was easier and more natural than lever pressing. The PR paradigm was translated to evaluate liquid food reward value in humans. An instrument (the sipometer) was devised that initially permitted a few seconds access to small amounts of a sweet beverage as the participants increased the time to obtain it in 3-5-sec increments. The device went through two refinements and currently delivers the reinforcer and measures the pressure exerted to obtain it. The sipometer is compared with other techniques for measuring motivation and reward. The use of the sipometer and the PR method are discussed in relation to the theoretical challenges inherent in measuring motivation and pleasure, from both psychological and behavioral economics perspectives, and why it is or is not important to separate these processes for both theoretical and practical applications.

Keywords: Wanting, Liking, Motivation, Hedonic, Reward value, Reinforcement

1. Introduction

This is the last paper in a series that honors the contributions of Anthony (Tony) Sclafani and Karen Ackroff both to the Columbia University Seminar on Appetitive Behavior and to the field of ingestive behavior in general. The previous papers in this series have described Tony’s and Karen’s work primarily in animals, with little translation to humans. In order to achieve balance and to highlight recent applications of their work, this paper describes how and why the application of the progressive ratio (PR) licking technique for measurement of reward value in rodents was applied to the development of a human analog, the sipometer (Hogenkamp, Shechter, St-Onge, Sclafani, & Kissileff, 2017). The following topics will be presented: a) a description of the animal studies that led to the development of the sipometer; b) the evolution of the sipometer with its advantages and limitations; c) the theoretical constructs underlying the interpretation of the data collected; d) comparison of the sipometer with other methods; and e) problems with existing techniques and future design of experiments that employ both natural and arbitrary, effort-requiring tasks.

2. Progressive ratio licking in rats: How and why it was employed for measuring reward value of sucrose

In order to test the hypothesis that sweet taste reward increases monotonically with concentration of sucrose solutions, and thereby demonstrate that reinforcing value of sweet taste, rather than post-ingestional effects, was driving rats’ increased responses to obtain a sweet taste, Sclafani and Ackroff (Sclafani & Ackroff, 2003) utilized a novel progressive ratio licking task. Their goal was to verify, with a simpler procedure than had been used in the past (e.g. alley running (Sheffield, Roby, & Campbell, 1954) or lever pressing (Guttman, 1953) that sweet taste is always positively motivating, and that intake measures mask the motivating effects because increased satiation at high sucrose concentrations reduces intake. The PR licking task minimizes intake, but increases the ratio of work output to reinforcer obtained. (It should be noted that the terms “reinforcement” and “reinforcer” have historically been defined differently in the psychological and behavioral economics literature. This issue is discussed in more detail in section 4.2.) This task was built on the original demonstration of Hodos (Hodos, 1961) that progressive increase in both the number of lever presses and the break point (the number of presses at which responding stopped) increased with the concentration of the milk used as reinforcer. Subsequently, Reilly (Reilly, 1999) applied the PR lever-press technique to sweet caloric (sucrose) and non-caloric (saccharin) solutions. McGregor et al (McGregor, Saharov, Hunt, & Topple, 1999). were the first to use a PR operant licking task in a study that compared the motivation of rats to consume alcoholic (beer) and non-alcoholic (sucrose) solutions. It remained for Sclafani and Ackroff (Sclafani & Ackroff, 2003) to demonstrate that both intake and breakpoint increased with sugar concentration when sucrose solutions were the reinforcers in the PR-licking task, whereas in a fixed ratio (FR) task, intake increased then decreased with concentration. The advantage of the licking over the lever-pressing task and runway tasks was that it required no learning or transfer of the animal to a special cage. Sclafani and Ackroff demonstrated that the progressive licking technique produced identical results to lever-pressing, and was easier to use. Their report ends with the prescient comment that “It is possible that a PR sipping task using an operant drinking straw may be an effective means of evaluating liquid food reward in human subjects.”

3. Development and evolution of the sipometer

3.1. First attempt

Tony asked the first author of this paper shortly after the PR licking paper (Sclafani & Ackroff, 2003) was published, whether he would be interested in testing the PR licking concept in humans. Since he was already doing human feeding studies and had extensive prior experience with reward concepts based on speed of eating in humans (Bobroff & Kissileff, 1986), he agreed to test any device Tony could build. In 2006 Kissileff and Sclafani began testing the concept that sipping on a straw could be an indicator of motivation and/or reward value. Tony constructed a device that utilized pulling on a straw as a proxy measure for sipping. The participant was positioned in front of a box from which the straw emerged. The straw was connected to tubing that passed through a solenoid-activated pinch valve, and was held by a clamp connected to a micro-switch, which was activated when the straw was pulled forward. A computer monitored switch activated and opened the solenoid to allow reinforcer flow through the straw, depending upon the reinforcement schedule, when the subject exerted sufficient pressure. On the Continuous Reinforcement (CR) schedule, the solenoid was opened as long as the switch was activated. On the PR 5-sec schedule, the switch had to be activated by pulling on the straw for successive periods increasing by increments of 5 s (i.e. 5, 10, 15, etc.), to open the solenoid for a 2-s period during which the reinforcer was available. The reinforcer was a strawberry-flavored yogurt shake that was unsweetened or sweetened with aspartame (12 g Equal® per 1106 g shake). The human and animal paradigms differed slightly in that effort in human procedures would be captured by progressive increases in the amount of time the response must be sustained, while in animal PR procedures, the effort is measured by the number of responses, not the amount of time responding. Eight men and eight women were studied under two schedules of sipping (CR and PR) and two levels of sweetness (sweet and non-sweet shakes) presented in counterbalanced order by means of a Latin square for each group of four.

Since it was not feasible to test several concentrations, as in the animal work, Kissileff and Sclafani chose a sweetener difference (non-sweet vs. optimally preferred sweet based on previous studies) (e.g(Bobroff & Kissileff, 1986; Drewnowski, Grinker, & Hirsch, 1982).) that would be expected, based on liking ratings, to generate intake differences in humans. The prediction was that on the PR compared to the CR schedule, participants would sip significantly longer and therefore consume more, when they received the sweet, compared to the non-sweet shake, but that overall, intake would be reduced on the PR schedule, thereby reducing post-ingestional effects.

The result was not exactly what we had expected (all values from (Gondek-Brown et al., 2007). Under both PR and CR schedules, subjects rated liking of the sweet shake significantly higher than the non-sweet by 2.55 units ± 0.57SE, (p ≤ 0.0001; t39 = 4.46), and on average, consumed approximately 100 g more per 1 unit difference on a 9 point scale of liking (Peryam & Pilgrim, 1957), thus replicating a previous finding (Bobroff & Kissileff, 1986). However, intake of the sweet shake was significantly higher than that of the non-sweet shake only under the CR condition (208 g ± 68 g SE; p = 0.004; t39 = 3.07), and not the PR (56 ± 68 g SE; p = 0.41; t39 = 0.83). In addition, there was an unanticipated sex × treatment interaction, such that the intake difference in males was greater than that of females under both conditions combined by 240 g ± 96 SE (p = 0.02; t39 = 2.5). Presaging future findings, there were significant regressions of intake from palatability ratings taken after the meal. The slopes of intake by liking rating were 65 g/unit ±22 SE (p = 0.02; t = 2.99) for CR and 30 g/unit ± 9 SE; p = 0.02; t = 3.24), for PR.

A second study (see (Hogenkamp et al., 2017) for details) with the sipometer was more consistent with expectations. Subjects consumed more of a yogurt shake drink when it was offered ad libitum after 21 h, than 1 h, of food deprivation, but when they were required to work on PR schedule, they worked harder (i.e. spent more time sipping), but did not consume more after 21 h than 1 h of deprivation. There was no difference in rated liking between the deprivation conditions. Thus deprivation changed the motivational, but not the hedonic aspects (see below for further discussion of this issue) of ingestion, and the sipometer was able to measure the motivational effect.

3.2. The second model

As a consequence of the first study, it was decided that a device that required detection of actual sipping, not simply moving the straw, might be a better measure of the reward value. Tony was able to arrange for a new pressure sensing device to be constructed at the Pierce Laboratory with the help of Dana Small, who supervised the construction by engineer John Buckley. We piloted a limited number of subjects (6 female and 4 male) with a chocolate milkshake and found that, as in the first study, sweet intake was significantly higher than non-sweet intake for the CR (difference = 170 g, ± 75 SE, t (41) = 2.27, p = 0.03) but not PR (difference = 61 ± 75 SE). Although the results were similar to those in the first study, participants consumed more sweet shake than non-sweet shake per second of reinforcement. This behavior resulted in differences in reward size, that depended on the sweetness of the shake, so we decided that a third modification was necessary.

3.3. The current sipometer

The sipometer went back to the Pierce Lab in 2011, and was modified so that liquid reinforcement was delivered at a fixed rate using a peristaltic pump. Two types of studies were conducted with the new device. In the first (Hogenkamp et al., 2017), eight women were tested in a sham-drinking paradigm (i.e., sip and spit) with an aspartame sweetened Kool-Aid compared to an unsweetened Kool-Aid, and in the second, (Merriam & Kissileff, in preparation) a milkshake was consumed. In the sham drinking paradigm, more of the sweet than non-sweet taste beverage was consumed under PR, but paradoxically, the difference reversed with the CR schedule. A new measure of motivation or reward value was introduced since we now had the ability to measure the actual sipping pressure exerted. The pressure increments of each sip were totaled and the cumulative pressure provided a measure of effort. Although the cumulative pressures during the appetitive phase were not greater for the sweet than non-sweet beverage, there was a significant positive correlation between the cumulative pressure and the rated enjoyment of the beverage, as well as the amount subjects wanted which they rated in a taste test before they sipped.

In the second sipometer study, twelve men and twelve women were given chocolate milkshakes to consume with the sipometer under two schedules of reinforcement (PR-3 sec increment, and CR) crossed with two levels of sweetness, unsweetened vs. 10% sucrose sweetened. Enjoyment rating, liking, and sweetness intensity ratings were higher for the sweetened than non-sweetened shake, across all participants, regardless of reinforcement schedule. Intake was significantly higher for sweet than non-sweet beverage during the CR schedule, but not different on the PR schedule, as we found with the less refined version of the sipometer. However, when differences in measures that require musculo-skeletal action as opposed to cognition or thinking, such as intake, cumulative pressure or time spent sipping, were regressed against differences between sweet and non-sweet, hedonic measures (e.g. enjoyment or liking), the differences between sweet and non-sweet emerged. It therefore appears that measures of motivation in the absence of corresponding measures of hedonics could give misleading negative results. Absence of motivation should not be inferred from absence of responding, unless the hedonic values of the reinforcers are also measured.

3.4. Advantages, disadvantages and potential applications

The big advance in translating animal to human work is that the sipometer is a semi-natural way for reinforcement to be given to humans, which has a strong parallel to reinforcers given to animals. The consummatory response of sipping is used as an operant just as licking is used in animal studies. Sipping for small quantities of fluid is a common behavior in humans, particularly when they are completing consumption of a palatable beverage and little remains in the cup. The disadvantage is that people easily tire when the reinforcer is not delivered, and they are not as persistent as hungry rats that have little else to do other than work for food. Another disadvantage is that we do not really know what aspect (e.g. motivation vs. pleasure) of ingestive behavior is being measured by the response. We consider this issue in the next section. Nevertheless, the potential value of the sipometer device remains to be seen where it could be used clinically to test hypotheses and potential treatments for eating disorders and obesity, and industrially where it could be used to test drugs and beverages. We will discuss the advantages and limitations further, after we describe other methods and a theoretical framework.

4. Theoretical background

4.1. Constructs and measurements

In order to understand how the sipometer could be employed to solve problems related to mechanisms controlling ingestive behavior, it is important to recognize that there are two main constructs to which its use could be applied. Since Craig’s (Craig, 1918) early analysis of behavior, it has been traditional to divide ingestive behavior into two phases; those that 1) precede the act of consuming, called variously “appetitive”, “motivational”, “instrumental” “incentive”, or “wanting,” and 2) those that follow consumption, called “hedonic”, “affective”, “rewarding”, or “liking”. Much of the experimental work in the last four decades has been an attempt to unravel the neuropsychology of these two phases by means of clever instrumentation and experimental design to obtain the measurements that test the relationships among the constructs.

4.2. Integration of psychology and behavioral economics: motivation component

Although he did not recognize it at the time, the fields of behavioral economics and psychology converged in the work of (Hodos, 1961) and both are related to ancient wisdom: “The reward is according to the exertion” (Mishna Avot 5:26 (Scherman & Zlotowitz, 1985), p. 581). The ambiguity of this proverb underscores current problems in the field of ingestive behavior. Does the exertion make the reinforcer more rewarding, or does the reward induce greater exertion to obtain it? This ancient Hebrew proverb also encapsulates modern psychological theory of motivation, equivalent to: “the rationale for the obstruction box procedure ((Moss, 1924; Warden, 1931)) was that the greatest intensity of electric current which the animal would cross should correlate with variations in reward and deprivation” [(Hodos, 1961), p.943]. Hodos developed the PR responding paradigm, in which an animal must increase its number of responses in order to receive successive reinforcements, to overcome the variability in outcome engendered by the use of electric shock to measure motivation. The PR schedule was subsequently used in several fields outside of ingestive behavior, particularly for measuring reinforcing value of brain stimulation and drug usage, but the remainder of this review will be on use of the PR schedule in ingestive behavior. However, before proceeding to that review, we note that, in rats, clusters of licks for sucrose increase, with increasing effort requirements. This phenomenon, known as the “effort justification effect”, has been interpreted as an indication that effort increases reward (i.e. reinforcing) value (Dwyer, 2012). Thus, it is possible to demonstrate that effort can be both an indicator of reward value and generator of it.

An important application of the PR schedule for ingestive behavior was how best to measure the intervening variables that enabled animals and humans to change responding when either their internal environment, or the stimulus that generated the response, was changed. This problem was couched in the languages of reinforcement and behavioral economics, each of which developed unique vocabularies.

In the psychological literature, “reinforcement” was used early on as both the object obtained (i.e. a reinforcer) when a response was made to a stimulus which strengthened the response and the description of a theory (see (Skinner, 1945)p. 272 for example.). The term “reinforcer” refers to the object or reward obtained when a response was made to a stimulus. The imposition of an obstacle or a task to obtain the reinforcer (as opposed to the reinforcer being freely available), particularly those generated by sweet tastes, was used to separate reflex from behavior, which had purpose and expectation (see A.N. (Epstein, 1982), cited by (Berridge, 2004; Teitelbaum, 1966)). A persistent issue was how to separate behavioral measurement of motivation (drive, incentive, or “wanting”) from pleasure (hedonic value, reward, or “liking”) as the underlying set of processes that accounted for the behavior.

In behavioral economics (see (Epstein, Leddy, Temple, & Faith, 2007), for review, particularly p. 891, from which the following was extracted): reinforcement theory was extended so that reinforcers were considered as commodities and behavioral cost as equivalent to price. A basic principle of economics is that increasing the price of a commodity reduces purchases, just as increasing the behavioral cost of obtaining a reinforcer reduces consumption (Bickel, Marsch, & Carroll, 2000). Consistent with this notion, L. Epstein and colleagues (Epstein, Dearing, Paluch, Roemmich, & Cho, 2007) hypothesized that willingness to pay more for a commodity was related to reinforcing value of that commodity (Epstein et al., 2007).

4.3. Relation of hedonics to motivation in reward: problems and solutions

When the amount an animal consumed was the measure of either motivation or reward, experimenters could not determine from the behavior alone whether intake effects were attributable to a change in motivation (e.g. by deprivation, lowering of blood glucose, increased extracellular osmotic pressure), onset of satiation (due to gastric distention, secretion of hormones, or neurotransmitter release), or change in hedonic value of the item being consumed. The interpretation of results from these sorts of experiments was not clear, because either a) the neural interpretation of the sensory quality of the stimulus, or b) the animal’s internal state, changed during the course of the experiment.

In order to resolve conflicting interpretations (e.g., attribution to pleasantness of the reinforcer or motivation to consume it) of the mechanism underlying intake changes during experimental manipulations, a variety of techniques were introduced, all of which attempted to reduce the effects of the accumulation of stimulus in the animal’s body (i.e, mainly in the stomach and intestine), although a similar critique could be leveled at these procedures as well, because of prolonged stimulation of receptors in the mouth. Procedures that were employed to reduce or eliminate post-ingestive signals included a) sham feeding/drinking, wherein the reinforcer drained out a fistula in the esophagus or stomach or spit out in human studies; b) the use of schedules of reinforcement that involve arbitrary motor responses (e.g. nose poking, lever pressing, running down an alley) for receipt of the reinforcer after varying amounts of effort are expended; c) reinforcement of consummatory responses such as licking; d) intra-oral infusions coupled with observations of oromotor and somatic responses responses; or e) reinforcers that directly stimulate the neural structures that presumably underlie the rewarding effects of sensory stimuli, such as electrical or chemo-stimulation of the brain.

5. Hedonics and motivation in humans

5.1. Basic ideas for separation of motivational and hedonic aspects of reward

The principles described above, mainly derived from animal studies, led to attempts to separate the motivating from hedonic aspects of ingestion in humans. Berridge (Berridge, 1996) proposed that the reward system (mainly from an animal perspective) is comprised of both “liking” and “wanting” elements which are undergirded by both overlapping and distinct neural structures. In this framework, “liking” is the conscious perception of pleasantness, manifested by objective evaluation of oromotor and somatic responses that indicate “reward value” of food. The construct of liking has been linked to opioid/endocannabinoid/orexin networks in the nucleus accumbens, ventral pallidum, and other forebrain limbic structures, while “wanting” is a motivational drive for foods linked to dopamine/opioid networks in the nucleus accumbens, striatum, amygdala, and mesolimbic systems (Berridge, Ho, Richard, & DiFeliceantonio, 2010).

5.2. Application of the principles to measurements

5.2.1. Choice tasks as measures of reward

Finlayson and colleagues (Finlayson, King, & Blundell, 2007a) applied Berridge’s theory of liking and wanting by measurement of the hedonic aspect with a questionnaire, and the motivational aspect with a computerized choice task in which explicit wanting was determined by how many times an item was selected, while implicit (i.e. non-cognitive) wanting was measured by reaction time. Subsequently, Dalton and Finlayson (Dalton & Finlayson, 2014) found that “trait binge eating”, was attributable, in part, to enhanced implicit wanting for sweet foods with high fat content.

An advantage of choice tests with computerized photo-based paradigms is that they can be used in tandem with fMRI technology which can potentially illuminate the neural basis for subjective self-report ratings. Lemmens et al. (Lemmens et al., 2009), found that consumption of highly liked sweet dessert foods produced greater reduction in wanting of foods across categories (i.e. sweet, dessert, and general), than did consumption of a neutral, sufficiently liked food. Consumption of dessert, but not a neutral food, also decreased liking for other foods in the dessert category. Born et al (Born et al., 2011). also linked liking on a food choice paradigm (task-related signaling in a pre-meal in fasting state) to the thalamus and occipital visual areas by means of fMRI. Wanting-related activity in the brain was reduced in the caudate from fasting to sated conditions, but liking activity reduction was seen in the anterior insula. Thus, liking and wanting mechanisms could be controlled by both overlapping and distinct brain structures.

5.2.2. Use of variable ratio schedules for “values” measures of reward

“Reinforcing value” has been defined as “the amount of work an individual will perform to receive a given reinforcer in the presence of alternative reinforcers and changing schedule demands” ((Bulik & Brinded, 1994) p. 666). Subjects worked on a variable ratio schedule in which the requirement for reward increased progressively from VR5, to VR300 for points exchangeable at the end of a computer game for either food or cigarette reinforcement. Reinforcements were consumed after every two games. Subjects worked harder for food when deprived, but did not consume more. This result is consistent with the deprivation study with the sipometer reported above (Hogenkamp et al., 2017). Leonard Epstein (Epstein, Truesdale, Wojcik, Paluch, & Raynor, 2003) and colleagues followed up on these studies but added a 10-point hedonic-value scale, to measure reinforcing value of food after deprivation, compared to no deprivation. The deprived group made significantly more responses than the non-deprived group on the VR32 schedule, but showed no difference in their hedonic evaluation rating. In Epstein’s framework (see (Epstein et al., 2003) p.221), tmotivation or wanting was equated with the incentive or reinforcing value of food, and measured by the effort expended to obtain food, while the hedonic value of food (i.e. liking) was measured by subjective ratings of palatability or the pleasantness of food. In a study of liking and wanting in obese versus non-obese individuals, the number of responses was higher for obese than non-obese at reward ratios above 128 responses per reinforcement. Furthermore, the reinforcing value of food was a significant predictor of energy intake (p = 0.002), but self-reported liking of the favorite food was not (p = 0.72). The results were interpreted to mean that relative reinforcing value of food may be more influential in determining food intake (Epstein et al., 2003, 2007) than liking.

5.2.3. Progressive ratio schedules

Nasser et al (Nasser, Evans, Geliebter, Pi-Sunyer, & Foltin, 2008). adapted a variant of the Bulik-Brinded (Bulik & Brinded, 1994) procedure by changing the schedule from VR to PR 20 (i.e. after each reinforcement, an additional 20 taps on the computer key was required to obtain the next reinforcement). As in the Bulik-Brinded study, participants chose between food and money. Obese individuals with and without binge eating disorder (BED) were tested, and reinforcing value of food (i.e. breakpoint) dropped after feeding in the non-BED group, but increased in the BED group. These changes resulted in significantly higher breakpoints for food after feeding in the BED group. The breakpoint (the highest ratio achieved in the 10 trial session) was the primary metric in Nasser et al. whereas the number of responses was the metric in the Bulik and Brinded study. In both studies, obese individuals were more motivated than controls to obtain food.

Utility of the PR approach was also demonstrated in another clinical study (Schebendach, Broft, Foltin, & Walsh, 2013) in which binge eaters were instructed to work for yogurt shake on a PR schedule, under binge and non-binge instructions. Participants ate the shake in the amount that they earned after completion of the task. Binge eaters worked significantly harder and consumed more of the milkshake under binge instructions than did healthy controls, but binge eaters worked significantly less and consumed less than non-binge eaters when they were asked not to binge. It should be noted that Schebendach et al (Schebendach et al., 2013) use the term “hedonics” to refer to reinforcing value, in contrast to mainstream authors in the field who use “hedonics” and “liking” interchangeably, and consider both distinct from “wanting” or motivational processes. Consequently, it is unclear whether differences in effort expended by binge eaters was related to differences in wanting or liking in this study.

The PR schedule was also used to measure “reward value” for M&M candies in a paradigm validated with a depression model (Willner et al., 1998) on patients after Roux-en-Y gastric bypass. Patients worked as hard as controls before surgery (equal break points), but after surgery, reduced responding at a rate below the level of controls (Miras et al., 2012). Sweet taste detection thresholds were lower after surgery than before, but “just right concentration” was the same as controls’ (Bueter et al., 2011). These results could be interpreted as greater reduction in the motivational aspect (“wanting”) than the hedonic (“liking”), as a result of the operation, but the rating of “just right” concentration is not the same as evaluation done by a sensitive hedonic rating scale (e.g(Kalva, Sims, Puentes, Snyder, & Bartoshuk, 2014).), which employs magnitude estimation logic across concentrations. This paradigm is the solid-food counterpart of the sipometer that used liquid reinforcers.

5.2.4. Delay discounting of reward

Another aspect of reward measurement that deserves consideration is the relationship between the effort expended and the temporal delivery of the reward. Several procedures have been developed to compare the reward value of immediately available rewards with delayed delivery. This research, originally pioneered by Mischel (Mischel, Ebbesen, & Raskoff Zeiss, 1972), led to the concept of “delay discounting,” which is defined as the preference for smaller, proximal rewards rather than larger delayed ones (Epstein, Salvy, Carr, Dearing, & Bickel, 2010) p. 438) Delay discounting is a direct consequence of Herrnstein’s matching law that choice depends on relative rate of reinforcement, which is lower when there is a delay that devalues the reward (see (Bickel, Johnson, Koffarnus, MacKillop, & Murphy, 2014) for review). The value of the reward diminishes with the length of the delay, and this devaluation is measurable. This phenomenon is important because “obese versus lean humans also show behavioral response inhibition deficits on stop-signal and go/no-go tasks involving both food and non-food stimuli, and a preference for immediate food reward over larger delayed rewards” (See ((Stice, Lawrence, Kemps, & Veling, 2016) for references and review, p. 18). The sipometer is a valuable tool for measuring reward because the reward is immediate rather than delayed, as in the paradigms described above.

5.2.5. Affective reactions measured by facial (i.e. oro-skeletal) movements

In order to assess hedonic reactions independently from motivated behavior in response to food or drink, investigators have used facial affective reactions in human newborns elicited by taste stimuli as a hedonic measure of liking (Steiner, Glaser, Hawilo, & Berridge, 2001). These studies were based on animal work by (Grill & Norgren, 1978). Facial affective reactions of human newborns and non-human primates in videos were compared during sucrose administration to both species, and the spontaneous facial expressions elicited by the taste were coded for frequency and duration of distinct facial movements. The use of facial reactions as measures of hedonic response has been widely used in the animal literature (e.g (Myers, 2017).), but has been relatively limited in humans, with the exception of (Winkielman, Berridge, & Wilbarger, 2005). Winkielman et al. employed a subliminal projection of facial reaction to test the hypothesis that there are implicit (i.e. non-cognitive) affective components that control behavior. Their participants drank more beverage when they had been shown happy compared to unhappy faces for a duration too short to process cognitively (16 ms) and their interpretation was: “basic affective reactions can be unconscious and interact with incentive motivation to influence assessment of value and behavior toward valenced objects” ((Winkielman et al., 2005) p. 121).

5.3. Comparative merits and limitations of the sipometer compared with other measures

Given the variety of methods that are available for measuring rewarding properties of foods and beverages and motivation of individuals to consume them, the sipometer has several advantages. First, it can directly measure the effort expended by the individual for an immediately received reinforcement. It therefore allows translation of the many paradigms, such as changes in beverage properties, administration of drugs, and variations in body weight, used in animals, to similar manipulations in humans. It is an uncomplicated instrument that allows, in addition to the number of responses (i.e. times the individual sips and swallows, similar to licking in the rat), measures of the actual pressure patterns exerted and the energy expended in physical units (force per unit area). Second, the reinforcer does not require intermediary mental processing, or devaluation as a result of delay. Third, the value of the reinforcer is related directly to the number of reinforcements obtained and does not require comparison with another reinforcer as in choice tests. The sipometer can also be used in conjunction with neuroimaging studies to test hypotheses about motivation while a subject is actually working for the reinforcer. The last advantage is that sipping is a natural behavior connected to consuming beverages, and therefore no special training is required for the use of the technique. This advantage could be a particularly important for the measurement of reward in individuals who are cognitively impaired, immature, or simply not capable of performing some of the complex behavioral tasks required by the other measures of motivation and reinforcing value. There are two main limitations. First, individuals report that working for the reinforcers is slightly aversive and this aversion can potentially cause their devaluation. This effect has been measured by comparison of ratings of liking after individuals work for equal amounts of reinforcer on continuous reinforcement or progressive ratio schedules. The other limitation is that only liquid reinforcers can be used. However, given that one fifth of energy intake in the US is attributable to beverages (Barquera et al., 2008), this is not a severe limitation.

6. Critique of wanting and liking dichotomy and PR schedule sipping

6.1. Ambiguity in existing theory and practice

Both hedonic and motivational processes are altered in obese compared to non-obese individuals. However, it is not clear a) whether it is productive to dichotomize these processes in humans, although the separation in animals is clearer, and b) whether work for a reinforcer measures “reward value”, “reinforcing value” and “food reinforcement”, as motivational or hedonic (.i.e. experienced pleasure) processes, or both. The main problem is whether the operations used to measure liking and wanting simply capture different manifestations of the same underlying process, or two different processes. Ambiguity exists because the transition from deprived to non-deprived state (or the converse), is accompanied by processes that are both behavioral (interest in and willingness to work for, reinforcer) and cognitive (pleasure from reinforcer), both of which are reflected in the operations that supposedly measure wanting and liking. Alternatively, changes in the composition and consequent perception of the reinforcer (e.g. sweet or non-sweet) can change both hedonic and motivational processes. There has been a running debate in the field since Finlayson and Blundell first proposed their operational separation with their questioning paper (Finlayson et al., 2007a) and follow up review in 2007 (Finlayson, King, & Blundell, 2007b). Without going into details which are available in the following: (Finlayson & Dalton, 2012), (Havermans, 2011) (Havermans, 2012), there is still no consensus about whether wanting and liking can be separately assessed by non-subjective operations.

There have been further attempts to fill both the practical and theoretical gaps necessary to determine whether PR responding measures reward value, motivation, or both, and whether the distinction is unnecessary, but there appears to be no consensus. One recent review (Pool, Sennwald, Delplanque, Brosch, & Sander, 2016) proposed that expected pleasure could be an important determinant, but this construct has its own set of difficulties (Kahneman & Snell, 1992). In our own work (Hogenkamp et al., 2017), we previously proposed that the manipulations, deprivation to measure motivation, sucrose concentration to measure hedonics, determined what the procedure actually measured. However, given that any manipulation affects both systems, in the absence of a neurological intervention that actually inhibits or excites pathways known for specificity of the two constructs, it remains an open question what responding on the PR schedule actually measures. Nevertheless, the weight of evidence favors the PR schedule or any action to obtain a reward as a measure of the motivational, rather than the affective/hedonic axis, because motivation requires action, whereas pleasure is simply experienced, and is therefore passive. It is not possible to infer liking or pleasure simply from the fact that a subject is willing to work for a reinforcer. Nor can one conclude that because a reinforcer no longer supports action, that it is no longer eliciting pleasure. After consuming a large amount of a pleasing reinforcer, individuals may stop working to receive the reinforcer, and while their momentary liking (i.e. pleasantness), measured independently, after consumption, declines, it does not decrease to zero (Rolls, Rolls, Rowe, & Sweeny, 1981). Consequently, although sweet tasting beverages may elicit more pleasure than non-sweet tasting beverages, in our studies with the sipometer ((Hogenkamp et al., 2017) the motivating aspect of that pleasure was captured by the sipping response, and questionnaires used in conjunction with the task captured the hedonic aspect. Hence attribution of reinforcing or reward “value” to a stimulus because work was used to obtain it, implies that the “value” supports motivation, but does not necessarily imply that it was pleasant. It is thus possible to have motivation in the absence of pleasure and pleasantness in the absence of motivation.

6.2. Use of the PR response to discriminate liking from wanting can still be useful

The inability of an intervention, or even a theory, to identify a process with certainty, does not mean we should give up trying to create better theories or find the underlying mechanisms by which rewards generate behavior. It is important to continue the effort to develop theoretical pathways by which reinforcers exert their effects. If such development can identify biological mechanisms that underlie disturbances in motivated/rewarded behaviors, interventions targeting motivational disturbances could be developed and tested experimentally. The sipometer and other measures simply tell us how strongly the subject responds to the stimulus, but they do not measure the intensity of any of these variables (liking, wanting, etc.) by themselves. If the items in the experimental model are amenable to measurement and direction of action, it is possible to evaluate the entire model or its individual component’s contributions, but because there are linkages, it is not possible to measure the contribution of independent components by themselves unless the components can be somehow unlinked. The importance of differentiating mechanisms of wanting and liking was highlighted in a recent editorial (Finlayson, 2017), which provides a rationale for continuing the search for mechanisms underlying these constructs in humans.

6.3. Future applications

Measurement of both the motivational and hedonic properties of foods and beverages is important for understanding how both motivation and hedonics can be disturbed in patients, who can then be treated for the disturbance. The sipometer has potential application for both measurement and treatment. Individuals could be trained to tolerate delay of reward and the instrument could be used to measure progress in treatment for impulsive eating. More generally, there is a need for development of equipment that can both monitor and manipulate rewarding aspects of ingestion. The application of microchip technology for the treatment and monitoring of ingestive behavior disorders should be a high priority for the scientific community through the detection and provision of feedback for motivation to consume, and for the pleasure that consumption induces.

Acknowledgments

The first author thanks Anthony Sclafani for his continuous colleagueship over the past 40 years, and specifically for supporting the development and testing of the sipometer at St. Luke’s Hospital. The first author also thanks Xavier Pi-Sunyer, director of New York Obesity Nutrition Center, NIH/NIDDK DK26687 currently at Columbia University Medical Center for continued support of his work. This paper is based on a presentation given to the Columbia University Seminar on Appetitive Behavior, H. Kissileff, Chair, M. Herzog, Rapporteur, May 5, 2016. The Seminar was supported by DuPont Nutrition and Health, New York Obesity Nutrition Research Center, PepsiCo, and the Columbia University Seminars, Robert Pollack, Director. Anthony Sclafani (Emeritus Distinguished Professor, Brooklyn College, Dept of Psychology), and Rebecca Klatzkin (Assistant Professor of Psychology, Neuroscience Program, Rhodes College) are thanked for their constructive comments and suggestions on the manuscript.

References

  1. Barquera S, Hernandez-Barrera L, Tolentino ML, Espinosa J, Ng SW, Rivera JA, et al. Energy intake from beverages is increasing among Mexican adolescents and adults. The Journal of Nutrition. 2008;138:2454–2461. doi: 10.3945/jn.108.092163. [DOI] [PubMed] [Google Scholar]
  2. Berridge KC. Food reward: Brain substrates of wanting and liking. Neuroscience & Biobehavioral Reviews. 1996;20:1–25. doi: 10.1016/0149-7634(95)00033-b. [DOI] [PubMed] [Google Scholar]
  3. Berridge KC. Motivation concepts in behavioral neuroscience. Physiology and Behavior. 2004;81:179–209. doi: 10.1016/j.physbeh.2004.02.004. [DOI] [PubMed] [Google Scholar]
  4. Berridge KC, Ho CY, Richard JM, DiFeliceantonio AG. The tempted brain eats: Pleasure and desire circuits in obesity and eating disorders. Brain Research. 2010;1350:43–64. doi: 10.1016/j.brainres.2010.04.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bickel WK, Johnson MW, Koffarnus MN, MacKillop J, Murphy JG. The behavioral economics of substance use disorders: Reinforcement pathologies and their repair. Annual Review of Clinical Psychology. 2014;10:641–677. doi: 10.1146/annurev-clinpsy-032813-153724. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bickel WK, Marsch LA, Carroll ME. Deconstructing relative reinforcing efficacy and situating the measures of pharmacological reinforcement with behavioral economics: A theoretical proposal. Psychopharmacology. 2000;153:44–56. doi: 10.1007/s002130000589. [DOI] [PubMed] [Google Scholar]
  7. Bobroff EM, Kissileff HR. Effects of changes in palatability on food intake and the cumulative food intake curve in man. Appetite. 1986;7:85–96. doi: 10.1016/s0195-6663(86)80044-7. [DOI] [PubMed] [Google Scholar]
  8. Born JM, Lemmens SG, Martens MJ, Formisano E, Goebel R, Westerterp-Plantenga MS. Differences between liking and wanting signals in the human brain and relations with cognitive dietary restraint and body mass index. The American Journal of Clinical Nutrition. 2011;94:392–403. doi: 10.3945/ajcn.111.012161. [DOI] [PubMed] [Google Scholar]
  9. Bueter M, Miras AD, Chichger H, Fenske W, Ghatei MA, Bloom SR, et al. Alterations of sucrose preference after Roux-en-Y gastric bypass. Physiology & Behavior. 2011;104:709–721. doi: 10.1016/j.physbeh.2011.07.025. [DOI] [PubMed] [Google Scholar]
  10. Bulik CM, Brinded EC. The effect of food deprivation on the reinforcing value of food and smoking in bulimic and control women. Physiology and Behavior. 1994;55:665–672. doi: 10.1016/0031-9384(94)90042-6. [DOI] [PubMed] [Google Scholar]
  11. Craig W. Appetites and aversions as constituents of instincts. The Biological Bulletin. 1918;34:91–107. [Google Scholar]
  12. Dalton M, Finlayson G. Psychobiological examination of liking and wanting for fat and sweet taste in trait binge eating females. Physiology & Behavior. 2014;136:128–134. doi: 10.1016/j.physbeh.2014.03.019. [DOI] [PubMed] [Google Scholar]
  13. Drewnowski A, Grinker JA, Hirsch J. Obesity and flavor perception: Multidimensional scaling of soft drinks. Appetite. 1982;3:361–368. doi: 10.1016/s0195-6663(82)80054-8. [DOI] [PubMed] [Google Scholar]
  14. Dwyer DM. Licking and liking: The assessment of hedonic responses in rodents. The Quarterly Journal of Experimental Psychology. 2012;65:371–394. doi: 10.1080/17470218.2011.652969. [DOI] [PubMed] [Google Scholar]
  15. Epstein AN. The physiology of thirst. In: Pfaff D, editor. Physiological mechanisms of motivation. New York: Springer-Verlag; 1982. pp. 25–55. [Google Scholar]
  16. Epstein LH, Dearing KK, Paluch RA, Roemmich JN, Cho D. Price and maternal obesity influence purchasing of low-and high-energy-dense foods. The American Journal of Clinical Nutrition. 2007;86:914–922. doi: 10.1093/ajcn/86.4.914. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Epstein LH, Leddy JJ, Temple JL, Faith MS. Food reinforcement and eating: A multilevel analysis. Psychological Bulletin. 2007;133:884–906. doi: 10.1037/0033-2909.133.5.884. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Epstein LH, Salvy SJ, Carr KA, Dearing KK, Bickel WK. Food reinforcement, delay discounting and obesity. Physiology & Behavior. 2010;100:438–445. doi: 10.1016/j.physbeh.2010.04.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Epstein LH, Temple JL, Neaderhiser BJ, Salis RJ, Erbe RW, Leddy JJ. Food reinforcement, the dopamine D2 receptor genotype, and energy intake in obese and nonobese humans. Behavioral Neuroscience. 2007;121:877–886. doi: 10.1037/0735-7044.121.5.877. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Epstein LH, Truesdale R, Wojcik A, Paluch RA, Raynor HA. Effects of deprivation on hedonics and reinforcing value of food. Physiology and Behavior. 2003;78:221–227. doi: 10.1016/s0031-9384(02)00978-2. [DOI] [PubMed] [Google Scholar]
  21. Finlayson G. Food addiction and obesity: Unnecessary medicalization of hedonic overeating. Nature Reviews Endocrinology. 2017;13:493–498. doi: 10.1038/nrendo.2017.61. [DOI] [PubMed] [Google Scholar]
  22. Finlayson G, Dalton M. Current progress in the assessment of “liking” vs. “wanting” food in human appetite. Comment on “You Say it’s Liking, I Say it’s Wanting”. On the difficulty of disentangling food reward in man. Appetite. 2012;58:373–378. doi: 10.1016/j.appet.2011.10.011. [DOI] [PubMed] [Google Scholar]
  23. Finlayson G, King N, Blundell JE. Is it possible to dissociate “liking” and “wanting” for foods in humans? A novel experimental procedure. Physiology & Behavior. 2007a;90:36–42. doi: 10.1016/j.physbeh.2006.08.020. [DOI] [PubMed] [Google Scholar]
  24. Finlayson G, King N, Blundell JE. Liking vs. wanting food: Importance for human appetite control and weight regulation. Neuroscience & Biobehavioral Reviews. 2007b;31:987–1002. doi: 10.1016/j.neubiorev.2007.03.004. [DOI] [PubMed] [Google Scholar]
  25. Gondek-Brown M, Sclafani A, McNally S, Gordon R, Farkas J, Torres M, et al. Measuring the reward value of aspartame-sweetened yogurt shake via continuous and progressive ratio schedules in humans. Columbia Undergraduate Science Journal. 2007;2 [Google Scholar]
  26. Grill HJ, Norgren R. Taste reactivity test. I. Mimetic responses to gustatory stimuli in neurologically normal rats. Brain Research. 1978;143:263–279. doi: 10.1016/0006-8993(78)90568-1. [DOI] [PubMed] [Google Scholar]
  27. Guttman N. Operant conditioning, extinction, and periodic reinforcement in relation to concentration of sucrose used as a reinforcing agent. Journal of Comparative and Physiological Psychology. 1953;46:213–224. doi: 10.1037/h0061893. [DOI] [PubMed] [Google Scholar]
  28. Havermans RC. “You Say it’s Liking, I Say it’s Wanting”. On the difficulty of disentangling food reward in man. Appetite. 2011;57:286–294. doi: 10.1016/j.appet.2011.05.310. [DOI] [PubMed] [Google Scholar]
  29. Havermans RC. How to tell where “liking” ends and “wanting” begins. Appetite. 2012;58:252–255. [Google Scholar]
  30. Hodos W. Progressive ratio as a measure of reward strength. Science. 1961;134:943–944. doi: 10.1126/science.134.3483.943. [DOI] [PubMed] [Google Scholar]
  31. Hogenkamp PS, Shechter A, St-Onge MP, Sclafani A, Kissileff HR. A sipometer for measuring motivation to consume and reward value of foods and beverages in humans: Description and proof of principle. Physiology & Behavior. 2017;171:216–227. doi: 10.1016/j.physbeh.2017.01.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Kahneman D, Snell J. Predicting a changing taste: Do people know what they will like? Journal of Behavioral Decision Making. 1992;5:187–200. [Google Scholar]
  33. Kalva JJ, Sims CA, Puentes LA, Snyder DJ, Bartoshuk LM. Comparison of the hedonic general Labeled Magnitude Scale with the hedonic 9-point scale. Journal of Food Science. 2014;79 doi: 10.1111/1750-3841.12342. [DOI] [PubMed] [Google Scholar]
  34. Lemmens SG, Schoffelen PF, Wouters L, Born JM, Martens MJ, Rutters F, et al. Eating what you like induces a stronger decrease of ‘wanting’ to eat. Physiology & Behavior. 2009;98:318–325. doi: 10.1016/j.physbeh.2009.06.008. [DOI] [PubMed] [Google Scholar]
  35. McGregor IS, Saharov T, Hunt GE, Topple AN. Beer consumption in rats: The influence of ethanol content, food deprivation, and cocaine. Alcohol. 1999;17:47–56. doi: 10.1016/s0741-8329(98)00033-0. [DOI] [PubMed] [Google Scholar]
  36. Miras AD, Jackson RN, Jackson SN, Goldstone AP, Olbers T, Hackenberg T, et al. Gastric bypass surgery for obesity decreases the reward value of a sweet-fat stimulus as assessed in a progressive ratio task. American Journal of Clinical Nutrition. 2012;96:467–473. doi: 10.3945/ajcn.112.036921. [DOI] [PubMed] [Google Scholar]
  37. Mischel W, Ebbesen EB, Raskoff Zeiss A. Cognitive and attentional mechanisms in delay of gratification. Journal of Personality and Social Psychology. 1972;21:204. doi: 10.1037/h0032198. [DOI] [PubMed] [Google Scholar]
  38. Moss FA. Study of animal drives. Journal of Experimental Psychology. 1924;7:165. [Google Scholar]
  39. Myers KP. The convergence of psychology and neurobiology in flavor-nutrient learning. Appetite. 2017;122:36–43. doi: 10.1016/j.appet.2017.03.048. [DOI] [PubMed] [Google Scholar]
  40. Nasser JA, Evans SM, Geliebter A, Pi-Sunyer FX, Foltin RW. Use of an operant task to estimate food reinforcement in adult humans with and without BED. Obesity. 2008;16:1816–1820. doi: 10.1038/oby.2008.281. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Peryam DR, Pilgrim FJ. Hedonic scale method of measuring food preferences. Food Technology. 1957;11:9–14. [Google Scholar]
  42. Pool E, Sennwald V, Delplanque S, Brosch T, Sander D. Measuring wanting and liking from animals to humans: A systematic review. Neuroscience & Biobehavioral Reviews. 2016;63:124–142. doi: 10.1016/j.neubiorev.2016.01.006. [DOI] [PubMed] [Google Scholar]
  43. Reilly S. Reinforcement value of gustatory stimuli determined by progressive ratio performance. Pharmacology Biochemistry and Behavior. 1999;63:301–311. doi: 10.1016/s0091-3057(99)00009-x. [DOI] [PubMed] [Google Scholar]
  44. Rolls BJ, Rolls ET, Rowe EA, Sweeny K. Sensory specific satiety in man. Physiology and Behavior. 1981;27:137–142. doi: 10.1016/0031-9384(81)90310-3. [DOI] [PubMed] [Google Scholar]
  45. Schebendach J, Broft A, Foltin RW, Walsh BT. Can the reinforcing value of food be measured in bulimia nervosa? Appetite. 2013;62:70–75. doi: 10.1016/j.appet.2012.11.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Scherman N, Zlotowitz M. The complete ArtScroll Siddur: Weekday, sabbath, festival: A new translation and anthologized commentary. Mesorah Publications; 1985. [Google Scholar]
  47. Sclafani A, Ackroff K. Reinforcement value of sucrose measured by progressive ratio operant licking in the rat. Physiolology and Behavior. 2003;79:663–670. doi: 10.1016/s0031-9384(03)00143-4. [DOI] [PubMed] [Google Scholar]
  48. Sheffield FD, Roby TB, Campbell BA. Drive reduction versus consummatory behavior as determinants of reinforcement. Journal of Comparative and Physiological Psychology. 1954;47:349. doi: 10.1037/h0054471. [DOI] [PubMed] [Google Scholar]
  49. Skinner BF. The operational analysis of psychological terms. Psychological Review. 1945;52:270. [Google Scholar]
  50. Steiner JE, Glaser D, Hawilo ME, Berridge KC. Comparative expression of hedonic impact: Affective reactions to taste by human infants and other primates. Neurosci Biobehav Rev. 2001;25:53–74. doi: 10.1016/s0149-7634(00)00051-8. [DOI] [PubMed] [Google Scholar]
  51. Stice E, Lawrence NS, Kemps E, Veling H. Training motor responses to food: A novel treatment for obesity targeting implicit processes. Clinical Psychology Review. 2016;49:16–27. doi: 10.1016/j.cpr.2016.06.005. [DOI] [PubMed] [Google Scholar]
  52. Teitelbaum P. The use of operant methods in the assessment and control of motivational states. Operant Behavior: Areas of Research and Application. 1966:565–608. [Google Scholar]
  53. Warden CJ. Animal motivation; experimental studies on the albino rat 1931 [Google Scholar]
  54. Willner P, Benton D, Brown E, Cheeta S, Davies G, Morgan J, et al. Depression increases craving for sweet rewards in animal and human models of depression and craving. Psychopharmacology. 1998;136:272–283. doi: 10.1007/s002130050566. [DOI] [PubMed] [Google Scholar]
  55. Winkielman P, Berridge KC, Wilbarger JL. Unconscious affective reactions to masked happy versus angry faces influence consumption behavior and judgments of value. Personality and Social Psychology Bulletin. 2005;31:121–135. doi: 10.1177/0146167204271309. [DOI] [PubMed] [Google Scholar]

RESOURCES